Using Texting to Encourage Participant Engagement and Retention

Bri Hightower worked for EDC’s Center for Children and Technology, where she contributed her methodological expertise to a wide range of EDC studies, including EDC’s and SRI Education’s ongoing summative evaluation of the Ready To Learn initiative. In this post, Bri shared findings from her team’s experience using texting to engage, inform, and retain participants in a recent Ready To Learn home study on the impact of PEG+CAT transmedia on preschoolers. PIs Naomi Hupert and Shelley Pasnik presented findings from the study at the upcoming 2016 AERA Annual Meeting in Washington, DC.

Researchers are beginning to explore the use of texting as a tool to engage families in their children’s education. In an investigation of the impact of PEG+CAT transmedia resources (videos and interactive games) reported on in an EDC press release, our team found that texting helped facilitate New York City families’ engagement and retention in our study. We used texting to share reminders with families and to encourage use of the transmedia resources. From this experience, we have suggestions for other studies that I outline below.   

About the Study
Since 2006, EDC and SRI Education have conducted a series of studies as part of our summative evaluation of the Ready To Learn initiative. For Year 5 of a Ready To Learn grant, our team—led by Naomi Hupert and Shelley Pasnik—conducted a home study with 197 families (87 families in the New York City metropolitan area and 110 families in San Francisco Bay area) with a preschool aged child in a Head Start center, public preschool, or community-based program where 50 percent or more of the children served were from low-income backgrounds. In the study, we randomly assigned families to a PBS KIDS treatment group or a non-treated Business As Usual (BAU) group. We provided the families in the PBS KIDS group with technology resources (a tablet and Chromebook) and asked them to follow a 12-week long curated PEG+CAT transmedia experience designed to help build children’s early mathematics and problem solving skills. Then, we asked families in both groups to record their media use once a week in a media diary that we collected at the midpoint and end of the study.

To remind families to complete their media diaries on the assigned day, and to encourage PBS KIDS families to engage with the PEG+CAT resources, we sent families regular text messages and emails. It is important to note that when families consented to participate in the study, we also asked them to consent to receiving text message and/or email reminders; 92% (n=80) of the New York City metropolitan area families requested to receive text message reminders throughout the study.

How We Communicated
To communicate with families in the study, we added a texting plan to one of our in- house Android tablet devices. We sent messages to New York City families via the Android messages app, an app that is already downloaded on every Android device and allows sending messages to 5 different numbers at a time. The numbers could be blind copied so that participants weren’t able to see the phone numbers of other participants.

Each week, we sent BAU families and PBS KIDS families a text message that reminded them to complete an entry in their media diary. The text message was sent at 7:00 p.m. on a rotating day of the week corresponding with the assigned entry date. At the beginning of the study, the families were provided with a calendar that indicated what days the researchers wanted them to fill out their media diaries and when they would receive text messages from the research team. The evening texts were designed to be delivered around a preschooler’s bedtime in hopes that the timing of the reminder would be convenient for caregivers.

Every Monday at 4:00 p.m., we sent PBS KIDS families an additional text that was related to the theme of the week’s PEG+CAT content and aimed to inspire families to engage with the materials. We sent the text on Monday to give families a full week to engage children in the study resources. We chose 4:00 p.m. based on our belief that caregivers and children were likely to be together—in case caregivers and children wanted to explore the PEG+CAT content right away.

Lessons Learned
Positive: Throughout the study, we only received positive feedback about the text messages from participating families. The study experienced minimal attrition (2.3% of our New York City sample) which may have been influenced by our frequent text message contact with families. Texting families allowed us to get in touch with participants who weren’t responding to more traditional forms of communication, such as phone calls or email messages. In fact, some participants requested to be contacted exclusively via text to accommodate their work schedule, other commitments, or simply, preference.
Suggestion: Families will vary in their preferred communication method, so allow them to select from a variety of options (such as phone calls, email messages, or text messages) if feasible.

Neutral: Some participants requested a desire to control what time text messages were sent to them so that the time they received the reminder texts was more personally convenient.
Suggestion: While you may have chosen a time that you think is convenient for the population you’re working with, allow participants to select the general time (e.g., early morning, late morning, afternoon, early evening, or late evening) they would like to receive text messages to accommodate their individual schedules.

Negative: When we initially attempted to send our text messages via a Google Voice account, Google’s bots labeled us as spam for sending the same message to multiple numbers. Upon searching online for mobile applications or services that would allow us to send mass text messages to individual numbers, all identified options were cost-prohibitive.
Suggestion: For our needs, purchasing an unlimited texting plan on a device that we already had in-house was the most cost-effective option for our study.

EDC and SRI will present more lessons learned from our Ready To Learn and early learning research at the upcoming 2016 Annual Meeting of the American Educational Research Association (AERA). We hope to see you there!

Date: 

Wednesday, April 6, 2016 - 3:30pm