I. Introduction
Autism spectrum disorders (ASD) refers to an umbrella neurodevelopmental disorder spectrum (American Psychiatric Association [APA], 2013). The most recent estimate shows that 1 in 54 children was diagnosed with ASD in 2016 (Maenner, 2023). This estimate is two times higher than the one in 2000, 1 in 150 children (CDC). Such increase in the prevalence of ASD has intensified the demands to support the needs of students with ASD.
Literacy plays a critical role in the life success of children with ASD. However, previous research consistently shows that students with ASD demonstrate difficulties in reading comprehension. Scholars in the field explain the difficulties are resulted from the lack of oral language skills and their unique cognitive profile. It is well documented that students with ASD are known as having relatively higher decoding abilities and lower level of reading comprehension (Chiang & Lin, 2007). Given that proficient readers require a higher level of decoding and language comprehension (Gough & Turner, 1986), it is not surprising why students with ASD hardly become proficient readers. In addition, students with ASD are described as having unique cognitive profiles, including the lack of joint attention and dysfunctioning theory of mind (ToM), which negatively impacts their reading comprehension (Williamson et al., 2012). Not being able to keep up with the attention of others hinders students with ASD from not only having enriched vocabulary but also transitioning from emergent readers to more interactive or independent readers (Kadervek & Rabidoux, 2004). Furthermore, impaired ToM hinders individuals with ASD from engaging in social activities or sharing experiences because they have challenges in understanding others’ perspectives and emotions (Baron-Cohen et al., 1985). Given the substantial negative effects of limited oral language skills and impairments of ToM of children with ASD, it is evident that special education teachers need to use effective strategies that address these two factors: joint attention and ToM.
Dialogic reading (DR) is an instructional technique that helps young children to become an active storyteller while interacting with teachers during the reading (Whitehurst & Lonigan, 1998). DR starts with the teacher’s questions that facilitate the child’s thinking process about the text and come to their own interpretation. Then, the teacher evaluates the student response, expands on the student’s understanding, and repeats the reading process as necessary. DR encourages the student to engage and actively participate in the reading process. To guide student responses, DR uses a specific type of questions (i.e., CROWD: completion, recall, open-ended, wh-questions, and distancing) within the instructional procedure (PEER: Prompt, Evaluate, Expand, and Repeat) (Whitehurst & Lonigan, 1998). DR demonstrates substantial and significant effects in improving young children’s oral language skills, vocabulary, print knowledge listening comprehension and supporting children with language impairments become active storytellers (Dickinson & Smith, 1994; National Early Literacy Panel [NELP], 2008; Whitehurst & Lonigan, 1998). While DR is recognized as an effective strategy to improve students’ oral language development, it did not provide specific tools for securing students’ joint attention, which is critical for young readers to transit to more independent readers. In the interactive to independent literacy model, Kadervek and Rabidoux(2004) posit that without securing the joint attention, young readers cannot engage in literacy activities. In addition to joint attention, for students with ASD, incorporating strategies that facilitate functioning ToM is needed.
The purpose of modified dialogic reading (MDR) is to maximize academic and communicative needs of students with ASD. MDR integrates three different Evidence-based Practices (EBPs): DR, visual support, and systematic prompt (see Figure 1) (Whalon et al., 2015). To acknowledge and motivate student participation, in MDR, adult readers prompt CROWD plus wh-inference or emotion questions within PEEP instructional sequence (Prompt, Evaluate, Expand, Praise [PEEP]). While DR is an EBP that shows effects on oral language skills of students with language impairments and (What Works Clearinghouse [WWC], 2015), visual supports and prompts are the ones that demonstrate strong effects on changing behaviors of students with ASD (Wong et al., 2014). These three strategies are EBPs that special education teachers feel comfortable using in their classrooms (Accardo & Finnegan, 2019). Three existing studies show positive effects of MDR on students with ASD (e.g., Fleury et al. 2016; Whalon, 2015, 2016). Whalon et al. (2015, 2016) have researchers or caregivers implement MDR and showed significant effects on the listening comprehension of students with ASD. Fluery et al. (2016) had paraeducators implement MDR to students with ASD and showed its large effects on listening comprehension. Despite its positive effects on students with ASD, none of these studies were not implemented by teachers. Given that teachers are critical indicators for student outcomes (National Institute of Child Health and Human Development [NICHD], 2000), we need to know more about how accurately teachers can implement MDR with students with ASD.
Implementation fidelity is the extend to using EBPs as intended. Given its critical role to maximize the effects of EBPs (Cook & Cook, 2013), it is logical to provide supports for teachers to improve the correct use of EBPs. Furthermore, the lack of knowledge of how to maintain and generalize skills is known as one of the reasons why teachers do not use EBPs (Cho & Oh, 2016; Scheeler et al., 2016). Professional development (PD) plays a decisive role in supporting teachers’ use of EBPs in fidelity (National Reading Council [NRC], 2001). However, one-time PD showed limited effects on supporting teachers using EBPs (e.g., Hall et al., 2010) because of the lack of reinforcement available to the teachers. In response, multiple scholars (e.g., Fixsen et al., 2005; Kretlow & Bartholomew, 2010; Shin, Park, & Lee, 2021) highlighted providing ongoing support in addition to the one-time PD as a package for teachers to not only transfer their course knowledge into practice but also sustain their skills. To facilitate this process, teachers need at least 20 hours of contact time through a variety of PD formats, including workshops, lectures, and ongoing coaching (Desmione, 2009; Joyce & Showers, 1982; Leko & Brownell, 2009).
Although we know PD and coaching are essential in guiding teachers to use of EBPs, little information is available on teachers using MDR and related support for them. To fill the gap in the literature, the purpose of this study is to examine the effects of PD and ongoing coaching on a special education teacher’s use of MDR in fidelity. The following research questions used to guide this study: How did a MDR PD package impact the special teacher’s correct use of MDR with students with ASD?
II. Methods
This study took place in a self-contained classroom of a private school in the Southeastern metropolitan of the United States. The classroom targets PK to 1st grade students with ASD. Students came to this classroom for their daily reading instructions for 15 to 20 minutes. The classroom had a carpet area for small group readings, including MDR sessions.
This current study includes one special education teacher as a participant. The special education teacher, Melinda(pseudonym), was a Hispanic/Latino female and in her early 40s. Melinda earned a Bachelor of Science in Elementary Education from abroad and a Master of Science in speech therapy. Melinda was certified as a cross-categorical special education teacher and had more than ten years of teaching experience as a special education teacher. She decided to participate in this study since her students were struggling with reading comprehension and communications. She reported that she did not have any previous training related to MDR.
Four students participated in this study as a student participant, and all of them were diagnosed with ASD. Tom was 6.2 years old African-American boy and received speech therapy. He engaged in impulsive behaviors, so Melinda developed a behavioral intervention plan for Tom at the time of his participation in this study. Grace was 4.0 years old Hispanic girl and received speech therapy. She showed difficulty in expressive communication, interpersonal skills, and turn taking. Kevis was 7.4 years old Caucasian boy and received speech therapy. He had limited attentive skills, and this lack of skills led him to difficulty in following teacher’s instructions. Leo was 7.0 years old Caucasian boy and received speech therapy. He showed limited attention and lack of comprehension skills in abstract concepts.
The current study is a sub-project of a large intervention study that focuses on the effects of MDR on student outcomes. In the intervention study (Kang et al., under review), students moved across three experimental phrases: baseline, intervention, and maintenance based on the changing criterion with five consecutive student reading comprehension data.
The purpose of this current study is to examine the effects of the PD package on the teacher’s use of MDR in fidelity while working with students with ASD. Different from the main intervention study (Kang et al., under review) is that this study only focuses on teacher’s fidelity of implementation. Fidelity plays a critical role in maximizing the intended student outcomes while using EBPs (Cook & Cook, 2013). To examine the effects of the PD package on the teacher’s implementation fidelity, we used a case study design by using multiple data sources. For example, we collected data on the teacher’s correct use of the MDR procedures while she was teaching students with ASD. Second, we conducted informal observations to gather contextual information. Case study design is appropriate to this study since it intends to “investigate a contemporary phenomenon in depth and within its real-content, especially when the boundaries between phenomenon and context are not clearly evident” (Yin, 2009, p. 18). It is challenging with one teacher participant to set up methodological rigor when using a multiple baseline design across multiple participants, which also demonstrates higher level of internal validity (WWC, 2015). There are some previous studies that took this stance as well (e.g., Cheek et al., 2002; Whalon et al., 2016).
In the larger project, student participants received MDR with the focus of reading comprehension and initiation skills (Kang et al., under review). As its sub-project, the data collection of this study was conducted within three experimental phases, including baseline, intervention, and maintenance. Between baseline and intervention phases, the teacher received an initial PD training.
During baseline, Melinda continued her typical reading lessons. Once any student participant demonstrated a stable therapeutic direction over five data points (Horner et al., 2005; Krotchwill et al., 2012), the student became the first student participant who needed to receive the MDR from the teacher. In this study, Tom entered the intervention phase at first.
Right before starting the MDR intervention with Tom, Melinda had a two-hour PD session (see Figure 2).
Melinda started using MDR to student participants in the intervention phase. After Tom showed stable five consecutive data points in terms of level, variability, and trend, Melinda chose Grace to enter the intervention next. The same procedures were repeated for the rest of the students. During the intervention, Melinda received informal coaching as necessary from the primary researcher.
All of the student participants entered the maintenance at the same time when six data points were collected on Leo in the intervention phase. During the maintenance, the PD package was not available to Melinda. The effects of MDR training on teacher’s implementation fidelity was documented as present or not in the fidelity checklist and calculated as a percentage of accuracy (see Figure 2).
During intervention and maintenance phases, Melinda used 13 popular children’s trade books and matching question card sets to implement MDR.
Before the intervention, Melinda and the primary researcher selected thirteen books using the following criteria: (a) age-appropriate topics, (b) appropriate for developing emergent literacy skills, (c) story content with colored illustrations, and (d) inclusion of new vocabulary (Hargrave & Sénéchal, 2000). The lengths of all selected books were about 15 pages.
Melinda used a total of thirty-four question sets, twenty-three question sets from Whalon et al. (2015), and 11 question sets developed by the primary researcher of this study. Each question set was expected to be used per session. Each question set included wh-inference and emotion questions and CROWD questions to support students’ ToM (e.g., making inferences, understanding feelings). Each question page had three possible answers with representative pictures made using Boardmaker software (see Figure 1). Some of the question pages included colored dots to prompt the teacher to facilitate students’ participation. For example, red dots marked with I prompted the teacher to use an intentional pause of three to five seconds before turning the page. Blue dots marked with J indicated the time for teachers to remind children to use their joint attention (e.g., [pointing out the picture] look!), while yellow dots marked with Q prompted the teacher’s use of question prompts (Whalon et al., 2015)(see Figure 1).
To evaluate the effects of the PD package on the teacher’s use of MDR in fidelity, the primary researcher collected data on the implementation fidelity during the training and teaching students with ASD as well. Based on MDR elements (Whalon et al., 2015), the primary researcher created the fidelity checklist (Figure 3). The researcher used the fidelity checklist in monitoring the teacher’s fidelity of implementation throughout the study. When the teacher used each element as intended, the researcher recognized the occurrence as fidelity.
The independent variable was a PD package, consisting of an initial two-hour training and informal coaching.
The initial training was intended to improve Melinda’s knowledge about MDR (e.g., using CROWD prompts, PEEP, systematic instruction, attention prompts, and intentional pause). The primary researcher provided direct instruction on how to implement MDR, integrating each component by using power point presentation and visual resources, including a visual map (Figure 4). During the initial training, Melinda had an opportunity to practice how to implement MDR procedures. The primary researcher provided the correct procedure via verbal feedback when her implementation deviated. When she reached 100% correct implementation, the training was over.
During the intervention, Melinda received ongoing informal coaching as necessary when the teacher fidelity checklist indicated potential issues with teacher implementation. The imformal coaching as a format of quick conversation took about 10 to 15 minites in the classroom after school. Each coaching session started with positive statements about her performance. Melinda watched video clips of her MDR implementation, and received positive, specific, and corrective feedback about her implementation. There were several times the first primary researcher of this study reminded her of the content from the initial PD. Each session ends with encouragement to continuously use the strategy during the reading session.
In this study, Melinda led 78 reading sessions. All sessions were video recorded and uploaded on a secured online data cloud system because (a) the primary researcher needed to analyze dependent variables, and (b) the authors need to check interobserver agreement (IOA). In this study, IOA on teacher’s fidelity of implementation was measured across all sessions. The IOA check began by training a second coder. The primary researcher trained a second coder to code sample videos using the fidelity checklist. IOA was calculated by dividing the total number of agreements between first and second coders by the total number of agreements and disagreements, which was then multiplied by 100 (Cooper et al., 2007). The IOA between the first and second coders during the training session was 100% across all phases. This is much higher than 80%, the criteria of the field required for IOA Cooper et al., 2007). Out of a total of 78 videos in this study, we calculated IOA for 28% (n=22) of videos across phases (i.e., baseline, intervention, maintenance). The primary researcher completed fidelity checklists for every session. The IOA between the coders were 97.55%, which met the criteria of acceptable IOA in the field (Cooper et al., 2007).
To investigate Melinda’s perceived effectiveness of PD on MDR, she was asked to complete a 10 question, 5-point Likert-scale (1 = strongly disagree, 5=strongly agree) social validity survey upon the completion of this study. Before completing the survey, Melinda watched selected video clips of her implemented MDR sessions on all phases: baseline, beginning and final interventions, and maintenance. Melinda was asked to response to the questions about: (a) the usefulness of PD package, (b) the impact of PD on teacher knowledge and instructional skills (c) the impacts of MDR on student participants’ targeting skills, including listening comprehension and initiation skills, and (d) the usability of PD.
Melinda strongly agreed that the 2-hour training and on- going coaching was useful for her classroom practice. Regarding the statements asking about her knowledge and confidence for this specific strategy, the teacher strongly agreed that she knew the steps to implement the strategy and she felt confident in implementing the reading strategy with her students. In response to the statement claiming that she would use this strategy in the future, she responded strongly agree. She also strongly agreed that she learned how to implement MDR easily and she felt confident in using MDR with her students. The teacher strongly agreed that she would continue to use MDR in her class.
To examine the effects of PD package on the fidelity of implementation, mean percentage of implementation fidelity was calculated and analyzed across baseline, intervention, and maintenance phases. To do this, the following steps were enacted. First, we calculated the total number of MDR cycles per session, consisting of PEEP. In other words, we considered each MDR cycle as the moment that the teacher asked questions to the point when the teacher praised student participation. The total number of MDR cycles was used as the total number of opportunities for the teacher to implement each MDR component. Second, we counted the frequency of the teacher’s use of MDR components: (a) expanding student responses, (b) praising student participation, (c) using systematic instruction, (d) using joint attention, and (e) using intentional pause. The data were considered as the actual frequency of teacher’s implementation regarding each component. Given each reading session had varied lengths, frequency of each component was converted into a percentage. To calculate the percentage of teacher’s use a specific MDR component, we divided the frequency of each instructional component by the total number of MDR cycles of the session. Then we multiplied it by 100. The same procedure repeated across all MDR components (e.g., expanded student response, praising students). We also conducted two levels of visual analysis (e.g., within-phases and between-phases) to investigate the effects of the PD package on the teacher’s intervention implementation.
III. Results
In this MDR study, Melinda led 78 MDR sessions, including 32 sessions for Tom, 28 sessions for Grace, 29 sessions for Kevin, and 23 sessions for Leo. Depending on student schedules, each student had different group members.
During baseline, Melinda hardly used MDR components except expanding student responses (range = 55.5% - 63.8%). Melinda hardly used praise (range = 26.2% - 30.5%), systematic instruction (range = 5.6% - 10%), attention prompts (range = 10% - 22.3%), and intentional pause (range = 0% - 11.2%) across students. Right after the PD, Melinda showed the fidelity of implementation data indicated high fidelity of teacher implementation during the intervention (84.4% - 100%) and the maintenance (82.3%-100%) phases. Right after an initial 2-hour PD, Melinda showed an immediate increase in the mean percent of using all MDR components to Tom from baseline to intervention: expanding student responses from 55.5% to 83.4%, using praise from 26.2 in baseline to 87%, systematic instruction from 10% to 92.2%, attention prompts 20% and 98.2%, and using intentional pause from 0% to 95.1%. Her implementation fidelity for Tom was consistent across Grace, Kevin, and Leo (see Table 1). During the maintenance across the four students, Melinda maintained high fidelity implementation, 100%, in using multiple MDR components across students.
Our informal observation and visual analysis of her MDR implementation show some variability during her implementations of systematic instruction and praise across students. For example, Melinda was expected to provide least to most prompts when students did not correctly answer within five seconds. However, our informal observation data shows that Melinda asked clarification or repeated questions when the students did not answer correctly within five seconds, which increased variability in the fidelity of implementation. Melinda demonstrated a stable trend in providing systematic instruction to Tom while showing a relative decrease in using systematic instruction to Grace or a very sharp reduction to Kevin. On the other hand, Melinda showed the increase in using systematic instruction to Leo (see Figure 5).
Melinda also demonstrated variability in praising students (Figure 6). The teacher demonstrated an overall decreasing trend of praising Tom during the intervention phase, but she had a big drop during session 28. This occurred after she gave Tell maximum praise (100%) after three consecutive data points (session 26, 27, 28). During the maintenance phase, data was stable. The teacher had a relatively increasing trend during the intervention phase for Grace, with three dips at sessions 17, 21, and 25. During the intervention phase, the teacher participant showed an increasing trend of praising Kevin until session 27, but her use of praise had a sharp drop by session 30. At the beginning of maintenance, the teacher showed an increase in praising Kevin, but there was a big drop at session 32. Finally, the teacher’s use of praise for Leo was stable during the intervention phase and showed a relative increase during the maintenance phase.
IV. Discussion
The purpose of this study was to determine (a) the effects of the PD package on the fidelity of the teacher’s implementation of MDR and (b) the teacher’s perceived effectiveness of PD on the use of MDR. Findings from this study showed that the PD package positively affected the teacher participant’s use of MDR as intended, and the teacher felt that the PD and coaching helped her gain related skills and knowledge in correctly using MDR to students with ASD.
The findings of this study provide evidence that the PD package positively affected the teacher’s use of MDR as intended. In this study, Melinda participated in a 2-hour PD and ongoing coaching sessions. Melinda showed immediate gains in implementing each MDR component (e.g., expanding student responses, praising students, using visual support, using attention prompts, using intentional pause) and maintained the higher level of fidelity of implementations. Our findings are consistent with previous studies that examined the effects of PD and coaching on teachers’ use of literacy strategies, in particular for fluency and word knowledge (Brownell et al., 2017), shared reading (Dennis & Horn, 2014; Kaderavek et al.., 2014), and DR (Rezzonico et al., 2015).
Data also showed that MDR is a user-friendly intervention. The teacher indicated that after the 2-hour MDR training and ongoing feedback, she was able to implement this specific strategy with high fidelity. In addition, Melinda became confident in implementing the MDR procedures with her students suggesting that this type of instructional method could be feasible and easily applicable when teaching reading skills to students with ASD. This contrasts with the findings from the previous studies (e.g., Desmione, 2009; Joyce & Showers, 1982; Leko & Brownell, 2009) that emphasized securing more than 20 hours of contact time via different formats of PD to help teachers’ transit of PD knowledge into practice.
Despite improved percentages of the fidelity of implementation after PD, we observed variability in her MDR implementation of systematic instruction and praise across students. Data in this current study show that Melinda sometimes did not follow the intervention protocol, but rather provided systematic instruction or praised students based on their characteristics (e.g., behavior needs) and classroom contexts (e.g., student group dynamics).
Our informal observation indicated that Melinda’s implementation was affected by group dynamics and student behaviors. While Tom, Kevin, and Leo often participated in the small groups together, Grace did not have chances to interact with the other three students. This could explain why the teacher’s behavior was differentiated between three students (Tom, Kevin, and Leo) and Grace from sessions 27 to 30. In short, the teacher was becoming more flexible in praising student participants and using systematic instruction based on her instructional decision of how to better meet their needs. Her efforts could be interpreted as providing support within the zone of proximal development (Vygotsky, 1978).
This study shares multiple implications to researchers and teacher preparation programs. First, researchers need to investigate teachers’ use of MDR more. Three existing studies show that MDR is effective in improving listening comprehension and initiation skills of young children with ASD (Fluery et al., 2016; Whalon et al., 2015, 2016); however, none of the MDR studies were implemented by teachers, but by researchers (Whalon et al., 2015), paraprofessionals (Fluery et al., 2016) or caregivers (Whalon et al., 2016). Given that teachers are a critical indicator for student outcomes (NICHD, 2000), more researchers need to pay attention more about the effects of PD on MDR for students with ASD. In addition, researchers should examine what the most effective PD in MDR in terms of formats, frequency, and length would be. Although existing literature (e.g., Desmione, 2009; Joyce & Showers, 1982; Leko & Brownell, 2014) indicates PD and coaching at least twenty hours of contact time as the effective PD components, studies regarding the MDR are missing from the literature. Third, school districts and teacher preparation programs need to provide more PD opportunities for teachers to get to know about EBPs. In addition, schools and teacher educators continue to encourage teachers to use EBPs and monitor them using those correctly.
We identified limitations of this study, and this offers insights on the future research direction. In fact, having one teacher participants caused multiple limitations. First, the findings of this study are hard to generalize. Therefore, future research needs to replicate this research with more teacher participants to see consistent findings across teachers who have students with ASD. Second, this study has limitation in terms of methodological rigor. In fact, having one teacher implement MDR while students are transiting through baseline, intervention, and maintenance threatened and violated the internal validity. The teacher’s implementation fidelity to Grace, Kevin, and Leo on the baseline might be affected by PD and coaching that she already received to use MDR to Tom. Future research should make careful considerations that minimize the potential threats for internal validity.
According to the recent studies, MDR must be an effective strategy to meet the reading and communicative needs of young children wish ASD. Providing PD and ongoing coaching resulted in the overall higher level of teacher implementation fidelity, which confirms findings of previous studies (Brownell et al., 2017; Dennis & Horn, 2014; Kaderavek et al., 2014; Rezzonico et al., 2015). Furthermore, the current study expanded the previous studies by providing a special education teacher PD package, which help to promote teachers’ use of MDR in their reading class. Given the needs of students with ASD and the increase in the prevalence of the population, more teachers need to use MDR needs to use more in the classroom. This suggests the need for additional studies to general support teachers’ use of MDR and to generalize the effects of support on teacher implemented MDR on student.
