--> Europe's Journal of Psychology ejop.psychopen.eu | 1841-0413 Research Reports Treatment of Dyslexia in a Regular Orthography: Efficacy and Efficiency (Cost-Effectiveness) Comparison Between Home vs Clinic-Based Treatments

The outcomes of three treatments for dyslexia, one clinic-based and two home-based, were compared using a quasi-experimental design for their efficacy and efficiency in improving accuracy and fluency in reading in a large sample of Italian students. The efficacy comparison was based on gain scores in fluency and accuracy of reading texts, and lists of words and nonwords. The efficiency (cost-effectiveness) comparison was based on the ratio of gain scores to the number of hours of treatment. Efficacy and efficiency measures yielded very different results. The efficacy comparison showed a clear superiority of the clinic-based treatment over home-based treatments. The efficiency comparison, on the other hand, showed the superiority of a home-based treatment. The importance of considering both efficacy and cost-effectiveness in any comparison of treatment outcomes is discussed within the framework of the dissemination of evidence-based treatments.

The aim of this study is to compare the efficacy and efficiency (cost-effectiveness) of two home-based versus one clinic-based treatment for developmental dyslexia.There is considerable evidence showing that specific interventions can improve reading in terms of accuracy and fluency for children with developmental dyslexia from both regular and less regular orthographies.The following references are only a sample of all studies carried out to date: Bakker (2006), Scammacca et al. (2007), Tijms (2007), Tressoldi, Lorusso, Brenbati, & Donini (2008), Wexler, Vaughn, Edmonds, & Reutebuch (2008).
Very few studies however have added cost-effectiveness information to their efficacy analysis.As regards accuracy, Torgesen et al. (2001) compared the results of an intervention that placed primary instructional emphasis on building skills in phonemic awareness and phonemic decoding (i.e., phonemes discrimination, spelling, etc.) and another intervention placing more emphasis on their application while reading meaningful text, with the results obtained in another six studies using similar training with students of the same ages.They compared gains in standard score points per hour of instruction for three measures of reading skills.The gains ranged from 0.30 to 2.57 for phonemic decoding, from 0.13 to 0.23 for word identification and from 0.12 to 1.7 for passage comprehension.These differences are not trivial, but were not discussed further by the authors.Tijms (2007)  receiving treatment for dyslexia for twelve to fifteen months.The treatment was computer-based and focused on learning to recognise and use the phonological and morphological structure of Dutch words.The treatment consisted of several modules, each addressing specific links between phonological concepts and the writing system.A cost-effectiveness analysis revealed that the treatment produced a gain of 0.60 to 0.74 in standard score points (M = 100, SD = 15) per teacher hour.
The importance of studying efficiency is crucial for the generalisation and wide implementation of empirically-supported treatments.For each unit of time, i.e., one hour, there is a corresponding economic cost in terms of charges to the National Health Service or educational economic resources and/or to customers.If the question "Is this treatment effective with respect to....?" is answered positively, the natural corollary is to ask "How much does it cost with respect to ....?" (Cambridge & Knapp, 1997).Two well-known factors that limit the dissemination of effective treatments are cost and treatment availability (Higa & Chorpita, 2008;Henggeler, Lee, & Burns, 2002).Customers and private or public insurers cannot pay, regardless of cost, even if the treatments concerned are proven to be efficient.Increased cost-effectiveness would make treatments accessible to more individuals in need of assistance.Patients would enjoy rapid treatment gains, and this would also improve the credibility of the treatment and increase the motivation for further change.
The aim of this study is to compare efficiency in terms of cost-benefit analysis, among three treatments with previous evidence of efficacy in improving the accuracy and fluency of reading in a regular orthography.One of the treatments was clinic-based, that is, children received treatment at a centre run by professional speech therapists or psychologists.Apart from the costs levied by the clinic, there were further costs, including travel expenses from the patient's home to the clinic and non-monetary costs, such as the time necessary to travel from home to the clinic.
The other two treatments were home-based.After assessment, participants and their parents or tutors were trained to implement the treatment activities at home.Treatment integrity and treatment adjustments were monitored periodically, usually fortnightly to monthly, by the treatment supervisor.Costs were clearly lower, because the number of treatment sessions on site was less.Although there was a reduction in the number of treatment sessions directly delivered by experts (thus reducing associated costs), there was no guarantee that the same benefits were obtained when the treatment was implemented by parents or tutors who were not specialised in this field.
This study is the first to be carried out in Italy, but is potentially of interest to all those involved in the assessment of treatment efficiency in terms of cost-effectiveness.
In sum, the principal aim of this study was to answer the question "In term of efficiency (cost-effectiveness), can home-based treatments obtain similar improvements in reading accuracy and fluency as clinic-based treatments?".
We sought to answer this question by first comparing efficacy, that is, differences (gains) from pre-to post-treatment and second efficiency, that is, the ratio of these gains to the hours of intervention, among three different treatments.Home vs Clinic Treatments Efficacy and Efficiency Comparison the consequences of this condition for their everyday lives, particularly at school (see Instruments and Procedure).

Method
To avoid the inclusion of participants who were simply at risk of being diagnosed as dyslexic, we selected participants attending the third to eighth grades, corresponding to a chronological age of eight to thirteen years old and consequently with a minimum of two years of regular schooling and reading instruction.All participants were born in Italy and used Italian as their first oral language.They were enrolled in the different treatment regimens according to their willingness to participate in the treatment offered by the participating clinics and the availability of opportunities for regular follow-up.Given the geographical distance between the different clinical centres, it was impossible to assign participants randomly to the different treatment regimens.However clinical and demographic conditions at baseline were considered in the efficacy and efficiency comparison analysis (see

Results
) with gain scores as the dependent variable covariated with grade i .The different treatments were offered by different clinical centres located in different parts of northern Italy (two) and in the centre of Italy (one) with comparable socio-economic conditions.

Instruments
Reading of all participants was assessed at baseline and after intervention with the following tasks: • text reading drawn from the MT Battery (Cornoldi, Colpo, & Gruppo, 1998) related to the grade of each participant.Each participant was individually tested being required to read as fast and correct as possible, but also to comprehend a text ranging from 250 to 500 syllables for a maximum of four minutes.Fluency (measured in syllable x sec) in reading the text and number of errors were collected.
• reading of isolated words and nonwords, using the lists presented in the DDE-2 Battery (Sartori, Job, & Tressoldi, 2007).Each participant was individually tested being required to read as fast as correctly possible two lists of 28 high frequency words, two lists of 28 low frequency words and three lists of sixteen legal nonwords of different orthographic complexities.Fluency (measured in syllable x sec) and number of errors were collected.
All these instruments obtained good reliability (test-retest correlation above 0.85) and concurrent and predictive validity scores.
IQ was measured with the Italian version of WISC-R or WISC-III and compared with Italian norms (Orsini, 1993;Orsini & Picone, 2006).To be included in this study, a participant had to have a total IQ above 85 to exclude participants with intellectual disabilities.IQ was not included in the analysis because previous studies showed that IQ did not predict the responsiveness of children to therapy (Ferrer, Shaywitz, Holahan, Marchione, & Shaywitz, 2010;Francis et al., 2005;Jimenez et al., 2003;Stuebing, Barth, Molfese, Weiss, & Fletcher, 2009).

Procedure
All treatments were provided on a 1:1 basis.
For all treatments, parents were informed about the requirements of participation in the treatment prior to children enrolment, i.e. the expected duration and treatment requirements, such as regular attendance at the clinic or daily training at home.
The common theoretical framework of all three treatments was the Simple View model (Hoover & Gough, 1990;Joshi & Aaron, 2000) and Goswami's development of reading model (Goswami, 2008).
Common characteristics in the treatments, consisted of reading texts facilitating the identification of syllables, the sublexical units that are more consistent in regular orthographies (Carreiras, Alvarez, & Devega 1993;Carreiras & Grainger, 2004), to build up orthographic representations of recurrent syllables to achieve faster, automatic direct word recognition.In Italian, the correspondence between syllables and phonology approximates 99% regularity.For example, the syllable'pa' is pronounced /pa/ in whichever word and position, as in patate (potatoes), scarpata (escarpment), or scarpa (shoe).
The choice to present syllables within connected texts was justified by evidence that shows there is greater generalizability if words are presented in context than in lists.Martin-Chang and Levy (2005), for example, showed that training words in context, as compared with training in isolation, led to the faster reading of those words when they were later encountered in a new context for both good and poor readers.

Treatments Home-1 and Home-2
Both of these clinics, adopted home-based treatments.The minimal requirements were twofold: the availability of a computer with hardware characteristics sufficient to run the special software designed to promote fluency and accuracy of text reading (mainly WinABC® or Reader® ii ) at home; the commitment to practice for at least ten minutes a day under the supervision of a tutor (usually a parent) for three months.Exercises consisted mainly of reading text of different length, difficulty level, and content, to meet the child's preferences.The software options facilitate the visual identification of each syllable (i.e.inserted in a box or coloured differently).For example, with the word "palazzo" (palace), the identification of the three syllables could be facilitated as follows: palazzo, palazzo, palazzo.The shift of the target syllable from left to right could be obtained at a self-paced speed, by pressing the space bar of the computer keyboard or automatically, setting the shift-time using the software options.The participant was invited to read the text accurately and as fast as he or she could, but still paying attention to its content.
If the advancement of the target syllables was self-paced, the participant was invited to aim for the velocity goal defined by the therapist.If the syllable advancement was automatic, the participant was invited to maintain the fluency imposed by the computer.
Reading errors were registered by the tutor and used for subsequent feedback.When the child met the fluency goal with a percentage of errors below 3% of the number of words in the text, the treatment supervisor increased the velocity goal gradually, usually adding 0.2 syllables per second at each increment.
To check treatment fidelity, parents or tutors of participants were requested to keep a diary of the type and amount of daily exercises and were monitored by experienced clinicians approximately every fifteen days by phone, email or direct interview.This enabled the clinicians to monitor the correct implementation of their recommendations, support parents, motivate participants to continue the treatment and change software parameters to improve accuracy and fluency when necessary.The two treatments differed only in the modality used to present the target syllables.In treatment Home-1, the syllable was tackled at a self-paced rate, whereas in treatment Home-2, it was presented at a fixed rate.This difference may have had important implications for the rapid identification of syllables and automatisation of their recognition (Tressoldi, Vio, & Iozzino, 2007).Home vs Clinic Treatments Efficacy and Efficiency Comparison

Treatment Clinic
Differently from the Home-1 and Home-2 treatments, specific exercises were applied to reduce errors according to their position during text reading.For example, participants were trained to identify errors in the initial or final graphemes, or syllables, of words in texts when there were recurrent errors in this part of the word.When accuracy was considered sufficient, participants were trained in fluency, by means of the same special software designed to present text rather than isolated words used in the home-based treatments).Participants attended two sessions a week, lasting approximately 45 minutes each, for approximately two months.

Results
The descriptive statistics of the main variables that were identified at baseline assessments, are presented in Table 1.

Table 1
Means and standard deviations (in parenthesis) at baseline of main demographic and reading measures of participants of the three treatments.For each treatment we registered the mean number of hours of effective intervention and the months of its duration.

Nonwords
The ratio between hours and months gives an index of treatment intensity (TI), hours x months (see Table 2).

Table 2
Mean and standard deviation (in parenthesis) of hours, months and treatment intensity (TI) of each treatment

Baseline Comparison
We report the clinical significance of the differences.For dyslexic children, a difference of at least 0.3 syllables/second (syll/sec) is considered clinically significant (Tressoldi, Stella, & Faggella, 2001), which corresponds to the annual expected change in fluency reading of texts and isolated words.In addition, a difference of at least 0.15 syll/sec, is considered clinically significant in relation to reading nonwords without specific intervention.
According to this criterion, participants in treatments Home-1 and Home-2 show lower fluency in reading text and participants in treatments Home-1 show lower fluency in reading isolated words, whereas participants in treatment Home-1 and Home-2 are considered to be at the lower level of fluency in reading nonwords.For accuracy, we chose a difference of at least 20% to be clinically significant.The choice was based on the fact that this difference corresponds to a different level of accuracy severity according to Italian normative data.With this criterion, participants in treatments Clinic and Home-2 show a lower accuracy in reading text, and participants in treatments Home-1 and Home-2 show a lower accuracy in reading words whereas no clinical differences are observed for nonwords.For the variable 'grade', participants in treatments Home-1 were shown to be more than one school grade younger than participants in the two other treatments.

Outcome Comparison Among the Three Treatment Approaches
In order to take into account the differences at the baseline assessment, in relation to each reading task, text, words and nonwords, we calculated the gain scores weighted for grade (used as a covariate) and the corresponding confidence intervals of fluency expressed in syll/sec and accuracy, expressed as number of errors at outcome.
It is important to remember that for regular orthographies such as German or Italian, accuracy is not the main problem to be solved.Children with dyslexia may read in a relatively correct fashion but their reading is characteristically slow and laborious (Wimmer,1993;Zoccolotti et al., 1999).
From a clinical and practical point of view, it is more interesting to calculate the effect sizes of the comparisons between the outcomes obtained with the three treatments.We calculated the effect size d (Cohen, 1988) and the improvement index (II) as suggested by Valentine and Cooper (2003).Effect size d was calculated with the formula Mt1-Mt2/pooled SD, where Mt1 and Mt2 represents the means of the gain scores of the two groups being compared (Morris, 2008).The improvement index (II) represents the difference between the percentile rank corresponding to the intervention group mean, and the percentile rank, corresponding to the comparison group mean (that is, the 50th percentile) in the comparison group distribution.Alternatively, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if the student had received the intervention.
Results of fluency are presented in Figure 1 and results of accuracy are presented in Figure 2 (results in tabular form are reported in the Appendix).It is important to consider that not all participants to Home-2 treatment were assessed for words and nonwords reading (see number in the Appendix).Although we report both accuracy and fluency treatment outcomes, it is important to remember that the more important are those related to fluency.For text and word fluency treatment Clinic obtained the best results with the following effect size differences with respect the other two treatments: for text fluency, the differences of Clinic with treatment Home-1 and Home-2, are respectively d=1.17; II= 38% and d=0.70;II= 26%; for word fluency the differences of Clinic with treatment Home-1 and Home-2, are respectively, d=0.80;II= 29% and d=0.50;II= 19%.
For nonword fluency, treatment Clinic obtained the best results.The differences of treatment Clinic with Home-1 and Home-2, are respectively d=0.76; II= 28%, d=0.32;II= 13%.
The superiority of treatment Clinic confirms the importance of the quality of treatment.Even if treatment intensity was identical to treatment Home-2, an intervention delivered by an expert is clearly more tailored to the individual reading and personality needs of each child.For text accuracy, treatment Home-2 obtained the best results.The differences of treatment Home-2 with treatments Clinic and Home-1, are respectively d=0.25; II = 10% and d=0.62;II = 23%.For words and nonwords accuracy all treatments reduced errors to the same amount.
To summarise, treatment Clinic, obtained the best results for fluency.For accuracy, differential efficacy among treatments was observed only in reading text where treatments Home-2 obtained the best results.

Efficiency Comparison
The measure of efficiency is the relationship between amount of treatment and outcome results.To compare efficiency among treatments, we calculated the ratio between gain scores and hours of treatment.The choice between hours and months can be justified in economic terms because consumers pay per hour of treatment, irrespective of the TI.Similarly to the efficacy comparisons, we calculated the effect size d and the improvement index (II) for each statistically significant difference.
For text fluency, treatment Home-1 outperformed all other treatments with a difference with treatment Clinic of d=0.57;II = 22%, and a difference of d= 1.85; II= 47% with Home-2 treatment.Almost identical results are observed for word fluency.The difference between treatment Home-1 and treatment Clinic is d=0.65;II=24%, whereas the difference with Home-2 treatments is d=1.2;II=38%.For nonword fluency, treatment Clinic outperformed with a small difference, treatments Home-1 and Home-2 corresponding to a d=0.30;II= 12%.For accuracy, treatment Home-1 obtained the best outcomes in all three types of reading tasks.For text accuracy, the difference with treatment Clinic is d=0.40;II=16%, whereas the difference with Home-2 is d=0.77;II=28%.For word accuracy the difference of treatment Home-1 with respect to the two other treatments equals to d=1.0; II=34%, whereas the difference for nonword accuracy with respect to the two other treatments equals to d= 0.80; II=29%.To summarize, treatment Home-1 obtained the best results x hour of intervention in both fluency and accuracy in all three reading measures, followed by treatment Clinic.

Discussion
The comparisons of outcomes obtained with the different treatments changed dramatically depending on whether we used efficacy or efficiency measures.Using efficacy measures, treatment Clinic, obtained the best results.
Conversely, using efficiency measures, treatment Home-1 obtained the best outcomes.
The importance of considering efficacy parameters as well as efficiency (cost-effectiveness) parameters in The superiority of treatment Home-1 over treatment Home-2, can partially be explained by the differences in the training characteristics, for example the modality used to present the target syllables, self-paced versus fixed rate respectively.This is an important topic to study, but at present there is insufficient evidence to support the superiority of one of these modalities.However, even if this difference may have contributed to the outcomes, we cannot exclude the possibility that other variables, such as individual differences in reading level, motivation, etc., contributed to the differences in outcomes.
If we assume that each hour of treatment will obtain similar outcomes iv , we can estimate that the clinic-based treatments and treatment Home-2 need respectively approximately half an hour (0.094/0.069) and three and a half hours (0.094/0.025) more to obtain the fluency results in reading texts obtained with treatment Home-1 and approximately half an hour (0.7/0.45) and two hours (0.7/0.38) more respectively to obtain the accuracy results achieved in treatment Home-1.From these simple comparisons which can be extended to the outcomes related to words and nonwords, the differences in cost-effectiveness among treatments are quite apparent.
The data observed in this study cannot be generalised to other treatments.We believe however, that we have presented sufficient reason to support the importance of complementing efficacy comparisons with efficiency comparisons.Efficiency (cost-effectiveness) measures are the basis for any economic decision, both at a governmental level and at a private level and efficiency measures are fundamental to the successful dissemination of previously proven effective treatments.
This approach is emphasized by different authors (i.e.Duncan & Magnuson, 2007;Ludwig & Phillips, 2007).For example Duncan and Magnuson state: Although effect sizes can help standardize (and compare) results and ensure that statistical significance will not be the sole arbiter of meaningful effects, we argue that they provide incomplete and at times misleading guidance to policymakers.
A cost-benefit approach is more useful because evidence-based policy decisions must compare the value of a program's effects with the costs incurred in achieving them.An inexpensive program that produces small but economically valuable outcomes may make for good policy, whereas a very expensive program that produces larger but not proportionately larger effects may not (Duncan, & Magnuson, 2007, p. 46).
Given the feasibility of home-based treatments, clinicians should devise means of training parents, teachers and/or educators to deliver trainings with a sufficient standard of quality to achieve the best possible outcome for every dyslexic child.

Limitations and Future Research
If we compare our experimental design with those recommended by the What Works Clearinghouse (WWC), namely (a) the use of random assignment, (b) evidence of the use of a check of fidelity of treatment, and (c) the use of standardised measurement, we see that our experimental design suffers from the lack of random assignment of participants to the three treatments.The justification for this lack of random assignment was presented in the Participants section and we consider that the use of gain scores covariate with grade may sufficiently have taken into account the differences observed at baseline although we hope to replicate this study using the recommended random assignment of participants.Tressoldi, Brembati, Donini et al.
We hope that further comparative studies of reading interventions from different countries will analyse the outcomes observed not only in terms of efficacy but also in terms of cost-effectiveness.
obtained improvements in reading accuracy and fluency with effect sizes ranging from d = 0.81 to d = 1.49 in a sample of 140, 10 to14-year-old Dutch children with reading and spelling difficulties who had been --> Europe's Journal of Psychology ejop.psychopen.eu| 1841-0413

Figure 1 .
Figure 1.Means of gain scores in fluency, expressed in syll/sec, obtained by the three treatments in reading text, words and nonwords.

Figure 2 .
Figure 2. Means of errors reduction obtained by the three treatments in reading text, words and nonwords.

Figure 3 Figure 3 .
Figure 3 show the mean gain of syll/sec iii weighted for grade (used as covariate) x hour of intervention related to fluency, whereas Figure 4 show the main gain of errors reduction x hour of intervention obtained by the different treatments.

Figure 4 .
Figure 4. Means of errors reduction x hour of intervention, obtained by the three different treatments in reading text, words and nonwords.
comparisons of treatment regimens was corroborated by our results.Using efficacy parameters, the clinic-based treatments, showed the best outcomes in relation to the improvement of reading fluency with respect to other treatments with an average Improvement Index difference of 23% in fluency.The superiority of the clinic-based treatment over home-based treatments is expected if we consider that every treatment session is delivered by a professional at a ratio of 1:1.When we divide the gains in accuracy and fluency measured at the end of treatment by the number of hours of treatment employed, a different picture emerges.With respect to the clinic-based treatments, treatment Home-1 obtained an average Improvement Index difference in fluency and accuracy of 11% of 26% respectively.With respect to treatment Home-2, the average Improvement Index difference of treatment Home-1 was 35% and 30% respectively for fluency and accuracy.For text reading, treatment Home-1 obtained an average gain of 0.094 syllables per second and an average of 0.7 error reduction x hour of treatment.