© Borgis - Postępy Nauk Medycznych 1/2013, s. 45-50
*Monika Suchowierska1, Gary Novak2
Nauka i pseudonauka w terapii autyzmu
Science and pseudoscience in the treatment of autism
1Department of Psychology, University of Social Sciences and Humanities, Warszawa, Polska
Head of Department: Ewa Trzebińska, PhD
2Department of Psychology, California State University Stanislaus, USA
Head of Department: William Potter, PhD
Pojęcia, takie jak „metody oparte na dowodach” lub „empirycznie udowodnione metody” odnoszą się do ważnego aspektu pracy klinicznej – mówią one o znaczeniu nauki w ocenianiu efektywności danego oddziaływania. Niestety, jeśli chodzi o terapię zaburzeń rozwoju, to często niesprawdzone, lub wręcz szarlatańskie, techniki są wykorzystywane do pracy z dziećmi. W obecnym artykule omówiono założenia metody naukowej i skontrastowano ją z pseudonauką, wymieniono powody, dla których rodzice dzieci z autyzmem skłaniają się ku eksperymentalnym oddziaływaniom, streszczono wyniki najnowszego raportu dotyczącego praktyki klinicznej opracowanego przez Narodowe Centrum Autyzmu w USA oraz zaproponowano kilka wytycznych dla profesjonalistów mogących mieć do czynienia z rodzicami dzieci o zaburzonym rozwoju.
Terms such as “evidence-based practice” or “best practices” reflect a very important aspect of clinical work, as they point to science as means for evaluating treatment effectiveness. Unfortunately, in the area of developmental disabilities dubious and pseudoscientific interventions are all too prevailing. In the present article, we describe premises of the scientific method and contrast it with pseudoscience, list the reasons for popularity of unsubstantiated claims in autism treatment, summarize the results of a recent report on practice guidelines prepared by the National Autism Center, and offer recommendations for practitioners who may come in contact with parents of children with developmental disabilities.
Despite the fact that terms such as “evidence-based practice”, “empirically supported treatments”, and “best practices” have become ubiquitous in education as well as other human services disciplines their definitions are not generally agreed upon, and worse, the principles are frequently not applied (1). Whatever the name that is used, the requirement for scientific knowledge to function as a filter for recommendations regarding treatment should be a fundamental principle guiding clinical work. Science has become the dominant judge of value in many disciplines, but in psychology interventions based on scientific evidence are often not sought by consumers, and – even worse – not endorsed by service providers. Dubious interventions and pseudo treatments are especially popular in the area of developmental disabilities (2). In the following paragraphs, we will 1) describe the scientific and pseudoscientific approach, 2) list the reasons for the proliferation of unsubstantiated claims in autism treatment, 3) summarize the results of the latest report on educational interventions for children with autism prepared by the National Autism Center, and 4) offer guidelines for clinicians regarding their own practice treating autism.
Science is virtually the only means by which interested individuals can test and evaluate the effectiveness or any proposed intervention (3). The premises of the scientific method are few and straightforward. They include (but are not limited to) 1) conducting quantitative and direct measurements of observable events, 2) testing of verifiable hypotheses, 3) using analytic means to establish relations between events, 4) relying on data to draw conclusions, 5) insisting on replication, 6) publishing of results in peer reviewed journals, and 7) self-correcting their work based on the above (4). Through the scientific process, assertions are confirmed or disconfirmed resulting in an accumulated body of knowledge and technology. Our understanding of phenomena under study is constantly changing as new facts come in from additional research. Despite the fact, that the scientific method has brought many contributions to society, it is often distrusted and disdained (5). The reasons are related to the fact that science does not offer “quick fixes” – the accumulation of evidence is an arduous and slow process. The methods of conducting experiments and analyzing results are not well understood by lay people. Experiments sometimes do not conform to conventional wisdom and common sense and at times seem to defy logic. Moreover, since scientific inquiry is painstaking, gathering data piece-by-piece may appear to be a self-serving endeavor, detached from real-world problems. Finally, science does not offer “good” or “bad” information with unequivocal conclusions – facts are facts, they should have neutral value. This is difficult to embrace by audiences who are hoping for a much-wanted answer, simple, and unambiguous. These characteristics of science are apparent in the scientific findings on climate change, where despite an accumulation of evidence based on the preponderance of careful scientific research, politicians and other non-scientists choose to ignore the findings.
In contrast to science, pseudoscience has many compelling features. They include 1) portraying their interventions as extraordinarily effective with little or no risk associated with their use, 2) basing claims on anecdotes, case studies, surveys, and other non-experimental methods, 3) using methods of inquiry that include uncontrolled, descriptive studies with qualitative data, 4) disseminating results directly to consumers via Internet, mass-media, brochures, reports in self-published books and journals and without peer-review, 5) promoting interventions through endorsements by individuals with apparent credibility or by celebrities (4, 6). Pseudoscience is, unfortunately, very wide-spread in the area of human services. It is also a big business for providers of the pseudoscience. Because of their often desperate need for help for their children, the population of parents of children with autism is especially vulnerable to interventions touted as “cures” or “breakthroughs” (7). Because it has attracted so many treatments of dubious merit, autism is called “a late 20th century fad magnet” by Metz, Mulick and Butter (8). What may be some reasons for such desperation and gullibility of parents of autistic children?
First, the nature of autism makes it a fertile ground for pseudoscience. Despite the fact that this disability was described 70 years ago by Kanner (9), it is still rather unknown and perplexing (10). Autism is treatable, but incurable (11). Thus, parents who are told that their child has a life-long disability, that is quite severe and puzzling reach out for everything and anything that may help. Additionally, parents of autistic children experience much more stress than parents of children with other disabilities (12) and this disorder often has far-reaching adverse effects on the whole family, which motivates parents to find anything that will remedy this situation. Further, since autism is not distinguished by any physical abnormalities, children with this disorder do not look disabled. They also act typical in many instances and show, in some areas, exceptional abilities (13). It is easy, thus, to believe that the child is “almost normal” and will reveal himself or herself with some sort of special individual or technique. Maurice (14) points out the myth of “hidden inner child.” That is the myth that there is a normal child hidden under the autism who may be awakened if only the right conditions happen. This and other misconceptions of autism encourage parents to undertake treatments that offer quick cures. Heterogeneity of the population of children with autism is also one reason for popularity of nonscientific therapies. Because of the inexact and ever-evolving diagnostic criteria for autism, children with autism exhibit a variety of different symptoms and, as individuals, they also are likely to be subjected to at least several therapies (15), in a way as to make it impossible to determine what the cause of the change is. If there is a testimonial supporting one therapy stating that child X advanced, parent of child Y may also want to subject his child to the same treatment that child X received, even though it is not known whether this treatment had anything to do with the changes in that one child. There are also additional reasons, not particular to autism, for the popularity of nonscientific therapies but they certainly also affect the field of developmental disabilities (16). They include the following: the incomplete effectiveness of available therapies, the best available treatment is onerous for clients, alternative treatments are supported by ideology, the treatment is promoted by proprietary professional group, and the treatment is recommended by authority or chosen on the grounds of tenacity.
A good measure of the success parents have in their child’s improvement is choosing an appropriate treatment. However, how to choose an effective evidence-based practice is not easy for most parents. It is not feasible to expect most of the parents to be able to evaluate the strength of research evidence on their own. A much better way is to inform consumers about the current status of practice guidelines for autism intervention. Such guidelines are established according to recommendations of teams of experts based on criteria for “empirically supported therapies” (EST). In 1995, a Division 12 (Clinical Psychology) Task Force on Promotion and Dissemination of Psychological Procedures and the American Psychological Association (APA) Task Force on Psychological Intervention Guidelines created three categories to describe degrees of evidence: Well-established Treatments, Probably Efficacious Treatments, and Experimental Treatments to describe the degree of evidence we have on the effectiveness of treatments. The information below comes from Chambless and Ollendick (17).
Category I: Well-established treatments
At least two good between-group design experiments must demonstrate efficacy in one or more of the following ways:
– Superiority to pill or psychotherapy placebo or to other treatments,
– Equivalence to already established treatment with adequate sample sizes.
A large series of single-case design experiments must demonstrate efficacy with both of the following:
– Use of good experimental design,
– Comparison of intervention to another treatment.
– Experiments must be conducted with treatment manuals or equivalent clear description of treatment,
– Characteristics of samples must be specified,
– Effects must be demonstrated by at least two different investigators or teams,
Category II: Probably efficacious treatments
Two experiments must show that the treatment is superior to a waiting-list control group.
One or more experiments must meet criteria Ia or Ib, III, and IV for well-established treatments (Category I), but the criterion 5 is not met.
A small series of single-case design experiments must meet criteria II, III, and IV for well-established treatments.
Category III: Experimental treatments
Treatment not yet tested in trials meeting task force methodology criteria.
Using the above-mentioned hierarchy of evidence method to make recommendations for interventions with strong scientific support must be part of professional practice guidelines. The ultimate goals is to set higher standards for care and to make better use of research findings to benefit consumers. The methodology to develop practice guidelines has been applied to autism only since the late 1990s. The first three examples of reviewing a variety of treatments for children with autism were: the New York State Department of Health Early Intervention Clinical Practice Guidelines (18), the Report of the Maine Administrators of Services for Children with Disabilities Autism Task Force (19), and the Recommendations of the Committee on Educational Interventions for Children with Autism of the National Research Council (20). Although these reports represented a very important step forward for evidence-based practice in autism treatment, their utility is limited due to 1) insufficient transparency of the process of guidelines’ creation, 2) incomplete comprehensiveness of reviewed studies, and 3) at this point in time – simply being out-of-date (7). Thus, we will describe in detail the latest effort to establish evidence-based practice guidelines dedicated to parents of individuals with autism and professionals working in the field of developmental disabilities – the National Standards Project undertaken by the National Autism Center. The goals of the project were four-fold: 1) to provide the strength of evidence supporting educational and behavioral treatments that target the core characteristics of autism spectrum disorders (ASD), 2) to describe the age, diagnosis, and skills/behaviors targeted for improvement associated with treatment options, 3) to identify the limitations of the current body of research on autism treatment, and 4) to offer recommendations for engaging in evidence-based practice for ASD (21). The report addressed some of the limitations of previous publications by reviewing the educational and behavioral treatment literature that targeted the core characteristics and associated symptoms of ASD and was published between 1957 and the 2007. Additionally, the report provided information about treatment effectiveness based on age, diagnostic groups, and treatment targets. Finally, the process of creating practice guidelines was completely transparent.
The report included 775 research studies which were reviewed by a group of experts. The inclusion criteria for studies were as follows: 1) the studies involved individuals with ASD (Autistic Disorder, Asperger’s Syndrome, and Pervasive Developmental Disorder – Not Otherwise Specified; PDD-NOS) under age 22; 2) they were published in peer reviewed journals prior to Fall 2007; 3) treatments could be implemented in or by school systems, or early intervention, home-, hospital-, and community-based programs; 4) interventions were educational and behavioral treatments – biomedical interventions (i.e., medication trials, nutritional supplement studies, and complementary and alternative medical interventions) were largely excluded, with the exception of curative diets; 5) studies needed to have empirical data; 6) research was published in English.
The Scientific Merit Rating Scale (SMRS) was developed as a means of objectively evaluating if the methods used in each study were strong enough to determine whether or not a treatment was effective for participants on the autism spectrum. The SMRS involved five critical dimensions of experimental rigor: 1) research design (i.e., degree to which experimental control was demonstrated), 2) measurement of the dependent variable (i.e., the extent to which accurate and reliable data were collected and these data represent the most direct and comprehensive sample of the target skill or behavior that is possible) 3) measurement of the independent variable (i.e., the extent to which treatment fidelity was adequately established), 4) participant ascertainment (i.e., the degree to which well-established diagnostic tools and procedures were used to determine eligibility for participant inclusion in the study), and 5) generalization (i.e., the extent to which researchers attempted to objectively demonstrate the spread of treatment effects across time, settings, stimuli, responses, or persons). For each dimension, reviewers, using clearly defined criteria, assigned a score from 0 (poor score) to 5 (strong score). The dimension scores were compiled to create a composite score for each article. Criteria were also developed to determine if intervention had 1) beneficial treatment effects (i.e., sufficient evidence that favorable outcomes resulted from the treatment), 2) unknown treatment effects (i.e., not enough information to confidently determine the treatment effects), 3) ineffective treatment effects (i.e., sufficient evidence that favorable outcomes did not result from the treatment), and 4) adverse treatment effect (i.e., sufficient evidence that the treatment was associated with harmful effects). All treatment studies were classified into one of 38 treatment categories. The treatment categories are shown in table 2.
A four-level Strength of Evidence Classification System criteria was applied based on the quality, quantity, and consistency of research findings for each treatment category. The four levels in this classification system were 1) Established – sufficient evidence that these treatments produce beneficial effects for individuals with ASD; 2) Emerging – one or more studies shows beneficial treatment effects, but additional high quality studies are needed; 3) Unestablished - little or no evidence about treatment effectiveness; and 4) Ineffective/Harmful – sufficient evidence exists to show treatment is ineffective or harmful. Additionally, each study was evaluated taking into consideration 14 treatment targets: 10 goals in the area of skills increased (i.e., academic, communication, higher cognitive functions, interpersonal, learning readiness, motor, personal responsibility, placement, play, self-regulation), and 4 goals in the area of behaviors decreased (i.e., general symptoms, problem behaviors, repetitive patterns of behavior or interests, sensory or emotional regulation).
The National Standards Project identified 11 Established Treatments, 22 Emerging Treatments, 5 Unestablished Treatments, and zero Ineffectual/Harmful Treatments. The treatments categorized by the National Standards Committee are shown in table 1.
Table 1. Strength of Evidence Classification System and its results as presented in the National Standards Project.
|Strength of evidence classification|
|3||Comprehensive Behavioral Treatment for Young Children|
|4||Joint Attention Intervention|
|6||Naturalistic Teaching Strategies|
|7||Peer Training Package|
|8||Pivotal Response Treatment|
|11||Story-based Intervention Package|
|1||Augmentative and Alternative Communication Device|
|2||Cognitive Behavioral Intervention Package|
|3||Developmental Relationship-based Treatment|
|8||Language Training (Production)|
|9||Language Training (Production & Understanding)|
|13||Peer-mediated Instructional Arrangement|
|14||Picture Exchange Communication System|
|18||Social Communication Intervention|
|19||Social Skills Package|
|22||Theory of Mind Training|
|2||Auditory Integration Training|
|4||Gluten- and Casein-Free Diet|
|5||Sensory Integrative Package|
Table 2. 38 Treatment Categories, their description and number of studies included in each category, as presented in the National Standards Project.
|No.||Treatment Name||Treatment Description||No. of studies|
|1||Academic Interventions||Traditional teaching methods are used to improve academic performance.||10|
|2||Antecedent Package||Modifications to the stimuli preceding target behaviors are made in order to increase the probability of desirable behaviors occurring and decrease the probability of undesirable behaviors occurring.||99|
|3||Auditory Integration Training||Modulated sounds are presented through headphones with the goal of improving distortions in hearing or sensitivities to sound.||3|
|4||Augmentative and Alternative Communication Device (AAC)||A variety of devices (e.g., pictures, symbols, computers) are used to facilitate communication.||14|
|5||Behavioral Package||Techniques based on basic principles of behavior are used to teach desirable behaviors and reduce undesirable behaviors.||231|
|6||Cognitive Behavioral Intervention Package||Negative or unrealistic thought patterns are changed in order to improve a person’s every day functioning.||3|
|7||Comprehensive Behavioral Treatment for Young Children||Inclusive and intensive programs that rely on a combination of behavior analytic techniques and are offered to young children with autism.||22|
|8||Developmental Relationship-based Treatment||A combination of procedures based on developmental theory is used with the goal of building social relationships.||7|
|9||Exercise||The use of physical activity to increase the probability of desirable behaviors occurring and reduce the probability of undesirable behaviors occurring.||4|
|10||Exposure Package||Conditions are arranged in which a person faces anxiety-provoking situations while the therapist is preventing the use of maladaptive strategies and reinforcing appropriate behaviors. ||4|
|11||Facilitated Communication||A facilitator supports the hand or arm of an individual with limited communication skills, helping them express words, sentences, or complete thoughts by using a typing device.||5|
|12||Gluten- and Casein-free Diet||An individual’s intake of naturally occurring proteins gluten and casein is limited.||3|
|13||Imitation-based Interactions||An adult is imitating the actions of a child.||6|
|14||Initiation Training||A child is taught to initiate interactions with his/her peers.||7|
|15||Joint Attention Intervention||A child is taught to respond to bids for attention from others and to initiate joint attention interactions with others.||6|
|16||Language Training (production)||Teaching strategies are used to increase speech production.||13|
|17||Language Training (production and understanding)||Teaching strategies are used to increase speech production and understanding of language.||7|
|18||Massage/touch Therapy||Deep tissue stimulation is provided to an individual.||2|
|19||Modeling||A model (demonstration of the target behavior) is presented to an individual with the goal of having it imitated. ||50|
|20||Multi-component Package||An individual is taught using a combination of multiple treatment procedures that are derived from different fields of interest or different theoretical orientations.||10|
|21||Music Therapy||Music is used as means of teaching an individual a variety of skills.||6|
|22||Naturalistic Teaching Strategies||A variety of techniques are used during teaching interactions that are primarily child-directed, focused on functional skills and conducted in the natural environment.||32|
|23||Peer Training Package||Typically developing children are taught strategies to facilitate play and social interactions with individuals with special needs.||33|
|24||Peer-mediated Instructional Arrangement||Academic skills are taught with the help of peer tutors.||11|
|25||Picture Exchange Communication System (PECS)||Behavioral principles are used to teach functional communication via a system of symbols to children with limited vocal and/or verbal repertoire or lack thereof.||13|
|26||Pivotal Response Treatment||“Pivotal” or fundamental areas are targeted during intervention.||14|
|27||Reductive Package||Strategies are used to reduce unwanted behavior in the absence of teaching appropriate behavior.||33|
|28||Schedules||A task list that communicates a series of activities or steps required to complete a specific activity is presented to an individual.||12|
|29||Scripting||A verbal and/or written script about a specific skill or situation which serves as a model is presented to an individual and practiced repeatedly.||6|
|30||Self-management||An individual is taught to regulate his/her behavior by recording the occurrence/nonoccurrence of the target behavior, and securing reinforcement for doing so.||21|
|31||Sensory Integrative Package||Procedures that stimulate or challenge the individual to effectively use all of their senses are used with the goal of addressing overstimulation or understimulation from the environment.||7|
|32||Sign Instruction||Sign language is taught as means of communication.||11|
|33||Social Communication Intervention||Teaching is focused on social communication impairments such as pragmatic communication skills, and the inability to successfully read social situations.||5|
|34||Social Skills Package||Social interaction skills are targeted via teaching separate social skills, from the most basic to very complex ones.||16|
|35||Story-based Intervention Package||A written description of the situations under which specific behaviors are expected to occur is presented to an individual.||21|
|36||Structured Teaching Approach||A combination of procedures that rely heavily on the physical organization of a setting, predictable schedules, and individualized use of teaching methods is used.||4|
|37||Technology-based Treatment||Instructional materials using the medium of computers or related technologies are used.||19|
|38||Theory of Mind Training||An individual is taught to recognize and identify mental states (i.e., a person’s thoughts, beliefs, intentions, desires and emotions) in oneself or in others and to be able to take the perspective of another person in order to predict their actions.||4|
The report concluded with several interesting facts about the type of interventions found to be “established”. First, less than 30% of the reviewed interventions were evaluated as “established.” But within those, about 67% were developed exclusively from the behavioral literature, and of the remaining 33% three-quarters represent treatments for which research support comes predominantly from the behavioral literature. Only less than 10% of established treatments came from theory of mind perspective. “This pattern of findings suggests that treatments from the behavioral literature have the strongest research support at this time” (21). Close to 60% of the reviewed interventions were evaluated as “emerging” and a little over 10% as “unestablished”. There were no interventions which were deemed “harmful” based on the reviewed literature.
The concluding remarks of the report state that parents of children with autism and professionals working with those children should seriously consider established treatments when making decisions about treatment choice. The report, it should be noted, cautioned that these treatments will not universally produce favorable outcomes for all individuals with ASD. Further, the authors suggest that children should not be subjected to the emerging treatments at the beginning of therapy. Instead, they may be an option later in therapy or if established treatments are not appropriate or acceptable for the child’s family. Lastly, unestablished treatments should not be used until additional research has been conducted and has shown favorable effects for children with ASD.
The National Standards Report provides parents and practitioners with solid professional judgments of the kinds of treatment that have evidence of efficacy. However, as the report acknowledges, not all treatments will work uniformly well in all cases. Knowing the results of the latest practice guidelines for treatment of autism, what should practitioners do in their daily work? As O’Donohue (22) suggests they should behave scientifically themselves. That means, they should take an experimental approach to all intervention efforts, empirically establishing the efficacy of the treatment they are using and being willing to change the treatment in light of evidence that it is not working. Adhering to scientific methods is of paramount importance. This means educating oneself on science-based interventions and understanding that employing pseudoscientific treatment not only can be of no benefit but also can cause direct (i.e., health deterioration) or indirect (i.e., loss of time and resources) harm. It also means seeking feedback about clinical work from peers, especially those with greater relevant scientific expertise and giving corrective feedback to those who are failing to use sound practice. Finally it means making data-based decisions and recommending to other professionals and parents of children with ASD sources where they can obtain knowledge on empirically validated treatments. In case of autism, some of such sources are the Association for Science in Autism Treatment (www.asatonline.org), the National Autism Center (www.nationalautismcenter.org), the Autism Help Center of the Cambridge Center for Behavioral Studies (www.behavior.org), and the Autism Special Interest Group of the Association for Behavior Analysis International (www.abainternational.org). Most importantly, however, every clinician should be guided in their professional work by a statement by Leonardo Da Vinci: “Those who fall in love with practice without science are like a sailor who enters a ship without a helm or a compass, and who never can be certain whither he is going” (23).
1. Detrich R: Evidence-based, empirically supported, or best practice? A guide for the scientist-practitioner. [In:] Luiselli JK, Russo DD, Christian WP, Wilczynski SM (ed.): Effective practices for children with autism. Education and behavioral support interventions that work. New York: Oxford University Press 2008; 3-36.
2. Favell JE: Sifting sound practice from snake oil. [In:] Jackobson JW, Foxx RM, Mulick JA (ed.): Controversial therapies for developmental disabilities. Fad, fashion, and science in professional practice. Mahwah, New Jork: Lawrence Erlbaum Associates 2005; 19-30.
3. McFall RM: Manifesto for a science in clinical psychology. The Clinical Psychologist 1991; 44(6): 75-88.
4. Green G: Science and ethics in early intervention for autism. [In:] Ghezzi PM, Williams WL, Carr JE, editors. Autism: Behavior analytic perspective. Reno, NV: Context Press 1999; 11-28.
5. Danforth S: On what basis hope? Modern progress and postmodern possibilities. Mental Retardation 1997; 35: 93-106.
6. Smith T: The appeal of unvalidated treatments. [In:] Jackobson JW, Foxx RM, Mulick JA, editors. Controversial therapies for developmental disabilities. Fad, fashion, and science in professional practice. Mahwah, New Jork: Lawrence Erlbaum Associates 2005; 45-60.
7. Wilczynski SM, Christian L, and the National Autism Center: The National Standards Project: Promoting evidence-based practice in autism spectrum disorders. [In:] Luiselli JK, Russo DD, Christian WP, Wilczynski SM (ed.): Effective practices for children with autism. Education and behavioral support interventions that work. New York, Oxford University Press 2008; 37-60.
8. Metz B, Mulick JA, Butter EM: Autism: A late 20th century fad magnet. [In:] Jackobson JW, Foxx RM, Mulick JA (ed.): Controversial therapies for developmental disabilities. Fad, fashion, and science in professional practice. Mahwah, NJ: Lawrence Erlbaum Associates 2005; 237-264.
9. Kanner L: Autistic disturbances of affective contact. Nervous Child 1943; 2: 217-250.
10. Frith U: Autyzm. Wyjaśnienie tajemnicy. Gdańsk, GWP 2008.
11. Centers for Disease Control and Prevention; http://www.cdc.gov/NCBDDD/autism/facts.html
12. Dunn ME, Burbine T, Bowers CA, Tantleff-Dunn S: Moderators of stress in parents of children with autism. Community Ment Health J 2001; 37(1): 39-52.
13. Howlin P, Goode S, Hutton J, Rutter M: Savant skills in autism: psychometric approaches and parental reports. Phil Trans R Soc B 2009; 364: 1359-1367.
14. Maurice C: Let me hear your voice. New York, Knopf 1993.
15. Goin-Kochel RP, Myers BJ, Mackintosh VH: Parental reports on the use of treatments and therapies for children with autism spectrum disorders. Research in Autism Spectrum Disorders 2007; 1(3): 195-209.
16. Vyse S: Where do fads come from? [In:] Jackobson JW, Foxx RM, Mulick JA (ed.): Controversial therapies for developmental disabilities. Fad, fashion, and science in professional practice. Mahwah, New Jork, Lawrence Erlbaum Associates 2005; 3-18.
17. Chambless DL, Ollendick TH: Empirically supported psychological interventions: controversies and evidence. Annual Review of Psychology 2001; 52: 685-716.
18. New York State Department of Health, Early Intervention Program. Clinical practice guideline: Guideline technical report. Autism/pervasive developmental disorders, assessment and intervention for young children (ages 0-3 years) (No. 4217). Albany, NY: New York State Department of Health 1999.
19. Report of MADSEC: Autism Task Force. Manchester, ME: Maine Administrators of Services for Children with Disabilities 2000.
20. National Research Council: Educating children with autism. Washington, DC: National Research Council 2001.
21. National Autism Center: National Standards Report. Randolph, MA: National Autism Center 2009.
22. O’Donohue W: On behaving scientifically: Fallibilism, criticism, and epistemic duties. Clinical Science Newsletter (on-line) 1997; 2-7.
23. da Vinci L: The Notebooks of Leonardo da Vinci; http://www.gutenberg.org/cache/epub/5000/pg5000.txt