Evidence-Based Policy

Author Guidelines

Background

The Evidence Based Policy Brief series is a joint initiative of the National Association of State Directors of Developmental Disabilities Services and the Association of University Centers on Disability. It provides critical reviews of research on topics of importance in developing and implementing sound public policy affecting individuals with developmental disabilities and their families. The series is designed primarily to meet the information needs of state and federal policymakers, advocates, and practitioners. It does so by selecting topics of policy significance, reviewing the research evidence related to the topic in an objective and scientific manner and presenting a summary of that evidence and its implications to policy in a brief, clear, non-technical, but authoritative manner.

The policy questions addressed in the Evidence-Based Policy Brief (EBPB) series are derived from nominations received and prioritized by the Research Committee of the National Association of State Directors of Developmental Disabilities Services. High priority topics are then matched to scholars of corresponding expertise. EBPB authors are recruited from the programs affiliated with the Association of University Centers on Disability as well as the broader community of scholars with relevant expertise. Each EPBP goes through a careful editing and revision process by an editorial board made up of representatives of NASDDDS, AUCD and other stakeholders groups. The editorial board assures quality in the science included in reviews, the comprehensiveness of reviews and presentation of the evidence. The NASDDDS Research Committee is responsible for selecting the managing editor of the series. The managing editor is responsible for matching authors to topics, initial orientation of authors, reviewing and selecting editors for each EBPB, managing the editorial process, and the publication and dissemination of each EBPB. The managing editor also works with authors to assure relevant background information is posted on the Evidenced-Based Policy website.

Guidelines for EBPB authors

Authors of reviews in the EBPB series are expected to follow a fairly prescribed set of guidelines for the development of publications. These guidelines are intended to maintain consistency in the nature, quality and usability of each topical brief by members of the primary target audiences. While the guidelines will in some ways limit the authors from creating the standard academic publications with which they are experienced, the series’ website will provide an opportunity to provide a more detailed presentation of the review. Authors will also be encouraged to seek more traditional academic publication of the outcomes of their review. The steps below are intended only to guide the development of an EBPB publication.

There are 10 basic stages in developing and disseminating each EBPB: 1) selecting and validating the research question; 2) establishing inclusion criteria for research to be reviewed; 3) identifying the relevant research; 4) choosing of an appropriate methodology for the research synthesis/meta-analysis; 5) developing a format for data abstracting; 6) abstracting and synthesizing the research; 7) creating tables and charts to present findings in non-technical formats; 8) writing a clear and succinct narrative to accompany the data presentation, 9) submitting a draft EBPB of 4 pages for editorial review, and 10) finalizing the EBPB based on editorial feedback.

  1. Select and validate the research question to investigate:

    • Select a topic from recommended topics of the NASDDDS Research Committee or submit a topic for review by the Research Committee (see topics at: http://evidencebasedpolicy.umn.edu )

    • Develop a precise operational definition of the question to be addressed.

    • Assure that sufficient high quality research exists to reach a valid evidenced-based conclusion and recommendations.

    • Establish the operational definitions of the target population, interventions, outcomes, etc. for the studies to be included.

    • Assure that the topic has sufficiently consistent operational definitions to allow the results to be synthesized.

  2. Establish inclusion criteria/ standards for research to be reviewed:

    • Develop specific criteria about the nature of studies to include/exclude in the review (sample size, sampling process, ages, characteristics of sample members, nature of intervention, program, environment, policy, etc.).

    • Establish specific standard for the sources of research that will be included. (Remember that in policy-relevant research good work is often unpublished in academic journals, but is found in evaluation reports and other “fugitive literature”.)

    • Refine and record decision rules as needed for studies not clearly allowed/disallowed by the initial criteria or for expansion of the search domain.

    • Keep a running notation of changes in the criteria and sources.

    • Be as inclusive as possible when initially collecting literature (see part 3 below).

  3. Identify the relevant research with a comprehensive approach that may include:

    • Look for other comprehensive reviews related to the topic.

      • Examine approaches/definition used in related reviews

      • Review the comprehensiveness and validity of other reviews

      • Use the bibliographies as a source of relevant studies

    • Use the “invisible college” approach by identifying the leading researchers in the country/world on the topic and locate their work on this topic.

    • Use “ancestry approach” by searching the citations in related studies.

    • Examine published/online bibliographies on the topic for additional sources of information.

    • Conduct internet searches on the topic (recording key words used)

    • As appropriate identify and contact governmental program agencies that may know of policy relevant research and evaluation studies on the topic, such as:

      • State developmental disabilities program agencies

      • State Developmental Disabilities Planning Councils

      • State DVR Agencies

    • Identify and contact as relevant other national and state level advocacy agencies that may know of policy-relevant research and evaluation studies on the topic. These might include:

      • The Arc and affiliates

      • ANCOR and affiliates

      • APSE

    • Access indexing and abstracting services as appropriate (note the key terms you used).

      • ERIC

      • Medline

      • Google Scholar

      • Psych Lit, Psychological Abstracts

      • Dissertation Abstracts International

      • Social Sciences Citation Index and other data based searches to locate studies

    • Obtain copies of the studies identified.

  4. Choose a methodology for the research synthesis/meta-analysis that is appropriate to the available research:

    • Identify the best common denominator features of the research data to select the most appropriate method of data summary (e.g., standard deviations for formal meta-analysis)

    • Based on the summation possibilities develop an analysis plan for the research reviews and abstracts

  5. Develop a format for data abstracting:

    • Develop a data collection (data abstracting) form to record the studies reviewed, identify their status with reference to the criteria for selection and record the relevant data/findings .

    • Include on the data collection form a place to record key descriptive information, e.g.,

      • A brief description of the background of the study.

      • a complete citation

      • the research design

      • the number of subjects

      • relevant characteristics of subjects

      • the service setting/program/intervention type

      • relevant time intervals

      • study outcomes

      • other information of relevance to the specific study or the large summary

    • Include cells to note the research design

      • type of study (e.g., experimental pretest-post test control group design; quasi-experimental non-equivalent control group design, etc).

      • how independent and dependent variables were measured

      • titles of any instruments and tests used

      • pre-experimental (e.g., case study),

    • Note the study outcomes

    • Note any information that is specific to the study that might be footnoted for a table.

  6. Abstract and synthesize the research reviewed:

    • If appropriate, establish adequate reliability among data abstractors

    • Identify studies meeting/not meeting criteria

    • Summarize findings according to planned abstracting/summary approach

    • Examine internal and external threats to validity of the studies

    • Examine study reports to determine if some were done by the same author with the same subjects to assure each study included is included just once.

    • Evaluate the studies to identify valuable information, observations and recommendations.

  7. Create tables and charts to present the findings in non-technical formats:

    • Select modes of graphic presentation appropriate to review that are concrete and easily interpreted.

    • Using abstracted data create 2-4 tables or charts that clearly summarize the findings of the review

    • Analyze and interpret results.

    • For complexities that may be lost in the simplified EBPB, create a back-up data summary.

  8. Write a clear and succinct 4-page narrative*, including:

    • Background:

      • What is circumstance that makes this research question significant today?

      • How does this background lead to the research question posed?

    • The question:

      • A short repeat of the question and its importance

    • The method:

      • Sources used to gather information

      • Dates included in the search

      • Key words used in the computer search

      • Inclusion/exclusion strategies

      • Describe the prototypical research methods used in the studies

      • Define how you separated the studies (by report, by author, by location)

      • Describe the characteristics of the primary research studies

      • Note what information you coded even if it was not included

      • Note conventions used in collecting and analyzing the data

      • Note strategies used to ensure internal validity of the data

      • Note inter-rater reliability if applicable

    • The findings:

      • Allow the table(s)/chart(s) speak for themselves as much as possible.

      • Provide summary description of the research.

      • Note the basis for any cumulative finding.

      • Note the total number of independent studies sampled and the number actually used in the report.

      • Provide descriptive information about the literature (e.g., # of participants, geographical location)

      • Note the outcome of the comparison between the studies.

      • Note the possible variables that may have interacted to produce the result

    • The discussion and recommendations

      • Summarize the main results.

      • Point out the substantive meaning and implications of the results.

      • Consider the effect these study characteristics had on the outcomes

      • What are the policy implications of the findings?

    • Edit the length of text and charts to 4 pages or approximately 2200-2500 words.

  9. Submit the draft EBPB for editorial review

    • The managing editor will assign and issue editor to each brief to answer questions of authors and to receive the completed draft

    • The assigned editor will share the draft EBPB with at least two other members of the editorial board for review and comment.

    • Comments and suggestions will be returned to the lead author within one month of submission

  10. Finalize the Brief based on editorial feedback

    • The author revises the Brief attending to the editorial comments and recommendation

    • The assigned editor reviews the brief with attention to the response to editorial comments and recommendation

    • The assigned editor approves the document or returns it for further revision

    • Once approved the completed EBPB is put into the template for the series

    • The author reviews the EBPB in the publication format

    • The EBPB is disseminated by NASDDDS and placed on the EPBP website with links from NASDDDS and AUCD

    • The EBPB author is assisted to provide (if desired) a longer more detailed description of the synthesis for the EPBP website and or a full bibliography of the studies reviewed if not possible to include them in the EBPB.

    • The EBPB author is assisted (if desired) to write and submit the content of the EPBP for a traditional scholarly review of literature and submission to an academic publication.

* Authors may also wish to develop a longer review document suitable for publication. Often this will be done prior to developing the condensed brief.