Start of content
Skip to main content

Learn with Us

At the Evidence Resource Center, we aim to help Pennsylvania’s school communities thoughtfully apply high-quality, relevant research to their own settings. Explore these topics to brush up on research basics, ESSA tiers, and the fundamentals of evidence-based decision making.

Glossary

See also:

action research

A wide variety of evaluative, investigative, and analytical research methods designed to diagnose problems or weaknesses—whether organizational, academic, or instructional—and help educators develop practical solutions to address them quickly and efficiently. Action research may also be applied to programs or educational techniques that are not necessarily experiencing any problems, but that educators simply want to learn more about and improve.

alternating treatment design

A single-case design experiment that rapidly alternates between two or more interventions to examine how outcomes change; outcomes are measured with only one or two data points within each phase.
See also:

attrition

Attrition occurs when an outcome variable is not available for all subjects initially assigned to the intervention and comparison groups. If a randomized controlled trial (RCT) or regression discontinuity design (RDD) study has high levels of attrition, the validity of the study results can be called into question. For single-case design research, attrition occurs when an individual fails to complete all required phases or data points in an experiment, or when the case is a group and individuals leave the group.
See also:

baseline

A point in time before the intervention was implemented in group design research and in regression discontinuity design studies. In a single-case design experiment, the baseline condition is a period during which participants are not receiving the intervention.
See also:

benchmark assessment

An assessment administered periodically throughout a school year, at specified times during a curriculum sequence, to evaluate students’ knowledge and skills relative to an explicit set of longer-term learning goals. These tools are low-stakes assessments used to predict how students will do on high-stakes summative assessments. The design and choice of benchmark assessments are driven by the purpose, intended users and uses of the instruments. Benchmark assessment can inform policy, instructional planning and decision-making at the classroom, school and/or district level.The benchmark assessment provides information on the degree to which students have mastered a given concept or how students are progressing toward demonstrating proficiency on grade-level content standards.

bias

A systematic difference between the true impact of the intervention and the estimated impact, which can lead to incorrect conclusions about the effect of an intervention.
See also:

career and college readiness

The academic preparation that would be sufficient to allow a student to participate successfully in postsecondary education or a career without the need for remedial academic support.
See also:

career and technical education

Career and technical education (CTE) offers students the opportunity to develop critical skills through a combination of classes and hands-on learning experiences, which allow them to apply academics to real-world problems.

case study

A process or record of research in which detailed consideration is given to the development of a particular person, group, or situation over a period of time.

causal

Evidence that an observed effect is a consequence of the intervention (What Works Clearinghouse Glossary, n.d.).
See also:

chronic absenteeism or chronic absences

Chronic Absenteeism is calculated in Pennsylvania for all public schools with 20 or more students based on the number of students who have missed more than 10 percent of enrolled school days across the academic year; this represents 18 days in a 180-day school year. (State-to-state definitions may vary.)

clustered randomized controlled trial

A type of randomized controlled trial in which participants are grouped or clustered, rather than individually randomized (Wikipedia, n.d.).

comprehension

Making meaning of what is viewed, read, or heard. Comprehension includes understanding what is expressed outright or implied as well as interpreting what is viewed, read, or heard by drawing on one's knowledge and experiences. Comprehension may also involve application and critical examination of the message in terms of intent, rhetorical choices, and credibility.

control group

The analysis group that does not receive the intervention being tested.
See also:

correlation

A relation existing between phenomena, or things or between mathematical or statistical variables, which tend to vary, be associated, or occur together in a way not expected on the basis of chance alone.

countervailing evidence

Research, contextual information from research, and findings that offset or counteract evidence for a specific program, practice, or strategy. For ESSA purposes, countervailing evidence is derived from a high-quality study meeting Tiers 1 or 2 (meeting WWC standards with or without reservations), and demonstrates that an intervention had unfavorable impacts on outcomes. The discovery of countervailing evidence disqualifies for an ESSA tier the outcome(s) it counters.

data

Factual information (such as measurements or statistics) used as a basis for reasoning, discussion, or calculation.

delivery method

The method by which an intervention was delivered (i.e. individual, small group, whole class, or whole school).
See also:

diagnostic assessment

This assessment ascertains, prior to instruction, each learner’s strengths, weaknesses, knowledge and skills in specific academic areas. This assessment type helps guide units, lesson plans and other resources based on students’ needs. The diagnostic assessment provides a more complete picture of a student’s or group of students’ strengths and weaknesses so that instruction can be directly targeted to meet students’ needs.

differential attrition

The difference in attrition rates between the intervention and comparison groups.
See also:

early warning system

An early warning system, based on research from John Hopkins and Dr. Robert Balfanz, identifies students at risk for dropout based on attendance, behavior and course grades. (The Pennsylvania Department of Education, n.d.).

effect size (d or g)

The size of the increase from the average of the post-test, and the average of the pre-test. This measurement can help school leaders set an expectation for how much growth they can expect to see as a result of an intervention.

English Learners

Students whose dominant or primary language is not English. Also referred to as English Language Learners or English as a Second Language.
See also:

ESSA

The Every Student Succeeds Act (ESSA) of 2015 replaced the No Child Left Behind Act of 2001.

evidence

Research, contextual information from research, and findings for a specific program, practice, or strategy.
See also:

experimental study design

Experimental design refers to how participants are allocated to the different groups and conditions of an experiment.

face validity

When an outcome measures what it claims to be measuring.
See also:

favorable effects

Desirable outcomes reported by a study. In the research world, this term is used instead of 'positive effects' because in some cases, a negative or lowered result is desirable (such as reduced suspensions).

fidelity

The extent to which delivery of an intervention adheres to the protocol or program model originally developed.
See also:

fluency

The ability to act (speak, read, write) with ease and accuracy. Research indicates that oral reading fluency is the ability to read text accurately, with sufficient speed, prosody, and expression.
See also:

group randomized controlled trial

A type of randomized controlled trial in which participants are grouped or clustered, rather than individually randomized.

group formation

The methods used to determine which participants are allocated to a control group and a treatment group, or the groups receiving and not receiving an intervention. Group formation is one of the core underpinnings of high-quality experimental design; as group formation gets further from random, confidence in the results usually decreases due to baseline equivalence and confounding factors.

intervention

An educational program, product, practice, or policy aimed at improving student outcomes.
See also:

intervention group

The group in a [study] that receives the intervention being tested.
See also:

mean

The average; the most common. Mean is just one type of ‘measure of central tendency’, and educators may not always find the information they need by looking at the mean. Look beyond the mean of a study by evaluating statistical dispersion and standard deviation. Looking at a normal curve, 0 represents the average; a standard deviation (SD) tells you how far from the zero an individual student was, and can indicate how far spread out from 'average' the study participants are. If SD is low, that means students are clustered together; researchers like to see participants clustered together both in the beginning (which means at baseline they were very similar) and still clustered together at the end (which indicates improvement for all).).

measures of central tendency

Mean, median, mode. Though mean (average) is mostly commonly used to analyze student samples, mean is just one type of ‘measure of central tendency’, and educators may not always find the information they need by looking at the mean. Mode may be useful in determining what student characteristics are truly typical/most common, as opposed to average; median can also be useful. Educators can also look beyond the mean of a study by evaluating statistical dispersion and standard deviation. Looking at a normal curve, 0 represents the average; a standard deviation (SD) tells you how far from the zero an individual student was, and can indicate how far spread out from 'average' the study participants are. If SD is low, that means students are clustered together; researchers like to see participants clustered together both in the beginning (which means at baseline they were very similar) and still clustered together at the end (which indicates improvement for all).).

meta-analysis

The examination of data from a number of independent studies of the same subject, in order to determine overall trends. Per ESSA regulations, meta-analyses are not eligible for an ESSA tier.

metacognition (or metacognitive awareness)

Knowledge of self, task, and genre. Also, awareness and knowledge of one's mental processes that allows one to monitor, regulate, and direct self to a desired end.

multi-site

A study that is conducted in multiple locations during the study period. What Works Clearinghouse defines a site as a school building; the USDOE’s ESSA non regulatory guidance indicates that multi-site means more than one LEA, locality or state.

multi-tiered systems of support (MTSS)

A standards-aligned, comprehensive school improvement framework for enhancing academic, behavioral and social-emotional outcomes for all students.

outcome

Knowledge, skills, attitudes, and other desired benefits that are attained as a result of an activity. An outcome measure is an instrument, device, or method that provides data on the outcome. An outcome domain is a group of closely related outcome measures, believed to provide information on the same underlying skill or ability.
See also:

overlapping population

Participants in the study that overlap in number from the intervention to the control group; or, when the study population is counted multiple times due to subgroup identification.

phonemic awareness

The ability to detect and manipulate the smallest units (i.e., phonemes) of spoken language.
See also:

phonics

An approach to teaching reading that emphasizes the systematic relationship between the sounds of language and the graphemes (i.e., letters or letter combinations) that represent those sounds.
See also:

p value

In statistical hypothesis testing, the p-value or probability value is the probability of obtaining test results at least as extreme as the results actually observed during the test, assuming that the hypothesis is correct.
See also:

qualitative

Qualitative research is a process of naturalistic inquiry that seeks in-depth understanding of social phenomena within their natural setting. It focuses on the 'why' rather than the 'what' of social phenomena and relies on the direct experiences of human beings as meaning-making agents in their every day lives' (University of Utah, College of Nursing, n.d., para. 1).

quasi-experimental study design

A design in which groups are created through a process that is not random.
See also:

random sample

A set or an element of a set, each of whose elements has equal probability of occurrence; characterized by procedures designed to obtain such sets or elements.

randomized controlled trial

A design in which groups are created through a process that is random. Carried out correctly, random assignment results in groups that are similar on average in both observable and unobservable characteristics, and any differences in outcomes between the groups are due to the intervention alone.
See also:

rationale

An explanation of controlling practice or phenomena.

RTI or RTII

RTI is a three-tiered approach designed to address the learning needs of all students. The first tier of support, classroom instruction, aims to provide all students with high-quality teaching. Those who do not make sufficient learning progress are then provided additional intervention teaching and, perhaps, special education depending upon their response to the intervention.

reliability

An outcome measure is reliable if it produces the same scores when it is administered at different times to different people in different settings.
See also:

reservations, with (What Works Clearinghouse with reservations)

Studies receiving this rating provide a lower degree of confidence that an observed effect was caused by the intervention. Randomized controlled trials that are not as well implemented or have problems with attrition, along with strong quasi-experimental designs, may receive this rating.
See also:

reservations, without (What Works Clearinghouse without reservations)

The highest possible rating for a group design study reviewed by the WWC. Studies receiving this rating provide the highest degree of confidence that an observed effect was caused by the intervention. Only well-implemented randomized controlled trials that do not have problems with attrition may receive this highest rating.
See also:

sample

The number of participants included in the analysis, defined by a measured characteristic (i.e. grade level, age, gender, race, ethnicity, etc.).

setting

A setting is the location or locations of where a study took place (i.e. country, state, region, city etc.). Setting also indicates the type of school setting (i.e. public, private, charter, cyber, etc.).

single case design

A design in which an outcome variable is measured repeatedly within and across different conditions that are defined by the presence or absence of an intervention. This design involves repeated, systematic measurement of a dependent variable before, during, and after the active manipulation of an independent variable. It is identified by the following features: an individual case is the unit of intervention administration and data analysis, the case provides its own control for purposes of comparison, and the outcome variable is measured repeatedly within and across different conditions or levels of the independent variable.
See also:

social-emotional learning

The process through which children and adults acquire and effectively apply the knowledge, attitudes, and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions.
See also:

social skills

Any competence facilitating interaction and communication with others where social rules and relations are created, communicated, and changed in verbal and nonverbal ways.
See also:

standard deviation

A measure of variation across observations in a sample. A low standard deviation indicates that the observations in the sample tend to be very close to the mean. A high standard deviation indicates that the observations in the sample tend to be spread out over a large range of values.
See also:

statistical analysis

Collecting data and then examining that data as a whole or in parts for trends (Business News Daily, 2014).

statistical significance

The likelihood that a finding is due to a cause (i.e. intervention, strategy, practice, etc.) rather than chance or other confounding factors. Statistical significance is usually defined by a p value < .05.

stratified random

A strategy used during randomization, in which researchers separate the sample into groups called blocks based on certain characteristic(s) (like gender) and then randomly assign sample members within each block to either the intervention or comparison condition. By doing so, the researchers ensure that both conditions will contain units that have the characteristic.
See also:

study

An examination of the effect of an intervention on a particular sample and set of outcomes. In most cases, the results of a study are presented in a single article or manuscript. A study may also present results over multiple manuscripts, such as an examination of a beginning reading program that analyzes both immediate and long-term effects of an intervention on a common sample that is reported on in manuscripts produced over several years.
See also:

study design

The method by which intervention and comparison groups are assigned or the method by which an outcome measure is assessed repeatedly within and across different phases that are defined by the presence or absence of an intervention.
See also:

subsample

A portion of the analysis sample, defined by a measured characteristic (like grade level, gender, or race)
See also:

substantively important

A term coined by What Works Clearinghouse, this label is assigned to 'an effect size of 0.25 or greater, regardless of statistical significance' (What Works Clearinghouse Glossary, n.d.).
See also:

summative assessment

This assessment seeks to make an overall judgment of progress made at the end of a defined period of instruction. These tools produce clear data on a student’s accomplishments and occur at the end of a grade or course. Summative assessments may also be administered at certain grades for purposes of state or local accountability. Scores on these assessments provide information on students’ mastery of a given set of content standards.

ESSA Tier I

Demonstrates a statistically significant effect on an outcome based on strong evidence from at least one well-designed and well-implemented experimental study. As tiers are also based on similarity of setting, school leaders should verify this tier for an outcome in the context of their school and community.

ESSA Tier II

Demonstrates a statistically significant effect on an outcome based on moderate evidence from at least one well-designed and well-implemented quasi-experimental study. As tiers are also based on similarity of setting, school leaders should verify this tier for an outcome in the context of their school and community.

ESSA Tier III

Demonstrates a statistically significant effect on improving this outcome based on promising evidence from at least one well-designed and well-implemented correlational study with statistical controls for selection bias. As tiers are also based on similarity of setting, school leaders should verify this tier for an outcome in the context of their school and community.

ESSA Tier IV

Demonstrates a rationale based on high-quality research findings or positive evaluation that such activity, strategy, or intervention is likely to improve student outcomes or other relevant outcomes and includes ongoing efforts to examine the effects of such activity, strategy, or intervention. As tiers are also based on similarity of setting, school leaders should verify this tier for an outcome in the context of their school and community.

treatment fidelity

Treatment fidelity is defined as the strategies that monitor and enhance the accuracy and consistency of an intervention to ensure it is implemented as planned and that each component is delivered in a comparable manner to all study participants over time.
See also:

unfavorable evidence

Research, contextual information from research, and findings that show negative or undesired results for a specific outcome of a program, practice, or strategy. For ESSA purposes, if unfavorable evidence comes from a well-designed, well-implemented, high-quality study, the evidence may be considered countervailing, and therefore disqualify an outcome from receiving an ESSA tier.

variance

The measurement of the spread between numbers in a data set.

What Works Clearinghouse

The What Works Clearinghouse is an investment of the Institute of Education Sciences (IES) within the U.S. Department of Education that was established in 2002. The work of the WWC is managed by a team of staff at IES and conducted under a set of contracts held by several leading firms with expertise in education, research methodology, and the dissemination of education research.
See also:

What Works Clearinghouse Potentially Positive Effects

This occurs when at least one study shows a statistically significant or substantively important positive effect, no studies show a statistically significant or substantively important negative effect, and fewer or the same number of studies show indeterminate effects than show statistically significant or substantively important positive effects.

well-executed randomized controlled trial

A randomized controlled trial that uses a valid randomization procedure, has low levels of sample attrition, and includes appropriate information to test program effectiveness.