Quasi-Experimental Research Designs: Comparison Groups, Interrupted Time Series, and Design Diagrams

A quasi-experiment has a treatment and an outcome, and it lacks random assignment. The people, classrooms, wards, or cities were already grouped, or a rule other than chance decided who received the program. You use the design when a randomized trial is impossible, unethical, or already too late: a campus policy that one residence hall adopted, a checklist a single ward started on a known Monday, a scholarship that turns on a published score.

What you can still do is choose a comparison that makes the main threat visible, measure the differences you can see, and say which differences you cannot see. Statistical control — covariates, matching, a difference-in-differences contrast, a model of the pre-intervention series — is part of the analysis. It is not a substitute for the diagram. If the diagram hides who was treated and when, the model will not repair that.

The notation on this page is the public Campbell and Cook shorthand. O is an observation. X is the intervention. A blank in the same column means “no treatment at that time,” not a missing data cell. R would mark random assignment; you will not see R on the quasi-experimental rows.

Random assignment is the line. Everything else — pretests, statistics, a carefully chosen comparison campus — can make a quasi-experiment more credible. None of it turns the design into an experiment. Write “quasi-experimental” in the methods section when you did not assign by chance, including when you wish you had.

The three figures in this section are the ones to learn first. Later sections add regression discontinuity and a staggered series, and a catalog table. The examples that follow use the same shapes with made-up series, labeled as illustrations.

Nonequivalent comparison group, pretest and posttest

Two intact groups. Both are measured before and after. Only one receives X. In symbols: O1 X O2 over O1 O2. “Nonequivalent” is the warning in the name: the groups were not formed by chance, so O1 is where you look for selection.

Interrupted time series

One unit, many observations, a sharp X. In symbols: O1 O2 O3 O4 X O5 O6 O7 O8. You need the stretch before X so you can see the level and the slope you are interrupting. A two-point before/after on a single group is not this design.

Interrupted time series with a comparison group

The same clock, two series, X in only one. A break that appears in both series at the same moment is a reason to suspect history (something else happened that week) rather than the program.

The names below are the standard internal-validity threats used in the Campbell and Cook tradition, plus contamination, which is the usual external leak between groups. Each line says what to circle on the design diagram before you trust a drop or a rise.

External validity is a separate question: would the same X do this in another hall, ward, or city? A clean diagram of one campus does not answer it. Say where you would not generalize, in a sentence next to the figure.

The regression-discontinuity sketch and the short selection list under the table are schematic. They are there so a proposal can pick a figure type before anyone estimates a model.

Multiple time series is the ITS-plus-comparison idea repeated: if X starts in March in one ward, June in a second, and October in a third, a cause that is “the program” should move with those dates. A cause that is “the new electronic record in April” will not line up with every start.

The counts below are illustrations invented for the diagrams. They are not results of a study, not administrative data, and not taken from any lecture handout. Round numbers are a teaching device. Do not cite them as findings.

Example 1 — nonequivalent comparison group

A university lets Residence Hall A adopt a weekday quiet-hours rule in week 6. Hall B, across the quad, does not. The outcome is noise complaints logged by student staff on one unchanged form. Students chose their halls. Nobody randomized the rule.

Weekly complaints, weeks 3–5 then weeks 6–8:

This is O1 X O2 over O1 O2, with three pretest weeks collapsed into the “before” description so the table stays readable. The design diagram is the comparison-group figure above. Selection is the threat to write in the proposal: Hall A may have been the hall that wanted quiet hours. A regression to the mean story is weaker here because Hall A’s pretest is only a little higher than Hall B’s, but you still show both O1 series. Do not describe this as a randomized dorm trial.

Example 2 — interrupted time series, then the same series with a comparison

A hospital ward starts a handoff checklist in month 5. The outcome is medication-order clarifications per month, same definition all year.

Illustrative counts: 22, 24, 21, 23, then the checklist, then 14, 13, 15, 12.

That is O1 O2 O3 O4 X O5 O6 O7 O8. The pre-checklist months are flat, which makes maturation a less natural story than a level shift — in this invented series. Instrumentation still matters: if the clarification log changed in month 5, the diagram needs a second mark on X.

Add a comparison ward that never starts the checklist and stays near 19, 20, 18, 21, 20, 19, 21, 18. That is the third figure. If both wards drop in month 5, look for a hospital-wide event before you credit the checklist.

A third shape, only as a pointer: a department awards a research stipend to every applicant whose entrance score is at least 80, and you compare later course completion for students just under 80 and just over 80. That is regression discontinuity. There is no numeric table, because the figure is the cutoff, not a league table of scores.

A design diagram shows assignment and time: who received X, who was observed, and in what order. A reporting flow shows how many units moved through a fixed report template.

A CONSORT flow diagram reports screening, randomization, allocation, follow-up, and analysis for a randomized trial. A PRISMA flow diagram reports how records were identified, screened, and included in a review. Neither figure is a quasi-experimental design diagram. Reporting flow is not the design.

If the study is quasi-experimental, draw the O/X diagram from this page first. If a proposal also wants the operational steps — recruitment, consent, the checklist, the analysis file — sketch that sequence in the research flowchart maker. The flowchart can say “nonequivalent halls” or “interrupted time series” in a box. It does not replace the design notation, and this guide does not replace the flowchart when you need a methods figure of the work itself.

Do not force quasi-experimental counts into a CONSORT template. CONSORT’s randomized-arm structure will invent a randomization you did not do.

This page is a Vizcept rewrite, not a reprint. Design names (nonequivalent control or comparison group, interrupted time series, regression discontinuity) are used in the public sense of the Campbell and Cook tradition. The examples, the counts, and the sentences are original to this page.

A set of public lecture notes for PPA 696 at California State University, Long Beach, formerly at http://www.csulb.edu/~msaintg/ppa696/696quasi.htm, is the page many methods reading lists still link. Those notes are thanked here as a widely cited teaching source. This guide does not reproduce their tables, assignments, or wording.

Archived copy, early 2012 snapshot via the Wayback Machine:

web.archive.org — CSULB PPA 696 quasi-experimental notes

Standard textbook treatments of the same designs remain the right citation for a thesis methods section. Point the archive link at the lecture notes; point this URL at the diagrams.