But what will happen to Birmingham City Schools and other districts when federal relief ends this September?
In 2022 Birmingham City Schools superintendent Mark Sullivan was stunned when he found out how much his students had fallen behind. Their reading scores had dipped modestly from prepandemic levels, but their math performance — already well behind national averages before COVID-19 — had plummeted to “dismal” lows. For Sullivan, a former math teacher, that felt like a “punch in the gut.”
His district wasn’t alone. Across the country, students suffered historic learning losses after COVID-19 shuttered classrooms, with dire losses in math. The impacts were especially felt in poor communities like Birmingham, where the vast majority of students are Black and receive free or reduced-price lunch.
New research by Harvard and Stanford researchers, with support from Carnegie Corporation of New York, finds that school districts are making a partial comeback. According to the Education Recovery Scorecard, the first nationwide study of pandemic education recovery, U.S. students made up roughly one-third of their pandemic loss in math, and one-quarter of their pandemic loss in reading during the 2022–23 academic year — aided by $189 billion of federal pandemic funding for elementary and secondary schools.
The study also revealed dramatic rebounds in some low-income districts, including Birmingham City, where students have nearly caught up to their prepandemic math scores. Can other school districts and policymakers learn from these success stories before the time to spend the federal aid runs out?
The COVID-19 pandemic was devastating for student achievement — and recovery has not been a straightforward task, as schools must generate more learning per day despite staff shortages and rising student absenteeism.
There hasn’t been a clear playbook, either. The majority of the federal Elementary and Secondary School Emergency Relief Fund (ESSER) has been directed at local school districts, which — while given wide latitude to make spending decisions — must do so before the program ends this September.
Tom Kane, a coauthor of the Education Recovery Scorecard and faculty director of the Harvard Center for Education Policy Research, was curious to find out which districts are seeing successful recoveries, and how their own districts’ recoveries stack up against state and national trends. But, because states use different tests and different proficiency definitions, it was not possible to compare the achievement gains of students in different parts of the country.
Recognizing the urgency, Kane joined with Stanford Educational Opportunity Project sociologist Sean Reardon, an Andrew Carnegie Fellow, to publish the Education Recovery Scorecard, the first study to directly compare school districts’ pandemic recoveries across the country. Using a method pioneered by Reardon, the researchers stitched together school districts’ local test scores with states’ performance on a national assessment to place every district’s scores on a common grade-level scale. Educators and decision-makers can use the scorecard to zoom in on more than 8,000 school districts and check out how many grade levels their math and reading scores dropped between 2019 and 2022, and how much they’ve recovered since then.
The scorecard reveals important national trends. Because U.S. school districts have, on average, recovered just one-third of their math loss and one-quarter of their reading loss, many school districts will find themselves still far short of their prepandemic achievement by the time federal relief ends this fall, according to the researchers.
The report also identifies outperforming districts, like Birmingham City. While the study could not draw conclusions about the specific intervention strategies used by districts, Kane hopes the successful districts they identified can serve as case studies for educators and policymakers. “Without this, people might say, ‘Okay, we’re still behind; there aren’t any bright spots to try to learn from,’” says Kane. “But now a district elsewhere in the country has a chance to learn from what Birmingham did.”
Before becoming superintendent, Mark Sullivan had spent nearly his whole life in Birmingham City Schools — first as a student, then a teacher, then a principal — and knew the high stakes firsthand. “We are 89 percent African American, 11 percent Latino, and at 86 percent free and reduced lunch,” he says. “For many of our students, this is their only shot to be successful.”
When the virus shut down schools, the superintendent knew his district would struggle to keep instruction going: “We have 20,000 students, but when we did an inventory, we had about 6,000 devices.” At first, they printed thousands of copies of classroom materials to send home. Then, when ESSER funding came, the district bought devices and Internet hotspots for every student.
It turned out the students faced even bigger challenges.
During virtual instruction, teachers saw their students’ home lives. “We had children whose parents were essential workers at McDonald’s and Walmart, leaving their 10-year-old at home on a Zoom lesson, babysitting a crying two-year-old on her hip,” Sullivan recalls. Many of the students were struggling with food insecurity. “We realized that the meals they were receiving in school were a vital part of their success,” he says. So to keep students fed, the district employed lunchroom staff and bus drivers to drive meals into neighborhoods, and opened a drive-through for students to pick up meals from the cafeteria.
As the virus raged — ultimately claiming the lives of eight Birmingham City Schools staff and one student — the district took on a strong public health role. In fall 2020, when the district opened to a hybrid model, it made masks mandatory and set up one of its high schools as a mass vaccination site. To overcome vaccine skepticism within the community, schools hosted a series of “COVID conversations” that included children of Tuskegee Experiment survivors as well as Second Gentleman Douglas Emhoff. “If we can get people vaccinated, we can get school started,” Sullivan thought.
When you have environments that have been deprived for so long, and you provide additional support, it can really address a lot of needs.
Then the first batch of test scores came in.
Birmingham City’s students had fallen badly behind — more than three-fifths of a grade level in math — after less than two years of the pandemic. Sullivan immediately knew the students would need more time with teachers in order to learn the content they missed. Kane agrees.
“It’s hard to imagine kids catching up without more instructional time,” and the best opportunity to add that time is summer, Kane says.
The superintendent had a bold idea. He wanted to restart school in July 2022, but parents and teachers balked.
So his district settled on a compromise: an August to June school year with three additional one-week breaks called “intersessions,” and a longer intersession over the summer, during which schools would offer extra instruction and after-school care paid for by federal funding. The district also hired tutors from local colleges to provide high-dosage tutoring for Birmingham students throughout the school year. And they doubled down on tier-one instruction — referring to core grade-level education. “Focus on standards, don’t assume anything about children based on where they come from, what they look like, their disability, what language they speak,” Sullivan explains. “Teach every child at high levels, and they will perform.”
The intersessions have proven popular: 7,000 students took part in the district’s most recent fall intersession. “That’s 7,000 students in school learning when they don’t have to be,” Sullivan says.
Birmingham schools’ preliminary data shows that students who have received tutoring or attended the intersessions are making “significant” achievement gains compared with those who haven’t, the administration says.
But absences were increasing as well. So Sullivan brought frontline staff — nurses, coaches, cafeteria staff, bus drivers — into his cabinet meetings, asking them, “What are you seeing?” The answer was that many children were still struggling with the pandemic’s impacts, from mental health issues to hunger to homelessness.
The schools doubled down on wraparound services. Using ESSER funds, the district hired paraprofessional support staff for every classroom between kindergarten to third grade. The city of Birmingham granted the district an additional one million dollars to add a mental health counselor to every school. When teachers learned some students had dropped out because the pandemic forced them to start working — like one student who lost both of her parents to the virus and needed to take care of her younger sister — the district expanded a dropout recovery program with sites across the city for students to catch up outside of regular school hours.
Sullivan says the pandemic validated the importance of becoming attuned to students’ individual and personal needs. Thanks to the dropout recovery program, the student who lost her parents ended up graduating, says Sullivan. She’s now in college studying to be a nurse.
In 2023, it was time to find out if the recovery had worked. Mark Sullivan thought about everything his district had tried: “I was terrified that we had spent all this money, and that we wouldn’t see a return.”
His chief academic officer came in holding Birmingham City Schools’ 2022–23 test results. The officer had “a very straight face,” Sullivan recalls, and he braced for bad news. Then the officer pulled out the data and announced, “We have made significant gains.”
Sullivan felt a wave of relief. “It was a joy that you wouldn’t believe,” he says.
According to the Education Recovery Scorecard, Birmingham has recovered more than half a grade level in math — nearly returning to its prepandemic levels, and outpacing Alabama’s statewide average by a quarter of a grade level. And the district’s own data shows its reading scores now exceed its reading scores prior to the pandemic.
The question is whether the momentum can continue after ESSER. “When you have environments that have been deprived for so long, and you provide additional support, it can really address a lot of needs,” says Sullivan. “And I’m concerned about losing the money.”
According to Kane and Reardon, there are a few ways school districts can maximize the funding’s benefits before the funds expire.
While the funding cannot be spent on employee salaries after September, districts can use the federal money to make payments on contracts signed before then, such as for tutoring and after-school providers. School districts should inform parents earlier if their students are falling behind; a 2022 Pew study found that parents are underestimating their children’s pandemic learning losses. Increasing parental awareness sooner could lead to higher enrollment in summer sessions, another critical recovery tool. And like Birmingham, districts should continue to investigate and address the root causes of absenteeism in their schools, the researchers suggest.
In late June, Kane and Reardon published new research to quantify the effect of pandemic relief dollars on achievement, by comparing the recoveries of similar schools that received different amounts of funding.
And Kane is using the scorecard to spark discussion among educators and policymakers: “What are we going to do to continue the recovery once federal money runs out?” Already, he’s spoken to superintendents who have used the scorecard to find out which other districts are performing well and to “draw some inferences about what kinds of policies they ought to be thinking about.”
Next year, many districts including Birmingham City may have to make tough decisions about which recovery programs to cut. “We’re a high-poverty school district, and we have no control over that,” Sullivan says. But the real secret of his district’s rebound, he explains, comes from something more fundamental: a belief in their students’ promise. “You can be a high-achieving school district if you view the children as capable of doing the work.”
A selection of actual assignments by eighth-grade students in Massachusetts shows how civics classes can lead to informed and engaged citizenship
Carnegie Reporter Summer 2024Citizenship
From researching the nation's legacy of poverty in rural areas to creating a user-friendly guide to environmental governance, these Andrew Carnegie Fellows are dissecting problems and offering solutions
Carnegie Reporter Summer 2024Andrew Carnegie Fellows