When the coronavirus pandemic disrupted school for millions of students last spring, studies projected alarming rates of learning loss as schools shifted to remote instruction, particularly for vulnerable students.
Complicating matters is that this shift poses challenges in accurately assessing student progress and participation. But as more data emerges, one thing continues to be clear, experts say -- the pandemic is amplifying inequities between students.
Opportunity Insights, a data project from Harvard and Brown universities, has been tracking the economic impact of the pandemic on consumer spending, small business revenue, employment and education at tracktherecovery.org.
On the education front, it has partnered with Zearn, a math curriculum publisher that provides digital lessons and instruction to school districts and teachers. Using data from Zearn, Opportunity Insights has measured the progress and participation in online math coursework of some 800,000 students in kindergarten through fifth grade who were already using the platform before the pandemic shuttered schools.
At the beginning of the pandemic, Zearn CEO Shalinee Sharma noticed that their data "went bananas." There were sharp drop-offs nationally in participation and progress in the classrooms using Zearn, but also a "huge divergence between affluent kids and poor kids" that didn't exist pre-pandemic.
"What it showed was affluent kids were doing more math than ever on our platform, and poor kids were doing less," Sharma told ABC News. "The pandemic was increasing the inequity in our data."
Fall education data on the tracker went live last week. So far, there are some bright spots. Overall, those initial drop-offs have become less severe. As of Oct. 11, total student participation in online math coursework was down 7.1% compared to January 2020. Between March 22 and May 2, those decreases ranged from 25.5% to 47.3%.
Progress -- measured by the number of lessons completed by active students each week -- has also improved. As of Oct. 11, total student progress in online math coursework increased by 0.8% compared to January 2020. Between March 22 and May 2, that number had dipped by as much as 29.2%.
Despite those encouraging signs, gaps between low- and high-income communities have continued since the spring, David A. Williams, director of policy outreach for Opportunity Insights, told ABC News.
As of Oct. 11, students from high-income ZIP codes increased participation in online math coursework by 3.9% compared to January 2020, while for students from low-income ZIP codes it decreased by 10.3%. Income levels are based on the median household income in the school's ZIP code using Census data.
Progress for high-income ZIP codes increased by 11.1% during that time, while for low-income ZIP codes it decreased by 4%. These numbers also diverge widely within regions. In Washington, D.C., which has had skyrocketing progress, high-income ZIP codes outperformed low-income ones by 30 percentage points.
"There's still a real issue if we are seeing those gaps persist," Miller said.
As executive director of the education nonprofit redefinED Atlanta, Ed Chang has fought to close opportunity and equity gaps in the city's public schools. The pandemic has only "amplified" those gaps, Chang told ABC News.
MORE: Digital divide amplified: Schools scramble to provide at-home learning optionsA June report redefinED Atlanta conducted with Learn4Life and EmpowerK12 projected that about 21,000 fewer Atlanta students in English language arts and 29,000 fewer in math were on track for grade-level proficiency in the 2019-'20 school year than prior to COVID-19.
The study also determined that "economically disadvantaged students and students of color, who were already behind their peers, will fall further behind" during the pandemic. It projected that only three out of 10 students from historically underserved groups would have demonstrated grade-level proficiency by the end of the 2019-'20 school year.
The jury is still out on how students fared, as state assessments were canceled in the spring and more data will need to be collected this fall, Chang said. So far this school year, he has been encouraged by increased engagement in online learning in the city. In Atlanta last spring, the average login rate was around 70%, he said. During the first week of school this year, that rate was around 90%.
"That's a huge increase in engagement," he said, though noted, "that's still 5,000 children that aren't really logging in on a regular basis."
"If you don't have access to devices or learning you're effectively shut out of the learning process," he said.
As remote learning continues indefinitely, students not engaging is a major concern for Megan Kuhfeld, senior research scientist with NWEA, a Portland-Oregon-based education research organization that develops pre-K-12 assessments.
"Students not showing up is definitely a huge challenge," Kuhfeld told ABC News. "I think a lot of school districts are struggling with what it means to measure attendance remotely."
Using research on summer learning loss to help determine the impact of school closures in the spring, NWEA projected in an April study that third- through eighth-grade students nationwide could return in fall 2020 with roughly 70% of their learning gains in reading compared with a typical school year. Depending on the grade, they could fall half a year to a full year behind in math.
The organization expects to have its most recent findings out in the coming weeks to compare to its projections for the fall, though it is unclear how much data will be available. School districts may not have the capacity to do fall or winter assessments, Kuhfeld said, as they adjust to new learning models or focus on meeting the emotional and social needs of students. And with in-person and remote students being tested in non-standardized environments, there's a question of how much of it will be comparable to standard in-school testing, Kuhfeld said.
"Many children don't have a quiet place to work," she said. "This is a problem for remote learning as well as for testing."
A report released earlier this month by the Aspen Institute and the National Center for the Improvement of Educational Assessment addressed the challenges in gathering trustworthy data this year, particularly from state assessments. The organizations advised that states design a system for collecting data "to document and understand students’ access to the resources, tools, and experiences they need to learn."
MORE: US schools reopen during the COVID-19 pandemicTo help close equity gaps, data is needed this fall "to understand where those gaps are now and to figure out how to actually support those gaps and intervene," Chang said.
"It would be helpful to know where the needs are greatest," Chang said. "That's why that assessment piece is going to be very important."
For Miller, the data available from Zearn, while just for math, is one of the "best metrics" available right now to get a sense of how students are engaged overall. The data, which is updated weekly, can be explored down to the city and county level, which could also help drive immediate decision-making for local leaders, Sharma said.
"They can make real-time improvements," Sharma said. "At any week we could make it better. It's data we could act on."
One way to act on the findings, Miller said, could be to look at outliers in high-performing low-income communities and see what lessons that they can help provide.
Kuhfeld would also like to analyze student learning in the context of what the school did.
"I think the most important piece of this in some ways is to try to figure out what strategies helped students during this time, given that remote learning probably is not going away in many places for a while," she said. "That's a real contribution to this."
