As Rapid Antigen Testing Grows, New York State's Virus Positivity Rate Comes Under Scrutiny

In New York and beyond, the coronavirus pandemic has ushered in an unusual new conversance with public health metrics, with government officials now using testing and hospitalization data as a gauge for the severity of the virus and important policy decisions.

No metric has drawn more attention—or controversy—than the positivity rate. The number, which measures the percent of tests that turn up positive and serves as a proxy for infection rate, has been used to justify economic reopenings as well as business and school shutdowns.

For example, Governor Andrew Cuomo has said that he would order schools to close should local positivity rates rise above 9 percent, an increasingly likely possibility given the surge of infections.

In spite of the high stakes, New York state officials have been calculating positivity on a flawed basis, according to several experts. The criticism stems from the state's decision to fold in a type of rapid test known as antigen tests, which are less sensitive than polymerase chain reaction (PCR) tests.

"That's not what the professional guidance is," said Dr. Jay Varma, Mayor Bill de Blasio's top health adviser. "It's not what the WHO does or the CDC or the Council of State and Territorial Epidemiologists does."

The Council of State and Territorial Epidemiologists, which advises the Center for Disease Control and Prevention and other health agencies, has said that a positive antigen test should be considered a probable case of COVID-19, not a confirmed case. In turn, the CDC has defined its positivity rate as based on PCR tests only.

Despite the limitation of antigen tests, their benefits are clear: the tests can provide results within 15 minutes and can cost as little as $5, whereas PCR tests cost around $100 at national labs. Antigen testing has proliferated across the country in nontraditional testing centers, like schools and nursing homes, especially as the demand for testing increases.

In New York City, the tests are now available at pharmacies, CityMDs and doctor's offices. Drawn by their quick turnaround time, many people are unaware of the caveats.

During a recent seven-day period, there were nearly 88,000 antigen tests performed in one week, although city health officials cautioned that not all antigen test results are being reported to the city.

According to Cuomo, the state performs “hundreds of thousands" of antigen tests per week.

Dr. Ian Lipkin, a renowned epidemiologist at Columbia University and testing expert, agreed with Varma that antigen tests should be separated from PCR tests. The latter are currently considered the gold standard in terms of accuracy.

"You need to distinguish those clearly," he said. "It's reasonable to say these are PCR, these are antigen, but I don't think you should mix the two."

Jonah Bruno, a spokesman for the state Department of Health, did not respond directly to questions on why the state has gone against CDC guidance in counting antigen tests as positive. He instead referred to remarks made by Gareth Rhodes, an adviser to the governor, during a press conference last month.

Rhodes said that the state counts "any test that is licensed by the FDA or the New York State Department of Health to detect the current infection."

He added that the state puts out "extensive guidance" on how to interpret antigen test results and noted that some who test positive on antigen tests might seek to confirm the results with a PCR test. Both tests would be counted by the state.

What makes antigen tests less reliable?

Antigen tests can miss positive cases as well as report false positives. Unlike PCR tests, which detect genetic material from the virus known as RNA, antigen tests work by identifying viral proteins, which are produced by the immune system in response to the virus.

As a result, most antigen tests are authorized for use only in people with symptoms, meaning they cannot accurately spot asymptomatic COVID-19 infections. More specifically, the tests are typically recommended within five days of symptoms, even though the period of infectiousness may last up to 10 days after symptoms start.

Also, unlike PCR tests, the tests are not reported by labs but by "point of care" centers, such as doctor's offices, community health clinics or nursing homes.

Dr. Jessica Justman, an epidemiology professor at Columbia University, noted that "there is no consistent system for reporting all antigen tests."

She said while laboratories have established reporting systems that report both positive and negative results, places that administer antigen tests usually only report positive tests to health authorities.

Like Varma and Lipkin, Justman said that antigen tests should not be included to calculate positivity, although she added that the key factor was transparency in reporting.

Along those lines, Dr. David Dowdy, an epidemiology professor at the Johns Hopkins Bloomberg School of Public Health, argued that antigen tests could be included in the positivity rate as long as cities and states recognized that an increasing share of antigen tests over time affects the positivity readings.

Should the ratio of antigen and PCR tests stabilize, he said the positivity rate could be a reliable indicator.

"In general, it’s much more useful to look at changes over time than absolute values."

How is New York City handling antigen tests?

New York City's Health Department began publishing antigen test results this week on its coronavirus data website. But positive antigen results are reported as probable cases while positive PCR tests are considered confirmed cases. The seven-day rolling average of cases now includes both, but they are clearly delineated as antigen versus PCR. Positivity rates are calculated using only PCR tests.

"These counts will ensure that we're keeping up with the science and making our data more complete," explained Dr. Dave Chokshi, the city's health commissioner, during a press conference on Monday.

Acknowledging the increasing role of antigen tests, the city recently started conducting a survey to compare the results of antigen tests against PCR tests.

Is this why the city and state report different positivity rates?

The inclusion of antigen tests by the state is only part of the reason why there is a disparity between the city and state positivity rates. The other has to do with methodology. State health officials report positivity based on all tests reported to the state on the previous day. New York City's positivity is calculated using the day the tests were performed. Because of this, the city's positivity has a two-day lag in order to allow time for results to come in.

The city and state also treat repeat tests differently. As one example, in New York City, people with more than one negative test within a seven-day period will only be included the count of total tests once. The state counts all multiple negative tests reported in a seven-day period.

Why is the calculation of the positivity rate so important?

As stated above, in New York, the positivity rate has been the main driver of policy in response to the pandemic. Cuomo has added and lifted restrictions based on this metric, as well as used it to tout the state's relative success in controlling the virus. New Yorkers have spent months attuned to the positivity dial, only to scratch their heads at seeing two different numbers, which was what happened in November when de Blasio closed schools after the city reached its 3% positivity trigger. Cuomo, meanwhile, stated the city's positivity as being 2.5%.

Cuomo later called the city's calculation “confusing" as well as “irrelevant” because only the state's metrics matter when it comes to most broader restrictions.

Experts have largely shaken their heads at the conflicting strategies.

"We just have this disconnect between the state health department and city health department," Lipkin said. "We need to have to a coherent policy."

He added that the city and state agencies are "two of the best health departments in the world."

To date, the U.S. has not established any federal standards for reporting COVID-19 testing data.

Some public health experts have criticized policies that lean too heavily on the positivity rate.

Dr. Wan Yang, an epidemiology professor at Columbia University who uses the city's health data to build projections on the course of the pandemic, has said that she pays little attention to it, preferring to focus instead on caseloads, hospitalizations and deaths.

Similarly, the Johns Hopkins University coronavirus data website warns that, "Policy decisions, like openings and closings or interstate travel, should not be determined based on test positivity alone."

Dowdy said that with any public health measure, people need to be aware of the limitations.

"It’s not the best metric in terms of learning overall how much transmission is occurring in the community or what percentage of people are infected at a certain time," Dowdy said. "The problem is that you’re wanting something that you can evaluate in real time. That’s the beauty of the percent positive. You can calculate that very simply."

Despite the anticipated start of vaccinations this month, New Yorkers should not expect the positivity rate to vanish in the near term.

Lipkin said that surveillance testing for coronavirus would likely continue through 2021 as health officials seek to see how much of the virus continues to circulate in communities.