Each month, the United States Department of Agriculture releases a series of reports that analysts, grain traders, investors and farmers use to make decisions or buy and sell commodities.
But on Friday, Nov. 8, two of those USDA reports - World Agriculture Supply and Demand Estimate (WASDE) and the Crop Production report – were delayed by at least 10 minutes.
These reports contain data that move the commodity markets the moment they are released, impacting the finances of farmers, grain traders on the Chicago Board of Trade (CBOT) and investors.
Analysts and traders wait at computer screens until 11:00 a.m. CT, clicking refresh on their internet browsers until the reports publish.
But that Friday, the reports weren’t there.
“USDA.GOV sites are temporarily down for maintenance,” the website read.
Brokers are poised to execute trades the moment they see the USDA data. But even as the data stalled, the markets began moving.
The price of December corn on the CBOT jumped $0.09 per bushel in the first six minutes the USDA reports were supposed to be published.
In that time, more than 31,000 contracts were bought and sold, according to data from commodities and futures market data company BarChart.
If two brokers were waiting on these reports to buy contracts for December corn, the one who received the data at 11:02 would have bought for $0.06 per bushel less than the one who didn’t see the report until 11:06. Each contract equals 5,000 bushels of corn.
That staggered access could mean big losses, or gains, depending on when traders saw the reports, which tell farmers and traders on the CBOT how much grain was produced in the U.S. and bought and sold around the world.
“Those 10 minutes are really valuable 10 minutes because the market is actively trading.” said Scott Irwin, professor of agricultural and consumer economics at the University of Illinois. “Millions of dollars, tens of millions of dollars are at stake.”
The USDA releases data in a number of ways. For the WASDE report, the only access is the report website that experienced the outage. The Crop Production report is also available via email newsletter, through the USDA’s data archives at the Albert R. Mann Library at Cornell University and the agency’s Quickstats data lookup tool. Some companies pipe the data directly into their computers via an API.
For some, the information became available in minutes.
For others, it was 10 minutes or more. Some found other ways of getting the data, emailing the reports to colleagues or pulling market-moving numbers from Twitter.
The Crop Production report wasn’t available on the USDA’s site for nearly an hour and a half.
“After a couple minutes have gone by, you know we're still all wondering. And then we're getting phone calls from customers. And then we're scrambling and searching the internet looking for what's going on,” said Jon Marcus, one of the founding partners of AI Source, a Chicago-based trading and market analysis company.
Dan Kerestes, director of statistics division, USDA National Agricultural Stastistics Service, which produces the monthly Crop Production Report, said that while there was a delay on the USDA reports website, the other methods of accessing the report were available immediately.
“We never want any data user to not get to the information right away,” he said. “The NASS (Crop Production) report got out there on time, and a lot of people did get it.”
Kerestes couldn’t confirm how many users access the data via each method.
“It could be a blessing or it could be a curse. It's dumb luck, depending on what the market was doing,” said Marcus. “If you were looking to hedge this thing, that's great, it worked out perfectly. If you were looking to buy that market, you missed out.”
Electronic trading changed report releases
The USDA’s WASDE and Crop Production reports have always moved the commodity markets, but until 2013, they were released after the markets closed, so a delay of 10 minutes wouldn’t have any impact.
Brokers weren’t on the trading floor of the Chicago Board of Trade, so no one could profit off of early access to the information.
As commodities futures began trading electronically, it became possible for brokers around the globe to trade on that data, at all hours. The USDA moved the report to midday in March 2013.
“In recent years, technology, speed of communications, and electronic trading platforms have changed significantly. The trading floor is now represented by computers operating 24 hours a day absorbing information and trading in fractions of a second,” said the agency in a July 2018 statement.
Marcus said electronic futures trading can make buy and sell decisions in a fraction of a second, so even a minute delay of market-moving information is huge.
Irwin said the agency has a legal mandate to provide fair, equal access to the data.
“For the last 5 years the USDA has been behind the curve in dealing with the reality of releasing these hugely important market-moving reports,” said Irwin. “I sense that they don't fully appreciate that millisecond matter.”
Trust in the USDA
The delay comes in a year when trust in the USDA among farmers has been strained.
“2019 has proven to be a year where those in the agricultural space and those in the USDA have not been on the best of terms,” said Marcus.
After an abnormally wet spring, the agency estimated more acres planted than what most farmers saw in the fields.
Then in August, it released an adjusted Crop Production report that caused a strong reaction in the commodities market. Later that month, a USDA staff member received a threatening phone call from a farmer while on a tour of crops in Iowa.
“I mean these guys (farmers) are busting their tails. If you're seeing one thing, and another 99 people are telling you one thing and there's one agency telling you another, you’ll have probably more people pissed off than less,” said Marcus.
The uneven release of data is one more erosion of public trust in the agency.
“As far as I'm concerned, it's the biggest black eye the USDA has ever had with a report release,” said Irwin. “This couldn't have come at a worse time.”
Mark Jekanowski, acting chairperson of the World Agricultural Outlook Board, which produces the WASDE report each month, said the USDA’s Office of the Chief Information Officer is still investigating the server error, while the agency works to prevent future delays.
“We’re putting in extra layers of redundancy to ensure the report goes out on time from here on forward,” said Jekanowski. “When something like this happens, you can’t just go on business as usual.”
Both Jekanowski and Kerestes acknowledged that this delay comes at the end of a tumultuous year for farmers.
“It has been quite a year for the farmer, and for USDA as well,” said Kerestes. “The problem we had on the last release we don’t want to have, on a good year or a bad year. We’re going to do everything we can to get that fixed.”
Irwin said the agency is still putting out quality data. But in a tough year for farmers, the USDA has become a target for frustration.
“There's nothing up to this point, I would say, that there's any evidence of a real systematic problem with USDA Estimates. But there's certainly a perception that there is. This will really compound that perception,” said Irwin.
The agency releases WASDE and Crop Production reports at the beginning of every month, so has a few weeks before a chance to redeem itself.
“They need to take a hard look,” said Irwin. “It can just never happen again.”