The Impact of COVID-19 on U.S. Food Delivery Services
Using Google Mobility and Trend data to determine structural changes in the industry
By Hongguin Kim, Evgeny Aleksandrov, and Ryan Young
Just a year ago, food delivery services were struggling financially. But as 2020 progressed (and much of the economy regressed), these companies saw much needed boosts.
With stay-at-home orders imposed and in-person dining shut down to varying degrees throughout the country, there was a food delivery demand shock last spring. And as the pandemic has subsisted, there continues to be more reliance on these services.
Our Mission
We want to provide an analysis that can be useful for food delivery services, new companies that are poised to enter this market, investors and anyone concerned with the ultimate impact of the pandemic on consumer behavior.
We want to figure out how demand for U.S. food delivery companies was influenced by the state of the pandemic and by the varying ways that people behaved around the country as a result.
The question on everyone’s mind (after when the pandemic will subside) is what life will be like in a post-Covid world. So our ultimate goal is to determine if the food delivery industry’s lift in 2020 could be the start of true shift in consumer behavior.
Our Data
We selected the following brands, which gave us a good sample of two types of food delivery companies:
Restaurants: Door Dash, Uber Eats, Grubhub, Postmates, Seamless, Chownow
Grocery: Instacart, Amazon Fresh, Shipt, Fresh Direct
In a perfect world, we would have accessible performance data such as weekly revenue or app activity, but as we learned in 2020, this is anything but a perfect world. So we decided to get creative.
Trends Data
We gathered Google Trends data to measure the interest in a given company within each state. Trends keeps records of the popularity of Google search terms over time. Although imperfect, we believe this is a suitable proxy for a company’s popularity. People generally will search the company name when they have interest in using the service.
In order to determine if the Trends data could additionally be used as an acceptable proxy for company performance, we examined how well it correlated with a company’s adjusted stock price.
The only food delivery company that was publicly traded in 2020 was Grubhub and its Trends and stock price data were not correlated. However, we believe Grubhub is somewhat of an unusual case because demand did indeed soar in 2020 compared to 2019, but operating costs rose at an even greater rate. Grubhub has also been losing market share in the U.S. in recent years as these other services have entered the market. Our Trends data show somewhat of a mix of these two factors (a significant increase during the onset of the pandemic, but a relatively quick descent to pre-pandemic levels unlike the other delivery services).
But we found many food and retail companies whose Trends data strongly correlated with its adjusted stock price. For example: Starbucks and Walmart (which had 0.483 and 0.514 correlation coefficients, respectively).
So it seems like the Trends data may be a decent proxy for company performance, but it is more reliable at corresponding to popularity and demand.
We set a script to pull the weekly Trends values for each company for each state dating to the start of 2019 and a separate set dating only to mid-February 2020. We will refer to these values as “Relative Interest” because they give interest (from 0–100), where 100 is the week where the company was searched the most in the given time period for that company in that state. So each data table we have for a company in a given location has exactly one week with the maximum popularity of 100. Thus, we cannot compare the absolute interest (or number of searches) across states. (For example, we cannot use our data to see if Uber Eats was more popular in New York or North Carolina.)
The biggest concern for using this data as a signal for [performance] is that sometimes Google users are searching terms simply to learn about the company or news involving the company. Therefore changes in interest can often be attributed to the “newsworthiness” of a search term. We looked for unexplained changes in the data as well as each company’s news events in 2020. We determined that we needed to adjust for DoorDash’s IPO announcement the week of December 6th and for rumors and the announcement of Postmates acquisition by Uber during the weeks of June 29th and July 5th We made these adjustments by replacing these values with the average of the weeks that came immediately before and after.
While Relative Interest will be our dependent variable, we want our inputs to account for the severity of Covid and the resulting behavioral changes.
Mobility Data
In order to track change in activity we used Google’s Community Mobility Reports, which Google has made publicly available during the pandemic. This dataset measures changes in movement through visits to different categories of places. The data is collected from users who have their device’s “Location History” turned on. Google has data from all over the world, but we will use the U.S. data, which features data at the country, state and county level.
The daily values are found by calculating how “visits and length of stay at different places change compared to a baseline.” The baseline is calculated as the median value for each corresponding day of the week during the first five weeks of 2020 (January 3 to February 6), when COVID-19 had not yet impacted behavior in the U.S.
We will focus on the following two categories of mobility (from February 16, 2020 to January 2, 2021):
Grocery & pharmacy (places like grocery stores, food warehouses and drug stores)
Retail & recreation (places like restaurants, shopping centers, and movie theaters)
We also used the CDC’s dataset to collect information about Covid cases and deaths. We aggregated the data so our values were on a weekly level.
For ease of analysis, we also focused on 11 states that represented a diverse sample of the country (geographically and demographically):
Alabama, California, Florida, Georgia, Iowa, Massachusetts, North Carolina, New York, Texas, Utah, Wisconsin
Exploratory Data Analysis
In looking at the mobility data, it is clear that restaurants, stores, and recreationally venues were less visited after the regulations and health risks that arrived with the pandemic. Here is a view of the second and third quarters of 2020:
It appears states that had harsher restrictions or more urban populations tended to be visit these places less. None of us are familiar with Montana or Wyoming, but it appears some of the residents of colder (and probably not unrelatedly, more rural states) hibernate in the winter. The weather is probably a factor for the aberrational increased summer mobility in these regions. (The Quarter 4 map looks nearly identical to the Quarter 2 map.)
Grocery Mobility is more of a complex subject. This is because Grocery stores were deemed essential and thus remained open across the country, even when stay-at-home orders were in place. So with restaurants closed and grocery store trips feeling like vacations, mobility at these types of venues was more varied:
Generally, the most mobile and least mobile states remained the same for both types of mobility.
Here is a week-by-week look at mobility in the U.S.
You can see the sharp decline in activity from the initial Covid-shock and that the reaction in the fall was not remotely proportionate to the increase in cases. You can see the “pandemic fatigue” when we directly compare covid cases to the mobility data:
Here’s a look at the mobility change over time at the state level:
And finally, let’s look at the Google Trends data. Here is the relative interest of each company in 2020, beginning with the third week in February:
The shock from the onset of the pandemic in the U.S. in mid-March is evident across all the services. We will hereinafter refer to this as the “Covid shock.” It had a much more distinct impact across states for Instacart than for the restaurant delivery services, such as Uber Eats:
Finally, when we compare our Trends data with our mobility data, we can see somewhat of an inverse relationship. (As grocery/retail mobility decreases, relative interest in delivery companies increases.)
We assume this relationship may hold for all states and companies over time, but we will explore this further.
Modeling
Having initially explored the data, we will now investigate the presence of a structural change in the levels of interest before and after the genesis of the pandemic in March and April. Further, we will explore whether causality is actually present between mobility and the interest variables. Next, we will determine the impact of the mobility on interest, controlling for the number of Covid cases. To conclude we propose a vector autoregression model based on multivariate time series which goal is to forecast the future of the interest for a certain company. Such forecasts are valuable for various stakeholders such as investors, employees, and journalists, to name a few.
T-Test
We recaptured our mobility data so it would date to the beginning of 2019. This would give us more data to work with from before the Covid shock, which we labeled as March and April for the purpose of this model. We observed the differences from before and after the onset of the pandemic in the U.S.
After the Covid shock, some of the Trends values returned to their pre-Covid level but some still remained higher than before. In order to see if there were significant differences between the two periods, we decided to conduct a t-test. Of the 10 brands, eight (DoorDash, Uber Eats, Grubhub, Postmates, Instacart, Chownow, Amazon Fresh, Shipt) showed statistical significant difference, while Seamless and FreshDirect did not. Based on our finding we were able to see people’s overall interest in delivery service remained higher than before the shock.
At this point, we want to further investigate if this higher interest is just due to the mobility itself or the behavior change of the people.
When we go deeper into the state level, we were able to see this trend becomes stronger in relatively urban states(NY, MA, FL). However when we see relatively less urban states(AL, IA, UT) the trend becomes weaker.
Causality Matrix
We all know the now-famous idiomatic expression “correlation does not imply causation”. Think of all the ridiculous examples of spurious correlation such as the number of people that drowned in a swimming pool and the power generated by US nuclear power plants, or total revenue generated by arcades and the computer science doctorates awarded in the US. Thus, while easy and accessible correlation is not an ideal value to be analyzed here, we want something that shows correlation. Enter Granger causality. According to Granger causality, if a series X “Granger-causes” a series Y, then past values of X should contain information that helps predict Y above and beyond the information contained in past values of Y alone. It is one of the best and most accessible econometric way to test for causality between two variables.
In this Figure we can see a matrix with the results of the Granger causality tests, in terms of p-values. Where the p-value is small, and we reject the null, we can say that the X variable Granger-causes the Y variable. Please note that this is not symmetric like the correlation number. This matrix shows the US level data for interest and mobility. We can see that the Retail and Recreation mobility series tends to cause the interest of the various delivery companies. The Grocery and Pharmacy mobility does not have such an effect.
Let’s further look at causation matrices at state levels. We found an interesting contrast between New York, Iowa, and Massachusetts. In the results for the state of New York we can see a clear distinction between how Retail and Recreation mobility affects the interest for food delivery companies (DoorDash, Uber Eats, and Grubhub) as opposed to the Grocery and Pharmacy mobility data (we fail to reject the null of no causation).
In Iowa we can see that Grocery and Pharmacy mobility causes more interest in the grocery delivery companies (Instacart, Amazon Fresh, and Shipt). For Massachusetts we see the same general causation trend with respect to the food delivery companies and the Retail and Recreation mobility. In some states, such as North Carolina, there is no distinction with respect to the six companies and the two types of mobility data: both respectively cause the interest in the six companies.
Having determined that some causation exists between our dependent and independent variables of interest let us investigate the magnitude and the direction of the effect that mobility, either Retail and Recreation or Grocery and Pharmacy, has on the interest of the food and grocery delivery companies we are investigating.
Rolling Beta Coefficient
Having determined that some causation exists between our dependent and independent variables of interest, let us investigate the magnitude and the direction of the effect that mobility, either Retail and Recreation or Grocery and Pharmacy, has on the interest of the food and grocery delivery companies we are investigating. For this purpose we will utilize a rolling linear regression model with independent variable Relative Interest, dependent variable mobility and COVID-19 cases as a control variable. We control for the cases in order to obtain the pure effect of mobility on interest, irrespective of the pandemic’s impact. We extract the beta coefficient of mobility recursively via means of a shifting window over our time series. This creates a new time series of the beta coefficients. This time series can be analyzed in order to determine the trend, and structural changes in how mobility affects interest.
Out of the 10 selected food delivery companies, on a US level, we find that for seven of them the trend is positive throughout the year. These companies are: Uber Eats, Grubhub, Postmates, Instacart, FreshDirect, Amazon Fresh, and Shipt. For the rest there is a non-significant or negative trend. In the case of the seven companies that show a positive trend throughout 2020, the end of year effect of mobility on interest is positive and significantly higher than the effect observed at the beginning of the year. The real-world/business interpretation of this is that COVID-19, when viewed from the perspective of the entirety of the US, resulted in a structural change in the behavior of individuals, namely, as we went through 2020, mobility had a stronger and positive effect on interest for most mainstream food and grocery delivery companies.
In looking at the State level data some interesting results emerge. Below is a table summarizing the findings.
We can see that in California and Massachusetts the most positive changes in the relation between mobility and interest are observed. States with the least pronounced changes are Florida, Utah, and Wisconsin. We also observe that in Georgia and Iowa there are many negative trends to the historical relationship between mobility and interest. The qualitative explanation may come from the differing COVID-related laws and regulations within each state, the per state performance and success of the companies, and several other idiosyncratic reasons. For instance, Massachusetts and California had some of the strictest regulations in the nation, while Iowa’s measures were among the weakest, which may account for some of these differences.
Forecasts
Having ascertained the factors at play that determine the structural changes present within the relationship between mobility and interest, it is important to understand what the future might bring. In order to have a proactive and forward-looking perspective it is paramount to construct a forecasting model, which allows investors, journalists and other stakeholders to have an empirically sound expectation of the future. As we are dealing with multiple time series, the model that best suits our needs, in terms of efficacy and interpretability is a vector autoregression (VAR) model. This model can be considered an extension of the famous ARMA model utilized in the context of univariate time series forecasting.
In order to implement the VAR model we must first satisfy some assumptions. Initially we check for the Granger causation within our selected variables. The variables must, on average, Granger cause each other. This assumption is satisfied and we have explored it in the previous subsection. We must further look at the cointegration test in order to determine the presence of a statistically significant connection between multiple time series. We then reserve the last four days of observations for the testing set, which will allow us to gauge the quality of the forecasts created using data from the training set. Yet another assumption needed for the vector autoregression is the stationarity of our series, which is checked via the Augmented Dickey-Fuller test. The results show the need for the data to be differentiated, namely all the data becomes X(t) = X(t) — X(t-1). The last step required before we fit our model is to determine the amount of lags to be included is the autoregressive part in the VAR model. Via Akaike information criteria we determine that lag amount to be 3. We then fit the model to a US-level subsample of our variables, namely ‘DoorDash’, ‘UberEats’, ‘Grubhub’, ‘Instacart’, ‘Retail_Recreation’, ‘Grocery_Pharmacy’, ‘Cases’, and ‘Death’. The Durbin-Watson statistic reassures us that the model was properly specified and it is ready for forecasting.
In above Figure we show the results of the forecast for the last four weeks of 2020, for the four companies we selected for our forecasting model demonstration, at US level aggregation. We can see that for DoorDash, Uber Eats, and Grubhub the general trend of interest is correctly predicted. Such information allows for value adding activities to be carried on by various stakeholders.
Conclusion
So overall, the retail mobility did have an effect on the popularity of food delivery and grocery mobility on grocery deliver. However, differing behaviors in different states allowed for varying impacts.
Throughout this post, we have tried to see how mobility impacts people’s interest (and thus demand) in delivery services. By comparing how the mobility and interest change overtime, we saw the two measures trend in opposite directions from each other, especially during the Covid shock. Next we sought to evaluate causality. We wanted to know if the recovery of activity would cause interest to go down based on our initial assumption that variables were inversely related. By conducting the causation matrix, we found that the change in the mobility actually causes the change in the interest. However, when running the rolling regression, the result was quite different from our initial assumption.
As 2020 progressed, the impact of mobility on the interest in many states (such as New York and California) changed from a negative to a positive relationship, which means even as mobility recovers, interest in delivery services remains the same or higher. Based on what we found, we believe public health regulations and closures are not the only direct driver of the growth in delivery app demand. Instead, we believe there is actual change in U.S. consumers’ perception and behavior since the Covid shock. This type of insight could not only be applied to delivery companies, but lasting behavior change could effect movie theaters, restaurants, concert venues, etc.
Moreover, the model we built also could be utilized to forecast how the interest will change in the future. The reason why we believe our model is meaningful is because predicting demand using mobility data is novel and powerful.
Data Sources: Google Trends, Google’s Community Mobility Reports, CDC, Yahoo Finance
The authors are students in Duke Fuqua’s MQM Program.
