Black Friday Web Scraping Insights

Less than a week ago, Black Friday madness took over the world, and for many, it was an excellent opportunity to grab irresistible money-saving deals. To paint the picture, Black Friday in the USA took the lead as the busiest day for online shoppers (93.2M), leaving previous historical frontrunner day – Cyber Monday (83.3M) behind. Impressive numbers, isn’t it?

To capture the lion’s share of sales, many companies turn to price tracking and product catalog monitoring practices to unlock a further competitive advantage over their direct rivals. Consequently, businesses timely adjust their pricing tactics and stock, which allows them to win the customer. And, to gather this knowledge, companies increasingly rely on web scraping practice, which is usually supported by a proxy infrastructure. If you are interested to learn more about the topic of scraping, check out our Python scraping tutorial.

In this article, we will reveal 2019 Black Friday web scraping insights, supported by actual usage statistics of our clients, that utilized Residential and Datacenter proxies, and our Web Scraper API data gathering solution. Also, a special thanks go to our Research Department for providing these valuable insights! Let’s begin. 

* Please note: Traffic volumes and request amounts are summarized into trend graphs, ensuring absolute anonymity. The findings don’t include usage statistics of the clients that prefer for their usage statistics to not be recorded.

Residential proxies traffic insights

A data analysis conducted by our Research Department revealed a noteworthy surge in traffic volume via Residential proxies on Black Friday (29th of November), as it can be observed from a noticeable spike in the graph below: 

To put this into perspective, let’s compare this year’s Black Friday Residential proxies traffic volume with historical traffic volume metrics:

• In comparison to the previous 7 days usual traffic volume, on the Black Friday, Residential proxies traffic volume increased by a significant 20%.

• When compared to the last 30 days’ usual traffic volume, on the Black Friday, Residential proxies traffic volume increased similarly, by 21%.

• Whereas, in contrast to 2018 Black Friday’s Residential proxies traffic volume, this year’s Black Friday Residential proxies traffic grew by a substantial 65%.

As you might be aware, our Web Scraper API is a robust web data collection tool optimized to capture data with high data delivery success rates. That said, our Research Department has also revealed a substantial requests growth to one of the most prominent marketplaces via the Web Scraper API solution on this year’s Black Friday: 

• In comparison to last year’s Black Friday Web Scraper API requests volume to one of the most significant marketplaces online, this year’s 2019 Black Friday requests volume grew by a staggering 93%! 

Furthermore, our Research Department has found out that during Black Friday, there was a significant amount of requests processed to “Clothes” category websites. As we can see from the graph below, a noticeable growth can be observed 10 days before Black Friday i.e., from the 19th of November.