OxyCon 2023: The Top Takeaways

Cybercriminal Footprint Erasure: Response Strategies

Javier Velandia, RA Product Manager at Appgate, captivated the audience’s attention from the very first seconds by bringing up one of the hottest topics in the cyber defense world: the ongoing war between cyber criminals and cybersecurity companies. 

He provided some alarming cyberattack figures: 4.7M reported attacks to APWG in 2022 and an overall 150% phishing attacks increase since 2019. Pretty disturbing, isn't it? 

According to Javier, cybercriminals never stop improving their strategies, which include hidden redirects, search engine poisoning, typosquatting or URL hijacking, and more. 

So, what actions can cybersecurity companies take to confront cyber fraudsters? Javier stressed the importance of reinforcing ML usage and growing multinational cooperation and mentioned some other proven tactics he and his team successfully employ to fight cybercrime. 

During the Q&A session, the cybersecurity expert gave tips on how to validate a domain, avoid sharing sensitive information with a fraudster website, and discussed the latest cybercrime trends.

Leveraging Machine Learning for Web Scraping

It would be considered a bad tone in 2023 to skip the omnipresent duo of web scraping and machine learning. Andrius Kūkšta, Data Engineer at Oxylabs, was the ML ambassador this year and explained how adaptive parsing and scraper block detection heavily depend on machine learning. 

Andrius started with basics: How to build a robust machine learning model? Then, he explained how ML is employed to develop reinforced fingerprinting, ML-based proxy rotation, and CAPTCHA solving. 

Later, the expert discussed how Oxylabs successfully implemented machine learning in the adaptive parser, block detection tools, and ML-based proxy management. 

And here's a little teaser for you: Andrius touched upon the untapped ML potential and mentioned some key areas where ML is yet to bring profit. 

Some curious questions popped up during the Q&A sessions, such as the potential of ML for product-matching across e-commerce platforms, how  LLMs are used in advanced parsing techniques, and others. 

Open-Source Technology for Extracting High-Quality Data at Scale

Browser or no browser? That is the question. If you ever felt lost in the data extraction world and its multiple choices, Glen De Cauwsemaecker, Senior Lead Crawler Engineer at OTA Insight, got your back. In his presentation, Glen guided us through the critical scraping choices everyone has to make.

And even though Glen doesn’t have a crystal ball, he tries to see into the future to answer the main question, “Can we continue extracting data without using browsers and accounts”? Well, let’s wait for the on-demand video with Glen's presentation to find a definite answer. 

Web Scraping, AI, and Evolving Legal Landscapes

To spice things up, after a flow of presentations, the Head of Legal at Oxylabs, Denas Grybauskas, held a lively panel discussion with top legal experts. 

Among many other topics, the panel discussed what laws will define the future of web scraping now that the hiQ Labs v. LinkedIn is over and what is important to consider regarding data scraping from the legal perspective. 

Additionally, the experts raised some thought-provoking questions, like how to distinguish if the data is public and whether IP blocking measures imply that the data isn’t publicly accessible. Join the expert discussion to learn their thoughts once the on-demand videos are live.

Finally, here's a bonus from the Q&A session - the legal experts explain what you should do if you receive an official complaint letter from the website owners asking to stop scraping their data. 

Accelerating Data-on-Demand Services with Async Python and AWS

Our legal experts passed the mic to Alexander Lebedev, Software Engineer at Hotjar, who talked about all things tech. 

How do you collect data on demand from tens of thousands of pages in a few minutes? Alexander’s presentation was just about that. He walked us through server and AWS matters, proxies, and how to save money while building the scraping infrastructure. He also explained what architecture to pick depending on one’s specific needs. 

Alexander matched his presentation with a real-life example of the Hotjar company. He also shared some tips and tricks that have saved him quite a few times:

• The easiest and safest way to extract data faster is to pay for more servers and better proxies;

• It’s almost never worth dealing with anti-bots yourself, as engineering time costs more than good proxies. 

In the Q&A session, Alexander shared some valuable resources that helped him improve his skills in Celery and FastAPI. So make sure you don't miss his presentation once we drop it off. 

Unlocking Insights from Video Data: Challenges and Solutions

Allen O’Neill, Co-founder at SocialVoice.ai, challenged our minds with another web scraping frontier to be unlocked: video data. 

According to Allen, video data is the new goldmine of valuable insights, such as pricing, availability, and sentiment, to name a few. The statistics further emphasize its importance: 93 % of video data is locked inside the video and not in the hashtags or descriptions under it. So, we should ask, “How do we unlock and analyze video data at scale?” Let’s find out the answer in the upcoming on-demand presentation by Allen. 

Web Scraping in 2023 and Beyond

Ever dreamed of time-traveling? Juras Juršėnas, COO at Oxylabs, and other industry experts made it possible. In their discussion, the panelists took us to the future to see what the web scraping industry will look like. 

The experts warned: Be prepared for the challenges. On the one hand, the lightspeed technological advancements and LLM development will make retrieving and consuming the data easier. On the other hand, the same technological advancements will contribute to making the data more locked down and hardly accessible. 

Is moving the content behind login a real possibility we should consider? The time will tell; meanwhile, you can check what our experts predict in the upcoming on-demand videos.