Featured Articles

Recent Articles

Sep 2, 2022

How to Scrape Glassdoor Job Data Legally Without Headless Browsers

Glassdoor stores over 100 million reviews, salaries, and insights; has 2.2 million employers actively posting jobs to the marketplace, and gets about 59 million unique visits per month. With so much data and demand, Glassdoor is a gold mine for job and company data. In today’s tutorial, we’ll be scraping

ScraperAPI Team
Aug 18, 2022

Building a Go Web Scraper Step-by-Step: The Beginners Guide to Colly

Go allows developers to create complicated projects with a simpler syntax than C, but with almost the same efficiency and control. Its simplicity and efficiency is why we decided to add Golang to our web scraping beginners series and show you how to use it to extract data at scale.

ScraperAPI Team
Aug 11, 2022

How to Use Python to Loop Through HTML Tables and Scrape Tabular Data

Tabular data is one of the best sources of data on the web. They can store a massive amount of useful information without losing its easy-to-read format, making it gold mines for data-related projects. Whether it is to scrape football data or extract stock market data, we can use Python

ScraperAPI Team
Aug 4, 2022

Extract Data from HTML Tables with Rvest [Export Table Data to a CSV in R]

Scraping tabular data it’s one of the most powerful skills data scientists can have to gather relevant information at scale. Nowaday, almost every set of data shown on the internet uses HTML tables to organize and display complex set of data in a more understandable format. In today’s article, we’re

ScraperAPI Team
Aug 1, 2022

Building a JavaScript Table Web Scraper Using Python without Headless Browsers

Web tables are some of the greatest sources of data on the web. They already have an easy-to-read and understand format and are used to display large amounts of useful information like employee data, statistics, original research models, and more. That said, not all tables are made the same and

ScraperAPI Team
Jul 4, 2022

How to Scrape HTML Tables in JavaScript [Export Table Data to a CSV]

HTML tables are the best data sources on the web. They are easy to understand and can hold an immense amount of data in a simple-to-read and understand format. Being able to scrape HTML tables is a crucial skill to develop for any developer interested in data science or in

ScraperAPI Team
Jun 27, 2022

How to Build a LinkedIn Scraper in Python [No Headless Browser Needed]

LinkedIn is a huge source of data that’s publicly available for users and non-users alike, and that, per the time of writing this piece, it’s legal to scrape. However, just like it was shown in the 2019 LinkedIn vs. HiQ case, that doesn’t mean LinkedIn is comfortable with it. For

ScraperAPI Team
Jun 15, 2022

Headers and Cookies for Web Scraping

As technology advances, web servers are getting better at differentiating user traffic from web scrapers, making it harder for all of us to access the data we need. Although there are many methods we can use to circumvent many security measures, there’s one, in particular, that doesn’t get enough attention:

ScraperAPI Team
May 24, 2022

Web Scraping vs Data Mining: Differences and Applications

Data mining is often confused with data extraction/web scraping, but they are, in fact, two different processes and use wildly different methods to accomplish their goals. In today’s article, we’ll explore the differences between web scraping and data mining by explaining what each one is, how they are used, and

ScraperAPI Team
May 13, 2022

How to Build a Football Data Scraper Step-by-Step

Scraping football data (or soccer in the US) is a great way to build a comprehensive dataset that can then be used to help us create stats dashboards, run cross-analysis and use the insights for sports journalism or fantasy leagues. Whatever your goal is, scraping football data can help you

ScraperAPI Team
Apr 20, 2022

Build an eBay Web Scraper: How to Extract Product Data Using Javascript

Build an eBay Web Scraper: How to Extract Product Data Using JavaScript eBay is one of the largest eCommerce platforms in the world. With over 1 billion active listings on its site, it is also one of the largest data gold mines for pricing analysis, online purchase trends, and more.

ScraperAPI Team
Apr 20, 2022

XPath Cheat Sheet for Web Scraping: Full Guide and XPath Examples

XML Path Language (XPath) is a query language and a major element of the XSLT standard. It uses a path-like syntax (called path expressions) to identify and navigate nodes in an XML and XML-like document. In web scraping, we can take advantage of XPath to find and select elements from

ScraperAPI Team