Contact Sales
Login
Back to blog

How To Scrape eBay with Python [Complete Guide]

Scraping eBay product data with Python tutorial

eBay is the third-largest e-commerce platform, only behind Amazon and Walmart, listing over a billion products and making it a massive source of information for businesses and analysts.

However, you need to access the data before you can even analyze or utilize it, and doing it manually is just not efficient — here’s where scrapping comes in handy.

In this tutorial, you’ll learn how to scrape eBay and how to extract valuable information (such as product details, prices, and descriptions) from eBay using Python and BeautifulSoup.

TL;DR: Full eBay Python Scraper

For those in a hurry, here’s the scraper we’ll build in this tutorial:

	import requests
	import json
	from bs4 import BeautifulSoup
	
	API_KEY = "YOUR_API_KEY"
	url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p4432023.m570.l1312&_nkw=airpods+pro&_sacat=0"
	
	payload = {"api_key": API_KEY, "url": url}
	r = requests.get("http://api.scraperapi.com", params=payload)
	html_response = r.text
	soup = BeautifulSoup(html_response, "lxml")
	
	
	result_list = []
	
	# Find all product items on the page
	listings = soup.find_all("div", class_="s-item__info clearfix")
	images = soup.find_all("div", class_="s-item__wrapper clearfix")
	
	for listing, image_container in zip(listings, images):
		title = listing.find("div", class_="s-item__title").text
		price = listing.find("span", class_="s-item__price").text
	
		product_url = listing.find("a")
		link = product_url["href"]
	
		product_status_element = listing.find("div", class_="s-item__subtitle")
		product_status = (
			product_status_element.text
			if product_status_element is not None
			else "No status available"
		)
	
		if title and price:
			title_text = title.strip()
			price_text = price.strip()
			status = product_status.strip()
	
			image = image_container.find("img")
			image_url = image["src"]
	
			result_dict = {
				"title": title_text,
				"price": price_text,
				"image_url": image_url,
				"status": status,
				"link": link,
			}
			result_list.append(result_dict)
	
	# print(result_list)
	
	# Output the result in JSON format
	output_json = json.dumps(result_list, indent=2)
	
	# Write the JSON data to a file
	with open("ebay_results.json", "w", encoding="utf-8") as json_file:
		json_file.write(output_json)
	
	print("JSON data has been written to ebay_results.json")

For this iteration, we’re sending our requests using ScraperAPI to bypass eBay’s anti-scraping mechanisms and handle the technical complexities when scaling the project.

Note: Replace API_KEY in the code with your own ScraperAPI API key.

Collect Data from All Major eCommerce Marketplaces

Collect millions of product details without sacrificing efficiency or data quality.

Start 7-Day Trial

How to Scrape eBay with Python?

Python stands out as one of the best languages for web scraping due to its user-friendliness, simple syntax, and wide range of available libraries.

To scrape eBay with Python, we will use the Requests library and BeautifulSoup for HTML parsing. Additionally, we’ll send our requests through ScraperAPI to avoid any anti-scraping measures.

Prerequisites

This tutorial is based on Python 3.10, but it should work with any Python version starting from 3.8. Ensure you have a compatible Python version installed before proceeding.

Now, let’s initiate a Python project within a virtual environment named “ebay-scraper.” Execute the following commands sequentially to set up your project structure:

	mkdir ebay-scraper 
	cd ebay-scraper
	python -m venv env

This is what each line of the commands above does for proper understanding:

  • First, t creates a directory named “ebay–scrapper”
  • Then, it switches your terminal to the actual directory where you are currently building
  • Finally, t locally creates the Python virtual environment for the project

Enter the project folder and add a main.py file. This file would soon contain the logic to scrape eBay.

You’ll also need to install the following Python libraries:

  • Requests: This library is widely regarded as the most popular HTTP client library for Python. It simplifies the process of sending HTTP requests and handling responses which will enable you to download eBay’s search result pages.
  • BeautifulSoup: This is a comprehensive Python HTML and XML parsing library. It helps to navigate through the webpage’s structure and extract the necessary data.
  • lxml: This library is known for its speed and feature-rich capabilities for processing XML and HTML in Python.

You can easily install these libraries together using the Python Package Installer (PIP):

	pip install requests beautifulsoup4 lxml

With these prerequisites in place, you are all set to move on to the next step!

Understanding eBay Search Results Page Structure

When you search for a specific term, like “airpods pro,” eBay will show results similar to the page below:

Product results in eBay example

All products listed under “airpods pro” can be extracted, as well as their links, titles, prices, ratings, and images.

Before start writing your script, it is critical to determine what data you intend to collect. For this project, we’ll focus on extracting the following product details:

  • Product Name
  • Price
  • Product Image
  • A Brief Product Description
  • The product URL

In the screenshot below, you’ll see where to find these attributes:

Elements we will scrape highlighted in red

Now that we know what information we’re interested in and where to find it, it’s time to build our scraper.

Step 1: Setting Up Our Project

Open your code editor and add the following lines of code to import our dependencies:

	import requests
	from bs4 import BeautifulSoup
	import csv

Note: we also imported the CSV library as we want to export our data in this format.

Next, we need to set up the URL of the eBay page we want to scrape. In this example, we will be searching for “airpods pro“. Your URL should be like this:

	url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p4432023.m570.l1312&_nkw=airpods+pro&_sacat=0"

Step 2: Send the Request and Parse the HTML

Now, let’s send a GET request to the eBay page and parse the HTML using BeautifulSoup.

	html = requests.get(url)
	print(html.text)

This code uses the requests library to send a “GET request” to our eBay search results page and retrieves the HTML content of the web page, and stores it in the html variable.

However, as soon as you run this code, you will realize that eBay has put some anti-bot measures in place.

You can bypass this anti-bot measure by sending a proper user-agent header as part of the request to mimic a web browser:

	headers = {
		"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
		"Accept-Language": "en-US, en;q=0.5",
	}
	
	html = requests.get(url, headers=headers)
	soup = BeautifulSoup(html.text, "lxml")

To parse the HTML content, the code creates a BeautifulSoup object by passing html.text and the parser type (lxml) to the BeautifulSoup() function. This allows us to navigate and extract data from the HTML easily.

Step 3: Find the Product Items on the Page

We generally need to understand the HTML layout of a page to scrape the results out. This is an important and critical part of web scraping.

Nonetheless, do not fret if you are not too skillful with HTML.

We will utilize Developer Tools, a feature available in most popular web browsers, to scrape our required data.

When on the search results page, do “inspect element” and open the developer tools window. Alternatively, press “CTRL+SHIFT+I” for Windows users or “Option + ⌘ + I” on Mac

DevTools opened on an eBay search page

In the new window, you will find the source code of the target web page. In our case, all the products are mentioned as list elements, so we have to grab all these lists.

In order to grab an HTML element, we need to have an identifier associated with it. It can be an id of that element, any class name, or any other HTML attribute of the particular element.

In our case, we are using the class name as the identifier. All the lists have the same class name, i.e. s-item.

On further inspection, we got the class names for the product name and product price, which are “s-item__title” and “s-item__price”, respectively.

CSS selectors highlighted using DevTools

The next step is to find the product items on the page. The code uses the find_all() method of the BeautifulSoup object to find all the div elements with the class “s-item__info clearfix“.

These div elements represent the product items on the page. The program also finds the corresponding image containers using the same approach.

Add the following lines of code to your file:

	import requests
	from bs4 import BeautifulSoup
	import csv
	url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p4432023.m570.l1312&_nkw=airpods+pro&_sacat=0"
	html = requests.get(url)
	print(html.text)
	headers = {
		"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
		"Accept-Language": "en-US, en;q=0.5",
	}
	
	
	html = requests.get(url, headers=headers)
	soup = BeautifulSoup(html.text, "lxml")
	listings = soup.find_all("div", class_="s-item__info clearfix")
	images = soup.find_all("div", class_="s-item__wrapper clearfix")

Step 4: Extract the Desired Information

For each product item, we will extract the title, price, product URL, product status, and image URL.

	import requests
from bs4 import BeautifulSoup
import csv
url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p4432023.m570.l1312&_nkw=airpods+pro&_sacat=0"
html = requests.get(url)
print(html.text)
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
    "Accept-Language": "en-US, en;q=0.5",
}


html = requests.get(url, headers=headers)
soup = BeautifulSoup(html.text, "lxml")
listings = soup.find_all("div", class_="s-item__info clearfix")
images = soup.find_all("div", class_="s-item__wrapper clearfix")
result_list = []

for listing, image_container in zip(listings, images):
    title = listing.find("div", class_="s-item__title").text
    price = listing.find("span", class_="s-item__price").text

    product_url = listing.find("a")
    link = product_url["href"]

    product_status_element = listing.find("div", class_="s-item__subtitle")
    product_status = (
        product_status_element.text
        if product_status_element is not None
        else "No status available"
    )

    if title and price:
        title_text = title.strip()
        price_text = price.strip()
        status = product_status.strip()

        image = image_container.find("img")
        image_url = image["src"]

        result_dict = {
            "title": title_text,
            "price": price_text,
            "image_url": image_url,
            "status": status,
            "link": link,
        }
        result_list.append(result_dict)

Inside a loop, the code extracts the title, price, product URL, product status, and image URL for each product item. It uses the find() method of the BeautifulSoup object to find specific elements within each product item div.

The extracted data is then stored in a dictionary called result_dict and appended to the result_list.

Step 5: Write the Results to a CSV File

After extracting the data, we create a CSV file called “ebay_results.csv” in write mode using the open() function.

	with open("ebay_results.csv", "w", newline="", encoding="utf-8") as csv_file:
    fieldnames = ["title", "price", "image_url", "status", "link"]
    writer = csv.DictWriter(csv_file, fieldnames=fieldnames)

    # Write the header row
    writer.writeheader()

    # Write the data rows
    for result in result_list:
        writer.writerow(result)

It creates a DictWriter object from the CSV library to write the data to the CSV file. The “fieldnames” parameter specifies the column names in the CSV file.

CSV file filled with ebay product data

We can also print the extracted data to the console.

	for result in result_list:
    print("Title:", result["title"])
    print("Price:", result["price"])
    print("Image URL:", result["image_url"])
    print("Status:", result["status"])
    print("Product Link:", result["link"])
    print("\n")

The code above iterates over each item in the result_list and prints the corresponding values for each key. In this case: the title, price, image URL, product status, and product URL.

Scraped eBay product details printed to the console

Full Scraping Code

Now that we have all of the puzzle pieces in place, let’s write the complete scraping code. This code will combine all the techniques and steps we’ve discussed, enabling you to collect eBay product data in a single step.

	import requests
	from bs4 import BeautifulSoup
	import csv
	
	
	
	url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p4432023.m570.l1312&_nkw=airpods+pro&_sacat=0"
	
	headers = {
		"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
		"Accept-Language": "en-US, en;q=0.5",
	}
	
	
	html = requests.get(url, headers=headers)
	soup = BeautifulSoup(html.text, "lxml")
	
	result_list = []
	
	# Find all product items on the page
	listings = soup.find_all("div", class_="s-item__info clearfix")
	images = soup.find_all("div", class_="s-item__wrapper clearfix")
	
	for listing, image_container in zip(listings, images):
		title = listing.find("div", class_="s-item__title").text
		price = listing.find("span", class_="s-item__price").text
	
		product_url = listing.find("a")
		link = product_url["href"]
	
		product_status_element = listing.find("div", class_="s-item__subtitle")
		product_status = (
			product_status_element.text
			if product_status_element is not None
			else "No status available"
		)
	
		if title and price:
			title_text = title.strip()
			price_text = price.strip()
			status = product_status.strip()
	
			image = image_container.find("img")
			image_url = image["src"]
	
			result_dict = {
				"title": title_text,
				"price": price_text,
				"image_url": image_url,
				"status": status,
				"link": link,
			}
			result_list.append(result_dict)
	
	# Write the result_list to a CSV file
	with open("ebay_results4.csv", "w", newline="", encoding="utf-8") as csv_file:
		fieldnames = ["title", "price", "image_url", "status", "link"]
		writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
	
		# Write the header row
		writer.writeheader()
	
		# Write the data rows
		for result in result_list:
			writer.writerow(result)
	
	# Print the result list
	for result in result_list:
		print("Title:", result["title"])
		print("Price:", result["price"])
		print("Image URL:", result["image_url"])
		print("Status:", result["status"])
		print("Product Link:", result["link"])
		print("\n")

Using ScraperAPI for eBay Data Scraping

Scraping eBay data can be a challenging task due to eBay’s anti-bot measures and frequent changes in their techniques.

You already saw a glimpse of this at the beginning of this tutorial, where a request without the proper headers was blocked by eBay.

However, this approach will only work for small projects (scraping a couple of tens of URLs).

To scale our project to collect millions of product details, we’ll use ScraperAPI. This solution is designed to help you bypass these challenges effortlessly. It also takes care of handling IP rotation, ensuring that your scraping process runs smoothly without interruptions.

Getting Started with ScraperAPI for easy eBay scraping

Using ScraperAPI is straightforward. All you need to do is send the URL of the webpage you want to scrape, along with your API key. Here’s how it works:

  • Sign Up for ScraperAPI: Go to the ScraperAPI dashboard page and sign up for a new account. This comes with 5,000 free API credits for 7 days, allowing you to explore the service’s capabilities.
ScraperAPI sign up form page
  • Obtain Your API Key: Upon signing up, you will receive your API Key.
ScraperAPI dashboard highlighting the API key

Now, you can use the following code to access eBay search results.

Note: Replace API_KEY in the code with your own ScraperAPI API key.

	import requests
	import json
	from bs4 import BeautifulSoup
	
	API_KEY = "YOUR_API_KEY"
	url = "https://www.ebay.com/sch/i.html?_from=R40&_trksid=p4432023.m570.l1312&_nkw=airpods+pro&_sacat=0"
	
	payload = {"api_key": API_KEY, "url": url}
	r = requests.get("http://api.scraperapi.com", params=payload)
	html_response = r.text
	soup = BeautifulSoup(html_response, "lxml")
	
	
	result_list = []
	
	# Find all product items on the page
	listings = soup.find_all("div", class_="s-item__info clearfix")
	images = soup.find_all("div", class_="s-item__wrapper clearfix")
	
	for listing, image_container in zip(listings, images):
		title = listing.find("div", class_="s-item__title").text
		price = listing.find("span", class_="s-item__price").text
	
		product_url = listing.find("a")
		link = product_url["href"]
	
		product_status_element = listing.find("div", class_="s-item__subtitle")
		product_status = (
			product_status_element.text
			if product_status_element is not None
			else "No status available"
		)
	
		if title and price:
			title_text = title.strip()
			price_text = price.strip()
			status = product_status.strip()
	
			image = image_container.find("img")
			image_url = image["src"]
	
			result_dict = {
				"title": title_text,
				"price": price_text,
				"image_url": image_url,
				"status": status,
				"link": link,
			}
			result_list.append(result_dict)
	
	# print(result_list)
	
	# Output the result in JSON format
	output_json = json.dumps(result_list, indent=2)
	
	# Write the JSON data to a file
	with open("ebay_results.json", "w", encoding="utf-8") as json_file:
		json_file.write(output_json)
	
	print("JSON data has been written to ebay_results.json")

This is the output, as seen in the ebay_results.json file

	[
	{
	  "title": "Apple AirPods Pro",
	  "price": "$200.00",
	  "image_url": "https://ir.ebaystatic.com/rs/v/fxxj3ttftm5ltcqnto1o4baovyl.png",
	  "status": "Brand New",
	  "link": "https://ebay.com/itm/123456?hash=item28caef0a3a:g:E3kAAOSwlGJiMikD&amdata=enc%3AAQAHAAAAsJoWXGf0hxNZspTmhb8%2FTJCCurAWCHuXJ2Xi3S9cwXL6BX04zSEiVaDMCvsUbApftgXEAHGJU1ZGugZO%2FnW1U7Gb6vgoL%2BmXlqCbLkwoZfF3AUAK8YvJ5B4%2BnhFA7ID4dxpYs4jjExEnN5SR2g1mQe7QtLkmGt%2FZ%2FbH2W62cXPuKbf550ExbnBPO2QJyZTXYCuw5KVkMdFMDuoB4p3FwJKcSPzez5kyQyVjyiIq6PB2q%7Ctkp%3ABlBMULq7kqyXYA"
	},
	{
	  "title": "Apple AirPods Pro 2nd Generaci\u00f3n PRECINTADOS",
	  "price": "USD213.60",
	  "image_url": "https://i.ebayimg.com/thumbs/images/g/uYUAAOSwAnRlOs3r/s-l300.webp",
	  "status": "Totalmente nuevo",
	  "link": "https://www.ebay.com/itm/296009215198?hash=item44eb862cde:g:uYUAAOSwAnRlOs3r&amdata=enc%3AAQAIAAAA4C51eeZHbaBu1XSPHR%2Fnwuf7F8cSg34hoOPgosNeYJP9vIn%2F26%2F3mLe8lCmBlOXTVzw3j%2FBwiYtJw9uw7vte%2FzAb35Chru0UMEOviXuvRavQUj8eTBCYWcrQuOMtx1qhTcAscv4IBqmJvLhweUPpmd7OEzGczZoBuqmb%2B9iUbmTKjD74NWJyVZvMy%2B02JG1XUhOjAp%2BVNNLH0dU%2Bke530dAnRZSd1ECJpxuqSJrH8jn6WcvuPHt4YRVzKuKzd8DBnu%2F0q%2FwkEkBBYy8AlLKj6RuRf4BOimd5C5QFRRey1p9D%7Ctkp%3ABFBMoLnxs_Ni"
	},
	{
	  "title": "Apple AirPods Pro 2da Generaci\u00f3n con Estuche de Carga Inal\u00e1mbrico MagSafe - SELLADO",
	  "price": "USD74.24",
	  "image_url": "https://i.ebayimg.com/thumbs/images/g/ce4AAOSwA6NlP5jN/s-l300.webp",
	  "status": "Totalmente nuevo",
	  "link": "https://www.ebay.com/itm/155859459163?hash=item2449f29c5b:g:ce4AAOSwA6NlP5jN&amdata=enc%3AAQAIAAAA4A9XuJkJREhc3yfSLaVZvBooRWQQJkXbWAXmnSwndqFI7UgsJgH2U98ZoS6%2BiExiDMoeL6W7E6l7y9KCZDHdxwCixGLPwJKvM7WiZNcXhuH5NsSgCjcvJt56K%2BXCLMuQhmJvfih%2BWeZlqRXTwfUKDE0NMNSXE98u1tAOMohQ1skdT%2FKS7RDJ7Dpo%2BcGb1ZCt9KRAoNMKOkmnr5BVTMHFpzAOOC6fQAlRFt8e5yXrlZYwbyMchnboMF3F3xODe5dEvxv3YiHinOAoLl93pNmz1Yn1ToO%2FrIef4ZCbsiECkfk3%7Ctkp%3ABFBMoLnxs_Ni"
	},
	{
	  "title": "AURICULARES BLUETOOTH APPLE AIRPODS PRO AURICULARES CON ESTUCHE DE CARGA INAL\u00c1MBRICA",
	  "price": "USD80.99",
	  "image_url": "https://i.ebayimg.com/thumbs/images/g/v78AAOSw-NFkS37t/s-l300.webp",
	  "status": "De segunda mano",
	  "link": "https://www.ebay.com/itm/144890564294?hash=item21bc268ec6:g:v78AAOSw-NFkS37t&amdata=enc%3AAQAIAAAA4LD7XbJuhs0J9HL02DzzbH8fZ7T5wxLjkMhC2%2B%2Bwg5WrsH3ni7TcbBIqImnuOYtcDKznUPm%2F2%2FHNoD43mA6%2Ffw23995rH3%2BW9Wb5QPimLoCn3cA1PUbPBO6zG99gyg04JbxmtGH6f%2FYaZlfXVo3vWLvSqwux3ZsP1okWWuuz5y1OjJvXQB9ACVoLSCC6au2RpqS3DdOPwi%2FGrP0Osj44XNbT8UHtSKF6N%2FmbqNTLXoaJVunBjlnpjvE%2BPTKAxSuFX3GvptHXOqFCKAkdKpmH8%2BW8hRTX0q9gVxv9Nw6epsC%2B%7Ctkp%3ABFBMoLnxs_Ni"
	},
	{
	  "title": "Auriculares AirPods Pro 2 Bluetooth con estuche de carga inal\u00e1mbrica MagSafe para tel\u00e9fonos",
	  "price": "USD58.53",
	  "image_url": "https://i.ebayimg.com/thumbs/images/g/ol8AAOSw~e9lLfXD/s-l300.webp",
	  "status": "Nuevo (otro)",
	  "link": "https://www.ebay.com/itm/295718525142?hash=item44da3298d6:g:ol8AAOSw~e9lLfXD&amdata=enc%3AAQAIAAAA4GHg1xYgyvcQRhTwQRVL2l5CvX3zSMsR281hXacuujxxm9F%2F5pYTo9aEp9ZL7e8jsJB9qlUMUF7kLuYgPHFoEh%2B0qQtajGFWTi3NqY3h8vYwhFCWqITUnrV%2BhJ0voFK4qvh3CmnzDVbYnkYPY36OehktFXjrS%2B6WWx24w7a1TXAJSCwEi0Oh9SWqDyrkSH19m7Ft0t2gMq3EpMYy7IidY2HOfQkZe7MUpHNqVoUnzaNiKv129JAfiWGkOPtwABqRt2b09iKMpoPjTfPUIHMj%2F5yL97%2B%2BCj5A4zGc5TRPsIrG%7Ctkp%3ABFBMoLnxs_Ni"
	},
  ]

With ScraperAPI, you can focus on extracting valuable eBay data without worrying about being blocked or keeping your scraper updated with eBay’s evolving anti-bot techniques.

You can perform a wide range of analyses with the collected data and use it for your specific purposes, including price comparison and market research.

Wrapping Up

This tutorial provides a quick rundown of how to scrape data from eBay using Python. It covered:

  • The steps involved in scraping eBay, including sending HTTP requests, parsing HTML responses with BeautifulSoup, and using ScraperAPI to handle anti-bot measures.
  • Understanding eBay’s page structure
  • Using custom headers to ensure a successful response at lower scale
  • How to extract product details from eBay

If you’re ready to scale your data collection to thousands or even millions of pages, consider exploring ScraperAPI’s Business Plan;, which offers advanced features, premium support, and an account manager.

Do you need 10M+ API credits? Contact sales for a custom plan.

Feel free to reach out if you have any further questions or need assistance!

Until next time, happy scraping!

 

FAQs about eBay scraping

Data obtained from eBay can offer insights into product trends, pricing, and competitor analysis, help users find the best deals, and enable data-driven decision-making.

Learn how to gain a competitive edge with ecommerce data.

Yes, it is legal to scrape data from eBay. What’s important to keep in mind is that you should only scrape publicly available data. As long as you are not bypassing login walls and you are using the data for a proper goal, like market research and academic research, there won’t be any legal action against you.

When scraping eBay, you can collect product details like:

  • Names
  • Descriptions
  • Prices
  • Seller information
  • Shipping details
  • Customer reviews
  • Product ratings

And much more.

As long as the data is publicly available, you can use web scrapers to collect such data.

About the author

John Fawole

John Fawole

John Fáwọlé is a technical writer and developer. He currently works as a freelance content marketer and consultant for tech startups.

Table of Contents

Ready to start scraping?

Get started with 5,000 free API credits or contact sales
Get Started For Free

Related Articles

Need More Than 3M API Credits per Month?

Talk to an expert and learn how to build a scalable scraping solution.

Book a call