Articles on: API

Scraping an E-commerce product page

Don't hesitate to take a look at our Python web scraping 101 to read a detailed introduction to these libraries. We also have a lot of tutorials in different languages in our web scraping blog.

We are going to extract the price, image URL, and product name for this product:

E-commerce product page with Chome dev tools opened.

We are going to use some basic CSS selectors to extract the price, title, and image URL. Then we simply print the result into the console.

import requests
from bs4 import BeautifulSoup

# Extract product data from a (dummy) E-commerce product page

# Replace with your ScrapingBee API
endpoint = ""
rows = []

params = {
    'api_key': SCRAPINGBEE_API_KEY,
    'url': '',

response = requests.get(endpoint, params=params)
if response.status_code != 200:
    print('Error with your request: ' + str(response.status_code))
    #import pdb;pdb.set_trace()
    soup = BeautifulSoup(response.content, 'html.parser')
    product = {
        'price':'.my-4 s')[0].text,
        'image_url':'.slick-track img')[0]['src'],
        'product_title':'.col-12 h2')[0].text

Updated on: 19/01/2022

Was this article helpful?

Share your feedback


Thank you!