9 min read
Technology

Scrape LinkedIn Jobs with Python: 2026 Step-by-Step Guide

If you want to scrape LinkedIn jobs with Python, there are two realistic routes in 2026: plain requests plus BeautifulSoup against LinkedIn's public guest job-search pages, or Selenium 4 driving a real browser. This guide shows both, saves the results to CSV with pandas, and is upfront about the part that matters most: LinkedIn's markup changes, and its Terms of Service limit what you can do.

The examples only touch public job listings that anyone can see without logging in. Read the legal and ethical section before you run anything at scale.

Which approach: requests + BeautifulSoup or Selenium?

Most tutorials on web scraping LinkedIn jobs using Python and Selenium jump straight to a browser. That works, but it is often not the better choice for public job search.

  • requests + BeautifulSoup (guest endpoints): fast, light, no browser or driver to maintain. The public job search is served as plain HTML fragments, so you can parse it directly. Downside: you depend on undocumented URLs and class names, and LinkedIn may return an error or a login wall if you send too many requests.
  • Selenium 4: renders the page like a user would, so it copes with content that needs JavaScript or scrolling. Downside: slower, heavier, needs a matching browser and driver, and it is easier for LinkedIn to detect and interrupt automated browsing.

Recommendation: start with requests and BeautifulSoup. Reach for Selenium only if the HTML you get back is missing the data you need.

Setup

Use Python 3.10 or newer and install the libraries in a virtual environment:

pip install requests beautifulsoup4 pandas selenium

Selenium 4.6 and later downloads a matching browser driver automatically through Selenium Manager, so you no longer need to install chromedriver by hand.

Method 1: scrape LinkedIn job postings with requests and BeautifulSoup

When you search for jobs without logging in, LinkedIn's page loads more results from a guest endpoint. You can open the same kind of URL yourself. It takes a keyword, a location and a start offset for pagination:

https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search?keywords=python+developer&location=Israel&start=0

This is not a documented API, so it can change or disappear. Treat it as a convenience, not a contract.

Step 1: fetch one page of results

import time
import random
import requests
from bs4 import BeautifulSoup

SEARCH_URL = "https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search"

HEADERS = {
    "User-Agent": "Mozilla/5.0 (compatible; job-research-script; contact: [email protected])",
    "Accept-Language": "en-US,en;q=0.9",
}


def fetch_page(keywords, location, start=0):
    params = {"keywords": keywords, "location": location, "start": start}
    response = requests.get(SEARCH_URL, params=params, headers=HEADERS, timeout=20)
    if response.status_code != 200:
        print(f"Stopped: HTTP {response.status_code} at start={start}")
        return None
    return response.text

Step 2: parse the job cards

Each result is a card. At the time of writing, the useful pieces sit in these elements. Check them in your browser's developer tools, because these class names are the first thing to break:

def parse_jobs(html):
    soup = BeautifulSoup(html, "html.parser")
    jobs = []
    for card in soup.select("li"):
        title = card.select_one(".base-search-card__title")
        company = card.select_one(".base-search-card__subtitle")
        place = card.select_one(".job-search-card__location")
        link = card.select_one("a.base-card__full-link")
        posted = card.select_one("time")
        if not (title and link):
            continue  # not a job card, or the markup changed
        jobs.append({
            "title": title.get_text(strip=True),
            "company": company.get_text(strip=True) if company else "",
            "location": place.get_text(strip=True) if place else "",
            "posted": posted.get("datetime", "") if posted else "",
            "url": link["href"].split("?")[0],
        })
    return jobs

Step 3: paginate politely and save to CSV with pandas

import pandas as pd


def scrape(keywords, location, max_pages=5):
    all_jobs = []
    for page in range(max_pages):
        html = fetch_page(keywords, location, start=page * 10)
        if not html:
            break
        jobs = parse_jobs(html)
        if not jobs:
            print("No more results (or the selectors need updating).")
            break
        all_jobs.extend(jobs)
        time.sleep(random.uniform(3, 6))  # be gentle: pause between requests
    return all_jobs


if __name__ == "__main__":
    rows = scrape("python developer", "Israel", max_pages=3)
    df = pd.DataFrame(rows).drop_duplicates(subset="url")
    df.to_csv("linkedin_jobs.csv", index=False, encoding="utf-8-sig")
    print(f"Saved {len(df)} jobs to linkedin_jobs.csv")

The utf-8-sig encoding keeps non-English characters readable when you open the file in Excel.

Optional: get the full job description

Each card links to a job page. The guest version of that page includes the description. The job ID is the number at the end of the URL, and you can request a lighter fragment for it:

def fetch_description(job_url):
    job_id = job_url.rstrip("/").split("-")[-1]
    url = f"https://www.linkedin.com/jobs-guest/jobs/api/jobPosting/{job_id}"
    response = requests.get(url, headers=HEADERS, timeout=20)
    if response.status_code != 200:
        return ""
    soup = BeautifulSoup(response.text, "html.parser")
    box = soup.select_one(".show-more-less-html__markup")
    return box.get_text(" ", strip=True) if box else ""

Call it sparingly, with the same pauses as before. One request per job adds up quickly, so only fetch descriptions for the jobs you actually care about.

Method 2: web scraping LinkedIn jobs with Python and Selenium 4

If you prefer a browser, here is the same idea with modern Selenium. Note the Selenium 4 style: find_element(By.CSS_SELECTOR, ...) and explicit waits. The old find_element_by_* methods were removed, so tutorials that use them will fail on current versions.

import time
import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

URL = "https://www.linkedin.com/jobs/search?keywords=python%20developer&location=Israel"

driver = webdriver.Chrome()  # Selenium Manager fetches the driver
try:
    driver.get(URL)
    WebDriverWait(driver, 15).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, "ul.jobs-search__results-list"))
    )

    # Scroll a few times so more public results load
    for _ in range(3):
        driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
        time.sleep(3)

    rows = []
    for card in driver.find_elements(By.CSS_SELECTOR, "ul.jobs-search__results-list li"):
        try:
            rows.append({
                "title": card.find_element(By.CSS_SELECTOR, ".base-search-card__title").text,
                "company": card.find_element(By.CSS_SELECTOR, ".base-search-card__subtitle").text,
                "location": card.find_element(By.CSS_SELECTOR, ".job-search-card__location").text,
                "url": card.find_element(By.CSS_SELECTOR, "a.base-card__full-link").get_attribute("href").split("?")[0],
            })
        except Exception:
            continue  # skip cards that do not match the expected layout

    pd.DataFrame(rows).drop_duplicates(subset="url").to_csv(
        "linkedin_jobs_selenium.csv", index=False, encoding="utf-8-sig"
    )
    print(f"Saved {len(rows)} rows")
finally:
    driver.quit()

Expect the occasional sign-in prompt or a stalled page. If LinkedIn asks you to log in, stop. Do not automate a logged-in session (see below).

Selectors will break: how to keep the scraper alive

  • Open the public search page in your browser, right-click a job, choose Inspect, and confirm each class name before you rely on it.
  • Keep every selector in one place at the top of the file so a fix is a one-line change.
  • Fail loudly: if a page returns zero cards, print a warning instead of silently saving an empty CSV.
  • Save the raw HTML of a failing page while you debug, so you can inspect it offline.

Is it allowed? LinkedIn's Terms of Service and ethical limits

This section is general information, not legal advice. LinkedIn's User Agreement prohibits scraping and the use of bots or automated methods to access the service without permission, and the rules differ by country and use case. Read the current terms yourself, and ask a qualified professional if you plan to use scraped data commercially.

Practical guardrails that keep a small research project on the more responsible side:

  • Public data only. Stick to job listings visible without an account. Do not scrape personal profiles.
  • Never scrape while logged in. Automating your own account is exactly what the terms are most concerned with, and it puts your account at risk.
  • Rate limit yourself. Add pauses, keep volumes small, and stop at the first error or block instead of retrying aggressively.
  • Do not bypass blocks. If you hit a CAPTCHA, login wall or rate limit, that is a signal to stop, not to rotate proxies.
  • Respect privacy and copyright. Do not republish full job descriptions or store personal data about recruiters.
  • Prefer official or alternative sources. LinkedIn offers official APIs for approved partners. Other job boards and aggregators provide APIs or feeds for the same listings, which are a safer base for anything long-lived.

What to do with the data

Once you have a CSV, pandas makes simple analysis easy: count jobs per company with df["company"].value_counts(), filter by keyword in the title, or track how many new postings appear week to week by saving a dated file each run.

FAQ

Can I scrape LinkedIn jobs with Python?

Technically yes, for public listings, using requests and BeautifulSoup or Selenium. Whether you are permitted to is a separate question: LinkedIn's terms restrict automated access, so keep it small, public and respectful, and consider official APIs or other job boards.

Is there a LinkedIn job scraper in Python that works without logging in?

Yes. The requests example above reads the public guest job-search pages and needs no account. It can stop working whenever LinkedIn changes its markup or blocks the traffic, so build in error checks.

Should I use Selenium or BeautifulSoup for scraping job postings in Python?

Use requests with BeautifulSoup first because it is faster and simpler. Switch to Selenium only when the content you need is not in the raw HTML and requires a real browser to render.

Why does my LinkedIn scraper return empty results?

The usual causes are changed class names, a login wall or rate limiting (check the HTTP status code), or searching too fast. Re-inspect the page, slow down, and print what the server actually returned.

If you want to learn this kind of automation properly, TovTech's data course teaches it.