TikTok Scraper in Python: Profiles, Videos and the Walls You'll Hit (2026)

●4 ●13
calendar_today • schedule5 min read

"How do I scrape TikTok?" usually means one of three things: follower counts for a list of creators, the latest videos from an account with their views and likes, or tracking how those numbers change over time. This guide builds that in plain Python, shows what works without a browser, and explains where the DIY approach stops working.

What you can get without logging in

TikTok's web app is a React app, but the first HTML response for a profile page (https://www.tiktok.com/@handle) contains a big JSON blob in a script tag:

<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__" type="application/json">{...}</script>

Inside it, __DEFAULT_SCOPE__["webapp.user-detail"]["userInfo"] holds the profile (user) and its counters (stats). That's public data, and you can read it with requests and the standard json module. You don't need BeautifulSoup.

The video list is different. The grid loads from an internal endpoint (/api/post/item_list/) that needs signed query parameters (msToken, X-Bogus, X-Gnarly, and so on) produced by obfuscated JavaScript. The signing changes often, so we'll treat that part as a known limit and come back to it later.

Setup

python -m venv .venv && source .venv/bin/activate
pip install requests

Step 1: Fetch a profile

import json
import re
import time
import requests

SESSION = requests.Session()
SESSION.headers.update({
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/129.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
})

DATA_RE = re.compile(
    r'<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__"[^>]*>(.*?)</script>',
    re.S,
)

class ProfileError(Exception):
    pass

def fetch_profile(handle: str) -> dict:
    handle = handle.lstrip("@")
    resp = SESSION.get(f"https://www.tiktok.com/@{handle}", timeout=20)
    resp.raise_for_status()

    match = DATA_RE.search(resp.text)
    if not match:
        raise ProfileError("rehydration blob missing (blocked or layout changed)")

    data = json.loads(match.group(1))
    detail = data.get("__DEFAULT_SCOPE__", {}).get("webapp.user-detail", {})
    if detail.get("statusCode", 0) != 0:
        raise ProfileError(f"TikTok statusCode {detail.get('statusCode')}")

    info = detail["userInfo"]
    user, stats = info["user"], info["stats"]
    return {
        "handle": user["uniqueId"],
        "id": user["id"],
        "sec_uid": user["secUid"],
        "nickname": user.get("nickname"),
        "verified": user.get("verified", False),
        "private": user.get("privateAccount", False),
        "bio": user.get("signature", ""),
        "followers": stats.get("followerCount"),
        "following": stats.get("followingCount"),
        "likes": stats.get("heartCount") or stats.get("heart"),
        "videos": stats.get("videoCount"),
    }

if __name__ == "__main__":
    print(fetch_profile("@tiktok"))

A few details that will save you some debugging:

  • Check statusCode, not just the HTTP status. TikTok returns HTTP 200 for missing or banned accounts. The blob then carries a non-zero statusCode (10221 or 10202 are common) and no userInfo.
  • Keep sec_uid. Handles can change, but the numeric id and secUid don't. Use them as your primary key.
  • Don't parse the visible "1.2M". The JSON holds exact integers, so use those.

Step 2: Store snapshots in SQLite

A single scrape is a fact. A series of scrapes shows a trend. Store every run as a row, so growth is just a SELECT:

import sqlite3
from datetime import datetime, timezone

def init_db(path="tiktok.db"):
    db = sqlite3.connect(path)
    db.execute("""
        CREATE TABLE IF NOT EXISTS profile_snapshots (
            user_id   TEXT NOT NULL,
            handle    TEXT NOT NULL,
            taken_at  TEXT NOT NULL,
            followers INTEGER,
            likes     INTEGER,
            videos    INTEGER,
            PRIMARY KEY (user_id, taken_at)
        )
    """)
    return db

def save_snapshot(db, p):
    db.execute(
        "INSERT OR IGNORE INTO profile_snapshots VALUES (?,?,?,?,?,?)",
        (p["id"], p["handle"],
         datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:00Z"),
         p["followers"], p["likes"], p["videos"]),
    )
    db.commit()

The taken_at value is rounded to the hour, so if a cron job accidentally runs twice in an hour, INSERT OR IGNORE drops the duplicate.

Step 3: Loop over many creators politely

import random

def run(handles, db):
    for h in handles:
        for attempt in range(3):
            try:
                save_snapshot(db, fetch_profile(h))
                print("ok", h)
                break
            except ProfileError as e:
                print("skip", h, e)
                break
            except requests.RequestException as e:
                wait = 2 ** attempt * 5
                print("retry", h, e, f"in {wait}s")
                time.sleep(wait)
        time.sleep(random.uniform(3, 7))

if __name__ == "__main__":
    db = init_db()
    run(["tiktok", "nasa", "duolingo"], db)

Then the growth query looks like this:

SELECT handle,
       MAX(followers) - MIN(followers) AS gained,
       COUNT(*) AS samples
FROM profile_snapshots
WHERE taken_at >= strftime('%Y-%m-%dT00:00Z', 'now', '-7 days')
GROUP BY handle
ORDER BY gained DESC;

The walls you'll hit

This is where most TikTok tutorials stop being honest. Here's what actually happens once the script runs on a schedule:

  1. Datacenter IPs get a different page. From a cloud VM you will often get a page with no rehydration blob, a "verify" screen, or a redirect. It works from a home connection and fails on a server. Residential proxies help, but they cost money and you have to rotate them.
  2. The blob moves. In the last few years the data has lived in SIGI_STATE, then in __UNIVERSAL_DATA_FOR_REHYDRATION__, with different key paths each time. Wrap the parsing in a single function (as above) and alert when it raises, so a layout change doesn't silently produce empty rows.
  3. Videos need signatures. The profile HTML sometimes includes a few items, but often none. Full pagination goes through item_list with signed parameters. The DIY options are running a headless browser (Playwright) and capturing the XHR responses while scrolling, or reverse-engineering the signer. Both work, both are slow or fragile, and both break when TikTok ships a new bundle.
  4. Region matters. The same handle can return different counts or videos depending on the exit country. Pin your proxy region if you compare numbers over time.
  5. Rate limits are soft. You won't get a clean 429. Instead, responses quietly degrade to empty blobs. Track the empty-response rate as a health metric.
  6. Respect the data. Only collect public data, store what you actually need, and check TikTok's terms and your local privacy rules (GDPR applies to creator data in the EU).

When DIY stops being worth it

If you only need follower counts for a few dozen accounts, the script above, a cheap VPS, and maybe a proxy are enough. Once you need full video history with views, likes and comments, the signing problem eats your time.

That's the point where I switch to an API. For example, ScrapeAtlas* has a /v1/tiktok/profile/videos endpoint that takes a handle, a sort order (latest, popular, oldest), an optional since date, and a max_cursor for pagination. It returns the same itemList shape TikTok uses internally:

import os
import requests

API = "https://api.scrapeatlas.com/v1/tiktok/profile/videos"
HEADERS = {"x-api-key": os.environ["SCRAPEATLAS_API_KEY"]}

def all_videos(handle, since=None):
    params = {"handle": handle, "sort_by": "latest"}
    if since:
        params["since"] = since          # ISO 8601 or Unix seconds
    while True:
        r = requests.get(API, headers=HEADERS, params=params, timeout=60)
        r.raise_for_status()
        page = r.json()
        for v in page.get("itemList", []):
            s = v.get("stats", {})
            yield {
                "id": v["id"],
                "created": v["createTime"],
                "desc": v.get("desc", ""),
                "views": s.get("playCount"),
                "likes": s.get("diggCount"),
                "comments": s.get("commentCount"),
                "shares": s.get("shareCount"),
            }
        if not page.get("hasMore"):
            break
        params["max_cursor"] = page["cursor"]

for v in all_videos("nasa", since="2026-09-01"):
    print(v)

Since the fields match TikTok's own stats object, the SQLite schema above only needs a video_snapshots table next to profile_snapshots, and the rest of your pipeline stays the same.

Wrap-up

  • Profiles: requests plus the __UNIVERSAL_DATA_FOR_REHYDRATION__ blob, keyed on id/secUid, with statusCode checks.
  • Store hourly snapshots in SQLite and compute growth with SQL.
  • Expect IP blocking, layout changes and signed video endpoints. Decide early whether you'll maintain a browser or signer, or hand that part off.

If you've found a more stable way to get the video list, let me know in the comments.


*I build it.

1 Comment

0 votes
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

Dashboard Operasional Armada Rental Mobil dengan Python + FastAPI

Masbadar - Mar 12

Twitter (X) Scraper in Python: Tweet Metrics, SQLite Snapshots, and the Walls You'll Hit (2026)

viktorhendelmann - Oct 14

Cloudflare Blocked My Server, So I Made My Visitors' Browsers Do the Fetching

bryanhamiltondevverified - Nov 15

Python for commerce students, from a former bio kid

rahuldk1105 - Oct 30

Free SEO Tools Every Small Business Should Use (2026) | FloxoLab

floxolab - Oct 20
chevron_left
246 Points • 17 Badges
5Posts
0Comments

Related Jobs

View all jobs →

Commenters (This Week)

2 comments
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!