"How do I scrape TikTok?" usually means one of three things: follower counts for a list of creators, the latest videos from an account with their views and likes, or tracking how those numbers change over time. This guide builds that in plain Python, shows what works without a browser, and explains where the DIY approach stops working.
What you can get without logging in
TikTok's web app is a React app, but the first HTML response for a profile page (https://www.tiktok.com/@handle) contains a big JSON blob in a script tag:
<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__" type="application/json">{...}</script>
Inside it, __DEFAULT_SCOPE__["webapp.user-detail"]["userInfo"] holds the profile (user) and its counters (stats). That's public data, and you can read it with requests and the standard json module. You don't need BeautifulSoup.
The video list is different. The grid loads from an internal endpoint (/api/post/item_list/) that needs signed query parameters (msToken, X-Bogus, X-Gnarly, and so on) produced by obfuscated JavaScript. The signing changes often, so we'll treat that part as a known limit and come back to it later.
Setup
python -m venv .venv && source .venv/bin/activate
pip install requests
Step 1: Fetch a profile
import json
import re
import time
import requests
SESSION = requests.Session()
SESSION.headers.update({
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/129.0 Safari/537.36"
),
"Accept-Language": "en-US,en;q=0.9",
})
DATA_RE = re.compile(
r'<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__"[^>]*>(.*?)</script>',
re.S,
)
class ProfileError(Exception):
pass
def fetch_profile(handle: str) -> dict:
handle = handle.lstrip("@")
resp = SESSION.get(f"https://www.tiktok.com/@{handle}", timeout=20)
resp.raise_for_status()
match = DATA_RE.search(resp.text)
if not match:
raise ProfileError("rehydration blob missing (blocked or layout changed)")
data = json.loads(match.group(1))
detail = data.get("__DEFAULT_SCOPE__", {}).get("webapp.user-detail", {})
if detail.get("statusCode", 0) != 0:
raise ProfileError(f"TikTok statusCode {detail.get('statusCode')}")
info = detail["userInfo"]
user, stats = info["user"], info["stats"]
return {
"handle": user["uniqueId"],
"id": user["id"],
"sec_uid": user["secUid"],
"nickname": user.get("nickname"),
"verified": user.get("verified", False),
"private": user.get("privateAccount", False),
"bio": user.get("signature", ""),
"followers": stats.get("followerCount"),
"following": stats.get("followingCount"),
"likes": stats.get("heartCount") or stats.get("heart"),
"videos": stats.get("videoCount"),
}
if __name__ == "__main__":
print(fetch_profile("@tiktok"))
A few details that will save you some debugging:
- Check
statusCode, not just the HTTP status. TikTok returns HTTP 200 for missing or banned accounts. The blob then carries a non-zero statusCode (10221 or 10202 are common) and no userInfo.
- Keep
sec_uid. Handles can change, but the numeric id and secUid don't. Use them as your primary key.
- Don't parse the visible "1.2M". The JSON holds exact integers, so use those.
Step 2: Store snapshots in SQLite
A single scrape is a fact. A series of scrapes shows a trend. Store every run as a row, so growth is just a SELECT:
import sqlite3
from datetime import datetime, timezone
def init_db(path="tiktok.db"):
db = sqlite3.connect(path)
db.execute("""
CREATE TABLE IF NOT EXISTS profile_snapshots (
user_id TEXT NOT NULL,
handle TEXT NOT NULL,
taken_at TEXT NOT NULL,
followers INTEGER,
likes INTEGER,
videos INTEGER,
PRIMARY KEY (user_id, taken_at)
)
""")
return db
def save_snapshot(db, p):
db.execute(
"INSERT OR IGNORE INTO profile_snapshots VALUES (?,?,?,?,?,?)",
(p["id"], p["handle"],
datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:00Z"),
p["followers"], p["likes"], p["videos"]),
)
db.commit()
The taken_at value is rounded to the hour, so if a cron job accidentally runs twice in an hour, INSERT OR IGNORE drops the duplicate.
Step 3: Loop over many creators politely
import random
def run(handles, db):
for h in handles:
for attempt in range(3):
try:
save_snapshot(db, fetch_profile(h))
print("ok", h)
break
except ProfileError as e:
print("skip", h, e)
break
except requests.RequestException as e:
wait = 2 ** attempt * 5
print("retry", h, e, f"in {wait}s")
time.sleep(wait)
time.sleep(random.uniform(3, 7))
if __name__ == "__main__":
db = init_db()
run(["tiktok", "nasa", "duolingo"], db)
Then the growth query looks like this:
SELECT handle,
MAX(followers) - MIN(followers) AS gained,
COUNT(*) AS samples
FROM profile_snapshots
WHERE taken_at >= strftime('%Y-%m-%dT00:00Z', 'now', '-7 days')
GROUP BY handle
ORDER BY gained DESC;
The walls you'll hit
This is where most TikTok tutorials stop being honest. Here's what actually happens once the script runs on a schedule:
- Datacenter IPs get a different page. From a cloud VM you will often get a page with no rehydration blob, a "verify" screen, or a redirect. It works from a home connection and fails on a server. Residential proxies help, but they cost money and you have to rotate them.
- The blob moves. In the last few years the data has lived in
SIGI_STATE, then in __UNIVERSAL_DATA_FOR_REHYDRATION__, with different key paths each time. Wrap the parsing in a single function (as above) and alert when it raises, so a layout change doesn't silently produce empty rows.
- Videos need signatures. The profile HTML sometimes includes a few items, but often none. Full pagination goes through
item_list with signed parameters. The DIY options are running a headless browser (Playwright) and capturing the XHR responses while scrolling, or reverse-engineering the signer. Both work, both are slow or fragile, and both break when TikTok ships a new bundle.
- Region matters. The same handle can return different counts or videos depending on the exit country. Pin your proxy region if you compare numbers over time.
- Rate limits are soft. You won't get a clean 429. Instead, responses quietly degrade to empty blobs. Track the empty-response rate as a health metric.
- Respect the data. Only collect public data, store what you actually need, and check TikTok's terms and your local privacy rules (GDPR applies to creator data in the EU).
When DIY stops being worth it
If you only need follower counts for a few dozen accounts, the script above, a cheap VPS, and maybe a proxy are enough. Once you need full video history with views, likes and comments, the signing problem eats your time.
That's the point where I switch to an API. For example, ScrapeAtlas* has a /v1/tiktok/profile/videos endpoint that takes a handle, a sort order (latest, popular, oldest), an optional since date, and a max_cursor for pagination. It returns the same itemList shape TikTok uses internally:
import os
import requests
API = "https://api.scrapeatlas.com/v1/tiktok/profile/videos"
HEADERS = {"x-api-key": os.environ["SCRAPEATLAS_API_KEY"]}
def all_videos(handle, since=None):
params = {"handle": handle, "sort_by": "latest"}
if since:
params["since"] = since # ISO 8601 or Unix seconds
while True:
r = requests.get(API, headers=HEADERS, params=params, timeout=60)
r.raise_for_status()
page = r.json()
for v in page.get("itemList", []):
s = v.get("stats", {})
yield {
"id": v["id"],
"created": v["createTime"],
"desc": v.get("desc", ""),
"views": s.get("playCount"),
"likes": s.get("diggCount"),
"comments": s.get("commentCount"),
"shares": s.get("shareCount"),
}
if not page.get("hasMore"):
break
params["max_cursor"] = page["cursor"]
for v in all_videos("nasa", since="2026-09-01"):
print(v)
Since the fields match TikTok's own stats object, the SQLite schema above only needs a video_snapshots table next to profile_snapshots, and the rest of your pipeline stays the same.
Wrap-up
- Profiles:
requests plus the __UNIVERSAL_DATA_FOR_REHYDRATION__ blob, keyed on id/secUid, with statusCode checks.
- Store hourly snapshots in SQLite and compute growth with SQL.
- Expect IP blocking, layout changes and signed video endpoints. Decide early whether you'll maintain a browser or signer, or hand that part off.
If you've found a more stable way to get the video list, let me know in the comments.
*I build it.