A weekly TikTok search report is useful when it shows which videos appeared in the latest results since the previous run. To do this, save the IDs of videos already seen and add new results to a CSV. Below is a reproducible Python script using the TikTok Scraper API: it searches videos by keyword, parses both documented response formats, and maintains a local ID database.
The goal: spot new posts instead of reviewing the entire results list
A support team, market researcher, or content analyst may need a regular sample of public videos for a product-related query. Exporting the full results every time makes repeated videos harder to review and new posts easier to miss among those already processed. An ID log makes each run comparable: the CSV contains only videos new to this process, along with the main fields returned in the response.
This log answers a limited but practical question: which items appeared in the current run’s response but were absent from the saved set of IDs? This is useful for a manual review queue and for sending data to a spreadsheet or subsequent analysis. The sample reflects the API results for the specified query; it does not, by itself, describe every TikTok post.
What you need
- Python 3.8 or later.
- Access to the TikTok Scraper API and a license key.
- A query you can repeat without changing its meaning, such as a product name or a specific phrase.
Save the key in the JSONSCRAPER_TIKTOK_LICENSE_KEY environment variable rather than in the script file. In the example below, the key is passed as the license_key query parameter, as specified by the API schema. Do not print it in execution logs.
export JSONSCRAPER_TIKTOK_LICENSE_KEY="your_key"
python -m pip install requests
In Windows PowerShell, you can set the variable for the current session like this: $env:JSONSCRAPER_TIKTOK_LICENSE_KEY="your_key". Do not add real secrets to Git or a shared CSV.
Integration: run a search and normalize the JSON
For video search, the documentation describes the GET route /searchVideo with the required keyword parameter. You can also set count, region, sort order, time filter, and a cursor for paging through results. This minimal example uses only a keyword, the number of items to request, and the US region. The TikTok API endpoint reference lists the exact parameter names.
The search response requires special attention to its structure: the aweme_list array may be empty, while video items may appear in search_item_list, with data nested under aweme_info. The code therefore checks both paths, combines the items, and removes duplicates using aweme_id. This prevents successfully parsed JSON from being mistakenly turned into an empty report.
import csv
import json
import os
from pathlib import Path
from datetime import datetime, timezone
import requests
BASE_URL = "https://tiktok.evelode.com/searchVideo"
KEYWORD = "product name"
REGION = "US"
COUNT = 30
STATE_FILE = Path("seen_video_ids.json")
OUTPUT_FILE = Path("new_tiktok_videos.csv")
license_key = os.environ.get("JSONSCRAPER_TIKTOK_LICENSE_KEY")
if not license_key:
raise RuntimeError("Set JSONSCRAPER_TIKTOK_LICENSE_KEY")
response = requests.get(
BASE_URL,
params={
"license_key": license_key,
"keyword": KEYWORD,
"count": COUNT,
"region": REGION,
},
timeout=60,
)
response.raise_for_status()
data = response.json()
# The example response contains an outer status and a nested tiktok object.
if "status" in data and data["status"] != "ok":
raise RuntimeError("API returned a status other than ok")
tiktok = data.get("tiktok") or {}
items = list(tiktok.get("aweme_list") or [])
for entry in tiktok.get("search_item_list") or []:
info = entry.get("aweme_info")
if isinstance(info, dict):
items.append(info)
# Unique items in the current response, preserving order.
current = {}
for item in items:
video_id = item.get("aweme_id")
if video_id:
current[str(video_id)] = item
if STATE_FILE.exists():
seen = set(json.loads(STATE_FILE.read_text(encoding="utf-8")))
else:
seen = set()
new_items = [(video_id, item) for video_id, item in current.items()
if video_id not in seen]
run_time = datetime.now(timezone.utc).isoformat()
fields = ["collected_at_utc", "query", "aweme_id", "created_at",
"author", "description", "share_url", "play_count",
"digg_count", "comment_count", "share_count"]
write_header = not OUTPUT_FILE.exists() or OUTPUT_FILE.stat().st_size == 0
with OUTPUT_FILE.open("a", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fields)
if write_header:
writer.writeheader()
for video_id, item in new_items:
author = item.get("author") or {}
stats = item.get("statistics") or {}
writer.writerow({
"collected_at_utc": run_time,
"query": KEYWORD,
"aweme_id": video_id,
"created_at": item.get("create_time"),
"author": author.get("unique_id") or author.get("nickname"),
"description": item.get("desc"),
"share_url": item.get("share_url"),
"play_count": stats.get("play_count"),
"digg_count": stats.get("digg_count"),
"comment_count": stats.get("comment_count"),
"share_count": stats.get("share_count"),
})
seen.update(current.keys())
STATE_FILE.write_text(json.dumps(sorted(seen)), encoding="utf-8")
print(f"Response: {len(current)} unique videos; new: {len(new_items)}")
The script writes missing JSON values as empty cells: fields and metrics may not be present in an individual record. The code does not replace a missing value with zero or treat zero as missing data. Collection time is recorded separately from publication time, which the API returns in the create_time field.
How change detection works
- The request returns a page of search results for the specified keyword.
- The normalizer collects records from
aweme_listandsearch_item_list[].aweme_info. - The script removes duplicates within the response using the video ID.
- IDs are compared with the local JSON log.
- New IDs are added to the CSV, then the full set of IDs seen is saved for the next run.


Visualizing this process is useful when handing the script to colleagues: the CSV is an incremental log, while the JSON file stores deduplication state. They can be stored separately from the code and backed up together. For a team or cloud-based run, replace the local JSON file with suitable persistent storage and use atomic state updates so concurrent runs do not overwrite one another.
Repeat runs and pagination
Schedule repeat runs using an operating system scheduler, a CI task, or an automation workflow. Do not change the keyword without recording its version: different queries produce different samples, and mixing them in a single log makes interpretation harder. The script shown here uses one query for simplicity; you can process a list of queries in a loop and save the query field for each row.
The API supports cursor for paging through search results. If you add pagination, pass the cursor from the previous response, store it as an opaque value, and stop paging when no further pages are needed. The count parameter sets the number of items to request, but the actual number of records should be determined from the response. Even multiple pages are still pages of search results, not a guarantee of full platform coverage.
This script deliberately does not automatically classify text or declare a “trend.” The CSV helps quickly identify newly found records; an analyst can then label them by topic, intent, or question type and review examples manually. If you connect a language model, pass it only the text fields it needs and ask it to return categories alongside the original ID so the result can be checked and linked to the record.
Checks and common errors
- HTTP error.
raise_for_status()stops execution for an HTTP status code outside the successful range. Check API availability, parameters, and account access before updating the state file. - API status is not
ok. The script stops before writing results if the response includes a top-levelstatusfield with another value. This preserves the previous log and helps avoid mistaking an erroneous response for an empty result set. - Empty list in
aweme_list. Checksearch_item_listtoo; the code handles both structures. - Videos repeat across runs. This is expected when search result pages overlap: videos are not added to the report again if their IDs are already saved.
- A metric is missing. Its cell remains empty. In calculations, distinguish missing values from actual zeros.
- State was lost. The script will treat all returned IDs as new. Back up
seen_video_ids.jsonalong with the CSV, or use a persistent database.
For automation, record diagnostic details without the key: run time, HTTP status code, API status, number of parsed records, and number of new IDs. This helps distinguish a lack of new videos from a failed request. You can archive the full raw response separately if you need to recheck the schema later; restrict access and retention time according to the team’s needs.
Result
After running the script, you have a CSV of videos with IDs new to the log, descriptions, links, and available public fields, along with JSON state for comparison during the next run. This result works as a compact intake queue for support, analytics, or a content team. You can review the API description and access options on the TikTok Scraper API page; check exact search parameters against the official endpoint reference.