Michael JohnsonAfter launching a campaign, a team may want to check which public videos appear in search results for selected terms. Manual searches are difficult to repeat and compare a few days later. A small collector can run the same queries on a schedule and archive responses with the date and search phrase, so an analyst can review the records and prepare their own summary.
This guide presents a Python archiving pipeline for saving search results for fixed phrases. Unlike monitoring individual hashtags or accounts, the key to comparison here is to repeat the same search phrases and save a separate snapshot of each response. The jsonscraper TikTok Scraper API page lists the base URL and the GET /searchVideo endpoint; its documentation says the search phrase is passed in the required keyword parameter.
Task and measurement boundaries
Suppose a marketer wants to collect results every day for several fixed phrases: a product name, a category, and a customer problem. For each query, the system saves the collection time and API response. After inspecting a sample response, you can add normalization for fields that are actually present in the JSON, such as a video ID or public counters, if available.
The archive answers a specific question: which records did the search return for the saved phrase during this run? If the response contains metrics, repeated observations may show differences between values recorded at different times. Such a difference pertains to the observed snapshots and the interval between them; for comparison, save both collection times and the same field.
Save the query alongside the response. The result set is tied to a specific search phrase and collection time. The archive is therefore useful for tracking results for selected queries, not as a complete overview of all TikTok content. IDs, original responses, and timestamps help verify where each row in a later table came from.
Check the endpoint and its parameters
The TikTok API endpoint map lists the base URL https://tiktok.evelode.com and GET /searchVideo. The current API specification identifies keyword as a required search parameter; optional parameters include cursor, count, cache_timeout, date_posted, sort_by, and region. Authorization uses license_key. Details about the endpoint are also available in the Python SDK documentation.
The count and date_posted parameters help set the response size and time filter; sort_by accepts the documented values most-liked, relevance, or date-posted. If you use pagination, pass the cursor received in the previous response as an opaque value rather than calculating it yourself. For a comparable daily archive, save the request parameters and your own code version alongside the data.
The example below uses a small count for a test run and a fixed US region. Before daily collection, test one query, its HTTP status, and the JSON structure. The response schema may contain matching videos in tiktok.aweme_list or in tiktok.search_item_list, where the video object is in aweme_info; the code supports both observed forms.
Set up the project
Create a virtual environment and install requests:
python -m venv .venv
source .venv/bin/activate
pip install requestsIn PowerShell, activate the environment with .venv\Scripts\Activate.ps1. Set the key and list of search phrases through environment variables. For the TikTok API, use the documented environment variable name JSONSCRAPER_TIKTOK_LICENSE_KEY:
export JSONSCRAPER_TIKTOK_LICENSE_KEY="your_key"
export TIKTOK_QUERIES="product name,product category,customer problem"Collect and save snapshots
The example sends one request for each phrase to the documented endpoint, then saves the original JSON, collection time, and parameters used. The filename includes the date and search phrase. The key is stored in the environment and is not included in the archive.
import json
import os
import re
import time
from datetime import datetime, timezone
from pathlib import Path
import requests
BASE_URL = "https://tiktok.evelode.com"
ENDPOINT = "/searchVideo"
LICENSE_KEY = os.environ["JSONSCRAPER_TIKTOK_LICENSE_KEY"]
QUERIES = [
item.strip()
for item in os.environ["TIKTOK_QUERIES"].split(",")
if item.strip()
]
OUT = Path("data")
OUT.mkdir(exist_ok=True)
def safe_name(text):
cleaned = re.sub(r"[^a-zA-Z0-9_-]+", "_", text).strip("_")
return cleaned[:80] or "query"
def fetch(query):
params = {
"license_key": LICENSE_KEY,
"keyword": query,
"count": 10,
"region": "US",
}
response = requests.get(
BASE_URL + ENDPOINT,
params=params,
timeout=60,
)
response.raise_for_status()
payload = response.json()
if payload.get("status") != "ok":
raise RuntimeError("API returned a status other than ok")
return payload
def extract_videos(payload):
tiktok = payload.get("tiktok") or {}
videos = list(tiktok.get("aweme_list") or [])
for item in tiktok.get("search_item_list") or []:
video = item.get("aweme_info") or {}
if video:
videos.append(video)
return videos
for query in QUERIES:
collected_at = datetime.now(timezone.utc).isoformat()
payload = fetch(query)
record = {
"collected_at": collected_at,
"query": query,
"request": {
"endpoint": ENDPOINT,
"keyword": query,
"count": 10,
"region": "US",
},
"videos_found": len(extract_videos(payload)),
"payload": payload,
}
filename = OUT / f"{collected_at[:10]}_{safe_name(query)}.json"
filename.write_text(
json.dumps(record, ensure_ascii=False, indent=2),
encoding="utf-8",
)
print(f"Saved response for query: {query}")
time.sleep(1)
print(f"Responses saved: {len(QUERIES)}")This minimal collector makes one request per phrase with count=10. For broader collection, add pagination using cursor: continue making requests with the cursor value from the response until the API indicates that there are no more results, and record each response separately. The cursor and result count define a particular pass; the saved JSON lets you later check exactly what each request returned.
The example checks the outer status, then reads the tiktok object. It collects items from aweme_list and also from search_item_list[].aweme_info, if that form is present. Possible video fields include an ID, description, author, link, and statistics; individual fields may be absent. Therefore, save the full response and check that each field exists when normalizing.
The one-second interval is a pause between requests made by the script, not a claim about the service’s limits. Set the frequency and retries with regard to access terms and documentation. The cache_timeout parameter is documented for search; it lets you set the cache duration in seconds, and according to the specification, a value of 0 disables caching. Add it to the request when that behavior suits your use case.
Add normalization based on the actual response
After the first run, open the saved file and inspect the structure of payload. The extract_videos function in the example collects items from two response locations documented for search forms. Find the exact field keys needed for your analysis. If a field is missing from a particular record, store it as a missing value rather than substituting an assumed number.
For further processing, it is useful to create one row for each object found, adding collected_at and query. You can save confirmed fields to a CSV or database and keep the original JSON in the archive. This combination supports convenient tables while preserving a way to verify the origin of each record.
If the API returns a video ID, use it to link observations of the same video across different runs. Store each observation separately rather than overwriting the past. If there is no stable ID, first check which attributes can be used to match records, and do not assume that matching description text guarantees a match.
Compare snapshots and prepare a report
After normalization, group records by a verified ID and sort them by collection time. For counter fields, you can calculate the difference between consecutive values with ordinary code. For example, a change from 12,400 to 18,900 means the counter value changed by 6,500 between those snapshots. Include the observation dates and original field in the report so readers can verify the calculation.
A useful final table includes the search phrase, post ID, collection time, and selected fields from the response. You can group the summary by query and sort it by a confirmed counter. This gives the team a verifiable list of returned records to review; implement the table or message as a separate step after inspecting the actual schema.
Start by storing JSON files locally. Once collection becomes regular, move normalized rows to PostgreSQL or an analytics warehouse, and store the original responses in object storage with the date and query in the filename. Keeping the original is especially useful when refining the parser: you can rerun it on historical data without querying the API again.
Schedule collection
For a daily archive, run the script with cron or another scheduler available in your environment. For example, a job could run the Python file once a day, passing the key and search phrases through environment variables. Test one query first, then the full list, and only then enable the schedule for regular use.
For a reliable run, save the status of each request, limit the number of retries for temporary errors, and flag an incomplete collection if one phrase fails. Do not include the key in error messages. It is also useful to separate files by date and query name so a rerun does not overwrite the previous snapshot.
Choose the frequency according to your task and query volume. One run per day is enough for a daily overview; more frequent collection makes sense when the team cares about changes during the day. Start with a small list of phrases, check whether the returned results are useful, and then expand collection within the applicable access terms and data-use rules.
Use AI for thematic grouping
Once the response schema has been confirmed, selected text fields can be sent to a model to group descriptions into predefined themes or draft a brief summary. Keep the original post ID, search phrase, and text excerpt on which the category is based alongside the classification. This lets an analyst trace the model’s output back to the original record.
Calculate numeric differences, sorting, and deduplication with ordinary code. A model is suitable for processing text and drafting report wording, while counters and timestamps should come from saved responses. Separating these operations makes the origin of each finding clear.
Manually review a portion of automated categories and save corrections as test examples. If different types of posts regularly end up in the same group, refine the category definitions and review again. This process helps assess classification quality on data from your task.
Conclusion: a repeatable archive and verifiable summary
After running the documented request with the keyword parameter, the team has an archive showing which phrases were queried, when they were queried, and what data the API returned for each one. It can be used to build a results table, compare confirmed fields across snapshots, and prepare a list of found posts for manual analysis.
Start with one test query, then configure the phrase list and schedule. The TikTok Scraper API description provides the base URL and endpoint map; the API specification describes the /searchVideo parameters and response format. Archiving verified responses provides a foundation for repeatable analysis using fixed search phrases.