Michael JohnsonComparing two IG creators using a single profile snapshot is easy; understanding who publishes consistently and which posts gain engagement over time is harder. One-off checks mix posts of different ages, and a manual spreadsheet quickly becomes outdated. A practical solution is to collect public media on a schedule, store each response as a separate observation, and calculate changes using your own data.
Below is a working approach for a marketer, analyst, or developer who needs to compare public accounts in the same niche each week. It uses jsonscraper’s IG Scraper API to retrieve data, while Python handles scheduling, normalization, and historical storage. This builds on the topic of monitoring profiles and competing posts, focusing here on a consistent observation model suitable for comparative reporting.
First, decide what to compare
The task is often framed broadly: “find effective creators.” For technically useful monitoring, narrow it down to measurable questions: how often posts are published, which formats appear, how many public likes and comments are visible at the time of a check, and how those numbers change after publication. This set of measures can help shortlist accounts for manual review or prepare a comparative summary.
Public counters are snapshots taken when a request is made. So the key architectural decision is not a complex model, but storing the collection time alongside each value. If you retrieve information about the same media every night and overwrite the previous row, you lose the history needed to calculate growth. Instead, store observations immutably: one row per post and collection time.
Limit comparisons to equivalent periods and comparable posts. For example, you can look at posts first observed in the same calendar month or account for a post’s age at each snapshot. Otherwise, yesterday’s video will appear weaker than an older post simply because it has had less time to accumulate engagement.
The pipeline: request, normalize, retain history
A minimal process has four steps: retrieve a list of public media from selected profiles, map the response to a common schema, write rows to storage, and generate a scheduled report. The IG Scraper API page shows the base URL and an endpoint map, including a user route for retrieving posts in chunks. Check the parameters for each route and sample responses in the Postman collection before integration.
The API base URL is https://ig.jsonscraper.com. For an initial check, the documentation shows a profile request by username: GET /v1/user/by/username with the parameters username, license_key, and cache_timeout in the published example. The route listed for retrieving posts is GET /v1/user/medias/chunk. Get its exact arguments, continuation fields, and response structure from the documentation for that specific request; do not transfer a schema from one route to another.
import os
import requests
BASE = "https://ig.jsonscraper.com"
KEY = os.environ["JSONSCRAPER_LICENSE_KEY"]
def get_profile(username: str) -> dict:
response = requests.get(
f"{BASE}/v1/user/by/username",
params={
"username": username,
"license_key": KEY,
"cache_timeout": 60,
},
timeout=30,
)
response.raise_for_status()
return response.json()
print(get_profile("example_account"))
This short example follows the documented profile route and shows how to pass parameters with HTTP GET. The published example includes cache_timeout=60; use it only on routes that support the parameter, and check the details in the documentation. Store the key in an environment variable, not in code or a browser application.
Connect post retrieval
After checking the profile, add a request for the media list. The API page lists GET /v1/user/medias/chunk; use the chunking logic specified in the documentation to process the result. Get parameter names and the pagination stopping condition from the current description of this request in Postman rather than guessing from the route name.
Below is a transport-call skeleton, not a complete endpoint contract: fill in params with documented arguments before running it. After receiving a response, the adapter should read only fields confirmed by the actual JSON for the selected route.
def fetch_media_chunk(username: str, params: dict) -> dict:
query = {
**params,
"username": username,
"license_key": KEY,
}
response = requests.get(
f"{BASE}/v1/user/medias/chunk",
params=query,
timeout=30,
)
response.raise_for_status()
return response.json()
Next, write an adapter function that maps the response to an internal format. Useful fields for analysis may include a stable post ID, author, link, publication date, media type, and available public counters. Check the exact field names and types in the documentation and sample JSON. If a metric is missing, store it as an empty value rather than substituting zero: zero means a measured count of zero, while an empty value means no data is available.
def normalize_media(item: dict, username: str, observed_at: str) -> dict:
# Specify keys only after checking the JSON for the selected endpoint.
return {
"username": username,
"media_id": item.get("DOCUMENTED_MEDIA_ID_KEY"),
"media_url": item.get("DOCUMENTED_MEDIA_URL_KEY"),
"published_at": item.get("DOCUMENTED_PUBLISHED_AT_KEY"),
"media_type": item.get("DOCUMENTED_MEDIA_TYPE_KEY"),
"likes": item.get("DOCUMENTED_LIKE_COUNT_KEY"),
"comments": item.get("DOCUMENTED_COMMENT_COUNT_KEY"),
"observed_at": observed_at,
}
Replace the placeholder names with the exact keys from the selected endpoint’s response, and pin the mapping with a test based on saved sample JSON. That way, schema changes become visible when the integration is updated, and the example does not present assumed field names as a confirmed contract.
Store observations, not just current values
CSV works for a prototype; for regular use, choose PostgreSQL, Supabase, or an analytics data warehouse. An observations table might contain username, media_id, published_at, observed_at, likes, comments, and the raw JSON. The pair media_id and observed_at identifies a unique snapshot. A separate profile table stores the list of tracked accounts and the date monitoring began.
Saving the raw response is useful for debugging and rebuilding normalized data. Set a retention period that matches the project’s purpose and organizational rules, control access to the tables, and do not collect information that is unnecessary for the chosen analysis. For CRM and direct marketing activities, separately establish the applicable legal basis: the technical availability of public data does not resolve that question.
CREATE TABLE media_observations (
username TEXT NOT NULL,
media_id TEXT NOT NULL,
media_url TEXT,
published_at TIMESTAMPTZ,
observed_at TIMESTAMPTZ NOT NULL,
media_type TEXT,
likes BIGINT,
comments BIGINT,
raw_json JSONB,
PRIMARY KEY (media_id, observed_at)
);
Insert records in batches after successfully validating the response. On a rerun of the same job, the primary-key combination prevents an accidental duplicate snapshot; the application can use an insert that skips conflicts. A new run at a different time will save a separate row and extend the time series.
Calculate metrics that answer the question
A basic publishing-cadence metric is the number of unique media items posted by an account during a chosen interval. It is easy to calculate from published_at, if that field is present and correctly interpreted. To assess changes in public engagement, compare two observations of the same media_id: the absolute change in likes is the later count minus the earlier count; the relative change is that difference divided by the earlier value. Apply the same calculation to comments.
If the earlier value is zero or missing, relative growth is undefined; show the absolute difference and clearly indicate that there is no baseline for a percentage. Counters can change between observations, so the report should state the interval between snapshots and the time of the most recent collection. These details help readers interpret the numbers correctly.
def growth(previous, current):
if previous is None or current is None:
return {"delta": None, "rate": None}
delta = current - previous
rate = None if previous == 0 else delta / previous
return {"delta": delta, "rate": rate}
Do not reduce accounts to a single ranking without explainable weights. A practical report typically shows the number of posts, median likes and comments per post, distribution of formats, and changes in engagement over the same post-age interval. The median reduces the influence of an outlier post; the number of posts shows the sample size behind the comparison.
There are two ways to account for a post’s age. The first is to capture counters regularly, for example at consistent intervals after the media is first detected. The second is to compare posts that fall within the same time range, using observations from roughly the same post age. If a scheduled run is missed, do not label the actual trend as an exact one-day increase: calculate the interval using the timestamps.
Automate the run and quality checks
Schedule collection with cron, GitHub Actions, or an ETL orchestrator. The job sequence is straightforward: read the account list, request the required chunks, validate the HTTP status and response structure, normalize the data, save the snapshot, and log the run. For each run, record the number of accounts and media items processed, errors, and duration. This helps distinguish a real change in activity from a collection failure.
For transient errors, use a limited number of retries with a pause, and save the context for failed responses without the secret key. Do not run an infinite cursor loop: stop according to the completion conditions specified in the endpoint response. If the documentation sets usage limits or rules, configure the frequency and sample size accordingly.
The cache_timeout parameter is listed on the IG API page for supported routes. Caching is useful when a repeat request in a short window adds no value; choose a collection interval that fits the analytical question. For example, a daily snapshot is more useful for a weekly review than frequent repeat requests to the same profiles without a specific purpose.
Test a small sample before enabling a regular schedule. Save a sample response, check IDs and dates, compare normalized fields with the source JSON, and manually review several rows. Add tests for missing values, empty lists, and unexpected formats. When the API changes, these checks will catch a problem faster than an inconsistent report at the end of the week.
Where AI can help
AI is useful at the final stage, to briefly describe results that have already been calculated. Provide the model with aggregates and links to posts, not the entire raw archive. For example, share the number of media items per week, median public engagement, and a list of posts with the greatest change over a specified interval. Ask the model to summarize the observations and include the record IDs they came from.
Keep metric calculations in regular code so the same rules produce reproducible results. A model can draft a written summary, group captions into predefined themes, or suggest questions for manual review. For thematic classification, store the model’s label separately from the original caption and record the prompt or classifier version, so later changes do not silently rewrite the history.
Do not turn an automated report into an automated decision about hiring, contracting, or evaluating a particular person. Use public data for a limited analysis of content and review conclusions manually, especially when they affect business decisions. This keeps monitoring useful and separates measured counters from interpretation.
What the result looks like
After several collection cycles, you will have a table showing when each post was detected, how many reactions it had at each observation, and how its metrics changed. A weekly summary can answer practical questions: who published consistently, which formats were used most often, which posts gained engagement between comparable snapshots, and where there is not yet enough data for a confident comparison.
This process helps replace occasional manual checks with repeatable research. It does not replace qualitative content analysis: metrics describe publishing frequency and available public engagement, while meaning, audience, and brand fit require separate assessment. But a consistent schedule, retained history, and explicit calculation logic make those assessments verifiable and useful to a team.
To get started, visit the IG Scraper API page, check the parameters for the post route in the Postman documentation, then implement an adapter with a test account and save your first snapshot. Once the schema is confirmed, add scheduling, error monitoring, and a report based on your chosen metrics.