How to Track Competitors’ IG Content Over Time: An API Pipeline in Python

A marketing team can see competitors’ posts in its feed, but manual browsing is a poor way to answer a specific question: which formats and topics regularly attract reactions, and which only stood out once? The solution is to save data on public posts daily and compare them over time by format and available counters.

Below is a basic pipeline in Python using the IG Scraper API. It retrieves posts from a selected account, saves responses as JSONL, and creates a summary of new content. Reaction metrics in this report reflect the values returned by the API at collection time; snapshots from different dates let you analyze changes separately.

First, decide what to compare

For a small competitor sample, it is enough to keep a list of public accounts and collect their posts on a regular schedule. Start with questions the data can answer: how often content is published, which posts are Reels or other formats, which topics recur, and how available like and comment counts change.

Keep two concepts separate in storage: a post and its snapshot. Identify a post with a stable ID, if one is present in the response; save each new run as a separate record with its collection time. This lets you see how available counts change instead of overwriting yesterday’s value with today’s.

Use the documented endpoint

For a basic request, jsonscraper’s documentation specifies the address https://ig.jsonscraper.com and the route GET /v1/user/by/username. The API map also lists GET /v1/user/medias/chunk for retrieving posts. Check the exact parameters and response fields in the product’s associated Postman collection: do not tie your handler to an assumed structure before examining the actual response for your request.

For an initial check, use the documented profile-request example; pass the key through an environment variable instead of writing it into your source code:

curl -G "https://ig.jsonscraper.com/v1/user/by/username" \
  --data-urlencode "username=apple" \
  --data-urlencode "license_key=$JSONSCRAPER_LICENSE_KEY" \
  --data-urlencode "cache_timeout=60"

The example on the product page shows username, license_key, and cache_timeout for the profile route. Use a parameter on other routes only if it is specified for them in the documentation. For a posts request, get the current parameter names from that route’s specification.

Save responses as timestamped snapshots

The following script is a starting point for a production run: the URL and parameters for the posts route are left in the configuration so they can be filled in strictly according to the Postman documentation. The handler saves the original JSON and collection time; add field normalization after checking the actual response.

import json
import os
from datetime import datetime, timezone
from pathlib import Path

import requests

BASE_URL = "https://ig.jsonscraper.com"
USERNAME = os.environ["IG_USERNAME"]
LICENSE_KEY = os.environ["JSONSCRAPER_LICENSE_KEY"]
OUTPUT = Path("instagram_snapshots.jsonl")

# Fill in the endpoint and parameters according to the documentation for
# GET /v1/user/medias/chunk in the Postman collection.
MEDIA_PATH = os.environ["IG_MEDIA_PATH"]
MEDIA_PARAMS = json.loads(os.environ["IG_MEDIA_PARAMS_JSON"])

params = {
    **MEDIA_PARAMS,
    "username": USERNAME,
    "license_key": LICENSE_KEY,
}

response = requests.get(
    f"{BASE_URL}{MEDIA_PATH}", params=params, timeout=30
)
response.raise_for_status()
payload = response.json()

record = {
    "collected_at": datetime.now(timezone.utc).isoformat(),
    "username": USERNAME,
    "response": payload,
}
with OUTPUT.open("a", encoding="utf-8") as file:
    file.write(json.dumps(record, ensure_ascii=False) + "\n")

There are deliberately no invented pagination parameters or field names here: copy them from the specific specification and verify them against an API response. Requests with a timeout and raise_for_status() help detect a network error before the result is recorded as ordinary data. For automated runs, add JSON validation and error logging.

Add a schedule and metrics

Run the collection once a day using cron or your automation scheduler. For multiple accounts, iterate over a configuration list, save a separate snapshot for each, and allow for a retry after temporary errors. Store the key in an environment variable or secrets store; the API page also recommends not embedding it in client code.

After checking the response, normalize records into a table with fields such as username, media_id, collected_at, content type, and available counters. Do not delete the original JSON: it will be useful if the response format changes or you need to verify a calculation.

In the analytics layer, calculate the number of new posts per period, the median available reactions by format, and the change in a post’s counter between snapshots. The median reduces the influence of a single viral post on the summary. Compare accounts across identical time windows and show the sample size alongside each aggregate.

What you’ll get

Your team will have a reproducible table of posts and a daily summary: publishing frequency, distribution by format, and trends in available reactions. This information helps generate hypotheses for your own editorial calendar, rather than simply collecting feed screenshots.

The next step is to send normalized records to Google Sheets, a database, or a visualization system, and generate a written summary as a separate stage. If you use AI, provide aggregated data and ask it to describe observed patterns with the period and number of posts specified; leave strategy decisions to the editor.

Start with one account and one route: check the parameters and response structure in the IG Scraper API description, then lock that configuration into your schedule. This approach turns competitor monitoring from a one-off manual review into a clear pipeline with a saved history.