Before you spend bandwidth on video bytes, you want a table of every candidate video with its duration, license, language and caption tracks. yt-dlp can build that table without downloading any media, but the flags overlap in confusing ways: -j vs -J, flat vs full extraction, --print vs --write-info-json. This guide maps each yt-dlp metadata option to the job it does, lists the info.json fields worth keeping, and shows how to store them as JSONL and Parquet and filter before downloading. Flags were checked against the yt-dlp README and source at release 2026.10.26.
TL;DR
- Two passes. First enumerate channels and playlists with
--flat-playlist, which makes no per-video requests and returns thin records. Then run full extraction per video with-jor--skip-download --write-info-json. - Use
-j, not-J, at scale.-jprints one JSON line per video.-Jprints a whole playlist as a single line. - Use
-O/--printwith%(.{field,field})jto emit only the fields you need as compact JSON. - Pace extraction with
--sleep-requests.--sleep-intervalsleeps before each download, and a metadata-only run has none. - Filter on metadata (
duration,license,language,live_status) in DuckDB or with--match-filters, then download only the IDs that pass.
The yt-dlp metadata flags at a glance
| Flag | What it does (per the README) | Use it for |
|---|---|---|
-j, --dump-json |
"Quiet, but print JSON information for each video. Simulate unless --no-simulate is used." |
Streaming one record per video into a JSONL file |
-J, --dump-single-json |
One JSON per URL. For a playlist, "the whole playlist information is dumped in a single line". | Small playlists, or when you need the playlist object itself |
--flat-playlist |
"Do not extract a playlist's URL result entries; some entry metadata may be missing" | Cheap enumeration of channels and playlists |
--skip-download |
"Do not download the video but write all related files" | Writing .info.json, subtitles and thumbnails without media |
--write-info-json |
Writes video metadata to a .info.json file |
One file per video, kept as raw provenance |
-O, --print [WHEN:]TEMPLATE |
Prints a field or output template. Implies --quiet and --simulate. |
Slim, custom records |
--print-to-file [WHEN:]TEMPLATE FILE |
Appends the template to a file | Writing a side file (e.g. a URL list) during another run |
--ignore-no-formats-error |
Ignores "No video formats" errors. The README calls it useful "for extracting metadata even if the videos are not actually available for download" (experimental). | Keeping metadata for videos whose formats fail |
Because -j and -O imply simulate mode, nothing is written to disk except what you redirect. --skip-download is different: it still writes the side files you ask for (.info.json, subtitles, thumbnails), just not the media.
Step 1: Enumerate channels and playlists with --flat-playlist
A flat run reads the channel or playlist pages and returns one stub per entry. It doesn't open any watch pages. In the YouTube tab extractor (extractor/youtube/_tab.py), each stub is a _type: "url" result with id, url, title and, when the page shows them, duration, view_count, channel_id, live_status and availability. There is no license, language, tags or category at this stage.
# one compact JSON object per upload on the channel's Videos tab
yt-dlp --flat-playlist --lazy-playlist \
-O "%(.{id,url,title,duration,view_count,live_status,availability,channel_id})j" \
"https://www.youtube.com/@CHANNEL/videos" > seeds/CHANNEL_videos.jsonl
# a playlist
yt-dlp --flat-playlist -j "https://www.youtube.com/playlist?list=PLAYLIST_ID" > seeds/PLAYLIST_ID.jsonl
Practical notes:
- Pick the tab. A bare channel URL returns nested playlists, one per tab. The extractor's own test expects "three playlists for videos, shorts and streams tabs". Point at
/videos,/shortsor/streamsexplicitly so you know what you're counting. --lazy-playlistprocesses entries as they arrive, so output starts streaming before the whole channel is paged. It disablesn_entries,--playlist-randomand--playlist-reverse.- Sampling.
-I 1:500(--playlist-items) limits the run to the first 500 entries, which helps when you're sizing a channel before committing. - Dates. Flat entries have no
upload_dateby default.--extractor-args "youtubetab:approximate_date"parses the "3 weeks ago" text into a timestamp. The README warns it "may cause date-based filters to be slightly off", so use it for ordering and coarse cuts only.
The %(.{a,b,c})j syntax comes from the README's output template section: an empty field name refers to the whole info dict, {} selects keys, and the j conversion emits JSON. It gives you a slim JSONL without a jq step.
Step 2: Full metadata per video, without media
Full extraction opens each watch page and player response. That's where license, categories, tags, chapters, captions and formats come from. You have two output styles.
Option A: stream JSONL with -j
yt-dlp -a urls.txt -j \
--ignore-no-formats-error \
--sleep-requests 1 \
--retry-sleep extractor:exp=1:120 \
>> meta/full.jsonl 2>> meta/errors.log
One line per video, appended. The default --no-abort-on-error keeps going after unavailable videos, and the errors land in the log for a retry pass. --retry-sleep accepts a type prefix (http, fragment, file_access, extractor) and linear=/exp= expressions.
Option B: one .info.json per video
yt-dlp -a urls.txt --skip-download --write-info-json \
--no-write-playlist-metafiles \
-o "meta/%(id).2s/%(id)s.%(ext)s" \
--sleep-requests 1 \
--download-archive meta_archive.txt --force-write-archive
%(id).2sshards files into subdirectories by the first two characters of the ID. Nobody wants a million files in one directory.--clean-info-jsonis the default and strips internal fields such as filenames.--no-clean-info-jsonkeeps everything.--no-write-playlist-metafilesskips the extra playlist-level.info.jsonwhen your input is a playlist URL.- Archive gotcha: in simulate or skip-download mode, archive entries are only written with
--force-write-archive. Also, yt-dlp records flat entries in the archive too when that flag is on (seeprocess_ie_resultinYoutubeDL.py). Keep a separate archive file per stage, or your enumeration run will mark every ID as already done.
Option A is easier to ingest. Option B gives you an immutable per-video record you can hash and keep as lineage. Many teams do B and derive a slim JSONL from it.
Which info.json fields matter
Field names below come from the README's output template list and the info dict spec in extractor/common.py. Example values come from the YouTube extractor's tests. Any field can be missing, and None means "unknown".
| Field | Type / example | Notes |
|---|---|---|
id, title, description |
string | The 11-character video ID is your primary key. |
duration |
seconds (int or float) | Also available as duration_string (HH:mm:ss). |
upload_date, timestamp |
"20080529", Unix time |
upload_date is UTC, YYYYMMDD. |
channel, channel_id, uploader_id |
"UCSMOQeBJ2RAnuFungnQOxLg", "@BlenderOfficial" |
Use channel_id for joins. Handles can change. |
license |
"Creative Commons Attribution license (reuse allowed)" or "Standard YouTube License" |
Treat a missing value as unknown, not as CC. |
categories |
list, e.g. ["Film & Animation"] |
One YouTube category per video, wrapped in a list. |
tags |
list of strings | Uploader-supplied and noisy, but useful for topic balancing. |
language |
"en" |
Not in the generic spec. For YouTube it is a format-level field (the audio track's language, or inferred from the original-language auto-caption track) that ends up on the info dict from the selected format. Often absent. |
view_count, like_count, comment_count |
int | A snapshot, so store epoch (extraction time) next to it. |
live_status |
not_live, is_live, is_upcoming, was_live, post_live |
Drop is_live/is_upcoming. Decide whether was_live streams fit your data. |
availability, age_limit |
"public", 0 |
Skip anything needing sign-in. |
chapters |
list of {start_time, end_time, title} |
Free, creator-made segmentation, useful for clip boundaries. |
heatmap |
list of {start_time, end_time, value} |
Normalized "most replayed" curve, values 0–1. |
subtitles, automatic_captions |
{lang: [{ext, url, name}]} |
Manual vs ASR tracks. Keep the key lists, drop the URLs. |
formats, thumbnails |
lists of dicts | Most of the file's size. Stream URLs are short-lived, so store the format properties you filter on (height, fps, vcodec), not the URLs. |
The caption keys are the cheapest language signal you have. A video with manual en subtitles, or an en-orig key in automatic_captions, very likely has English speech. The details of manual, ASR and translated tracks are in our yt-dlp subtitles guide.
Storing metadata as JSONL and Parquet
Raw -j lines are wide and nested, and the formats list varies from video to video. That makes schema inference messy. A pattern that works: keep the raw JSON compressed as provenance, and project a flat row for analytics.
import gzip
import json
import sys
from yt_dlp.version import __version__ as YTDLP_VERSION
KEEP = ["id", "title", "duration", "upload_date", "timestamp", "channel_id",
"channel", "license", "language", "categories", "tags", "view_count",
"like_count", "comment_count", "age_limit", "availability",
"live_status", "webpage_url", "epoch"]
def project(info):
row = {k: info.get(k) for k in KEEP}
row["chapters"] = [
{"start": c.get("start_time"), "end": c.get("end_time"), "title": c.get("title")}
for c in info.get("chapters") or []
]
row["manual_sub_langs"] = sorted(info.get("subtitles") or {})
row["asr_orig_langs"] = sorted(
k for k in (info.get("automatic_captions") or {}) if k.endswith("-orig"))
row["max_height"] = max(
(f.get("height") or 0 for f in info.get("formats") or []), default=0)
row["ytdlp_version"] = YTDLP_VERSION
return row
# usage: python project.py meta/full.jsonl meta/raw.jsonl.gz > meta/slim.jsonl
with open(sys.argv[1], encoding="utf-8") as src, \
gzip.open(sys.argv[2], "at", encoding="utf-8") as raw:
for line in src:
info = json.loads(line)
raw.write(line)
print(json.dumps(project(info), ensure_ascii=False))
Then convert to Parquet with DuckDB's JSON reader and query it directly:
duckdb -c "COPY (SELECT * FROM read_json('meta/slim.jsonl', format = 'newline_delimited'))
TO 'meta/slim.parquet' (FORMAT parquet)"
Record epoch and ytdlp_version on every row. Counts drift, and extractor behavior changes between releases. When a field looks wrong six months later, you'll want to know which version produced it. For the wider pipeline (sharding, checksums, lineage), see building a YouTube video dataset for multimodal training.
Pre-filter before downloading
This is where metadata pays for itself. Two places to filter:
In SQL, on the Parquet table
COPY (
SELECT 'https://www.youtube.com/watch?v=' || id
FROM 'meta/slim.parquet'
WHERE duration BETWEEN 60 AND 3600
AND live_status = 'not_live'
AND availability = 'public'
AND license LIKE 'Creative Commons%'
AND (language LIKE 'en%' OR list_contains(asr_orig_langs, 'en-orig'))
AND max_height >= 720
) TO 'download_urls.txt' (FORMAT csv, HEADER false);
SQL lets you rerun with different thresholds without touching YouTube again, check how many hours each filter removes, and balance categories before you commit bandwidth. To turn the surviving durations into a bandwidth estimate, see bandwidth for large-scale video downloads.
Inline, with --match-filters
yt-dlp -a urls.txt -j \
--match-filters "duration>=60 & duration<=3600 & !is_live & license*='Creative Commons' & language^=en" \
--sleep-requests 1 >> meta/filtered.jsonl
Rules from the README: any output-template field can be compared, !field checks that a field is not present, & joins conditions, and passing --match-filters twice ORs them. A comparison on a missing field fails unless you add ? (e.g. like_count>?100). For license that strict default is what you want. For language it can drop good videos that simply lack the field, which is one more reason to filter in SQL, where you can fall back to caption keys. More on license filtering in our Creative Commons guide. For speech-specific language rules, see the audio-only download guide.
Rate limiting and failure handling
--sleep-requests SECONDS: "sleep between requests during data extraction". This is the main throttle for metadata runs.--sleep-interval/--max-sleep-interval: a randomized sleep "before each download". With-jor--skip-downloadthere is no media download, so these do little here. Use them in the download stage.-t sleepexpands to--sleep-subtitles 5 --sleep-requests 0.75 --sleep-interval 10 --max-sleep-interval 20. That's a reasonable starting point when you also fetch captions.- Parallelism. Throughput comes from running several workers on disjoint ID shards, each with its own sleep settings and egress. A single process with a short sleep tends to get you blocked. If you see "Sign in to confirm you're not a bot", read our fix guide. For per-worker egress, see the yt-dlp proxy guide.
- Dependencies. The README lists
yt-dlp-ejsplus a JavaScript runtime (deno recommended) as required for full YouTube support. A-jrun still resolves formats, so install them for metadata jobs too, and stay on a current release. - Resumability. Use
--download-archivewith--force-write-archive, or dedupe onidagainst your JSONL before each run.
Compliance
YouTube's Terms of Service restrict automated access and downloading, and that applies to metadata collection as well as media. Copyright and licensing determine what you can train on and redistribute. Filtering on license helps, but record it per sample. The README notes that info.json "may contain personal information", and titles, descriptions and channel names can be personal data under GDPR. So can faces and voices in any media you fetch later. Honor takedown and opt-out requests, and keep the lineage (ID, channel, extraction time) you need to act on them. Get legal review for your jurisdiction and use case.
If you'd rather not run this yourself
Corpora offers dedicated 1–40 Gbps unmetered proxy capacity (HTTP(S) and SOCKS5, compatible with yt-dlp) for teams running their own metadata and download workers. There's also a fully managed option that delivers YouTube metadata, video, audio and subtitles as JSONL/Parquet with sha256 checksums and manifests to S3, GCS or Azure Blob, starting with a free sample batch. Contact us to scope it.
FAQ
How do I get YouTube video metadata with yt-dlp without downloading the video?
Run yt-dlp -j URL. It prints the full info dict as JSON and simulates, so no media is downloaded. Use --skip-download --write-info-json instead if you want a .info.json file per video.
What's the difference between -j and -J?
-j (--dump-json) prints one JSON object per video. -J (--dump-single-json) prints one object per input URL, so a playlist becomes a single line containing every entry. For large channels, -j streams better and fails per video instead of per playlist.
How do I list all video IDs on a channel quickly?
yt-dlp --flat-playlist -O "%(id)s" "https://www.youtube.com/@CHANNEL/videos". Flat mode doesn't open each video, so it's fast, but entries lack license, tags and exact dates.
Why is language missing from my yt-dlp JSON?
For YouTube, language comes from the selected format's audio track metadata or from the original-language auto-caption track. Many videos expose neither. Fall back to the subtitles and automatic_captions keys, and confirm with audio language ID after download.
Does --sleep-interval slow down metadata extraction?
Not meaningfully. It applies before each media download. For -j and --skip-download runs, use --sleep-requests.