YouTube subtitles look like free ASR labels, but a naive yt-dlp run mixes three different things: captions uploaded by the creator, YouTube's own speech recognition, and machine translations of either. This guide shows which yt-dlp subtitle flags fetch which tracks, how to tell them apart in info.json, and how to turn rolling auto-caption VTT into clean segment-level JSONL. Everything was checked against the yt-dlp README and source at release 2026.10.26.
TL;DR
--write-subsfetches creator-uploaded (manual) tracks.--write-auto-subsfetches YouTube's auto-generated captions. If both are on and a language exists in both, the manual track wins.- For training data, request the original-language ASR track as
<lang>-orig(e.g.en-orig). A plainenauto-caption on a non-English video is a machine translation. --sub-langstakes comma-separated, case-insensitive regexes that must match the whole code, plusalland-exclusions such asall,-live_chat.- Captions-only runs:
--skip-download --write-info-json, throttled with--sleep-subtitles. If you use--download-archive, add--force-write-archiveor nothing gets recorded. - Auto-caption VTT repeats every line across cues. De-duplicate before you use it (a stdlib script is below), or download
json3/srv3if you need word timings.
Manual subtitles vs auto-generated captions
yt-dlp's YouTube extractor fills two separate dictionaries in the info dict:
subtitles: tracks uploaded or edited by the creator (caption tracks whosekindis notasr).automatic_captions: YouTube's ASR track, plus machine-translated versions of it and, when requested, of the manual tracks.
The CLI flags map one-to-one onto these: --write-subs reads subtitles and --write-auto-subs (alias --write-automatic-subs) reads automatic_captions. In YoutubeDL.process_subtitles, manual tracks are added first and an automatic caption is only added for a language code that isn't already present. Either way the file is written as <name>.<lang>.<ext>, so the filename alone doesn't tell you which kind you got. Keep the .info.json and check the code against its subtitles keys (example below). Note that the cleaned info.json drops requested_subtitles, so you can't read the selection back from the file.
Without --sub-langs, yt-dlp downloads exactly one track: en if present, otherwise the first en* code, otherwise the first available language.
See what exists first: --list-subs
yt-dlp --list-subs "https://www.youtube.com/watch?v=VIDEO_ID"
This prints two tables (automatic captions and subtitles) with every language code and the formats on offer, and it only simulates unless you add --no-simulate. The same tables are available as the output-template fields automatic_captions_table and subtitles_table. On a typical video the automatic list is long: that's the translations, covered next.
Selecting languages with --sub-langs
The README defines --sub-langs as a comma-separated list of language codes, regexes, or all, where a leading - excludes a code. Two details from the source matter in practice:
- Each pattern is compiled case-insensitively and must fully match the code.
endoes not matchen-US.en.*matchesen,en-US,en-GB,en-orig, and everyen-xxtranslation. - Items are processed left to right, so
all,-live_chatmeans "everything, then removelive_chat". Live-stream replays expose chat as a subtitle track namedlive_chat. You can also use--compat-options no-live-chat.
# every manual track except chat replay
yt-dlp --write-subs --sub-langs "all,-live_chat" --skip-download URL
# English and Japanese manual tracks, any region variant
yt-dlp --write-subs --sub-langs "en.*,ja" --skip-download URL
Use broad regexes only with --write-subs alone. Combined with --write-auto-subs, they pull in translations.
Auto-translated tracks: how to spot them and why to drop them
In the YouTube extractor (extractor/youtube/_video.py), each caption track is expanded against YouTube's list of translation languages. The codes you'll see in automatic_captions:
| Key | What it is | Use for training? |
|---|---|---|
en-orig (any <lang>-orig) |
The ASR track in the video's spoken language. yt-dlp adds the -orig label, with the name "… (Original)", so it can be told apart. |
Yes, as a weak label |
en when the audio is English |
The same original ASR track, kept under the plain code for compatibility. | Yes (duplicate of en-orig) |
de, fr, … when the audio is English |
Machine translation of the ASR track (URL carries tlang=). |
No |
de-en (target-source) |
Machine translation of the manual en track. Only built when --write-auto-subs or --list-subs is used. |
No |
Translated tracks are text produced by a translation model, time-aligned to someone else's speech. As ASR labels they're simply wrong for the audio, and as translation pairs they're synthetic. Rules that hold up:
- Request
<lang>-origexplicitly for ASR captions. The-origkey is added when the ASR track is translatable. If it is missing, fall back to the plain code only after confirming the URL has notlangparameter. - Treat any requested track whose URL contains
tlang=as a translation and discard it. --extractor-args "youtube:skip=translated_subs"stops yt-dlp from building translations of manual tracks (thexx-yykeys) and saves extraction time. In the current source it does not remove the plain-code translations of the ASR track, so the two rules above still apply.
Subtitle formats: --sub-format, --convert-subs, --embed-subs
For YouTube, the extractor currently offers every track in seven formats: json3, srv1, srv2, srv3, ttml, srt and vtt. --sub-format takes a /-separated preference list. best means the last format the extractor lists, which for YouTube is vtt. If nothing matches, yt-dlp warns and falls back to best.
vtt: the default and the easiest to parse. Auto-captions include per-word start times as inline tags.json3,srv3: YouTube's own timed-text formats, useful for word-level timing (below). YouTube doesn't document them.--convert-subs FORMATconverts toass,lrc,srtorvttthrough ffmpeg. TTML is converted to SRT first, with a warning that styling is lost. There's no converter forjson3orsrv*, so keep those raw and parse them yourself.--embed-subsmuxes tracks into mp4, webm or mkv output. Used alone, it deletes the sidecar files after embedding. Add--write-substo keep them. It does nothing on a--skip-downloadrun, and for datasets sidecar files are easier to process than embedded streams.
Captions-only runs at scale
Caption files are kilobytes, so a captions-plus-metadata pass is a cheap way to decide which videos are worth downloading audio for:
yt-dlp -a ids.txt --skip-download \
--write-subs --write-auto-subs --sub-langs "en,en-orig" --sub-format vtt \
--extractor-args "youtube:skip=translated_subs" \
--write-info-json \
--sleep-requests 1 --sleep-subtitles 2 \
--download-archive subs_archive.txt --force-write-archive \
-o "meta/%(id)s.%(ext)s" \
-o "subtitle:subs/%(id)s.%(ext)s"
- Output templates. The
subtitle:prefix sets a separate template for subtitle files, and yt-dlp appends the language and extension:subs/VIDEO_ID.en-orig.vtt.-P "subtitle:subs"sets only the directory. The.info.jsonlands inmeta/(there's aninfojson:prefix too). - Archive gotcha. With
--skip-download, yt-dlp only writes archive entries if--force-write-archiveis set. Without it, every re-run starts from zero. - Rate limits.
--sleep-subtitlessleeps before each subtitle download and--sleep-requestsbetween extraction requests.--sleep-intervalapplies to media downloads, not subtitle files. The-t sleeppreset expands to--sleep-subtitles 5 --sleep-requests 0.75 --sleep-interval 10 --max-sleep-interval 20. - Missing tracks. If YouTube requires a PO token for subtitle requests and none is available, the extractor skips subtitles for that client and warns "There are missing subtitles languages because a PO token was not provided." The README documents a
subscontext for thepo_tokenextractor argument. Current yt-dlp also needs a JavaScript runtime andyt-dlp-ejsfor full YouTube support. - Bot checks and IPs. At volume, subtitle and metadata requests hit the same anti-bot systems as video. See fixing "Sign in to confirm you're not a bot" and the yt-dlp proxy guide.
Python API equivalents
The CLI flags map to YoutubeDL params (the README points to devscripts/cli_to_api.py for translating any command line):
| CLI | Python param |
|---|---|
--write-subs |
writesubtitles=True |
--write-auto-subs |
writeautomaticsub=True |
--sub-langs "en,en-orig" |
subtitleslangs=["en", "en-orig"] |
--sub-format vtt |
subtitlesformat="vtt" |
--list-subs |
listsubtitles=True |
--skip-download |
skip_download=True |
--sleep-subtitles 2 |
sleep_interval_subtitles=2 |
--sleep-requests 1 |
sleep_interval_requests=1 |
--force-write-archive |
force_write_download_archive=True |
--convert-subs srt |
postprocessors=[{"key": "FFmpegSubtitlesConvertor", "format": "srt", "when": "before_dl"}] |
--embed-subs |
postprocessors=[{"key": "FFmpegEmbedSubtitle"}] |
This script fetches captions only and writes a manifest line per track labeled manual, auto or auto-translated, using the precedence and tlang rules above:
import json
import sys
import yt_dlp
LANG = "en"
OPTS = {
"skip_download": True, # --skip-download
"writesubtitles": True, # --write-subs
"writeautomaticsub": True, # --write-auto-subs
"subtitleslangs": [LANG, f"{LANG}-orig"], # --sub-langs "en,en-orig"
"subtitlesformat": "vtt", # --sub-format vtt
"writeinfojson": True, # --write-info-json
"outtmpl": {"default": "subs/%(id)s/%(id)s.%(ext)s"},
"extractor_args": {"youtube": {"skip": ["translated_subs"]}},
"sleep_interval_requests": 1, # --sleep-requests 1
"sleep_interval_subtitles": 2, # --sleep-subtitles 2
"ignoreerrors": True, # log failures, keep going
# "proxy": "http://USER:[email protected]:8000",
}
def track_kind(info, lang, sub):
if lang in (info.get("subtitles") or {}):
return "manual" # manual wins when both flags are set
if "tlang=" in (sub.get("url") or ""):
return "auto-translated" # machine translation: drop for training
return "auto"
def main(ids_path, manifest_path):
with open(ids_path, encoding="utf-8") as f:
ids = [line.strip() for line in f if line.strip()]
with yt_dlp.YoutubeDL(OPTS) as ydl, \
open(manifest_path, "a", encoding="utf-8") as out:
for vid in ids:
info = ydl.extract_info(f"https://www.youtube.com/watch?v={vid}")
if not info:
continue # extraction failed; already reported
for lang, sub in (info.get("requested_subtitles") or {}).items():
out.write(json.dumps({
"id": vid,
"lang": lang,
"kind": track_kind(info, lang, sub),
"file": f"subs/{vid}/{vid}.{lang}.{sub['ext']}",
}) + "\n")
if __name__ == "__main__":
main(*sys.argv[1:3])
Run it as python fetch_captions.py ids.txt captions_manifest.jsonl. On an English video with uploaded captions you get en (manual) and en-orig (auto). On a Spanish video without English captions, en is labeled auto-translated and en-orig doesn't exist.
Cleaning rolling auto-caption VTT
YouTube's auto-caption VTT is built for a two-line karaoke display, not for parsing. A typical stretch looks like this (abridged):
00:00:00.160 --> 00:00:02.389 align:start position:0%
in<00:00:00.480><c> this</c><00:00:00.800><c> video</c>
00:00:02.389 --> 00:00:02.399 align:start position:0%
in this video
00:00:02.399 --> 00:00:04.950 align:start position:0%
in this video
going<00:00:02.720><c> to</c><00:00:03.040><c> talk</c>
Each cue repeats the previous line on top, and 10 ms "hold" cues repeat it again. (The seemingly blank line after the first timing line is a single space, so it doesn't end the cue.) Naively concatenating the cue text triples most words. The fix: strip tags, then drop the longest run of lines at the start of each cue that equals the end of the previous cue. Manual tracks don't have this layout, so the script detects auto-captions by their inline word timestamps and only merges identical back-to-back cues for manual tracks.
#!/usr/bin/env python3
"""YouTube WebVTT -> segment-level JSONL ({"start", "end", "text"}), stdlib only.
Auto-generated tracks use a "rolling" layout: each cue repeats the previous
line above the new words, and ~10 ms cues re-show text that was already
emitted. Those repeats are removed. Manual tracks are passed through, with
identical back-to-back cues merged.
usage: python vtt_to_jsonl.py VIDEO.en-orig.vtt > VIDEO.en-orig.jsonl
"""
import html
import json
import re
import sys
TS = r"(?:(\d+):)?(\d{2}):(\d{2})\.(\d{3})"
TIMING = re.compile(rf"^{TS}\s+-->\s+{TS}")
WORD_TS = re.compile(r"<(?:\d+:)?\d{2}:\d{2}\.\d{3}>") # only in auto captions
TAG = re.compile(r"<[^>]*>") # <c>, </c>, <00:00:01.234>, <i>, ...
NON_SPEECH = re.compile(r"\[[^\]]*\]") # [Music], [Applause]; None to keep them
def seconds(h, m, s, ms):
return int(h or 0) * 3600 + int(m) * 60 + int(s) + int(ms) / 1000
def clean(line):
text = html.unescape(TAG.sub("", line))
if NON_SPEECH:
text = NON_SPEECH.sub("", text)
return " ".join(text.split())
def parse_cues(lines):
"""Yield (start, end, [text lines]). A cue ends at an empty line."""
block = []
for line in lines + [""]:
if line:
block.append(line)
continue
for i, row in enumerate(block): # header/NOTE/STYLE blocks never match
m = TIMING.match(row)
if m:
g = m.groups()
text = [clean(t) for t in block[i + 1:]]
yield seconds(*g[:4]), seconds(*g[4:]), [t for t in text if t]
break
block = []
def overlap(prev, cur):
"""Length of the longest suffix of prev that is also a prefix of cur."""
for k in range(min(len(prev), len(cur)), 0, -1):
if prev[-k:] == cur[:k]:
return k
return 0
def segments(path):
with open(path, encoding="utf-8-sig") as f:
raw = f.read()
rolling = bool(WORD_TS.search(raw))
out, prev = [], []
for start, end, lines in parse_cues(raw.splitlines()):
if not lines:
continue
new = lines[overlap(prev, lines):] if rolling else lines
prev = lines
if not new:
continue # cue only repeats text that was already emitted
text = " ".join(new)
last = out[-1] if out else None
if last and last["text"] == text and start - last["end"] < 0.05:
last["end"] = round(max(last["end"], end), 3) # same caption held
continue
out.append({"start": round(start, 3), "end": round(end, 3), "text": text})
return out
if __name__ == "__main__":
if len(sys.argv) != 2:
sys.exit("usage: vtt_to_jsonl.py FILE.vtt > FILE.jsonl")
for seg in segments(sys.argv[1]):
print(json.dumps(seg, ensure_ascii=False))
On the excerpt above it emits one segment per spoken line, with no repeats:
{"start": 0.16, "end": 2.389, "text": "in this video"}
{"start": 2.399, "end": 4.95, "text": "going to talk"}
Notes: NON_SPEECH strips bracketed tags like [Music]. Set it to None if you want them as non-speech labels. Auto-captions may also contain >> speaker-change markers, which you can split on or strip. Segment boundaries follow YouTube's display lines, not sentences, so re-segment by pauses or punctuation if your model needs utterance-level samples.
Word-level timing: json3 and srv3
For forced-alignment checks or word-level supervision, request the native formats with --sub-format json3 or srv3 (they can't be converted, so request them directly). YouTube doesn't publish either schema, so treat what follows as observed structure and validate it on a sample:
- json3: a top-level
eventslist. Each event hastStartMs,dDurationMsand asegslist of text pieces inutf8. On auto-caption tracks, words are separatesegscarrying an offset from the event start (tOffsetMs). Open-source parsers such as this one read the same fields. - srv3: XML with
<p t="…" d="…">lines (milliseconds) and<s>spans. See this reverse-engineered format description.
import json
def json3_words(path):
with open(path, encoding="utf-8") as f:
events = json.load(f).get("events", [])
for ev in events:
t0 = ev.get("tStartMs", 0)
for seg in ev.get("segs") or []:
word = seg.get("utf8", "").strip()
if word:
yield (t0 + seg.get("tOffsetMs", 0)) / 1000, word
These are word start times only. Derive ends from the next word's start, capped by the event's end, and expect them to be approximate.
Which flags for which need
| Need | Flags |
|---|---|
| See what tracks exist | --list-subs |
| Captions only, no media | --skip-download + subtitle flags |
| Creator-uploaded English, any region | --write-subs --sub-langs "en.*" |
| Original-language ASR only | --write-auto-subs --sub-langs "en-orig" |
| Manual if present, plus original ASR | --write-subs --write-auto-subs --sub-langs "en,en-orig", then check info.json |
| All manual tracks except chat replay | --write-subs --sub-langs "all,-live_chat" |
| Skip translations of manual tracks | --extractor-args "youtube:skip=translated_subs" |
| Word-level timing | --sub-format json3 (or srv3) |
| SRT files | --sub-format srt or --sub-format vtt --convert-subs srt |
| Subtitles inside the video file | --embed-subs (add --write-subs to keep sidecars) |
| Separate subtitle folder | -o "subtitle:subs/%(id)s.%(ext)s" or -P "subtitle:subs" |
| Throttle subtitle requests | --sleep-subtitles N --sleep-requests N or -t sleep |
| Resumable captions-only runs | --download-archive FILE --force-write-archive |
Captions as ASR labels vs re-transcribing with Whisper
Neither caption type is a clean transcript. Manual captions are often edited for reading: they condense speech, drop disfluencies, add speaker names and sound tags, and can be offset in time. Auto-captions are another ASR system's output, so training on them distills its errors. They can also lack punctuation and casing. A practical split:
- Use caption availability and language as cheap filters before you download any audio (pair with the audio-only download guide).
- Use manual captions as labels only after alignment and filtering, e.g. drop segments where your own ASR disagrees beyond a WER threshold you calibrate per language.
- Where label quality matters, re-transcribe with Whisper and keep the captions as a cross-check. Disagreement between the two is a useful quality signal.
The Whisper side (model choice, language ID, per-language thresholds) is covered in building a YouTube video dataset for multimodal training. Whatever you choose, store the track kind (manual/auto) and language with every segment so you can re-weight or drop sources later.
Compliance
YouTube's Terms of Service restrict automated access and downloading, and captions are content like any other. Copyright and licensing apply: some teams restrict collection to Creative Commons videos (see our guide to Creative Commons YouTube data) and record the license per sample. Voices and transcribed speech can be personal data under regimes like GDPR. Honor takedown and opt-out requests, and keep the lineage you need to do so. Get legal review for your jurisdiction and use case.
If you'd rather not run this yourself
Corpora offers dedicated 1–40 Gbps unmetered proxy capacity for teams running their own yt-dlp stack. It also offers a fully managed option that delivers video, audio, subtitles and metadata as JSONL/Parquet with sha256 checksums and manifests to your S3, GCS or Azure bucket, starting with a free sample batch. Contact us to scope it.
FAQ
How do I download only subtitles with yt-dlp, without the video?
Add --skip-download to your subtitle flags, e.g. yt-dlp --skip-download --write-subs --write-auto-subs --sub-langs "en,en-orig" URL. Add --write-info-json if you need to tell manual and auto tracks apart later.
What is the difference between --write-subs and --write-auto-subs?
--write-subs downloads tracks the creator uploaded (the subtitles key). --write-auto-subs downloads YouTube's automatic captions and their machine translations (the automatic_captions key). With both, a manual track takes precedence for the same language code.
What does en-orig mean in yt-dlp?
It's yt-dlp's label for the auto-generated caption track in the video's original spoken language. Other plain codes in the automatic list are usually machine translations of that track.
Why does my auto-generated VTT repeat every line?
YouTube's auto-caption VTT is a rolling two-line display: each cue repeats the previous line and short hold cues repeat it again. Strip the inline tags and remove the overlap between consecutive cues, as the script above does.
Can yt-dlp give word-level timestamps from YouTube captions?
For auto-captions, yes, indirectly. Download json3 or srv3 (or read the inline timestamps in VTT) and parse the per-word offsets yourself. Manual tracks are usually timed per line only.