Tag: ffmpeg

  • Split Video into Clips with PySceneDetect for Training Data

    Long videos are not training examples. Before you caption, filter or train, you have to split video into clips that each show one continuous shot and fit the window your model samples. This guide covers PySceneDetect's detectors (CLI and Python), TransNetV2 as a learned alternative, how Panda-70M split and re-stitched clips, frame-accurate cutting with ffmpeg, and what to record per clip.

    TL;DR

    • Detect, then cut. Run shot detection once, store boundaries as timestamps, and treat file cutting as a separate, repeatable step.
    • Start with AdaptiveDetector, add ThresholdDetector for fades, and tune thresholds from a stats file.
    • Raw shot boundaries over-split. Panda-70M stitched similar neighbors back together and roughly doubled average clip length.
    • Re-encode for frame-accurate cuts. Stream copy (-c copy) can only cut at keyframes.
    • Derive min/max length from your model's sampling window, drop short clips, split long shots, trim edges, and record why every boundary exists.

    Why splitting comes first

    Almost every downstream signal assumes a clip shows one continuous shot. A motion score on a clip containing a hard cut measures the cut; a caption describing two unrelated scenes teaches the model to blend them. The Stable Video Diffusion paper (arXiv 2311.15127) ran PySceneDetect in a cascaded manner at three frame rates to keep cuts and fades out of generated videos, and reports roughly 4x more clips after cut detection, meaning many source clips contained cuts that metadata didn't reveal.

    Splitting also defines your unit of data: every later filter, caption and dedup decision is keyed by clip ID. The broader pipeline is in building a YouTube video dataset for multimodal training, and the filters that run after splitting are in video data curation filters for generative models. This article goes deep on the splitting step.

    PySceneDetect: the detectors that matter

    PySceneDetect (BSD-3-Clause, v0.7.3 at the time of writing) is the tool both SVD and Panda-70M used. Install with pip install scenedetect --upgrade and put ffmpeg (or mkvmerge) on your PATH for splitting. Version 0.7 reworked timestamps to support variable-frame-rate video and moved some imports; if you have v0.6 code, read the migration guide. The docs recommend pinning scenedetect~=0.7.

    The three detectors you'll use most (defaults from the detector reference):

    Detector (CLI command) What it measures Default threshold Use it for
    ContentDetector (detect-content) Weighted frame-to-frame change in hue, saturation and luma (edges optional), 0–255 27.0 Hard cuts, steady cameras
    AdaptiveDetector (detect-adaptive) Same score, compared to a rolling average of neighboring frames 3.0 ratio, min_content_val=15.0 Hard cuts with fast camera motion; the CLI default
    ThresholdDetector (detect-threshold) Average pixel intensity crossing a fixed level 12.0 Fades to and from black

    One trap: min_scene_len defaults to 15 frames in Python but 0.6 s on the CLI, and ThresholdDetector's add_final_scene and min_out_length also differ. If you prototype on the CLI and productionize in Python, set them explicitly. None of these detectors is designed for cross-dissolves or wipes, where change is spread over many frames; that gap is why learned detectors and Panda-70M's extra heuristics exist.

    The CLI: fastest way to look at your data

    Global options go before commands. This detects cuts, drops (rather than merges) scenes shorter than 2 seconds, writes a CSV scene list and splits with ffmpeg:

    scenedetect -i input.mp4 -o clips/ -m 2s --drop-short-scenes \
      detect-adaptive list-scenes split-video

    To tune a threshold, write per-frame metrics first. -s saves a stats CSV with columns such as content_val; list-scenes -n prints scenes without writing a file:

    scenedetect -i input.mp4 -s input.stats.csv detect-content list-scenes -n

    Plot content_val for a few videos per source category. True cuts are isolated spikes; a threshold inside the noise floor of handheld footage over-splits, one above low-contrast cuts misses them. Thresholds that work for studio interviews rarely work for drone footage, so keep them per category.

    split-video re-encodes by default with -map 0:v:0 -map 0:a? -map 0:s? -c:v libx264 -preset veryfast -crf 22 -c:a aac. Options include -hq (CRF 17, preset slow), -crf, -p, -a for custom ffmpeg arguments, -f for filenames, and -c/--copy, which the config reference calls "significantly faster, but can result in inaccurate splits due to keyframe positioning".

    The Python API: what you'll run at scale

    In a pipeline you want boundaries as data. detect(path, AdaptiveDetector()) returns (start, end) FrameTimecode pairs. To run a cut detector and a fade detector in one decoding pass, register both on a SceneManager:

    from scenedetect import open_video, SceneManager, AdaptiveDetector, ThresholdDetector
    
    video = open_video("input.mp4")
    sm = SceneManager()
    sm.add_detector(AdaptiveDetector(min_scene_len="0.6s"))
    sm.add_detector(ThresholdDetector(threshold=12.0, min_scene_len="0.6s"))
    sm.detect_scenes(video, show_progress=True)
    
    for start, end in sm.get_scene_list(start_in_scene=True):
        print(start.get_timecode(), end.get_timecode(), end.seconds - start.seconds)

    start_in_scene=True makes a video with no cuts return one scene instead of an empty list. In v0.7, frame_num, seconds and frame_rate are properties, and video.frame_rate is an exact Fraction such as 30000/1001. Decoding is usually the bottleneck; frames are downscaled automatically, and the docs advise against frame_skip except for very high-frame-rate video.

    TransNetV2: a learned shot boundary detector

    TransNetV2 (arXiv 2008.04838) is a neural network built for shot transition detection, including gradual transitions. In the authors' re-evaluation it scores F1 77.9 on ClipShots, 96.2 on BBC Planet Earth and 93.9 on RAI. Those are their benchmarks; validate on a labeled sample of your own sources before switching.

    The reference inference code pins tensorflow==2.1, old enough that you'll likely want the provided Docker image or the inference-only PyTorch port in inference-pytorch/, which converts the TF weights. Weights are stored with git-lfs. Usage:

    from transnetv2 import TransNetV2
    
    model = TransNetV2()
    frames, single_frame_pred, all_frame_pred = model.predict_video("input.mp4")
    scenes = model.predictions_to_scenes(single_frame_pred)  # threshold defaults to 0.5
    # scenes: array of [start_frame, end_frame], zero-indexed, both inclusive

    The model sees frames resized to 48x27 via ffmpeg, so decoding dominates and a GPU helps. Output is frame indices, so convert with the source frame rate and take care with variable-frame-rate files. Its advantage over thresholded detectors is a per-frame transition probability: store the peak near each boundary as a confidence score and you can re-threshold later without re-running the model. A common compromise is PySceneDetect on everything and TransNetV2 on the subset that later filters flag as suspicious.

    How Panda-70M split and stitched

    Panda-70M split 3.8M long videos from HD-VILA-100M into 70.8M clips averaging 8.5 seconds. Its splitting code is public and encodes several practical lessons:

    1. Shot detection: ContentDetector with threshold=25, min_scene_len=15 frames and start_in_scene=True.
    2. Forced cuts: to handle undetected fades and unedited footage, it "recursively cut[s] the first 5 seconds as a new clip if the video clip is longer than 7 seconds".
    3. Stitching: ImageBind embeddings of frames near each piece's start and end; adjacent pieces with close embeddings are merged. Pieces whose own start and end differ sharply (likely a transition inside) and near-static events are dropped.
    4. Length and trimming: the README says Panda-70M itself was built with a 2.5 s minimum, a 60 s maximum and 10% trimmed from both ends; the repo's default line uses different values.
    5. Cutting: ffmpeg with -ss before -i and -t, re-encoded rather than stream-copied.

    On 1K long videos, the paper reports plain PySceneDetect splitting gave a 4.1 s average clip with max running LPIPS (the largest perceptual jump inside a clip) of 0.247, while its splitting gave 7.9 s at 0.256: nearly double the length for a small loss in consistency. To download Panda-70M itself, see downloading Panda-70M, HD-VILA and InternVid.

    Cutting with ffmpeg: keyframes, stream copy and re-encoding

    Compressed video stores full pictures only at keyframes; other frames are deltas. The FFmpeg seeking wiki spells out the consequences:

    • Re-encoding is frame-accurate. Since FFmpeg 2.1, -ss before -i is both fast and frame-accurate when transcoding.
    • Stream copy is not. With -c copy ffmpeg may only split on keyframes. The wiki's example: requested 157 s, next keyframe at 159 s, so the output starts with about 2 s of audio and no video. In training data that means lead-ins from the previous shot or frozen frames.
    • Keyframes can be far apart. libx264's default GOP of 250 is one keyframe every 10 s at 25 fps. Web uploads vary, so measure.

    To see where keyframes are, list video packets; keyframes have K in the flags field:

    ffprobe -v error -select_streams v:0 \
      -show_entries packet=pts_time,flags -of csv=p=0 input.mp4

    Rule of thumb: re-encode the clips you train on, at a low CRF so compression artifacts don't become something the model learns. Stream copy is acceptable only if your data loader decodes and trims to stored boundaries anyway.

    Clip length targets and short-clip filtering

    Pick lengths from your model, not the detector. If it samples N frames at F fps, a clip shorter than N/F seconds can't fill a window without looping or padding: 64 frames at 8 fps needs 64 / 8 = 8 s. Published reference points:

    Dataset Average clip length Source
    Panda-70M 8.5 s Panda-70M paper, Table 1
    SVD's LVD (before filtering) 11.58 s SVD paper, Table 1
    HD-VILA-100M (subtitle-aligned) 13.4 s Panda-70M paper, Table 1
    • Drop too-short clips. Don't merge them into neighbors without a stitching signal; merging across a cut reintroduces the problem.
    • Split too-long shots into windows at your maximum, and mark those boundaries as forced, not detected.
    • Trim the edges. Boundaries can be a frame off and fades bleed into neighbors; dropping a few frames per side is cheap insurance.

    A complete pipeline: detect, plan, cut, record

    This script detects boundaries, applies the length rules, cuts frame-accurately and writes one JSONL row per clip. The numbers are starting points, not recommendations for your data.

    import hashlib, json, subprocess
    from importlib.metadata import version
    from pathlib import Path
    from scenedetect import open_video, SceneManager, AdaptiveDetector, ThresholdDetector
    
    SRC = Path("raw/abc123XYZ_-.mp4")
    OUT = Path("clips"); OUT.mkdir(exist_ok=True)
    MIN_LEN, MAX_LEN, EDGE_TRIM = 2.0, 20.0, 0.1   # seconds
    
    video = open_video(str(SRC))
    sm = SceneManager()
    sm.add_detector(AdaptiveDetector(min_scene_len="0.6s"))
    sm.add_detector(ThresholdDetector(threshold=12.0, min_scene_len="0.6s"))
    sm.detect_scenes(video)
    scenes = sm.get_scene_list(start_in_scene=True)
    
    def plan(shot_start, shot_end):
        s, e = shot_start + EDGE_TRIM, shot_end - EDGE_TRIM
        reason = "forced_max_len" if (e - s) > MAX_LEN else "shot"
        t = s
        while e - t >= MIN_LEN:
            seg_end = min(t + MAX_LEN, e)
            yield t, seg_end, reason
            t = seg_end
    
    rows = []
    for shot_idx, (start, end) in enumerate(scenes):
        for s, e, reason in plan(start.seconds, end.seconds):
            clip_id = f"{SRC.stem}_{len(rows):05d}"
            out = OUT / f"{clip_id}.mp4"
            subprocess.run([
                "ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
                "-ss", f"{s:.3f}", "-i", str(SRC), "-t", f"{e - s:.3f}",
                "-map", "0:v:0", "-map", "0:a?",
                "-c:v", "libx264", "-preset", "veryfast", "-crf", "18",
                "-c:a", "aac", str(out)], check=True)
            dur = subprocess.run(
                ["ffprobe", "-v", "error", "-show_entries", "format=duration",
                 "-of", "csv=p=0", str(out)],
                capture_output=True, text=True, check=True).stdout.strip()
            rows.append({
                "clip_id": clip_id, "source_video_id": SRC.stem, "shot_index": shot_idx,
                "start_s": round(s, 3), "end_s": round(e, 3),
                "shot_start_frame": start.frame_num, "shot_end_frame": end.frame_num,
                "source_fps": str(video.frame_rate), "boundary_reason": reason,
                "detector": "adaptive(3.0)+threshold(12.0)",
                "scenedetect_version": version("scenedetect"),
                "encode": "libx264 veryfast crf18, aac",
                "probed_duration_s": float(dur),
                "sha256": hashlib.sha256(out.read_bytes()).hexdigest(),
            })
    
    with open(OUT / f"{SRC.stem}.clips.jsonl", "w", encoding="utf-8") as f:
        for r in rows:
            f.write(json.dumps(r) + "\n")

    Two production checks: compare probed_duration_s to end_s - start_s (a large gap means a decode problem), and inspect the first and last frames of a random sample. Measuring dataset accuracy with audits and Wilson intervals shows how to size that sample and report a boundary error rate.

    Metadata to record per clip

    Splitting owns provenance; later stages add scores to the same row, as in the curation filters guide. At minimum:

    • Provenance: clip ID, source video ID, source checksum, collection date, license at collection time.
    • Boundaries: start/end in seconds and source frames, exact source frame rate, whether the source is variable frame rate.
    • Boundary reason: detected cut, fade, forced split or stitched, plus a confidence if your detector provides one.
    • Reproducibility: detector, thresholds, library version, encode settings, pipeline version.
    • Output facts: probed duration, resolution, audio present, sha256.

    Keep boundaries even for clips you drop. Changing a threshold should be a query over stored rows, not a re-run of detection over millions of videos.

    Compliance

    Splitting turns each source into many derived files, so provenance matters for compliance too. YouTube's Terms of Service restrict automated access and downloading; copyright and licensing apply to every clip (filtering for Creative Commons videos is one option); faces and voices can be personal data under GDPR and similar laws; and takedown or opt-out requests must reach every derived clip, which requires each clip to carry its source ID. Get legal review for your jurisdiction and use case.

    If you'd rather not run this yourself

    Corpora collects YouTube video, audio, subtitles and structured metadata and delivers deduplicated datasets as JSONL/Parquet with sha256 checksums and manifests to S3, GCS, Azure Blob or private object storage, with a free sample batch to check quality first. See pricing or get in touch.

    FAQ

    How do I split a video into clips automatically by scene?

    Install PySceneDetect and ffmpeg, then run scenedetect -i input.mp4 detect-adaptive split-video. For training data, add -m 2s --drop-short-scenes and keep the list-scenes CSV as metadata.

    Which PySceneDetect detector should I use?

    AdaptiveDetector is the CLI default and copes with camera motion better than ContentDetector. Add ThresholdDetector if sources fade to black, and tune thresholds per source category with -s stats.csv.

    Why do my ffmpeg clips start early or show black frames?

    You are probably stream-copying (-c copy), which cuts at keyframes. Re-encode, with -ss before -i, for frame-accurate cuts.

    Is TransNetV2 better than PySceneDetect?

    It is a learned model that targets gradual transitions as well as hard cuts and outputs per-frame probabilities, but it needs a GPU at scale and its reference code pins an old TensorFlow. Compare both on a labeled sample of your own videos.

    How long should training clips be?

    At least your model's sampling window (frames divided by sampling fps), with a maximum so long shots don't dominate. Panda-70M averages 8.5 s, built with a 2.5 s minimum and 60 s maximum.

  • yt-dlp Audio Only: Download YouTube Audio for Speech Datasets

    For ASR, TTS and speech-LLM training you rarely need the video, yet many pipelines still download it, or download audio and then re-encode it to MP3 "for compatibility". This guide covers the yt-dlp audio-only path: how to select the right stream, keep it bit-for-bit, pick the original-language track on videos with dubbed audio, filter before downloading, and do the 16 kHz conversion once, at training time. Flags were checked against the yt-dlp README and source at release 2026.10.26.

    TL;DR

    • Use -f ba (bestaudio). It selects audio-only formats. Don't use ba*, which can return a muxed video file.
    • -x with the default --audio-format best doesn't re-encode. YouTube's Opus-in-WebM gets stream-copied to .opus, and M4A stays as it is. Only an explicit target like mp3 triggers a lossy re-encode.
    • On videos with several audio tracks, add -S lang. yt-dlp marks the original track with the highest language preference, but YouTube's default sort only uses it as a tiebreaker after quality and codec.
    • Store the native Opus/M4A. Convert to 16 kHz mono with ffmpeg -ac 1 -ar 16000 when you build training shards, not when you download.
    • Audio is cheap on bandwidth: 100k hours at 160 kbps is 7.2 TB. At that size, request rate limits cost you more time than the link does.

    Select the audio stream, not the video

    YouTube serves audio and video as separate DASH streams, so you can download just the audio. The yt-dlp format selection docs define ba/bestaudio as "the best quality audio-only format", equivalent to best*[vcodec=none]. ba* means "best format that contains audio" and may include video. The README marks it "Do not use!".

    Before you script anything, look at what a video actually offers:

    yt-dlp -F 'https://www.youtube.com/watch?v=VIDEO_ID'

    The audio-only rows show the codec, bitrate, sample rate (asr) and, on videos that have it, a language in brackets plus track notes such as (original) or (default).

    YouTube's audio-only formats

    There are two families: Opus in a WebM container and AAC in an M4A (MP4) container. yt-dlp no longer ships a hardcoded itag table. It now reads codec, bitrate and sample rate from YouTube's player response. The last version that had the table (2024.12.23) listed these nominal values:

    itag Container Codec Nominal bitrate (old yt-dlp table)
    249 webm Opus 50 kbps
    250 webm Opus 70 kbps
    251 webm Opus 160 kbps
    139 m4a AAC 48 kbps
    140 m4a AAC 128 kbps

    Treat these as labels, not measurements. Opus here is variable-bitrate, so the real average depends on the content and is often below the nominal figure. Which itags exist also varies by video and player client. The current extractor's own test suite marks itag 141 (256 kbps AAC) as "not served anymore". Check -F for real numbers.

    Some videos also offer DRC (dynamic range compression) variants, which show up as format IDs like 251-drc with "DRC" in the note. The YouTube extractor ranks them slightly below the regular stream. If you want uncompressed dynamics, exclude them explicitly with -f 'ba[format_id!*=drc]/ba'.

    Format sorting for audio (-S)

    -S changes what counts as "best". The sorting docs list these audio-relevant fields:

    • acodec: flac/alac > wav/aiff > opus > vorbis > aac > mp4a > mp3 > …
    • abr (average audio bitrate, kbps), asr (sample rate, Hz), channels
    • size (exact or approximate file size)
    • lang: "the language preference as determined by the extractor (e.g. original language preferred over audio description)"

    Fields sort descending; a + prefix reverses that. Your -S fields are placed ahead of the extractor's order. That matters on YouTube, whose extractor sets its own order (quality, res, fps, hdr:12, source, vcodec, channels, acodec, lang, proto), so lang normally comes near the end. Useful combinations:

    • -f ba -S lang: original track first, then the usual quality order.
    • -f 'ba[asr>=16000]' -S lang,+size: the smallest audio-only file that still has at least a 16 kHz sample rate. This makes sense if you're storage-bound and only train at 16 kHz. Measure WER on a sample before committing to it.
    • -f 'ba[ext=m4a]/ba': prefer AAC/M4A, for example if your decoder stack handles MP4 more easily than WebM/Ogg.

    Dubbed and multi-language audio tracks

    YouTube can attach several audio tracks to one video: the original, AI or human dubs, and audio description. For a speech dataset, a dub labeled with the original video's metadata is a data-quality bug. From the extractor source, this is how yt-dlp exposes them:

    • Each audio format's language is the part of YouTube's audio track ID before the dot.
    • A track whose display name contains "original" gets language_preference 10 and an (original) note. The default track gets 5, audio-description tracks get −10 with a language like en-desc, and all others get −1.
    • On single-track videos, yt-dlp fills language on audio formats from the original-language auto-caption track when one exists. Without one, language can be empty.

    So -S lang picks the original track whenever YouTube labels it. To require a specific language, use a format filter. String filters support ^=, $=, *=, ~= and negation with !:

    # English audio, but never the audio-description track
    yt-dlp -f 'ba[language^=en][language!$=desc]' -S lang URL

    If no format matches, that video fails with "Requested format is not available". yt-dlp then moves on to the next URL, because --no-abort-on-error is the default. Append /ba only if you'd rather get some audio than none. A format filter also excludes formats whose language is unknown, unless you write it as language^=?en.

    -x, --audio-format and why not MP3

    -x/--extract-audio needs ffmpeg and ffprobe. --audio-format accepts best (default), aac, alac, flac, m4a, mp3, opus, vorbis and wav. Here is what the FFmpegExtractAudioPP source actually does:

    • best + .m4a: does nothing ("already in a common audio format").
    • best + .webm: ffprobe reports Opus, so ffmpeg runs with -acodec copy into a .opus file. It's lossless and quick.
    • A different target (e.g. mp3): a full decode and re-encode with libmp3lame.

    Re-encoding Opus or AAC to MP3 stacks one lossy codec on top of another. Quality can only go down, and you pay CPU for every hour of audio. --audio-quality (0 best to 10 worst for VBR, or a bitrate like 128K; default 5) only matters when a re-encode happens. The built-in preset -t mp3 expands to -f 'ba[acodec^=mp3]/ba/b' -x --audio-format mp3. That suits a music player, not a corpus.

    Convert to 16 kHz mono at training time

    Many ASR models, including Whisper and wav2vec 2.0, take 16 kHz mono input. Do the conversion when you build shards, not at download:

    # 16 kHz, mono, 16-bit PCM WAV
    ffmpeg -nostdin -i abc123.opus -vn -ac 1 -ar 16000 -c:a pcm_s16le abc123.wav
    
    # Same, as FLAC (lossless, smaller than WAV)
    ffmpeg -nostdin -i abc123.opus -vn -ac 1 -ar 16000 -sample_fmt s16 -c:a flac abc123.flac

    -ac sets the channel count and -ar the sample rate (ffmpeg docs). Keeping the native file means you can later produce 24 kHz for TTS, keep stereo, or try another resampler without downloading again.

    If you really only want WAV on disk, you can do the conversion inside yt-dlp with --postprocessor-args (alias --ppa). The ExtractAudio+ffmpeg_o key passes the arguments to ffmpeg just before the output file, and only when the ExtractAudio step runs it:

    yt-dlp -f ba -S lang -x --audio-format wav \
      --ppa 'ExtractAudio+ffmpeg_o:-ac 1 -ar 16000' \
      -o '%(id)s.%(ext)s' URL

    Cut segments with --download-sections

    --download-sections takes a chapter-title regex or, with a * prefix, a time range. Negative timestamps count from the end, *from-url uses the URL's start/end time, and the flag can be repeated. It needs ffmpeg.

    # Minutes 10–20 only, plus any chapter whose title matches "interview"
    yt-dlp -f ba -S lang -x \
      --download-sections '*10:00-20:00' --download-sections 'interview' \
      -o '%(id)s_%(section_start)s.%(ext)s' URL

    Use this when you already have timestamps, for example from a dataset manifest. Treat the cut edges as approximate, and trim to exact sample boundaries in your own preprocessing if your labels need it.

    Filter before you download

    --match-filters accepts any output-template field, compared with the same operators as format filtering. & joins conditions, !field means "field absent or false", and repeating the flag ORs the filters:

    yt-dlp -f ba -S lang -x \
      --match-filters '!is_live & duration>=60 & duration<=7200 & language^=en' \
      URL_OR_PLAYLIST

    A detail that matters for language: YouTube doesn't set a video-level language, but language is a format field. yt-dlp evaluates match filters once before format selection, where missing format fields pass, and again in process_info after selection, before any media is downloaded. In practice language^=en tests the audio track you're about to download. Together with -S lang, that means "the original audio is English". Videos with no language info are rejected unless you write language^=?en.

    Filters on duration and is_live can drop playlist entries early, before a full extraction, when the playlist listing already includes those fields. For license filtering (Creative Commons), see our Creative Commons guide.

    Provenance: info.json, embedded metadata, archive

    • --write-info-json writes the full metadata next to the file, including the selected format_id, language, asr, abr, the license and the channel. That's your per-sample lineage.
    • --embed-metadata writes title, uploader, date and similar fields into the audio file's tags, so they stay with the file if it gets separated from its JSON. (The infojson attachment only applies to mkv/mka.)
    • --download-archive archive.txt records IDs so reruns skip completed videos.
    • Name files by %(id)s, not by title.

    Transcripts are a separate topic: our guide to yt-dlp subtitles and auto captions covers manual vs ASR tracks, the -orig track and VTT cleanup.

    Throttling: audio is request-bound

    An audio file is a small fraction of the size of its video, so per-video request overhead and YouTube's rate limits set your throughput, not bandwidth. The built-in -t sleep preset expands to --sleep-subtitles 5 --sleep-requests 0.75 --sleep-interval 10 --max-sleep-interval 20. For "Sign in to confirm you're not a bot", PO tokens, cookies and spreading load across IPs, see the troubleshooting guide.

    Storage math for 100k hours

    Formula: bytes = bitrate (bits/s) ÷ 8 × seconds. 100,000 h × 3,600 s = 360,000,000 s. The bitrates below are assumptions; your real Opus/AAC averages will differ.

    Storage format (assumed bitrate) Per hour 100k hours
    Native AAC/Opus at 128 kbps 57.6 MB 5.76 TB
    Native Opus at 160 kbps 72 MB 7.2 TB
    16 kHz × 16-bit × mono WAV = 256 kbps 115.2 MB 11.52 TB
    48 kHz × 16-bit × stereo WAV = 1,536 kbps 691.2 MB 69.12 TB

    For example, 160,000 ÷ 8 × 360,000,000 = 7,200,000,000,000 bytes = 7.2 TB. FLAC at 16 kHz mono comes in below the WAV figure, but the ratio depends on the content, so measure it on a sample. Transfer time: 7.2 TB is 57.6 Tbit, or 57,600 s ≈ 16 h at a sustained 1 Gbps.

    The request side looks different. Assume an average video length of 10 minutes: 100k hours is then 600,000 videos. With -t sleep the per-video sleep averages 15 s, so one worker starts at most 240 videos/hour. That's 2,500 worker-hours, about 104 hours of wall-clock time with 24 workers. For link and egress sizing, see bandwidth for large-scale video downloads.

    Goal → command

    Goal Command (add URL)
    Best native audio, original track, no re-encode yt-dlp -f ba -S lang -x
    Same, excluding DRC variants yt-dlp -f 'ba[format_id!*=drc]/ba' -S lang -x
    AAC/M4A only yt-dlp -f 'ba[ext=m4a]' -S lang
    Smallest file, still ≥16 kHz yt-dlp -f 'ba[asr>=16000]' -S lang,+size -x
    English original audio only yt-dlp -f ba -S lang -x --match-filters 'language^=en'
    16 kHz mono WAV straight away yt-dlp -f ba -S lang -x --audio-format wav --ppa 'ExtractAudio+ffmpeg_o:-ac 1 -ar 16000'
    Only a time range yt-dlp -f ba -S lang -x --download-sections '*10:00-20:00'
    Corpus run with provenance and pacing yt-dlp -f ba -S lang -x --embed-metadata --write-info-json --download-archive archive.txt -o '%(id)s.%(ext)s' -t sleep

    Python API

    The same setup through yt_dlp.YoutubeDL. The repo's devscripts/cli_to_api.py translates any CLI flags into these parameters.

    import yt_dlp
    from yt_dlp.utils import match_filter_func
    
    URLS = ['https://www.youtube.com/watch?v=VIDEO_ID']
    
    ydl_opts = {
        'format': 'ba[format_id!*=drc]/ba',
        'format_sort': ['lang'],                      # original audio track first
        'match_filter': match_filter_func('!is_live & duration>=60 & duration<=7200'),
        'paths': {'home': 'audio_raw'},
        'outtmpl': {'default': '%(id)s.%(ext)s'},
        'writeinfojson': True,
        'download_archive': 'archive.txt',
        'sleep_interval_requests': 0.75,
        'sleep_interval': 10,
        'max_sleep_interval': 20,
        'postprocessors': [
            # 'best' = stream-copy WebM/Opus to .opus, leave .m4a untouched
            {'key': 'FFmpegExtractAudio', 'preferredcodec': 'best'},
            {'key': 'FFmpegMetadata', 'add_metadata': True},
        ],
        # For 16 kHz mono WAV instead: set preferredcodec to 'wav' and add
        # 'postprocessor_args': {'extractaudio+ffmpeg_o': ['-ac', '1', '-ar', '16000']},
    }
    
    with yt_dlp.YoutubeDL(ydl_opts) as ydl:
        retcode = ydl.download(URLS)

    Postprocessors run in list order, so audio extraction happens before metadata embedding. Keys in postprocessor_args are lowercase.

    Compliance

    YouTube's Terms of Service restrict automated access and downloading. Copyright and licensing apply to audio just as to video; some teams collect only Creative Commons material and record the license per sample. Voices are personal data under regimes like GDPR, and a voice can identify a speaker even without a name attached. That matters especially for TTS and voice-cloning use. Honor takedown and opt-out requests, and keep the lineage (video ID, channel, info.json) you need to act on them. Get legal review for your jurisdiction and use case.

    If you'd rather not run this yourself

    Corpora offers dedicated 1–40 Gbps unmetered proxy capacity (HTTP(S) and SOCKS5, compatible with yt-dlp) for teams running their own stack. There is also a fully managed option that delivers audio, subtitles and metadata as JSONL/Parquet with sha256 checksums and manifests to S3, GCS or Azure Blob, starting with a free sample batch. Contact us to scope it.

    FAQ

    How do I download only the audio from YouTube with yt-dlp?

    yt-dlp -f ba -x URL. -f ba selects the best audio-only stream. -x with the default best format turns the WebM into .opus by stream copy and leaves M4A untouched, with no re-encoding. Add -S lang to prefer the original track on dubbed videos.

    Is --audio-format mp3 lower quality?

    Yes. YouTube's audio-only streams are Opus or AAC, so yt-dlp has to decode the Opus or AAC and re-encode it, adding a second lossy generation. Keep the native format and convert to WAV/FLAC when you need PCM.

    Why did yt-dlp download a dubbed audio track?

    On YouTube, the extractor's default sort puts quality and codec ahead of lang, so the original-track preference only breaks ties. Pass -S lang to make it the first criterion, and check -F for the (original) note.

    What sample rate should I store for ASR training?

    Store whatever YouTube served (check asr in -F). Resample to the model's rate, typically 16 kHz mono for ASR, with ffmpeg -ac 1 -ar 16000 when building training shards.

    Does --download-sections still download the whole file?

    No. yt-dlp passes each range to ffmpeg as input-side -ss/-t, so ffmpeg seeks into the stream instead of reading it from the start. You can pull labeled spans from long videos without storing the rest.