Downloading YouTube at scale is a solved problem. Finding the right videos is not. "Natural two-person English conversation, no background music" or "live-action vertical short dramas" are easy to say and slow to find: search results drift off-topic after a few pages, titles lie, and most candidates fail on things you can only hear or see. Corpora Curation is the system we built to do that search continuously and deliver only what matches your spec. It is now open for early-access projects.
TL;DR
- You describe what you need in plain language; it becomes a structured, versioned filter spec that we review with you.
- Discovery runs continuously across search, channels, playlists and channel feeds, and shifts effort toward the sources that actually produce matches.
- Candidates pass a funnel from cheap to expensive: metadata rules, content classification, then an audio probe that measures speech, language, speaker count and music.
- Accuracy is measured, not promised: a human-labeled calibration set before the job runs, and an audit sample with a confidence interval in every batch.
- Delivery is metadata-only by default (video IDs plus structured metadata and scores), or media files when you need them.
The problem it solves
Teams building speech, audio and video models usually don't struggle with bandwidth. They struggle with selection:
- Keyword search returns a few hundred relevant results, then drifts. The good material lives in specific channels and playlists you have to discover.
- Metadata is unreliable. A video titled "podcast" can be a single narrator over music; a "short drama" can be a recap channel narrating someone else's footage.
- The requirements that matter most — two or more speakers, little background music, a specific spoken language — are properties of the audio, not the text.
- Checking everything by downloading it is slow and expensive, and checking a sample by hand doesn't scale past a few thousand videos.
Curation is a pipeline built around that reality: spend almost nothing on candidates that obviously fail, and save the expensive checks for the ones that might pass. It is the same principle we describe in video data curation for generative models, applied before download instead of after.
How a job works
1. Requirement → spec
You describe the data in plain language, optionally with a few example videos that should and should not qualify. The system drafts a structured spec: seed search queries in the target language, hard rules (duration, Shorts or not, live streams, category), a written definition of positives and negatives for content classification, audio thresholds, the target count and the delivery format. We review and adjust it with you before anything runs. Every change creates a new spec version, and every decision records which version made it.
A simplified spec for conversational English speech looks like this:
target: { count: 20000 }
rules:
- "duration_s between 300 and 3600"
- "not is_live and not is_short"
stages:
- type: text_cls
definition: |
Positive: natural, unscripted conversation between two or more people
(podcasts, interviews, casual talk, street interviews).
Negative: single-narrator monologue, audiobooks, news reading, songs,
music-heavy vlogs, gameplay commentary, ASMR.
- type: audio_probe
require:
lang: en
speech_ratio: ">= 0.8"
n_speakers: ">= 2"
music_ratio: "<= 0.1"
download:
mode: metadata
2. Continuous discovery
Discovery expands search queries, channels, playlists and channel feeds, and treats each one as a source with its own track record. Sources that produce qualifying videos get more of the crawl budget; sources that don't are paused. When a channel produces matches, its other videos become candidates too, which is usually where the best material is.
3. The filter funnel
| Layer | What it checks | Cost per candidate |
|---|---|---|
| List metadata rules | Duration, Shorts, live status, upload date — from listing pages, before any per-video request | Lowest |
| Detail metadata rules | Category, license, available caption and audio languages, resolution and aspect ratio | Low |
| Content classification | Title, description, tags and channel judged against your written definition by a language model | Low–medium |
| Audio probe | Short audio segments analysed on CPU: speech ratio, spoken language, estimated speaker count, music and instrumental ratio | Highest — only for candidates that passed everything else |
The audio probe is what makes speech requirements enforceable. It samples segments rather than the whole file (several positions for long videos), runs voice-activity detection, language identification, speaker diarization and audio-event classification, and records every value. A video that passes on text but turns out to be one narrator over a music bed is rejected here, before it costs a download.
If your requirement is purely about topic or text, the audio layer can be skipped entirely and jobs run much faster.
4. Calibration and audits
Before a job runs at volume, it runs in calibration on a small number of candidates, and a sample of its decisions is labeled by a person. That gives a measured precision with a 95% confidence interval for the spec, and shows exactly which layer is making which mistakes. The spec is adjusted until the measured precision meets the target you set.
After that, every delivery batch includes an audit sample and the same interval. If you want to understand the statistics behind it, see measuring curated dataset accuracy.
5. Delivery
Results arrive in batches while the job runs, not all at once at the end:
- Metadata mode (default): video IDs with structured metadata and every score — no media download, so it is fast and you collect the files on your own infrastructure.
- Media mode: audio or video files in the format your spec requests, for example 16 kHz mono FLAC for speech, with checksums.
- Every batch includes a
manifest.jsonl(one record per video with the scores that qualified it), abatch_report.json(funnel counts per layer, durations, audit results) and the audit sample. - No duplicates for you: a video delivered to you once is never delivered to you again in a later batch or another job.
A manifest record looks like this:
{
"video_id": "abc123XYZ00",
"url": "https://www.youtube.com/watch?v=abc123XYZ00",
"channel_id": "UC...",
"title": "...",
"duration_s": 1843,
"lang": "en",
"license": "youtube",
"scores": { "text_cls": 0.91 },
"features": { "speech_ratio": 0.93, "n_speakers": 2, "music_ratio": 0.02 },
"spec_version": 3,
"accepted_at": "2026-11-06T12:41:09Z"
}
What it is good at today
- Speech and conversation data by language, speaker count and amount of background music — see what makes audio ASR- and TTS-ready for the criteria that matter.
- Genre and format selection that can be judged from text and audio: interviews, lectures, tutorials, podcasts, music versus speech, specific content categories.
- License-constrained jobs, for example Creative Commons only, with the license recorded per video (see Creative Commons YouTube videos for AI training).
Requirements that can only be judged visually (live-action versus animation, on-screen layout, camera style) currently rely on text signals and are less precise; frame-level visual checks are next on the roadmap. Very subjective definitions ("interesting conversations") are hard to hold at a stable accuracy, which is exactly what calibration exposes before you pay for volume.
Pricing and early access
Curation is priced per qualified item delivered, with the price depending on duration range and on which layers your spec needs (text only, or text plus audio). We are taking a limited number of early-access projects. Send us the kind of data you need and a rough volume; we will draft the spec, run calibration and show you the measured precision and a sample batch before you commit. Contact sales.
Compliance
YouTube's Terms of Service restrict automated access and downloading, and copyright, licensing and privacy law apply to what is collected — voices and faces can be personal data. Curation records the source, license field and collection time for every item, so any removal request can be traced to the batches it appears in, and it does not attempt to identify the people in the content. Customers are responsible for their own legal review of their intended use.
FAQ
How is this different from buying a fixed dataset?
A fixed dataset was selected for someone else's model. Curation selects against your spec, keeps running until it reaches your target count, and measures its own precision on your definition.
Do I have to receive media files?
No. The default is metadata only: IDs, metadata and scores. Many teams prefer to download on their own infrastructure.
How accurate is it?
It depends on how objective your requirement is, which is why every job is calibrated with human labels first and every batch carries an audit sample with a 95% confidence interval.
Can the spec change while the job is running?
Yes. A change creates a new spec version; decisions record the version that made them, so you can see exactly which items qualified under which rules.