Supported Datasets and Annotations

⭐ Dataset Quick Reference ⭐

This table is provided as a guide for users to select appropriate datasets. The list of annotations omits some metadata for brevity, and we document the dataset’s primary annotations only. The number of tracks indicates the number of unique “tracks” in a dataset, but it may not reflect the actual size or diversity of a dataset, as tracks can vary greatly in length (from a few seconds to a few minutes), and may be homogeneous.

“Downloadable” possible values:

• ✅ : Freely downloadable

• 🔑 : Available upon request

• 📺 : Do-it-yourself download

• 🧮 : Features only

• ❌ : Not available

Find the API documentation for each of the below datasets in Initializing.

Dataset	Downloadable?	Annotation Types	Tracks	License
AcousticBrainz Genre	audio: 🧮 annotations: ✅	Genre	>4M	Custom
BAF	audio: 🔑 annotations: 🔑	Matches	3425
Beatles	audio: ❌ annotations: ✅	Beats Chords Sections Key Vocal Activity	180	❌
Beatport EDM key	audio: ✅ annotations: ✅	global Key	1486
Billboard (McGill)	audio: ❌ annotations: ✅	Chords Sections	890
Candombe	audio: ✅ annotations: ✅	Beats	35
cante100	audio: 🔑 annotations: ✅	F0 Vocal Notes	100	Custom
CIPI	musicXML: 🔑 embeddings: 🔑 annotations: 🔑	difficulty levels	652
(CompMusic) Carnatic Rhythm	audio: 🔑 annotations: 🔑	Beats Meter	176
(CompMusic) Hindustani Rhythm	audio: 🔑 annotations: 🔑	Beats Meter	151
(CompMusic) Indian Tonic	audio: 🔑 annotations: ✅	Tonic	2150
(CompMusic) Jingju A Cappella	audio: ✅ annotations: ✅	Lyrics Phonemes Syllables	82
(CompMusic) OTMM Makam	audio: ✅ annotations: ✅	F0 Tonic	1000
(CompMusic) Raga	audio: 🔑 annotations: ✅	F0 Segments Tonic	780
Dagstuhl ChoirSet	multitrack audio: ✅ annotations: ✅	F0 Beats Notes	108
DALI	audio: 📺 annotations: ✅	Lyrics Vocal Notes	5358
Da-TACOS	audio: 🧮 annotations: ✅	Lyrics Vocal Notes	benchmark 15000 tracks coveranalysis 10000 tracks
EGFxSet	audio: ✅ annotations: ✅	Notes	8970
Filosax	audio: 🔑 annotations: 🔑 midi: 🔑	F0 Beats Chords Tempo Notes	48	❌
Four-Way Tabla Stroke	audio: ✅ annotations: ✅	Tags	236
Freesound One-Shot Percussive Sounds	audio: ✅ annotations: ✅	Tags	10254
Giantsteps key	audio: ✅ annotations: ✅	global Key	500
Giantsteps tempo	audio: 📺 annotations: ✅	global Genre global Tempo	664
Good Sounds	audio: : ✅ annotations: ✅	instrument Instruments sound quality metadata instrument metadata	16308
Groove MIDI	audio: ✅ midi: ✅	Beats Tempo Drums	1150
Gtzan-Genre	audio: ✅ annotations: ✅	global Genre Beats Tempo	1000	❌
Guitarset	audio: ✅ midi: ✅	Beats Chords Key Tempo Notes F0	360
Ikala	audio: ❌ annotations: ❌	Vocal F0 Lyrics	252	Custom
Haydn op20	audio: N/A midi: ✅ scores: ✅ annotations: ✅	symbolic Chords symbolic Key	24
IDMT-SMT-Audio Effects	audio: ✅ annotations: ✅	instrument Instruments midi nr Notes metadata Effect	55044
IRMAS	audio: ✅ annotations: ✅	Instruments Genre	9579
MTG_jamendo_autotagging_moodtheme	audio: ✅ annotations: ✅	moodtheme annotations	18448
MAESTRO	audio: ✅ annotations: ✅	Piano Notes	1282
Medley-solos-DB	audio: ✅ annotations: ✅	Instruments	21571
MedleyDB melody	audio: 🔑 annotations: ✅	Melody F0	108
MedleyDB pitch	audio: 🔑 annotations: ✅	F0 Instruments	103
Mridangam Stroke	audio: ✅ annotations: ✅	Stroke Name Tonic	6977
Orchset	audio: ✅ annotations: ✅	Melody F0	64
PHENICX-Anechoic	multitrack audio: ✅ annotations: ✅	Aligned score Notes Original score Notes	4
Queen	audio: ❌ annotations: ✅	Chords Sections Key	51	❌
RWC classical	audio: ❌ annotations: ✅	Beats Sections	61	Custom
RWC jazz	audio: ❌ annotations: ✅	Beats Sections	50	Custom
RWC popular	audio: ❌ annotations: ✅	Beats Sections Vocal Activity Chords Tempo	100	Custom
Salami	audio: ❌ annotations: ✅	Sections	1359
Saraga Carnatic	audio: ✅ annotations: ✅	F0 Vocal F0 Tempo Phrases Beats (samas) Sections Tonic	249
Saraga Hindustani	audio: ✅ annotations: ✅	F0 Tempo Phrases Beats (samas) Sections Tonic	108
Saraga-Carnatic-Melody-Synth (SCMS)	audio: ✅ annotations: ✅	F0 Events	2460
Slakh	multitrack audio: ✅ annotations: ✅	Notes Notes Instruments Instruments	1710
Tinysol	audio: ✅ annotations: ✅	Instruments Technique Notes	2913
Tonality ClassicalDB	audio: 🧮 annotations: ✅	Global Key	881
TONAS	audio: 🔑 annotations: 🔑	F0 Notes	72	Custom
vocadito	audio: ✅ annotations: ✅	F0 Notes Lyrics	40

Annotation Types

The table above provides annotation types as a guide for choosing appropriate datasets, but it is difficult to generically categorize annotation types, as they depend on varying definitions and their meaning can change depending on the type of music they correspond to. Here we provide a rough guide to the types in this table, but we strongly recommend reading the dataset specific documentation to ensure the data is as you expect. To see how these annotation types are implemented in mirdata see Annotations.

Events

A generic annotation to indicate whether a particular event is happening at a given time. It can be used, for instance, to indicate whether a particular instrument is playing at a given time-step or whether a particular note is being played at a given time-step. In fact, it is implicit in annotations such as F0 or Vocal Notes (instrument is activated when the melody is non-0). However, some datasets provide it as a standalone event annotation.

Beats

Musical beats, typically encoded as sequence of timestamps and corresponding beat positions. This implicitly includes downbeat information (the beginning of a musical measure).

Chords

Musical chords, e.g. as might be played on a guitar. Typically encoded as a sequence of labeled events, where each event has a start time, end time, and a label. The label taxonomy varies per dataset, but typically encode a chord’s root and its quality, e.g. A:m7 for “A minor 7”.

Drums

Transcription of the drums, typically encoded as a sequence of labeled events, where the labels indicate which drum instrument (e.g. cymbal, snare drum) is played. These events often overlap with one another, as multiple drums can be played at the same time.

F0

Musical pitch contours, typically encoded as time series indicating the musical pitch over time. The time series typically have evenly spaced timestamps, each with a corresponding pitch value which may be encoded in a number of formats/granularities, including midi note numbers and Hertz.

Effect

Effect applied to a track. It may refer to the effect applied to a single stroke or an entire track. It can include the effect name, the effect type, the effect parameters, and the effect settings.

Genre

A typically global “tag”, indicating the genre of a recording. Note that the concept of genre is highly subjective and we refer those new to this task to this article.

Instruments

Labels indicating which instrument is present in a musical recording. This may refer to recordings of solo instruments, or to recordings with multiple instruments. The labels may be global to a recording, or they may vary over time, indicating the presence/absence of a particular instrument as a time series.

Key

Musical key. This can be defined globally for an audio file or as a sequence of events.

Lyrics

Lyrics corresponding to the singing voice of the audio. These may be raw text with no time information, or they may be time-aligned events. They may have varying levels of granularity (paragraph, line, word, phoneme, character) depending on the dataset.

Matches

Music identifications in a query audio. This term is used in Audio Fingerprinting to refer to identifications of music from a reference database. Matches include information about which reference audio has been identified and the start and end times of the query match.

Meter

Rhythmic meter for each measure. A classical example of meter in Western music would be 4/4. Details how many subdivisions and the length of this subdivisions that we do have per each measure.

Melody

The musical melody of a song. Melody has no universal definition and is typically defined per dataset. It is typically encoded as F0 or as Notes. Other types of annotations such as Vocal F0 or Vocal Notes can often be considered as melody annotations as well.

Notes

Musical note events, typically encoded as sequences of start time, end time, label. The label typically indicates a musical pitch, which may be in a number of formats/granularities, including midi note numbers, Hertz, or pitch class.

Phonemes

Sung phonemes of the lead vocal lyrics. Likewise the annotations of lyrics, it can be represented as a stream of characters, or it can be time-aligned by start and end times, and the phoneme comprised in each interval.

Phrases

Musical phrase events, typically encoded by a sequence of timestamps indicating the boundary times and defined by solfège symbols. This annotations are not intended to describe the complete melody but the musical phrases present in the track.

Sections

Musical sections, which may be “flat” or “hierarchical”, typically encoded by a sequence of timestamps indicating musical section boundary times. Section annotations sometimes also include labels for sections, which may indicate repetitions and/or the section type (e.g. Chorus, Verse).

Segments

Segments of particular musical events, e.g. segments of note stability, segments of particular melodic event, and many more.

Technique

The playing technique used by a particular instrument, for example “Pizzicato”. This label may be global for a given recording or encoded as a sequence of labeled events.

Tempo

The tempo of a song, typical in units of beats-per-minute (bpm). This is often indicated globally per track, but in practice tracks may have tempos that change, and some datasets encode tempo as time-varying quantity. Additionally, there may be multiple reasonable tempos at any given time (for example, often 2x or 0.5x a tempo value will also be “correct”). For this reason, some datasets provide two or more different tempo values.

Vocal Activity

A time series or sequence of events indicating when singing voice is present in a recording. This type of annotation is implicitly available when Vocal F0 or Vocal Notes annotations are available.

Stroke Name

An open “tag” to identify an instrument stroke name or type. Used for instruments that have specific stroke labels.

Syllables

Additional representation of the sung lyrics but structured as syllables instead of complete sentences. It can be annotated as time-aligned events where the events are the syllables happening at certain time intervals. Otherwise, they can be represented as a stream of strings, grouped by meaningful syllable structures.

Tags

This is a broad annotation type that is used to label music and sounds, that often spans multiple categories. For example, music can be labeled with tags pertaining to the instruments present, the musical style, the mood, etc. Tags are often free-form and may not have a structured taxonomy/set of labels. They are typically represented as a list of strings, sometimes with associated weights/confidences.

Tonic

The absolute tonic of a track. It may refer to the tonic a single stroke, or the tonal center of a track.