Does CLAP able to train on ASR dataset? #170

Activity

1. mrowebot commented on Apr 17, 2025

   

   mrowebot

   on Apr 17, 2025

   Last edited by mrowebot

   More actions

   @MonolithFoundation Yes, you should be able to train on such a dataset as the READMe mentions the following in the Pretrained Models section:

   For music and speech: music_speech_epoch_15_esc_89.25.pt
   For speech, music and general audio: music_speech_audioset_epoch_15_esc_89.98.pt

   You would need to convert whatever ASR dataset you have to the Webdataset format for training to use this data. Have a look at the processing steps and the following repo: https://github.com/LAION-AI/audio-dataset
2. lukewys commented on May 4, 2025

   

   lukewys

   on May 4, 2025

   Contributor

   More actions

   The speech dataset is not very "ASR"-heavy, as far as I remember they also contains the description of the speaker and language. I don't how CLAP performs in ASR-like transcription task.