whisper-large-v3-gl / README.md

asierhv

added description and "how to use" example

323307b verified 7 days ago

preview code

raw

history blame contribute delete

6.39 kB

metadata

language:
  - gl
license: apache-2.0
base_model: openai/whisper-large-v3
tags:
  - whisper-event
  - generated_from_trainer
datasets:
  - mozilla-foundation/common_voice_13_0
metrics:
  - wer
model-index:
  - name: Whisper Large-V3 Galician
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: mozilla-foundation/common_voice_13_0 gl
          type: mozilla-foundation/common_voice_13_0
          config: gl
          split: test
          args: gl
        metrics:
          - name: Wer
            type: wer
            value: 5.008278145695364

Whisper Large-V3 Galician

Model summary

Whisper Large-V3 Galician is an automatic speech recognition (ASR) model for Galician (gl) speech. It is fine-tuned from [openai/whisper-large-v3] on the Galician portion of Mozilla Common Voice 13.0, achieving a Word Error Rate (WER) of 5.01% on the evaluation split.

This model is intended for high-accuracy transcription of Galician audio in research, media, and accessibility applications.

Model description

Architecture: Transformer-based encoder–decoder (Whisper)
Base model: openai/whisper-large-v3
Language: Galician (gl)
Task: Automatic Speech Recognition (ASR)
Output: Text transcription in Galician
Decoding: Autoregressive sequence-to-sequence decoding

Fine-tuned on Galician speech data, leveraging Whisper’s multilingual pretraining for low-resource language transcription.

Intended use

Primary use cases

High-quality transcription of Galician audio recordings
Offline or batch ASR pipelines
Research and development in Galician ASR
Media, educational, and accessibility tasks

Intended users

Researchers working on Galician or low-resource ASR
Developers building Galician speech applications
Academic and institutional users

Out-of-scope use

Real-time or low-latency ASR without optimization
Speech translation tasks
Safety-critical applications without additional validation

Limitations and known issues

Performance may degrade on:
- Noisy or low-quality recordings
- Conversational or spontaneous speech
- Accents underrepresented in Common Voice
Dataset biases may be reflected in outputs
Occasional transcription errors can occur under difficult acoustic conditions

Training and evaluation data

Training data

Dataset: Mozilla Common Voice 13.0 (Galician subset)
Data type: Crowd-sourced, read speech
Preprocessing:
- Audio resampled to 16 kHz
- Text normalized using Whisper tokenizer
- Filtering of invalid or problematic samples

Evaluation data

Dataset: Mozilla Common Voice 13.0 (Galician evaluation split)
Metric: Word Error Rate (WER)

Evaluation results

Metric	Value
WER (eval)	5.01%

Training procedure

Training hyperparameters

Learning rate: 1e-5
Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
LR scheduler: Linear
Warmup steps: 500
Training steps: 20,000
Train batch size: 32
Eval batch size: 16
Gradient accumulation steps: 2
Total train batch size: 64
Seed: 42
Mixed precision: Native AMP

Training results (summary)

Training Loss	Epoch	Step	Validation Loss	WER
0.0176	5.0	1000	0.1563	5.2514
0.004	10.0	2000	0.1884	5.5653
0.0039	15.0	3000	0.2052	5.5377
0.0033	20.0	4000	0.2054	5.2997
0.0012	25.0	5000	0.2115	5.1031
0.001	30.0	6000	0.2195	5.2394
0.001	35.0	7000	0.2257	5.3446
0.001	40.0	8000	0.2178	5.4015
0.0008	45.0	9000	0.2250	5.4705
0.0008	50.0	10000	0.2320	5.2946
0.0002	55.0	11000	0.2368	5.3515
0.0	60.0	12000	0.2551	5.0997
0.0	65.0	13000	0.2634	5.0738
0.0	70.0	14000	0.2697	5.0359
0.0	75.0	15000	0.2752	5.0186
0.0	80.0	16000	0.2804	5.0066
0.0	85.0	17000	0.2852	4.9859
0.0	90.0	18000	0.2894	4.9893
0.0	95.0	19000	0.2927	5.0014
0.0	100.0	20000	0.2940	5.0083

Framework versions

Transformers 4.37.2
PyTorch 2.2.0+cu121
Datasets 2.16.1
Tokenizers 0.15.1

How to use

from transformers import pipeline

hf_model = "HiTZ/whisper-large-v3-gl"  # replace with actual repo ID
device = 0  # set to -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

Ethical considerations and risks

This model transcribes speech and may process personal data.
Users should ensure compliance with applicable data protection laws (e.g., GDPR).
The model should not be used for surveillance or non-consensual audio processing.

Citation

If you use this model in your research, please cite:

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.

Contact and attribution

Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
Base model: OpenAI Whisper
Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.