HiTZ/whisper-large-v2-gl

Whisper Large-V2 Galician

Model summary

Whisper Large-V2 Galician is an automatic speech recognition (ASR) model for Galician (gl) speech. It is fine-tuned from [openai/whisper-large-v2] on the Galician portion of Mozilla Common Voice 13.0, achieving a Word Error Rate (WER) of 5.99% on the evaluation split.

This model is designed for high-accuracy transcription of Galician audio in research, media, and accessibility applications.

---

Model description

• Architecture: Transformer-based encoder–decoder (Whisper)
• Base model: openai/whisper-large-v2
• Language: Galician (gl)
• Task: Automatic Speech Recognition (ASR)
• Output: Text transcription in Galician
• Decoding: Autoregressive sequence-to-sequence decoding

Fine-tuned on Galician speech data, this model leverages Whisper’s multilingual pretraining to deliver state-of-the-art transcription performance for low-resource languages.

---

Intended use

Primary use cases

• High-accuracy transcription of Galician audio recordings
• Offline or batch ASR pipelines
• Research and development in Galician ASR
• Media, educational, and archival transcription tasks

Intended users

• Researchers working on Galician or low-resource ASR
• Developers building Galician speech applications
• Academic or institutional users

Out-of-scope use

• Real-time or low-latency ASR without optimization
• Speech translation tasks
• Safety-critical applications without validation

---

Limitations and known issues

• Performance may degrade on:
  • Noisy or low-quality recordings
  • Conversational or spontaneous speech
  • Accents underrepresented in Common Voice
• Transcription errors may still occur under challenging acoustic conditions
• Dataset biases from Common Voice may be reflected in outputs

Users are encouraged to evaluate the model on their own data before deployment.

---

Training and evaluation data

Training data

• Dataset: Mozilla Common Voice 13.0 (Galician subset)
• Data type: Crowd-sourced, read speech
• Preprocessing:
  • Audio resampled to 16 kHz
  • Text normalized using Whisper tokenizer
  • Filtering of invalid or problematic samples

Evaluation data

• Dataset: Mozilla Common Voice 13.0 (Galician evaluation split)
• Metric: Word Error Rate (WER)

---

Evaluation results

Metric	Value
WER (eval)	5.99%

This demonstrates the expected transcription quality for a large V2 Whisper model fine-tuned for Galician.

---

Training procedure

Training hyperparameters

• Learning rate: 1e-5
• Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
• LR scheduler: Linear
• Warmup steps: 500
• Training steps: 20,000
• Train batch size: 32
• Evaluation batch size: 16
• Gradient accumulation steps: 2
• Total train batch size: 64
• Seed: 42

Training results (summary)

Training Loss	Epoch	Step	Validation Loss	WER
0.0185	4.01	1000	0.1896	6.3569
0.0067	9.01	2000	0.2083	6.3862
0.0038	14.01	3000	0.2291	6.4621
0.0022	19.01	4000	0.2412	6.4794
0.0013	24.01	5000	0.2515	6.4673
0.0023	29.01	6000	0.2570	6.6432
0.0018	34.01	7000	0.2474	6.6380
0.0017	39.01	8000	0.2530	6.9312
0.0001	44.01	9000	0.2758	6.2379
0.0001	49.01	10000	0.2952	6.1241
0.0001	54.01	11000	0.3056	6.0499
0.0	59.01	12000	0.3152	5.9948
0.0	64.01	13000	0.3244	6.0310
0.0	69.01	14000	0.3336	6.0586
0.0	74.01	15000	0.3428	6.0344
0.0	79.01	16000	0.3518	6.0017
0.0	84.01	17000	0.3601	5.9879
0.0	89.01	18000	0.3675	6.0103
0.0	94.01	19000	0.3729	6.0068
0.0	99.01	20000	0.3753	6.0172

---

Framework versions

• Transformers 4.33.0.dev0
• PyTorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3

---

How to use

from transformers import pipeline

hf_model = "HiTZ/whisper-large-v2-gl"  # replace with actual repo ID
device = 0  # set to -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

---

Ethical considerations and risks

• This model transcribes speech and may process personal data.
• Users should ensure compliance with applicable data protection laws (e.g., GDPR).
• The model should not be used for surveillance or non-consensual audio processing.

---

Citation

If you use this model in your research, please cite:

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

---

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.

---

Contact and attribution

• Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
• Base model: OpenAI Whisper
• Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.