wav2vec2-darija-tamazigh-test-41000-rows-test-1

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6580
• Wer: 1.0482
• Cer: 0.7652

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• total_eval_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
14.1754	1.0	141	14.1600	1.0058	0.9922
8.6081	2.0	282	7.1228	1.0	1.0
5.7555	3.0	423	4.9075	1.0	1.0
4.2454	4.0	564	3.9789	1.0	1.0
3.609	5.0	705	3.4499	1.0	1.0
3.2704	6.0	846	3.1211	1.0	1.0
2.9115	7.0	987	2.9271	1.0	1.0
2.8653	8.0	1128	2.8172	1.0	1.0
2.6786	9.0	1269	2.6638	1.0070	0.9475
2.558	10.0	1410	2.5710	1.0151	0.9465
2.5521	11.0	1551	2.5164	1.0128	0.9481
2.5375	12.0	1692	2.4802	1.0104	0.9488
2.4797	13.0	1833	2.4620	1.0244	0.9133
2.4535	14.0	1974	2.4450	1.0575	0.8859
2.3841	15.0	2115	2.4159	1.0754	0.8772
2.4295	16.0	2256	2.3848	1.0575	0.8731
2.3852	17.0	2397	2.3547	1.0511	0.8702
2.3214	18.0	2538	2.3039	1.0360	0.8648
2.338	19.0	2679	2.2528	1.0250	0.8556
2.2563	20.0	2820	2.1923	1.0186	0.8518
2.2599	21.0	2961	2.1332	1.0197	0.8415
2.1725	22.0	3102	2.0976	1.0192	0.8390
2.1236	23.0	3243	2.0374	1.0203	0.8340
2.172	24.0	3384	1.9997	1.0203	0.8327
2.1778	25.0	3525	1.9503	1.0209	0.8297
2.0151	26.0	3666	1.9168	1.0226	0.8276
2.0777	27.0	3807	1.8800	1.0226	0.8244
2.0232	28.0	3948	1.8420	1.0267	0.8164
2.0165	29.0	4089	1.8184	1.0267	0.8117
1.9716	30.0	4230	1.7938	1.0273	0.8048
1.9624	31.0	4371	1.7657	1.0366	0.7920
1.967	32.0	4512	1.7473	1.0377	0.7930
1.8925	33.0	4653	1.7212	1.0482	0.7853
1.9507	34.0	4794	1.7025	1.0493	0.7811
1.9125	35.0	4935	1.6876	1.0499	0.7713
1.8882	36.0	5076	1.6784	1.0459	0.7717
1.8878	37.0	5217	1.6712	1.0488	0.7709
1.872	38.0	5358	1.6628	1.0476	0.7684
1.9165	39.0	5499	1.6598	1.0476	0.7666
1.8736	40.0	5640	1.6580	1.0482	0.7652

Framework versions

• Transformers 4.48.3
• Pytorch 2.1.0+cu121
• Datasets 2.21.0
• Tokenizers 0.21.2