train_cola_42_1763998308

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1828
• Num Input Tokens Seen: 3668584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2134	0.5	962	0.2261	183584
0.2406	1.0	1924	0.1994	366856
0.2971	1.5	2886	0.1828	550664
0.1749	2.0	3848	0.2156	734320
0.0699	2.5	4810	0.1904	918128
0.0761	3.0	5772	0.2006	1100800
0.1427	3.5	6734	0.2277	1284064
0.127	4.0	7696	0.2105	1467824
0.0869	4.5	8658	0.3083	1650992
0.1786	5.0	9620	0.2800	1834632
0.0014	5.5	10582	0.2896	2018408
0.1096	6.0	11544	0.2951	2202264
0.0005	6.5	12506	0.4053	2386136
0.1895	7.0	13468	0.3727	2568880
0.1034	7.5	14430	0.4502	2751696
0.0005	8.0	15392	0.4559	2935520
0.0002	8.5	16354	0.5162	3119168
0.0009	9.0	17316	0.5324	3302192
0.0001	9.5	18278	0.5414	3485264
0.0975	10.0	19240	0.5423	3668584

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4