quantization.log

uv run python custom_mlx_lm/custom_convert.py --hf-path . --mlx-path MobileLLM-R1-950M-mixed-4bit-mlx --dynamic-quant --target-bpw 4.5 --group-size 64 --report-ppl
Loading model from ....
Loading calibration data...
Token indices sequence length is longer than the specified maximum sequence length for this model (110205 > 32768). Running this sequence through the model will result in indexing errors
Calculating perplexity of original model...
Original PPL: 50.262
Starting advanced mixed-precision quantization...
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Estimating sensitivities: 100%|████████████████████████████████████| 54/54 [02:03<00:00,  2.28s/it]
Calculating perplexity of quantized model...
Quantized PPL: 59.059

✅ Model saved to MobileLLM-R1-950M-mixed-4bit-mlx

uv run python custom_mlx_lm/quant_summary.py --model-path MobileLLM-R1-950M-mixed-4bit-mlx --show 8 
Method: mixed_precision_dynamic
Group size: 64
Total linear layers: 154
4-bit layers: 153
8-bit layers: 1

Examples (8-bit):
- layers.0.attention.o_proj

Examples (4-bit):
- layers.0.attention.k_proj
- layers.0.attention.q_proj
- layers.0.attention.v_proj
- layers.0.feed_forward.down_proj
- layers.0.feed_forward.gate_proj
- layers.0.feed_forward.up_proj
- layers.1.attention.k_proj
- layers.1.attention.o_proj

weights.npz contains quantized tensors: True