cpatonn

/

Qwen3-VL-8B-Instruct-AWQ-4bit

like 6

Hi @qqescac ,I used llmcompressor to quantize this model. Perhaps, it was due to your transformers version as I remember the similars occured to me too and it stopped happening after upgrading transformers.

I can't successfully quantize Qwen3-VL-8B using llmcompressor. The error seems to point to its Visual Encoder. Is it because llmcompressor hasn't been adapted to Qwen3-VL yet? What framework should I use for convenient quantization?

You can check the recipe.yaml in this repo and it actually skips the quantization of visual parts. This is default for quantizing vision models.