NVFP4?
#9
by
ktsaou
- opened
Here it is if you'd like to give it a try! I'm having issues running it in VLLM on an RTX Pro 6000 Blackwell instance. There appear to be some outstanding vllm issues with patches that aren't released yet that may fix this. I added a note on the model card. It should work as is with Transformers though.
https://huggingface.co/Firworks/Qwen3-Coder-REAP-25B-A3B-nvfp4
Wow! You rock!