ReactiveAI
/

RxT-Beta-Mini-Decoder-Base

🇪🇺 Region: EU

Model card Files Files and versions

AdamF92 commited on about 23 hours ago

Commit

051930a

·

verified ·

1 Parent(s): b725dd4

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -30,7 +30,7 @@
     "dense",
     "moe"
   ],
-  "stm_size": 1024,
   "use_attention_output_bias": false,
   "use_flash_attention": true,
   "use_gated": true,

     "dense",
     "moe"
   ],
+  "stm_size": 2048,
   "use_attention_output_bias": false,
   "use_flash_attention": true,
   "use_gated": true,