MhaWay commited on
Commit
afcea13
·
verified ·
1 Parent(s): 44e7a56

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -24,7 +24,7 @@ model-index:
24
 
25
  # Veronica-Polymorphic
26
 
27
- **Veronica-Polymorphic Soft Mixture-of-Functions(SMoF)** is a decoder‑only transformer featuring a **polymorphic MLP layer**: each token is processed by a soft mixture of specialized branches (SwiGLU, GLU, Depthwise Causal Conv) under an entropy‑regularized router. The design enables adaptive capacity, incremental expansion (adding new branches post‑pretrain), and targeted specialization (e.g. translation modules) without full retraining from scratch.
28
 
29
  ## TL;DR
30
  | Feature | Description |
 
24
 
25
  # Veronica-Polymorphic
26
 
27
+ **Veronica-Polymorphic Soft Mixture-of-Experts(SMoE)** is a decoder‑only transformer featuring a **polymorphic MLP layer**: each token is processed by a soft mixture of specialized branches (SwiGLU, GLU, Depthwise Causal Conv) under an entropy‑regularized router. The design enables adaptive capacity, incremental expansion (adding new branches post‑pretrain), and targeted specialization (e.g. translation modules) without full retraining from scratch.
28
 
29
  ## TL;DR
30
  | Feature | Description |