Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ model-index:
|
|
| 24 |
|
| 25 |
# Veronica-Polymorphic
|
| 26 |
|
| 27 |
-
**Veronica-Polymorphic Soft Mixture-of-
|
| 28 |
|
| 29 |
## TL;DR
|
| 30 |
| Feature | Description |
|
|
|
|
| 24 |
|
| 25 |
# Veronica-Polymorphic
|
| 26 |
|
| 27 |
+
**Veronica-Polymorphic Soft Mixture-of-Experts(SMoE)** is a decoder‑only transformer featuring a **polymorphic MLP layer**: each token is processed by a soft mixture of specialized branches (SwiGLU, GLU, Depthwise Causal Conv) under an entropy‑regularized router. The design enables adaptive capacity, incremental expansion (adding new branches post‑pretrain), and targeted specialization (e.g. translation modules) without full retraining from scratch.
|
| 28 |
|
| 29 |
## TL;DR
|
| 30 |
| Feature | Description |
|