https://huggingface.co/mente-ai/uyu-2-28B
I wanted to see if I could make Gemma 4 31B smaller without losing the parts that make it good at creative writing and role-play.
So I used Global Iterative Structured Pruning (GISP) and removed around 8% of the model’s overall structure. I didn’t prune everything evenly. I mostly targeted parts related to coding and math, while trying to keep as much of the model’s literary and creative ability as possible.
After that, I trained it further with reinforcement learning on role-playing data to improve dialogue, immersion, and narrative generation.
The final model performs about 6.4% worse than the original Gemma 4 31B on coding and math benchmarks. On the other hand, it performed better in creative writing and role-playing.
Maybe this could be a clue toward building more specialized AI models in the future.
[link] [comments]