Paper: how layer reuse in looped and depth-grown language models affects in-context learning (3B-parameter study)
Highly recommended read on looped and depth-grown models.
There is a lot of interest in looped and depth-grown language models for reasoning.
The question here is what reusing layers does to in-context learning.
At 3B parameters, both grown and looped models gain more fromXSources2
- @omarsar0XHighly recommended read on looped and depth-grown models. There is a lot of interest in looped and depth-grown language models for reasoning. The question here is what reusing layers does to in-context learning. At 3B parameters, both grown and looped models gain more fromView original ·
arXivResearchHow Depth Growth and Looping Shape In-Context Learning in Language ModelsView original ·
Headline by dalao.ai. The quoted post is the record; its author keeps authorship.