Projects

NexteraBERT

A fast, long-context bidirectional encoder model.
It combines SnowLily, an input-dependent gating liquid mixer derived from Liquid Time-Constant networks, with length-adaptive attention using SSSMax (Stable Scalable-Softmax). Pretrained on 15× fewer tokens than ModernBERT-base, it matches ModernBERT-base on GLUE, outperforms it on MTEB v2, and runs 5.22× faster at 65K tokens.

SSE: Stable Static Embedding Model series

A lightweight and high-speed sentence embedding model.
While traditional static embedding models have faced challenges regarding the trade-off between parameter count and inference speed, this model utilizes Separable Dynamic Tanh (DyT) to control gradient flow and achieve regularization of the representation space.