针对超长上下文、异构数据训练的线性可扩展性问题的分布式注意力机制。
A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training
