feifeibear / long-context-attention

USP：统一（也称为混合、2D）序列并行注意机制，用于长上下文Transformer模型的训练与推理

应用与完整系统通用PythonApache-2.0 · 宽松，可直接商用

用途与标签

项目原文 · GitHub

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference