Kavli Affiliate: Wei Gao | First 5 Authors: Jihu Guo, Jihu Guo, , , | Summary: Pipeline parallelism is widely used to train large language models (LLMs). However, increasing heterogeneity in model architectures exacerbates pipeline bubbles, thereby reducing training efficiency. Existing approaches overlook the co-optimization of model partition, model placement, and workload scheduling, resulting in […]