察元AI教程库 · 文章 · 大模型

LazyTrain：在大型语言模型训练中，面向零浪费产出优化的资源有限分配

Training large language models on limited hardware is increasingly a scheduling problem across GPU compute, host memory, PCIe transfer, and storage bandwidth. Existing offloading systems reduce GPU re…

作者:Xiaojun Wu, Cehao Yang, Honghao Liu · 语言:en

前往原址学习

简介

Training large language models on limited hardware is increasingly a scheduling problem across GPU compute, host memory, PCIe transfer, and storage bandwidth. Existing offloading systems reduce GPU residency, and MegaTrain shows that a CPU-master layer-streaming executor can train large models on a single GPU, but fixed checkpointing and placement heuristics still leave communication exposed on th

返回教程库 · 察元AI产品家族