Scaling AI at Cerebras

Long training times are the single biggest factor slowing down innovation in deep learning. Today's common approach of scaling large workloads out over many small processors is inefficient and requires extensive model tuning. Urs Kster explains why with increasing model and dataset sizes, new ideas are needed to reduce training times.

Long training times are the single biggest factor slowing down innovation in deep learning. Today’s common approach of scaling large workloads out over many small processors is inefficient and requires extensive model tuning. With increasing model and dataset sizes, new ideas are needed to reduce training times. Urs Köster explores trends in the computer vision and natural language processing domains and techniques for scaling with the Cerebras wafer scale engine—the largest chip in the world. Cerebras’s unique, purpose-built processor allows you to leverage sparsity for building larger models and enables model-parallel training as an efficient alternative to data-parallel training.