An Open Source Deep Learning Framework for Cerebras WSE

The Challenge

The AI industry faces a critical bottleneck: vendor lock-in.

NVIDIA's CUDA ecosystem and PyTorch have become the de facto standard for AI development but really only because they're the most convenient. This convenience comes at a cost, researchers and organizations remain tethered to a single vendor's hardware, pricing, and roadmap.

Meanwhile, revolutionary hardware like Cerebras's Wafer-Scale Engine offers transformative performance advantages that remain largely untapped:

• Massive Parallelism: 850,000+ processing elements working in concert on a single chip
• Unprecedented Memory Bandwidth: 20 petabytes per second, eliminating the memory wall that constrains GPU-based training
• Simplified Scaling: Train billion-parameter models on a single device without the complexity of distributed GPU clusters
• Energy Efficiency: Dramatically reduced power consumption compared to equivalent GPU infrastructure

The problem? Existing tools force GPU-centric programming paradigms onto this radically different architecture, leaving significant performance gains unrealized.

Our Solution: PyFlame

PyFlame is a complete deep learning framework written in C++ with full Python bindings, architected natively for the Cerebras platform.

Core Features

Native Architecture Support

• Tensor operations designed for 2D mesh topology
• First-class wavelet communication primitives
• Direct integration with Cerebras Software Language (CSL)
• Zero translation overhead, your code compiles directly to optimized WSE instructions

Complete ML Toolkit

• Comprehensive tensor library with NumPy-compatible API
• Full automatic differentiation (autograd) system
• Transformer blocks, attention mechanisms, and neural network layers
• Optimizers, loss functions, and training utilities

Developer Experience

• Python-ready via pybind11, feels familiar to PyTorch users
• Explicit control over PE mesh layouts when you need it
• Seamless NumPy interoperability for data pipelines
• Extensive documentation and examples

Production Ready

• Model checkpointing and serialization
• Mixed-precision training support
• Profiling and debugging tools
• Import/export compatibility with existing model formats

Why Open Source?

We could have kept PyFlame proprietary. Instead, we chose to give it away.

Accelerating Innovation

When tools are open, the entire community benefits. Researchers can build on each other's work. Startups can compete without massive infrastructure investments. Students can learn on real-world systems.

Breaking Monopolies

Healthy competition drives progress. By lowering the barrier to Cerebras development, we're helping create a more diverse AI hardware ecosystem where the best technology wins, not just the most entrenched.

Building Trust

Open source means transparency. You can inspect every line of code, verify our claims, and adapt the framework to your specific needs.

Get Started

PyFlame is available for download now under Apache License 2.0 open-source license.

Download PyFlame
Download PyFlameRT
Download PyFlameVision

Whether you're a researcher exploring novel architectures, a startup looking to differentiate on performance, or an enterprise seeking alternatives to NVIDIA dependency, PyFlame gives you the tools to harness Cerebras's full potential.

Run the examples. Build something amazing.