S3AI - SLAC Sandbox for Streaming AI

The SLAC EdgeAI team has partnered with a number of private companies and research institutions to demonstrate and benchmark unique hardware in the task of ultrahigh throughput AI inference and conventional algorithms for streaming data processing.  Such a sandbox allows for pair-programming with the domain experts who bring the algorithms together with the hardware engineers and architects who can creatively leverage their unique hardware to execute pieces of those algorithms in optimal low latency and high throughput ways.

process diagram of ai research at SLAC

In training, we have previously demonstrated a model training, using SambaNova RDUs, that fits both the sub-10 minute time frame of between runs of LCLS-II and between shots of DIII-D. In inference, we achieved a 10 microseconds per inference (100k inference rate) using a Graphcore POD 16 as a round robin for the CookieNetAE auto-encoder model used for attosecond x-ray pulse reconstruction at the LCLS-II. 

More recently, working with the team of Gregory Morse and Peter Rakyta at Etvos Lorand University in budapest, we have optimized the General Atomics CAKE-NN plasma shape model for 10 microsecond inference latency with a single NVIDIA LPU chip (former Groq GC1 card).  These microsecond timescales for inference are relevant both for real-time diagnostic information extraction at LCLS-II as well as ultra-low latency sensor processing for within-shot control at DIII-D and other fusion reactor facilities.

 

 

Latency Histogram
Latency histogram for the full round trip from data acquisition server over 10Gb ethernet to an FPGA that streams the data flow directly to the Groq (now NVIDIA) GC1 card.  The model is the DIII-D CAKE-NN fully connected few layer neural network used by General Atomics to infer the plasma Principle Component coefficients for the plasma shape.

 

 

Animation describing the S3AI workflow
Our current R&D is focusing on building the pieces required to make a high-level automated S3AI experience as shown in this concept animation.

 

 

We are building various "Hello World" example cases, initially as demonstrations, to serve as starting points for user engagement with the S3AI team.  The goal is to rethink how we flow data into information through a palette of computational inference acceleration tools.  One of our first of such demos features Dr. Peter Rakyta from Etvos Lorand University in Budapest Hungary.  A second demo features Jack Hirschman, Benjamin Mencer, and Luca Scomparin explaining our motivation for S3AI via a high-speed streaming camera demo of 90k frames/second acquisition of vibrationally controlled water jet breakup into droplets.

 

S3AI Facilities

servers mounted on a server rack
The S3AI contains a dedicated rack of servers as an extension of the S3DF facility. (Ryan Herbst / SLAC National Accelerator Laboratory)
server room
The S3AI data center allows for flexible cabling and the addition of processing engines into easily accessible servers.  (Ryan Herbst / SLAC National Accelerator Laboratory)
instrumentation that supports integration with detectors
The S3AI servers are co-located with instrumentation labs supporting optical tables for ease of integration with detectors.  (Ryan Herbst / SLAC National Accelerator Laboratory)

S3AI Partners

A number of partners are joining S3AI from all along the pixel-to-HPC continuum: 

  • NVIDIA DAQIRI -- Direct streaming of sensor signals into NVIDIA's AI acceleration ecosystem.

  • Ametek: 

    • Vision Research -- direct image streaming to AI inference acceleration hardware.

    • Abaco -- in-digitizer embedding and inference.

  • Internet2 -- Network overlays for secure cross-facility signal exchange.

  • Ultrata -- Distributed memory technology with foundational security.

  • Wind River -- Real-time OS for distributed and deployed inference based control systems.

  • DIII-D/General Atomics -- Sensor acquisition and analytic inference in tokamak reactors.

  • Commonwealth Fusion Systems -- Real-time streaming sensor-to-control inference.

  • Kove -- Low latency cluster-distributed memory technology.

  • Cornami -- Hardware acceleration for tight coupling of analysis and encryption.

  • Fusion Energy Data Ecosystem and Repository (FEDER) -- Integration of historical data into simulated sensor streams.