S3AI - SLAC Streaming Sandbox for AI

The SLAC EdgeAI team has partnered with a number of private companies and research institutions to demonstrate and benchmark unique hardware in the task of ultrahigh throughput AI inference and conventional algorithms for streaming data processing.  Such a sandbox allows for pair-programming with the domain experts who bring the algorithms together with the hardware engineers and architects who can creatively leverage their unique hardware to execute pieces of those algorithms in optimal low latency and high throughput ways.

In training, we have previously demonstrated a model training, using SambaNova RDUs, that fits both the sub-10 minute time frame of between runs of LCLS-II and between shots of DIII-D. 

In inference, we achieved a 10 microseconds per inference (100k inference rate) using a Graphcore POD 16 as a round robin. Again, these are timescales for inference that are relevant both for real-time diagnostic information extraction at LCLS-II and DIII-D.