Microsoft's ONNX Runtime Web brings Generative AI to the web browser

ONNX Runtime Web now supports WebGPU, a web API that enables hardware acceleration for machine learning models running in web browsers. ONNX Runtime Web is a JavaScript library that will allow web developers to deploy machine learning models directly in web browsers, offering multiple backends leveraging hardware acceleration. For CPU inference, it compiles the native ONNX Runtime CPU engine into the WebAssembly (WASM) backend. Now, one can imagine generating high-quality images or segmenting objects in real-time, all within your web browser, without relying on servers.

Previously, complex machine learning models, especially large generative models, couldn’t run efficiently in web browsers due to their limited computational power. But WebGPU unlocks the power of your device’s GPU for these tasks, which can significantly boost performance.

WebGPU is capable of handling more complex machine learning workloads in a more efficient way with advanced features such as compute shader. Its support for half-precision (FP16) reduces GPU memory usage and bandwidth requirements while accelerating arithmetic. WebGPU promises inferencing more efficient and scalable machine learning applications directly within the web browser by harnessing GPU power for parallel computation tasks.

What does this mean for you?

You can now experience faster and more powerful AI in your browser with real-time applications like image generation. This ensures your data is kept private and secure by eliminating sending it to servers for processing. 

WebGPU is now included by default in Chrome 113 and Edge 113 for Mac, Windows, ChromeOS, and Chrome 121 for Android. Just ensure browser compatibility and find resources for WebGPU acceleration with ONNX Runtime Web.

This advancement definitely paves the way for a future where powerful AI capabilities are readily available and accessible within your web browser.

More in-depth details are here.