WebGPU + ONNX Runtime Demo

This demo shows how to run AI models directly in the browser using WebGPU for hardware acceleration and ONNX Runtime Web.

What This Demonstrates

• Browser-based AI: Run models entirely in the browser
• WebGPU Acceleration: Hardware-accelerated inference when available
• Privacy-first: No data leaves your device
• Zero Install: Works in any compatible browser
• Graceful Fallback: Falls back to CPU if WebGPU unavailable

Requirements

Browser Compatibility:

• Chrome/Edge 113+ with WebGPU enabled
• Check WebGPU status: chrome://gpu
• Enable WebGPU: chrome://flags/#enable-unsafe-webgpu

Running the Demo

Option 1: Local Server (Recommended)

# Navigate to the demo directory
cd Module08\samples\04\webgpu-demo

# Start a local server
python -m http.server 5173

# Open browser to http://localhost:5173

Option 2: VS Code Live Server

1. Install "Live Server" extension in VS Code
2. Right-click index.html → "Open with Live Server"
3. Demo opens automatically in browser

What You'll See

1. WebGPU Detection: Checks browser compatibility
2. Model Loading: Downloads and initializes MNIST classifier
3. Inference Execution: Runs prediction on sample data
4. Performance Metrics: Shows load time and inference speed
5. Results Display: Prediction confidence and raw outputs

Expected Performance

Execution Provider	Model Load	Inference	Notes
WebGPU	~2-5s	~10-50ms	Hardware accelerated
CPU (WASM)	~2-5s	~50-200ms	Software fallback

Troubleshooting

WebGPU Not Available:

• Update to Chrome/Edge 113+
• Enable WebGPU in chrome://flags
• Check GPU drivers are current
• Demo will fallback to CPU automatically

Loading Errors:

• Ensure you're serving via HTTP (not file://)
• Check network connection for model download
• Verify CORS isn't blocking the ONNX model

Performance Issues:

• WebGPU provides significant speedup over CPU
• First run may be slower due to model download
• Subsequent runs use browser cache

Integration with Foundry Local

This WebGPU demo complements Foundry Local by showing:

• Client-side inference for ultimate privacy
• Offline capabilities when internet unavailable
• Edge deployment for resource-constrained environments
• Hybrid architectures combining local and server inference

For production applications, consider:

• Use Foundry Local for server-side inference
• Use WebGPU for client-side preprocessing/postprocessing
• Implement intelligent routing between local/remote inference

Technical Details

Model Used:

• MNIST digit classifier (ONNX format)
• Input: 28x28 grayscale images
• Output: 10-class probability distribution
• Size: ~500KB (quick download)

ONNX Runtime Web:

• WebGPU execution provider for GPU acceleration
• WASM execution provider for CPU fallback
• Automatic optimization and graph optimization

Browser APIs:

• WebGPU for hardware access
• Web Workers for background processing (future enhancement)
• WebAssembly for efficient computation

Next Steps

• Try with custom ONNX models
• Implement real image upload and classification
• Add streaming inference for larger models
• Integrate with camera/microphone input