Compare REST, Batch, and Streaming model deployment architectures interactively
Netflix encodes millions of videos in batch mode overnight. ChatGPT uses streaming to show you tokens as they're generated. Your bank's fraud detection uses synchronous REST — it must respond before your transaction completes. Same ML model, three completely different serving architectures.
Which architecture would you choose for a medical diagnosis tool that must respond in under 200ms?
| Mode | Latency | Throughput | Best for |
|---|---|---|---|
| REST | Low (~120ms) | Medium | Real-time inference |
| Batch | High (~500ms+) | High | Offline processing |
| Streaming | TTFT = Low | Medium | Generative AI UX |