A model is the start of a journey.
Follow a workload from sourced research to configuration analysis.
Step 01 of 06Define the workload
Start with the decision.
Describe the inference task, input/output lengths, request concurrency, hardware constraints, and what metric you want to optimize (e.g. Latency, Throughput, ttft).
workload context
Illustrative structure
- input_tokens
- 128
- output_tokens
- 32
- concurrency
- 8
01 / 06