Model research, request preparation, and inference optimization connected through existing, traceable workflows.
Nemotron assists model research and assessment using Hugging Face metadata and sourced Tavily research. Claims retain source URLs and verified, inferred, or unknown status.
Selection pins an immutable model revision.
requirements → sources → assessment → selected revisionResearch and runtime validation remain distinct.
Research suggests model candidates; you choose which one to explore. Confiron prepares a JSON request containing the selected model, workload, serving settings, and optimization objectives.
Choose a new run on Nebius Serverless AI Jobs, or reuse compatible existing results for that model when they are available. Preparing the request keeps those choices explicit.
intent.jsonrequest.jsonNew remote run or compatible prior-run data.
For a new run, LLM Optimizer executes the prepared run through Nebius Serverless AI Jobs. It manages trial scheduling, remote GPU jobs, checkpoints, and result collection.
For offline exploration, reuse a compatible previous run for the selected model. Analyze stored inference measurements and compare configurations without launching another GPU job.
The optimizer explores framework-specific configuration dimensions. Input/output lengths, prompt count, request concurrency, and seed stay attached to each comparison. Throughput, E2E latency, and TTFT objectives retain their measurement definitions.
Keep the benchmark fixed. Explore serving parameters. Compare throughput and latency with the context that makes the measurements meaningful.
Preparation profiles cover vLLM, SGLang, and TensorRT-LLM. Model and runtime compatibility must be checked.
01max_num_seqsServing capacity02max_num_batched_tokensToken budget03gpu_memory_utilizationMemory allocationA model-based sampler that uses observed outcomes to guide future proposals. Its behavior depends on budget and configuration.
A surrogate-based approach useful for exploring smooth objectives. Availability depends on the optimizer’s optional dependencies.
Useful baselines and systematic exploration strategies. They are not Bayesian optimization.
The public lab uses actual seeded Optuna samplers offline. It does not emulate engine execution or establish which sampler wins on inference workloads.
Compare the search traces
vLLM
SGLang
TensorRT-LLMNemotron
Nebius Serverless AIDynamo AI ConfiguratorDefine the comparison before searching for a configuration.