Flyte 2 is available today for local execution - distributed execution coming to open source soon. Preview Flyte 2 for production, hosted on Union.ai
2.0.11

flyteplugins.vllm

Directory

Classes

Class Description
VLLMAppEnvironment App environment backed by vLLM for serving large language models.

Variables

Property Type Description
DEFAULT_VLLM_IMAGE Image