LLM Inputs

vllm.inputs.PromptStrictInputs

The inputs to the LLM, which can take one of the following forms:

alias of Union[str, TextPrompt, TokensPrompt]

class vllm.inputs.TextPrompt(*args, **kwargs)[source]

Bases: dict

Schema for a text prompt.

  • prompt: str

    The input text to be tokenized before passing to the model.

  • multi_modal_data: typing_extensions.NotRequired[MultiModalData]

    Optional multi-modal data to pass to the model, if the model supports it.

class vllm.inputs.TokensPrompt(*args, **kwargs)[source]

Bases: dict

Schema for a tokenized prompt.

  • prompt_token_ids: List[int]

    A list of token IDs to pass to the model.

  • multi_modal_data: typing_extensions.NotRequired[MultiModalData]

    Optional multi-modal data to pass to the model, if the model supports it.