思维导图备注
vLLM v0.6.2 Documentation
首页 AI助手 白天 夜间 BookChat 小程序 小程序 阅读
  • 书签 我的书签
  • 添加书签 添加书签 移除书签 移除书签

Getting Started

 Sponsor  来源:vLLM 浏览 821 扫码 2025-02-10 14:08:55
  • Installation
  • Installation with ROCm
  • Installation with OpenVINO
  • Installation with CPU
  • Installation with Neuron
  • Installation with TPU
  • Installation with XPU
  • Quickstart
  • Debugging Tips
  • Examples
当前内容版权归 vLLM 或其关联方所有，如需对内容或内容相关联开源项目进行关注与资助，请访问 vLLM .
上一篇:
下一篇:
  • 书签
  • 添加书签 移除书签
  • vLLM Recipes
  • vLLM v0.7.2 Documentation
  • vLLM v0.7.1 Documentation
  • vLLM v0.7.0 Documentation
  • vLLM v0.6.6 Documentation
  • vLLM v0.6.5 Documentation
  • vLLM v0.6.4 Documentation
  • vLLM v0.6.3 Documentation
  • vLLM v0.6.2 Documentation
  • vLLM v0.6.1 Documentation
  • vLLM v0.6.0 Documentation
  • vLLM v0.5.5 Documentation
  • vLLM v0.5.4 Documentation
  • vLLM v0.5.3 Documentation
  • vLLM v0.5.2 Documentation
  • vLLM v0.5.1 Documentation
  • vLLM v0.5.0 Documentation
  • vLLM v0.4.3 Documentation
  • vLLM v0.4.2 Documentation
  • vLLM v0.4.1 Documentation
  • Getting Started
    • Installation
      • Installation with ROCm
        • Installation with OpenVINO
          • Installation with CPU
            • Installation with Neuron
              • Installation with TPU
                • Installation with XPU
                  • Quickstart
                    • Debugging Tips
                      • Examples
                        • API Client
                          • Aqlm Example
                            • Cpu Offload
                              • Gguf Inference
                                • Gradio OpenAI Chatbot Webserver
                                  • Gradio Webserver
                                    • LLM Engine Example
                                      • Lora With Quantization Inference
                                        • MultiLoRA Inference
                                          • Offline Chat With Tools
                                            • Offline Inference
                                              • Offline Inference Arctic
                                                • Offline Inference Audio Language
                                                  • Offline Inference Chat
                                                    • Offline Inference Distributed
                                                      • Offline Inference Embedding
                                                        • Offline Inference Encoder Decoder
                                                          • Offline Inference Mlpspeculator
                                                            • Offline Inference Neuron
                                                              • Offline Inference Neuron Int8 Quantization
                                                                • Offline Inference Pixtral
                                                                  • Offline Inference Tpu
                                                                    • Offline Inference Vision Language
                                                                      • Offline Inference Vision Language Multi Image
                                                                        • Offline Inference With Prefix
                                                                          • Offline Inference With Profiler
                                                                            • OpenAI Audio API Client
                                                                              • OpenAI Chat Completion Client
                                                                                • OpenAI Chat Completion Client With Tools
                                                                                  • OpenAI Completion Client
                                                                                    • OpenAI Embedding Client
                                                                                      • OpenAI Vision API Client
                                                                                        • Save Sharded State
                                                                                          • Tensorize vLLM Model
                                                                                        • Serving
                                                                                          • OpenAI Compatible Server
                                                                                            • Deploying with Docker
                                                                                              • Distributed Inference and Serving
                                                                                                • Production Metrics
                                                                                                  • Environment Variables
                                                                                                    • Usage Stats Collection
                                                                                                      • Integrations
                                                                                                        • Deploying and scaling up with SkyPilot
                                                                                                          • Deploying with KServe
                                                                                                            • Deploying with NVIDIA Triton
                                                                                                              • Deploying with BentoML
                                                                                                                • Deploying with Cerebrium
                                                                                                                  • Deploying with LWS
                                                                                                                    • Deploying with dstack
                                                                                                                      • Serving with Langchain
                                                                                                                        • Serving with llama_index
                                                                                                                        • Loading Models with CoreWeave’s Tensorizer
                                                                                                                          • Frequently Asked Questions
                                                                                                                          • Models
                                                                                                                            • Supported Models
                                                                                                                              • Adding a New Model
                                                                                                                                • Enabling Multimodal Inputs
                                                                                                                                  • Engine Arguments
                                                                                                                                    • Using LoRA adapters
                                                                                                                                      • Using VLMs
                                                                                                                                        • Speculative decoding in vLLM
                                                                                                                                          • Performance and Tuning
                                                                                                                                          • Quantization
                                                                                                                                            • Supported Hardware for Quantization Kernels
                                                                                                                                              • AutoAWQ
                                                                                                                                                • BitsAndBytes
                                                                                                                                                  • GGUF
                                                                                                                                                    • INT8 W8A8
                                                                                                                                                      • FP8 W8A8
                                                                                                                                                        • FP8 E5M2 KV Cache
                                                                                                                                                          • FP8 E4M3 KV Cache
                                                                                                                                                          • Automatic Prefix Caching
                                                                                                                                                            • Introduction
                                                                                                                                                              • Implementation
                                                                                                                                                              • Performance benchmarks
                                                                                                                                                                • Benchmark suites of vLLM
                                                                                                                                                                • Developer Documentation
                                                                                                                                                                  • Sampling Parameters
                                                                                                                                                                    • Offline Inference
                                                                                                                                                                      • LLM Class
                                                                                                                                                                        • LLM Inputs
                                                                                                                                                                        • vLLM Engine
                                                                                                                                                                          • LLMEngine
                                                                                                                                                                            • AsyncLLMEngine
                                                                                                                                                                            • vLLM Paged Attention
                                                                                                                                                                              • Input Processing
                                                                                                                                                                                • Input Processing Pipeline
                                                                                                                                                                                • Multi-Modality
                                                                                                                                                                                  • Adding a Multimodal Plugin
                                                                                                                                                                                  • Dockerfile
                                                                                                                                                                                    • Profiling vLLM
                                                                                                                                                                                    • Community
                                                                                                                                                                                      • vLLM Meetups
                                                                                                                                                                                        • Sponsors
                                                                                                                                                                                        暂无相关搜索结果！

                                                                                                                                                                                          本文档使用 BookStack 构建

                                                                                                                                                                                          文章二维码

                                                                                                                                                                                          手机扫一扫，轻松掌上读

                                                                                                                                                                                          文档下载

                                                                                                                                                                                          • 普通下载
                                                                                                                                                                                          • 下载码下载(免登录无限下载)
                                                                                                                                                                                          你与大神的距离，只差一个APP
                                                                                                                                                                                          APP下载
                                                                                                                                                                                          请下载您需要的格式的文档，随时随地，享受汲取知识的乐趣！
                                                                                                                                                                                          PDF文档 EPUB文档 MOBI文档
                                                                                                                                                                                          温馨提示 每天每在网站阅读学习一分钟时长可下载一本电子书，每天连续签到可增加阅读时长
                                                                                                                                                                                          下载码方式下载：免费、免登录、无限制。 免费获取下载码

                                                                                                                                                                                          微信小程序阅读

                                                                                                                                                                                          BookChat 微信小程序阅读
                                                                                                                                                                                          您与他人的薪资差距，只差一个随时随地学习的小程序

                                                                                                                                                                                          书签列表

                                                                                                                                                                                            阅读记录

                                                                                                                                                                                            阅读进度: 0.00% ( 0/0 ) 重置阅读进度
                                                                                                                                                                                              欢迎使用AI助手 AI助手
                                                                                                                                                                                              全屏 缩小 隐藏 清空