Examples

Scripts

• API Client
• Aqlm Example
• Cpu Offload
• Fuyu Example
• Gradio OpenAI Chatbot Webserver
• Gradio Webserver
• Llava Example
• Llava Next Example
• LLM Engine Example
• Lora With Quantization Inference
• MultiLoRA Inference
• Offline Inference
• Offline Inference Arctic
• Offline Inference Distributed
• Offline Inference Embedding
• Offline Inference Mlpspeculator
• Offline Inference Neuron
• Offline Inference Tpu
• Offline Inference With Prefix
• OpenAI Chat Completion Client
• OpenAI Completion Client
• OpenAI Embedding Client
• OpenAI Vision API Client
• Paligemma Example
• Phi3V Example
• Save Sharded State
• Tensorize vLLM Model