vLLM Documentation

<h2 id="vllm-0271-and-vllm-omni-on-amd-rocm-714">vLLM 0.27.1 and vLLM-Omni on AMD ROCm 7.14</h2>
This 1-Click image ships two serving engines side by side on an AMD ROCm 7.14 host stack running Ubuntu 24.04, preconfigured for AMD Instinct GPUs.
vLLM 0.27.1 is a high-throughput, memory-efficient inference and serving engine for large language models, exposing an OpenAI-compatible API. It delivers state-of-the-art serving throughput with PagedAttention and continuous batching.
vLLM-Omni 0.27.0rc1 extends that engine to omni-modal models, covering text, audio, image and video, and ships its own <code>vllm-omni</code> command line interface. It is built from source against ROCm inside this image and is version-paired with vLLM 0.27.1.
Both engines live inside a single Docker container named <code>rocm</code>, alongside a JupyterLab environment with example notebooks. The login banner prints the JupyterLab URL and token plus ready-to-paste serve commands for both engines.
A note on ROCm versions, which are deliberately different and are not a defect: the host runs ROCm 7.14 for the driver and <code>amd-smi</code>, while the vLLM container carries ROCm 7.2.3, because upstream vLLM does not publish a ROCm 7.14 container image.
This image was validated on a live AMD Instinct MI300X. Verified on that node: a real vLLM completion served over the OpenAI-compatible API, a successful <code>vllm_omni</code> import reporting a ROCm platform, the <code>vllm-omni</code> CLI present, and a real GPU matrix multiplication.

vLLM

vLLM-Omni

ROCm

PyTorch

JupyterLab

Docker

vLLM 0.27.1 for high-throughput LLM serving plus vLLM-Omni 0.27.0rc1 for omni-modal text, audio, image and video inference, on an AMD ROCm 7.14 host, Ubuntu 24.04, ready for AMD Instinct accelerators.

AMD Developer Cloud

vLLM 0.27.1 + vLLM-Omni 0.27.0rc1 (ROCm 7.14)

Focus on building great things, while you use these 1-Click Apps to build your business.

These 1-Click Apps make it simple to deploy and manage your clients' infrastructure. Save time and resources, while reducing headaches and passing the value onto your clients.

Apps for agencies and website development shops

Work more effectively from home or remotely with these 1-Click Apps.

Work more effectively from home or remotely

Stop spending hours editing YAML and Helm charts. Instead, use these popular, preconfigured solutions to add apps to your Kubernetes cluster in minutes instead of days.

1-Click Apps for managing Kubernetes environments

Stay focused on teaching and learning, instead of managing infrastructure. 

1-Click Apps for students and educational institutions

SaaS Add-Ons make your DigitalOcean infrastructure better! Check out the full list of SaaS Add-Ons here!  

New Feature Alert: Introducing Software License Subscriptions on DigitalOcean Marketplace!

We now support Droplet 1-Click updates via API. See documentation for the details

Check out Tabnine, the AI coding assistant that accelerates & simplifies software development.

Description

Software Included

Getting started after deploying vLLM 0.27.1 + vLLM-Omni 0.27.0rc1 (ROCm 7.14)