vLLM Documentation

<h2 id="vllm-0230-on-amd-rocm-724">vLLM 0.23.0 on AMD ROCm 7.2.4</h2>
<p>This 1-Click image ships vLLM 0.23.0, a high-throughput and memory-efficient inference and serving engine for large language models, running on an AMD ROCm 7.2.4 host stack on Ubuntu 24.04. It is preconfigured for AMD Instinct GPUs and packaged as a ready-to-run Docker container with the OpenAI-compatible vLLM server, alongside a JupyterLab environment with example notebooks.</p>
<p>vLLM delivers state-of-the-art serving throughput with PagedAttention, continuous batching, and an OpenAI-compatible API.</p>

vLLM

ROCm

JupyterLab

Docker

vLLM 0.23.0 high-throughput LLM inference and serving on AMD ROCm 7.2.4 host, Ubuntu 24.04, ready for AMD Instinct accelerators.

AMD Developer Cloud

vLLM 0.23.0 (ROCm 7.2.4)

Focus on building great things, while you use these 1-Click Apps to build your business.

These 1-Click Apps make it simple to deploy and manage your clients' infrastructure. Save time and resources, while reducing headaches and passing the value onto your clients.

Apps for agencies and website development shops

Work more effectively from home or remotely with these 1-Click Apps.

Work more effectively from home or remotely

Stop spending hours editing YAML and Helm charts. Instead, use these popular, preconfigured solutions to add apps to your Kubernetes cluster in minutes instead of days.

1-Click Apps for managing Kubernetes environments

Stay focused on teaching and learning, instead of managing infrastructure. 

1-Click Apps for students and educational institutions

Secure your network with a VPN, protect your database, and more, with this selection of 1-Click Apps.

SaaS Add-Ons make your DigitalOcean infrastructure better! Check out the full list of SaaS Add-Ons here!  

New Feature Alert: Introducing Software License Subscriptions on DigitalOcean Marketplace!

We now support Droplet 1-Click updates via API. See documentation for the details

Check out Tabnine, the AI coding assistant that accelerates & simplifies software development.

Description

Software Included

Getting started after deploying vLLM 0.23.0 (ROCm 7.2.4)