Inference Optimized image simplifies the setup and deployment of large language models (LLMs) by leveraging Docker and vLLM, with built-in support for Hugging Face model downloads, speculative decoding, prompt caching and FP8 quantization.
Users run one model on a single GPU, with launch settings optimized for hardware utilization. If a model isn’t already cached locally, it is automatically downloaded and stored for future use.
The image also includes special handling for FP8 quantization, enabling efficient low-precision inference. Speculative decoding is fully supported, including the use of draft models to enhance performance.
This makes it a powerful, plug-and-play solution for scalable and optimized LLM inference.
<h2 id="models-available-h100--h200-single-gpu">Models Available (H100 / H200, Single GPU)</h2>
<ul>
<li><code>Qwen/Qwen3Guard-Gen-0.6B</code></li>
<li><code>Qwen/Qwen3Guard-Gen-4B</code></li>
<li><code>Qwen/Qwen3Guard-Gen-8B</code></li>
<li><code>meta-llama/Llama-3.1-8B</code></li>
<li><code>meta-llama/Llama-3.1-8B-FP8</code></li>
<li><code>mistralai/Mistral-Nemo-Instruct-2407</code></li>
<li><code>mistralai/Mistral-Nemo-Instruct-2407-FP8</code></li>
<li><code>openai/gpt-oss-20b</code></li>
<li><code>openai/gpt-oss-120b</code></li>
<li><code>Qwen/Qwen3.6-27B-FP8</code> — TextOnly (32K), ToolCall, Full multimodal (128K), Full+ToolCall</li>
<li><code>Qwen/Qwen3.6-35B-A3B-FP8</code> — TextOnly (32K), ToolCall</li>
<li>Any custom model available on Hugging Face (manual specification)</li>
</ul>
<blockquote>
 ⚠️ Licenses: Please check the respective model licenses on Hugging Face. 
 Many models require license acceptance before running the automation script.
</blockquote>
<hr/>
<h2 id="hardware-support">Hardware Support</h2>
<ul>
<li>Only supports DigitalOcean GPU Droplets with Nvidia H100 or H200 GPUs</li>
<li>Single-GPU configuration (1 model per droplet)</li>
</ul>
<hr/>
<h2 id="huggingface-token">HuggingFace Token</h2>
<ul>
<li>A HuggingFace Token with Read access is required for downloading the models. Note that some models require applications on HuggingFace website to be downloaded.</li>
<li>The Hugging Face tokens are stored at /.cache/huggingface directory, under the stored_tokens and token file. Please remove them prior to sharing your droplet to others.</li>
</ul>

Ubuntu	

CUDA Toolkit	

NVIDIA Driver

vLLM (vllm-openai container)

Inference Optimized image simplifies the setup and deployment of large language models (LLMs) by leveraging Docker and vLLM, with built-in support for Hugging Face model downloads, speculative decoding, prompt caching and multi-model concurrency.

DigitalOcean AI

1-Click Inference Ready - Single GPU

Focus on building great things, while you use these 1-Click Apps to build your business.

These 1-Click Apps make it simple to deploy and manage your clients' infrastructure. Save time and resources, while reducing headaches and passing the value onto your clients.

Apps for agencies and website development shops

Work more effectively from home or remotely with these 1-Click Apps.

Work more effectively from home or remotely

Stop spending hours editing YAML and Helm charts. Instead, use these popular, preconfigured solutions to add apps to your Kubernetes cluster in minutes instead of days.

1-Click Apps for managing Kubernetes environments

Stay focused on teaching and learning, instead of managing infrastructure. 

1-Click Apps for students and educational institutions

Secure your network with a VPN, protect your database, and more, with this selection of 1-Click Apps.

SaaS Add-Ons make your DigitalOcean infrastructure better! Check out the full list of SaaS Add-Ons here!  

New Feature Alert: Introducing Software License Subscriptions on DigitalOcean Marketplace!

We now support Droplet 1-Click updates via API. See documentation for the details

Check out Tabnine, the AI coding assistant that accelerates & simplifies software development.

Description

Software Included

Getting started after deploying 1-Click Inference Ready - Single GPU

API Creation