Examples

Production examples for deploying LLMs with Ray Serve.

Tutorials

Complete end-to-end tutorials for deploying different types of LLMs:

{doc}Deploy a small-sized LLM <../../_collections/serve/tutorials/deployment-serve-llm/small-size-llm/README>
{doc}Deploy a medium-sized LLM <../../_collections/serve/tutorials/deployment-serve-llm/medium-size-llm/README>
{doc}Deploy a large-sized LLM <../../_collections/serve/tutorials/deployment-serve-llm/large-size-llm/README>
{doc}Deploy a vision LLM <../../_collections/serve/tutorials/deployment-serve-llm/vision-llm/README>
{doc}Deploy a reasoning LLM <../../_collections/serve/tutorials/deployment-serve-llm/reasoning-llm/README>
{doc}Deploy a hybrid reasoning LLM <../../_collections/serve/tutorials/deployment-serve-llm/hybrid-reasoning-llm/README>
{doc}Deploy gpt-oss <../../_collections/serve/tutorials/deployment-serve-llm/gpt-oss/README>

{toctree}

:hidden:

../../_collections/serve/tutorials/deployment-serve-llm/small-size-llm/README
../../_collections/serve/tutorials/deployment-serve-llm/medium-size-llm/README
../../_collections/serve/tutorials/deployment-serve-llm/large-size-llm/README
../../_collections/serve/tutorials/deployment-serve-llm/vision-llm/README
../../_collections/serve/tutorials/deployment-serve-llm/reasoning-llm/README
../../_collections/serve/tutorials/deployment-serve-llm/hybrid-reasoning-llm/README
../../_collections/serve/tutorials/deployment-serve-llm/gpt-oss/README