README.md
Agent Lightning was completely refactored in v1.0. For legacy releases earlier than v1.0, see this branch.
The following is an example installation on a CUDA 13.0 machine:
cd <this-repo>
uv sync
bash scripts/setup_verl.sh 0.8.0 cu130
See the Installation Guide for details.
Agent Lightning v1.0 keeps the training architecture simple with three lightweight components:
verl and vLLM, builds training samples, and updates the policy.The Trainer creates rollouts, the Controller launches agents, and the Gateway turns interactions into training data, while agents continue to run with their real harnesses.
We evaluate Agent Lightning v1.0 across several practical training domains, including Search R1, LLM-in-Sandbox, and Coding Agent. Pure RL delivers substantial improvements across all three domains, as shown below.
<p align="center"> </p>| Section | Content |
|---|---|
| Installation | Base environment and verl GPU stack |
| Quick Start | Local first run and end-to-end flow |
| Basics | Components, rollouts, events, and trajectories |
| Trainer Configuration | verl integration and trace aggregation |
| API Gateway Configuration | Gateway and model proxy settings |
| Controller Configuration | Local and Kubernetes runners |
| Asynchronous Training | Collocated async collection and pause/drain |
| Example | Description |
|---|---|
| Calc-X | POC math reasoning example with AutoGen and MCP calculator tools, requiring only one GPU. |
| GSM8K | POC grade-school math reasoning example. |
| ScienceWorld | Interactive science tasks in a text-based environment. |
| Search-R1 | Multi-turn retrieval and reasoning agent. |
| LLM-in-Sandbox | General agent with computer and code execution tools. |
| Coding Agent | Coding agent trained with repository tests. |
If you find Agent Lightning useful in your research or projects, please cite our paper:
@misc{luo2025agentlightningtrainai,
title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning},
author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang},
year={2025},
eprint={2508.03680},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2508.03680},
}
This project welcomes contributions and suggestions. Start by reading the Contributing Guide for recommended contribution points, environment setup, branching conventions, and pull request expectations. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
This project has been evaluated and certified to comply with the Microsoft Responsible AI Standard. The team will continue to monitor and maintain the repository, addressing any severe issues, including potential harms, if they arise.
Agent Lightning v1.0 is released under the MIT License.