MirroS-Lab/AgentGarten-renderer

AgentGarten renderer

The real-time neural renderer of AgentGarten: Code Worlds for Evolving Agents. A code world (a simulator or a game engine) keeps the state and runs the rules; this model turns the depth and surface normals it exports into the observations an agent sees, block by block.

• Blog: https://mirros.ai/blog/worlds-for-evolving-agents
• Technical report: https://mirros.ai/report/agent-garten.pdf
• Project page: https://mirros-lab.github.io/agent-garten
• Code: https://github.com/MirroS-Lab/AgentGarten

What is in this repository

File	Contents
student.safetensors	The few-step autoregressive generation tower (7.0 B parameters, BF16), distilled with Adversarial Forcing
und.safetensors	The frozen text tower of Cosmos3-Nano, unchanged
text_tokenizer/	The Cosmos3-Nano text tokenizer, unchanged
config.yaml, manifest.json	Network configuration and the file list the loader reads

The model starts from Cosmos3-Nano. It was adapted to depth and normal conditions, trained as a block-causal autoregressive model, and distilled into a four-step student. Video is encoded and decoded with the Wan 2.2 VAE, which is not included here.

Use

Install the AgentGarten code, then:

hf download MirroS-Lab/AgentGarten-renderer --local-dir weights/AgentGarten-renderer
hf download Wan-AI/Wan2.2-TI2V-5B Wan2.2_VAE.pth --local-dir weights/Wan2.2-TI2V-5B

from wm.inference.cosmos3 import inspect_artifact, load_artifact
from wm.inference.serving import prepare_serving
from wm.networks.cosmos3.streaming import Cosmos3Stream, StreamPolicy

artifact = load_artifact(inspect_artifact("weights/AgentGarten-renderer"))
prepare_serving(artifact.student)
stream = Cosmos3Stream(artifact.student, StreamPolicy())

The repository's README shows the streaming loop: each block of four latent frames is denoised in four steps and then committed to the key-value cache, so the conditions of the next block can depend on what was just generated.

License

The weights derive from Cosmos3-Nano and are released under the same OpenMDW License Agreement, version 1.1. The text tower and the tokenizer are NVIDIA's files, redistributed unchanged under that license.