sergiopaniego/trackio-training-agents-4-bucket
7.23 MB
Sergio Paniego PRO
sergiopaniego
AI & ML interests
None yet
Recent Activity
updated a collection about 11 hours ago
Training Agents · Class 4: RL Environments updated a collection about 11 hours ago
Training Agents · Class 4: RL Environments updated a collection about 11 hours ago
Training Agents · Class 4: RL EnvironmentsOrganizations
Posts 112
Post
2004
Can you do RL over taste?
I've spent some time reproducing, in the open, Surya N's idea of training a model to paint with code. It's a coding model that learns to paint watercolours by writing JS code, trained with GRPO. I used TRL and OpenEnv for this, with the whole pipeline running on Hugging Face.
The interesting part is that the reward has no correct answer, unlike a math problem. In this case it's based on the artistic preferences of the person who builds the dataset.
Everything is published: the environment, the reference pool, the trained adapters, every painting of every run with the code that made it, and a write-up with all the decisions, including the ones that went wrong.
Blog post: https://huggingface.co/blog/train-to-paint-with-code
I've spent some time reproducing, in the open, Surya N's idea of training a model to paint with code. It's a coding model that learns to paint watercolours by writing JS code, trained with GRPO. I used TRL and OpenEnv for this, with the whole pipeline running on Hugging Face.
The interesting part is that the reward has no correct answer, unlike a math problem. In this case it's based on the artistic preferences of the person who builds the dataset.
Everything is published: the environment, the reference pool, the trained adapters, every painting of every run with the code that made it, and a write-up with all the decisions, including the ones that went wrong.
Blog post: https://huggingface.co/blog/train-to-paint-with-code
spaces 191
pinned
Running on Zero
Agents
Featured
123
VLM Object Understanding
🦀
Explore object detection, visual grounding, keypoint Detecti
pinned
Runtime error
Agents
11
Qwen2-VL-7B
🔥
Ask questions about charts in images
pinned
Running on Zero
Agents
SmolVLM-trl-dpo-rlaif-v
🔥
Generate text from an image and question
pinned
Runtime error
Agents
SmolVLM-trl-sft-ChartQA
🔥
Ask questions about charts in images
Running
Huggingface Static 25ced2
🎯
View and explore your data with an interactive dashboard
Running on CPU Upgrade
RL
Coding Environment Server
💻
Run Python code in an interactive coding environment
models 141
sergiopaniego/qwen3-1.7b-mbpp-grpo
Text Generation • 2B • Updated • 587
sergiopaniego/rl-envs-youtube-livestream-4-scripts
Updated
sergiopaniego/qwen3-1.7b-wordle-grpo
Text Generation • 2B • Updated • 500
sergiopaniego/Qwen3-4B-claude-code-local-grpo
Text Generation • 4B • Updated • 10
sergiopaniego/Qwen3-4B-claude-code-deepcoder-grpo
Text Generation • 4B • Updated • 10
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B-judged
Text Generation • 2B • Updated • 9
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B
Text Generation • 2B • Updated • 10
sergiopaniego/Qwen3-8B-opencode-deepcoder-grpo
Text Generation • 8B • Updated • 22 • 1
sergiopaniego/grpo-youtube-livestream-3-scripts
Reinforcement Learning • Updated • 2
sergiopaniego/Qwen3.5-4B-sdpo-math-hints
Updated • 1
datasets 16
sergiopaniego/opencode-rollout-trace
Updated
sergiopaniego/pelican-svg-drawings
Viewer • Updated • 169 • 683 • 1
sergiopaniego/math-sdpo-hints-plain
Viewer • Updated • 600 • 26
sergiopaniego/math-sdpo-hints
Viewer • Updated • 600 • 32
sergiopaniego/gsm8k-sdpo-plain
Viewer • Updated • 700 • 9
sergiopaniego/gsm8k-sdpo-hints
Viewer • Updated • 700 • 30
sergiopaniego/pi-mono-chat
Viewer • Updated • 886 • 28
sergiopaniego/requests-pr-diff
Viewer • Updated • 1 • 120
sergiopaniego/trl-r2e-test
Viewer • Updated • 1 • 18
sergiopaniego/chain-sum-rollouts
Viewer • Updated • 50 • 14