RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

Papers

arxiv:2609.15364

Published on Sep 14

· Submitted by

shicheng on Sep 15

· AetherLabs-AI

Upvote

81

Authors:

,

Shicheng Fan ,

,

,

,

Abstract

RSIAgent is a training-free multi-agent framework that enables recursive self-improvement via autonomous memory construction and broad-then-deep exploration to adapt digital agents to new environments.

Generated by thinkingmachines/Inkling-Small

Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce RSIAgent, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a broad-then-deep exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.

View arXiv page View PDF Project pageGitHub 396 Add to collection

Community

Shichengf

Paper author Paper submitter 8 days ago

We’re excited to share RSIAgent: Autonomous Exploration for Recursive Self-Improvement in New Environments!

Instead of continuing to scale model parameters, we explore a different path: Scaling Experience. RSIAgent enables agents to autonomously decide what to explore, execute tasks, verify outcomes, and consolidate stable action–condition–outcome relationships into reusable memory, all while keeping the underlying model weights frozen.

RSIAgent coordinates curriculum, actor, and verifier agents through a Broad-to-Deep exploration strategy: broad exploration builds coverage across tools, workflows, and failure modes, while deep exploration targets difficult cases, hidden constraints, and boundary conditions.

When applied to Kimi-K3 and GLM-5.3, RSIAgent achieves:

• 78.98% Partial Score on OSWorld 2.0 (0808 offline), compared with 72.60% for GPT-6 Astra

• 84.82% on Agents’ Last Exam (Near-term), compared with 82.26% for GPT-6 Astra

These results suggest that agents can continue improving by autonomously acquiring, verifying, and reusing their own experience—even without updating model parameters.

• Blog: https://aetherlabs.ai/articles/rsiagent-autonomous-exploration-for-recursive-self-improvement

• Code: https://github.com/AetherLabsAI/RSIAgent

• Project page: https://aetherlabsai.github.io/RSIAgent/

• Paper: https://arxiv.org/abs/2609.15364

We’d love to hear your feedback and are happy to answer any questions!

librarian-bot

7 days ago

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents (2026)

• NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness (2026)

• AREX: Towards a Recursively Self-Improving Agent for Deep Research (2026)

• Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses (2026)

• SimSkill: A Self-Evolving LLM Agent for Skill and Knowledge Accumulation in Traffic Simulation (2026)

• PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents (2026)

• ADIAS: Automated Design of Interactive Agentic Systems (2026)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on HF Mirror checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

Upvote

81

Get this paper in your agent:

hf papers read 2609.15364

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2609.15364 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2609.15364 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2609.15364 in a Space README.md to link it from this page.

Collections including this paper 2