---
format: "aidr-story-markdown/v1"
id: "69c86193bb92b071bc97839f95fc10b7c61cf68aea7d482c84d7ae8634a3cf91"
canonical_url: "https://aidr.today/69c86193?lang=en"
title: "StructRL: Model-Free Reinforcement Learning with Replay Graphs"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-11-13T05:00:00.000Z"
category: "Research"
topics: ["agent"]
source_urls: ["https://arxiv.org/abs/2611.14599"]
summary: "StructRL turns reinforcement learning experience into an explicit replay graph. For DQN on three sparse-reward MiniGrid tasks, final success rose from 0.10 to 0.90 for DoorKey, 0.10 to 0.80 for KeyCorridor, and 0 to 0.50 for ObstructedMaze. For SAC on continuous PointMaze, graph-derived direction labels raised final success from 0.17 to 1.0 on seven matched runs."
---

# StructRL: Model\-Free Reinforcement Learning with Replay Graphs

> [Open the canonical story](<https://aidr.today/69c86193?lang=en>)

**Published:** 2026-11-13T05:00:00.000Z
**Category:** Research
**Topics:** agent

## Summary

StructRL turns reinforcement learning experience into an explicit replay graph\. For DQN on three sparse\-reward MiniGrid tasks, final success rose from 0\.10 to 0\.90 for DoorKey, 0\.10 to 0\.80 for KeyCorridor, and 0 to 0\.50 for ObstructedMaze\. For SAC on continuous PointMaze, graph\-derived direction labels raised final success from 0\.17 to 1\.0 on seven matched runs\.

## Sources

- [Story source](<https://arxiv.org/abs/2611.14599>)