NetHack Learning Environment as a Frontier Benchmark for General Agents

The NetHack Learning Environment (NLE) has been sitting at the frontier of general-agent research since its 2020 debut, and six years on, no method has come close to solving the game it presents. That staying power is the point of this piece. NLE isn't hard because it's complicated in the way a big codebase is complicated; it's hard because of a specific combination of properties that expose the actual weaknesses of every current agent paradigm, RL, large language models, symbolic systems, and every hybrid tried so far. Understanding why NLE resists progress tells you something concrete about what's still missing from "general" AI agents.

Researchers from Facebook AI Research, UCL, NYU, Oxford, and Imperial College London built NLE on top of NetHack 3.6.6 and presented it at NeurIPS 2020. NetHack itself is a decades-old open-source terminal roguelike with a long competitive community history, including the /dev/null/nethack tournament that ran from 1999 to 2016 and its successor, the November NetHack Tournament. The dungeons are procedurally generated, up to 50 levels deep, populated with hundreds of monster and item types that interact in ways that are often stochastic and rarely intuitive. The goal is simple to state: descend, grab the Amulet of Yendor, come back up alive. Getting there is not simple at all, and death is permanent: every mistake, however small, can send an agent back to a brand new world with nothing carried over.

The original paper was explicit about its dual aim: build something complex enough to drive years of research into exploration, planning, skill acquisition, and language-conditioned RL, while keeping it far cheaper to run than other testbeds demanding similar rigor. Formally, NLE is a stochastic, partially observable Markov decision process (POMDP) wrapped in a Gym-compatible API. The action space runs to 98 primitive actions in NLE's default setup (76 in the commonly used subset, up to 121 in the full keyboard-driven version), covering movement, compound commands, and menu navigation. Observations come as glyph grids, color and character grids, scalar status features, text messages, and inventory encodings. The baseline agent shipped with the original release used TorchBeast and showed early promise, alongside an immediate and stark gap against expert human play that has never really closed.

The structural properties that make NLE genuinely hard, not just complicated

No single feature of NLE is what breaks agents. It's the conjunction of five things happening at once: procedural generation, partial observability, stochastic dynamics, an extremely long horizon, and a combinatorial space of entities that interact contextually.

Procedural generation means every episode hands the agent a fresh dungeon seed. There is no floor plan to memorize, no fixed sequence of encounters to learn by rote. Procedural generation means every episode hands the agent a fresh dungeon seed, and generalization is the only way through. The agent only sees what its character can currently perceive, so partial observability compounds this: information about unexplored rooms, hidden passages, or nearby monsters has to be inferred, not read off a full map.

Then there's the problem of the long horizon. Matthews et al. (2026) note that successful human ascensions typically run for a very large number of turns, spanning a wide range. Standard RL methods rely on reward signals to shape behavior, and when the payoff for a decision made early in a run doesn't appear until many turns later, the gradient information connecting the two is close to useless. Adding stochastic dynamics, monster behavior, item identification, spell effects, and dozens of other systems that involve genuine randomness causes deterministic planning to break down fast. Layering on a combinatorial entity space, hundreds of object and monster types whose interactions depend heavily on context, makes effective play require something like semantic world knowledge: what a scroll of fire does, when a potion is safe to drink, which enemies to avoid at low health.

Permadeath raises the stakes on all of it. A single wrong move after ten thousand correct ones can wipe the slate clean. Human players don't do this from scratch either: they lean hard on external references like the NetHack Wiki and community strategy write-ups, and even with that help, mastering the game takes months or years. NLE keeps that difficulty intact by design rather than smoothing it over for the sake of faster benchmark progress.

The task suite and the reward design trap agents fall into

NLE ships with a full-game task called Score, where the objective is simply to rack up as much in-game score as possible. It's sparse and long-horizon, which sounds straightforward until you see what agents actually do with it. Alongside Score sit modular subgoal tasks with dense, event-driven rewards: Staircase, Pet, Eat, Gold, Scout, Oracle. Each isolates one capability, navigation, inventory management, exploration under partial observability, so researchers can test components separately rather than only judging the whole game at once.

Reward functions in NLE are configurable: score deltas, +1 for each newly revealed tile, +100 for reaching a new staircase down, nutrition increments, or completion of specific in-game events. That flexibility is useful, but it also raises a problem that appears repeatedly across the NLE literature, from the original Küttler et al. 2020 paper through Hambro et al. 2022 and on to Matthews et al. 2026: the in-game score metric is trivially gameable. An agent can park itself on dungeon level one and farm weak enemies indefinitely, accumulating points forever without descending a single floor.

That's not a hypothetical edge case; it's the default failure mode. An agent optimizing for score can look like it's improving on the benchmark's headline number while learning a policy that would never ascend under any amount of additional training. Matthews et al. (2026) now recommend the Scout reward, +1 per newly revealed map tile, as a more honest stand-in for real progress, since farming a single room can't inflate it. SkillHack (Matthews et al., 2022) takes a different approach, using curated MiniHack-based tasks with sparse terminal rewards, each built around a human-interpretable skill like PickUp, Wear, or FreezeLava, specifically to measure skill acquisition and transfer rather than score accumulation. The lesson that keeps resurfacing across all this work is that defining "progress" in NLE takes real care. Get the reward wrong, and you train the wrong behavior at scale, confidently and efficiently.

The paradigm gap revealed by the NeurIPS 2021 NetHack Challenge

The 2021 challenge asked participants to build a program or agent, using any method at all, that could ascend in NetHack through NLE. Neural RL, hand-coded symbolic bots, hybrids, all fair game. The result reported by Hambro et al. was blunt: symbolic bots beat deep RL methods by a factor of five.

Real progress happened at the community level. Many entries beat the previous NLE state of the art by a wide margin. But the gap to actually winning the game stayed enormous, and no entry came anywhere near an ascension. If anything, that failure made the case for NLE as a long-term benchmark stronger, not weaker: even the best systems submitted by a large, motivated research community couldn't clear the game's most basic win condition.

Symbolic NetHack bots have a long history, SWAGGINZZZ, TAEB, BotHack among them, but many of their past wins leaned on bugs in older NetHack versions that have since been patched. As far as the challenge organizers knew, no bot had ever ascended in version 3.6.6. The challenge's NLE variant also widened the action space from 23 actions to 113, exposing agents to more of the game's real complexity rather than a simplified slice of it.

The factor-of-five gap cuts against a common assumption in machine learning, that learned neural policies eventually overtake hand-coded rules given enough scale. In NLE, at least at current compute levels, the opposite held. The game's knowledge requirements and its long time horizons favor systems that encode rules explicitly over systems that have to discover them from scratch through trial and error.

Why reinforcement learning alone stalls in NLE

Credit assignment is the first wall RL hits. When a reward shows up a great many turns after the decision responsible for it, standard policy gradient and Q-learning methods get almost no useful signal connecting the two. The math behind these methods assumes reward is close enough in time to action that the relationship is learnable; NLE routinely violates that assumption by orders of magnitude.

Exploration makes it worse, not better. The dungeon is vast, different every episode, and only partially visible at any moment. Random exploration or curiosity-driven bonuses reach a small sliver of the relevant state space before permadeath ends the episode and resets the clock. Large-scale RL runs, even with heavy compute behind them, get stuck in the earliest stages of the game. That pattern holds consistently: in the original NeurIPS 2020 baseline, in the 2021 challenge results, and in follow-on work through ICLR 2026.

The score-farming trap described earlier hits RL especially hard, because RL optimizers are good at finding whatever produces dense reward, and farming weak monsters on level one produces exactly that. Genuine progress through the dungeon yields reward that's sparse and delayed by comparison, so the optimizer, doing what it's built to do, finds and exploits the degenerate loop instead.

Hierarchical and skill-based approaches try to work around the long horizon by breaking the game into learnable subskills. SkillHack is one example; various other hierarchical approaches have been proposed as well. Both outperform flat RL baselines, but neither closes the distance to human-level play. One promising angle comes from Motif (Klissarov et al., ICLR 2024), which uses an LLM's preferences over captioned gameplay snippets to build an intrinsic reward signal, then trains an RL agent purely to maximize that signal. Motif ends up beating an agent trained directly on game score, which suggests the LLM's implicit sense of what "good" gameplay looks like is a better progress proxy than the score metric NLE ships with by default.

The limits of LLM and VLM agents dropped into NLE

NetPlay, released in 2024, was the first zero-shot LLM agent built for NLE. It works by prompting the model to pick from a set of predefined skills while tracking the history of past interactions. The agent never got past the first few levels. That failure matters because it shows zero-shot LLM reasoning doesn't solve the problem just by adding skill abstractions on top.

BALROG, published at ICLR 2025 by Paglieri et al., took a broader approach: a unified benchmark running LLM and VLM agents across BabyAI, Crafter, TextWorld, Baba Is AI, MiniHack, and NetHack, spanning tasks a non-expert could solve in seconds up to tasks that take years to master. The findings split cleanly. Models manage partial success on the easier games, but performance drops sharply on the harder ones, NetHack worst of all. More surprising: models do worse with visual representations of the environment than with text, which cuts against the assumption that vision-language models are a natural step up from text-only agents for game-like tasks. And because NLE's procedural generation makes repeat instances vanishingly unlikely, models can't lean on memorized solutions the way they sometimes can on static benchmarks.

A community blog post from January 2026 describes BRAID, a fork of BALROG built around a more modern agentic loop, reaching 6.96% progression using Claude Opus 4.5, described there as possibly the highest LLM progression on NLE recorded at that point. That result hasn't gone through peer review, so it should be read as a data point marking a rough ceiling.

LLMs carry a lot of declarative knowledge about NetHack, thanks to the Wiki, strategy guides, and item descriptions sitting somewhere in their training data. What they can't do reliably is turn that knowledge into a working sequence of actions across thousands of turns inside a stochastic, partially observed environment. Knowing what a scroll of genocide does is not the same as knowing when, in this exact run, to read one. NetPlay's specific failure, struggling with ambiguous task phrasing and a lack of clear feedback, points to a wider truth: LLMs need structured, legible feedback loops to act competently, and NetHack's feedback is, by design, sparse, noisy, and often cryptic.

What symbolic methods get right but still cannot ascend with

Symbolic bots win by encoding decades of accumulated community knowledge directly into rules: what to pick up, which monsters to run from at low health, how to manage hunger, when it's safe to head down another level. That's effectively a compressed, hand-built version of the NetHack Wiki running as a policy, and it explains the factor-of-five edge these bots held over deep RL in the 2021 challenge.

The limits appear just as clearly, though. Symbolic bots can't generalize to combinations of entities their rule-writers never anticipated, they don't learn from experience, and their heuristics break the moment NetHack's procedural generation throws something genuinely novel at them, which it does constantly by design. Older bots like SWAGGINZZZ, TAEB, and BotHack sometimes relied on exploits in older NetHack versions that have since been patched out. The current version of the game is a harder target for rule-based systems than the versions those bots were built against.

Hybrid systems, symbolic logic paired with neural components or LLMs, are the current frontier for actually pushing progress forward. But as of the ICLR 2026 Blog Track in late April 2026, no hybrid approach has made significant headway toward completing the game either. Matthews et al., writing for that same venue, state that no method tried so far, RL, large pretrained models, hand-coded symbolic agents, imitation learning, or any combination of these, has made significant progress toward finishing NetHack.

The interface problem: how NLE's own design was hiding complexity from agents

Matthews, D'Oro, Kanervisto, Fujimoto, Foerster, and Henaff presented a finding at ICLR 2026 (April 25 to 27) that reframes a chunk of this story: some of NLE's apparent difficulty wasn't coming from the game itself, it was coming from the interface NLE used to expose the game to agents. Their core claim is that the interface, as originally built, made many basic and necessary behaviors intractable or outright impossible to express.

Two bottlenecks stand out. First, the observation space leaves out information the game clearly tracks internally but does not present in a form an agent can actually use. Second, the primitive action set forces agents to spell out complex behaviors as long chains of tiny atomic actions, chains that are genuinely hard to learn through trial and error.

One striking discovery: when agents use the full action space, they can learn macro-actions that compress a lot of in-game time into a short action string. The sequence "9, 9, s," for instance, tells the game to wait and heal for 99 timesteps in one shot, effectively packaging a whole recovery behavior into a handful of keystrokes. That's not a trivial fix, it meaningfully eases the long-horizon credit assignment problem RL struggles with. The team validated their proposed interface changes on a custom MiniHack setup and found that combining all the modifications improved performance for an RL agent trained from scratch using Sample Factory PPO.

The implication runs deeper than a technical patch. Some of the intractability researchers assumed was inherent to NetHack was actually a measurement artifact, the interface was filtering out agent capability as much as it was testing it. Fix the interface, and you get a clearer read on both how much real progress is possible and how much of the gap is genuine. That distinction matters well beyond NLE: in any real-world agent deployment, observation and action spaces are engineered choices, not facts of nature, and getting them wrong can make a solvable problem look unsolvable.

The NetHack Learning Dataset: what roughly 13 billion transitions do and don't solve

The NetHack Learning Dataset adds close to 13 billion transitions to the ecosystem around NLE, about 10 billion drawn from human play and roughly 3 billion from bot play. That's an enormous trove by the standards of interactive benchmarks, and it opens the door to imitation learning and offline RL approaches that don't require an agent to learn everything through live trial and error inside the game.

What it doesn't do is hand anyone a shortcut past the structural problems described above. A dataset of past play captures specific trajectories through specific procedurally generated worlds, and NLE's whole premise is that no two worlds look alike. An agent can study 10 billion human transitions and still meet a dungeon layout, monster combination, or item interaction it's never seen represented in that data. Imitation learning built on this dataset can teach an agent to mimic patterns of competent play, picking up useful items, avoiding obviously dangerous fights, managing hunger, but pattern-matching to prior play is not the same as the on-the-fly reasoning NetHack demands when something unfamiliar appears.

The bot-play portion of the dataset carries its own asterisk, too. If some of that data comes from symbolic bots with the same generalization limits described earlier, an agent trained on it risks inheriting those limits rather than transcending them. The dataset is a genuinely valuable resource for studying how agents learn from demonstration at scale, and for testing offline RL methods against a task where online exploration is punishingly expensive. But it sits alongside the other findings here rather than resolving them: procedural generation, partial observability, long horizons, and NetHack's combinatorial complexity remain fully intact regardless of how much recorded play an agent gets to study beforehand. Solving NLE was never going to be a matter of data volume alone, and the dataset's existence, useful as it is, doesn't change that.