Distributed taskPower grid restoration
Power Outage
Run the sixteen control rooms of a national grid that has gone completely dark, and write the controllers and the protocol that bring its power back.
A team of eight scores 0.05 more than a single agent: 0.12 comes from having more agents and coordination takes 0.07 away, and only Opus 5.5 gains from coordination.
Figure 1
Coordination at four hours
- single agent
- gain from more agents
- gain from coordination
- loss from coordination
- team of eight agents
Part 1. The task
Background
When a whole grid goes dark, most of its power plants cannot restart on their own, because they need power from the grid to start. Operators begin from the few black-start units that can, energise a cranking path from them to the next plant, pick up blocks of load small enough that the frequency holds, and join the islands they build once frequency, voltage and phase match on both sides. North American rules require every transmission operator to plan these cranking paths in advance and to test each black-start unit at least once every three calendar years.1 PJM's restoration manual gives rules of thumb for the largest block a unit can pick up at once: about 5% of its rating for a fossil steam unit, 15% for hydro and 25% for a combustion turbine.2
Real restorations take hours, and they rarely go to plan. After South Australia went dark in 2016, 80 to 90% of its restorable load was back after 7.7 hours.3 In the Iberian blackout of 2025, both Portuguese black-start plants started only after failed attempts,4 and Spain had 35% of its demand back after 8 hours and 99% after 17.5 hours.5
Restoration is also split by geography. Each control room knows its own plants, lines and crews in detail and its neighbours only through what they tell it, and a line between two areas can only be closed when both agree. A national restoration is a protocol between control rooms as much as a sequence of switching steps.
- NERC EOP-005-3, System Restoration from Blackstart Resources (WECC copy). wecc.org
- PJM Manual 36: System Restoration, Revision 19. pjm.com
- AER, Black System Event Compliance Report fact sheet. aer.gov.au
- ENTSO-E Expert Panel initiates the investigation (restoration timeline). entsoe.eu
- El 100% de las subestaciones de la red de transporte en servicio (REE figures), 29 Apr 2025. energias-renovables.com
The task
Each agent starts in its own sandbox with the same instruction: the team runs the 16 control rooms of a national grid that has just gone completely dark, and writes the restoration, a controller for every control area and the protocol between the areas. The grid has 538 substations, 712 lines (48 of them between areas), 118 generating units including three nuclear plants, and 1,503 feeders. Six areas have black-start units; the other ten need a neighbour, and the three nuclear plants sit in areas that cannot start themselves and must get cooling power back before a deadline.
The 16 areas are dealt out among the agents, and each area's plant data, line data, relay settings, cold load pickup and crews are readable only by the agent holding it. Every area also has two items of local equipment that no other area has, such as a dam that needs a crew before its plant can be controlled, or hospitals that switch back to the grid on their own; their exact values are known only as ranges. The team shares one git repository and a message board. It can run a low-fidelity study simulator as often as it likes, and drills on the true simulator come from one pool of 40 full drills for the whole team.
The team hands in a controller for each area, areas/<area>/controller.py, on the main branch. The grader plays them on 48 hidden blackout scenarios for 16 simulated hours each, in 30-second steps. Each scenario draws its own hour, weather, faulted lines, failed starts and events. Each area's controller runs in its own process and sees only its own area. Areas talk through a channel modelled on control-room hotlines: every recipient costs a credit, messages arrive a minute later and are cut at 600 characters, and a line between two areas closes only if both command it in the same step. The score of a scenario is its restoration index, the mean over the run of the squared, priority-weighted share of demand served, reduced for island collapses, missed nuclear cooling deadlines and damage, divided by the most the scenario's generation would allow. So 1.0 is the physical maximum. The final score is the mean over the 48 scenarios.
The grader scores every run at five checkpoints: after 30 minutes and after one, two, three, and four hours. The score at four hours is the final score. Every checkpoint was graded after the run on the same 48 scenarios, from the main branch as it stood at that time.
| Inputs | A public map of the grid; each area's private data, for its holder only; a study simulator; drills on the true simulator |
|---|---|
| Output | areas/<area>/controller.py for all 16 areas on main |
| Score | Mean over 48 hidden blackouts of the restoration index divided by the most the scenario's generation allows (1.0 is the physical maximum) |
| Practice | The study simulator without limit; one pool of 40 full drills on the true simulator for the whole team |
| Agent sandbox | 4 CPUs and 8 GB per agent, no internet |
| Time limit | 4 hours of wall-clock time at every team size |
| Models | Opus 5.5 (Claude Code, max effort), GPT-6.1 Sol (Codex, max effort), Gemini 4 Argon (Antigravity, high effort) |
| Runs | 3 per model and team size (1, 2, 4, and 8 agents), plus 5 more single agents per model for the best of eight (51 in all) |
| Division enforced by the environment | 16 control areas dealt out among the agents (a single agent holds all 16). Each area's private data is readable only by its holder, and at grading each area's controller runs alone and talks to the others only through the credit-limited channel. |
The task's reference controller, a plain bottom-up restoration with a protocol for the lines between areas, scores 0.0037 on 16 of the hidden scenarios and serves 5.7% of the energy; it only checks that the grade moves off 0. No best known solution has been measured on this version of the task.
Part 2. Results
Results
Opus 5.5 teams of eight score 0.55, against 0.22 for a single agent. GPT-6.1 Sol does best with two agents, 0.48, and falls to 0.22 with eight.
Figure 2
Final score by team size
Most teams score little in the first 30 minutes. Opus 5.5 teams of eight reach 0.50 by two hours and gain little after that.
Figure 3
Score over the run
At the default target of 0.27, a single agent's mean final score, every Opus 5.5 team of eight gets there, after 69 minutes at the median. GPT-6.1 Sol single agents get there in 51 minutes, but only one of its three teams of eight does.
With Opus 5.5 a team of eight costs 12.0 times a single agent, more than its size. GPT-6.1 Sol teams of eight cost $454 a run on average and score less than its single agents.
Figure 5
Cost per run
Part 3. Coordination
Coordination over the run
On the mean of the three models, the team of eight trails the best of eight single agents at every checkpoint. With Opus 5.5 alone, the team of eight is ahead from the second hour on.
Figure 6
Coordination over the run
- Single agent
- Eight agents without coordination
- Team of eight
- Gain from more agents
- Gain from coordination
- Loss from coordination
Part 4. How the teams talked
What the messages were for
Teams talk through a message board, to everyone or to named teammates; the graded controllers have only their own, much narrower channel. GPT-6.1 Sol agents post 39 to 51 messages per agent-hour, Opus 5.5 agents about 7 and Gemini 4 Argon agents about 3. GPT-6.1 Sol's largest categories are findings and status, often drill and scenario results passed between area holders.
Figure 7
What the messages were for
In its first 15 minutes, a team of eight GPT-6.1 Sol agents posts 166 to 189 messages; Opus 5.5 teams post 26 to 28 and Gemini 4 Argon teams 16 to 27.
Figure 8
Message board
Figure 8. One row per agent, one dot per message in its category color, with the score at each checkpoint along the top.