What better models changed
++ Since August 2026, I have been exploring how much reverse + engineering an agent can carry out with access to a program and a + way to check its work. Most of my experience comes from + reconstructing games, especially Touhou. The projects have also + become a record of what different models could make progress on. +
++ Looking back through that record, there are functions that later + models closed after earlier sessions had left them unresolved. The + source changes make those breakthroughs more interesting than a + progress percentage does. +
+The temporary values had to stay alive together
++ One TH09 attack function was a byte short. Much of the logic was + already in place. The stack frame had the right size. But three + temporary vectors occupied different slots from the original, and + the compiled function still did not match. +
++ A + GPT-6-Sol checkpoint + recorded that residual after testing several ordinary source + arrangements. Later, + GPT-6.1-Sol found a useful lifetime hypothesis. +
++ The function calculated three vectors and copied each into an + animation's position. C++ keeps a returned temporary alive until + the end of the expression using it. Expressing the real operations + together kept those values alive together. The compiler could no + longer reuse the same storage for all three. +
++ A separate, real collision workspace also needed to keep its own + storage. With that source arrangement, the seventh targeted + candidate in the closing session matched. A cold build confirmed + the complete 1,436-byte function and all 44 encoded + fields. Here, an exact reconstruction means that the maintained source, + compiled under the pinned historical toolchain, reproduces the + checked target function. +
++ It was a useful discovery followed by several targeted tests. The + final result depended on both. +
+A different control-flow graph
++ Another TH09 function had a different obstacle. + GPT-6.1-Sol's earlier work + reproduced two diagnostic regions, but the complete candidate was + 1,510 bytes against a 1,491-byte target. The way its branches + shared a continuation still differed. +
++ GPT-6-Astra changed the surrounding control-flow graph. Completion and frame processing became complementary branches, + followed by one common frame increment and return. In that + context, the compiler produced the required block sharing. The + entire 1,491-byte function matched, and a + recompile replay was accepted. +
++ A different representation made a different result + possible. + That is one reason I take general model capability seriously. An + agent can spend a long time trying variations within a source + structure that is keeping it stuck. +
++ These histories also contain accumulated work. Later sessions + inherited source and recorded failed experiments. They show the + discoveries and how we checked them. A controlled comparison would + tell us how much came from changing the model. +
++ Even so, the cases leave me with a question. How much of this work + needs a model with broad reasoning ability? How much could a model + learn from repeated experience with this particular kind of code? +
+Why might a specialist do well?
++ Touhou is an interesting place to ask. Much of the game code comes + from the same author, ZUN. That gives us a reasonable expectation + of recurring habits. The repositories contain more concrete + examples: related loops, helper functions and source arrangements + have supplied useful hypotheses across games. +
++ The compiler contributes its own habits. Under a particular + version and set of options, small differences in source can + produce repeatable differences in machine code. +
+
+ TH095 gave us a wonderfully awkward example:
+ even the identifier backing a real local variable could change
+ where VC7.1 put it on the stack. Moving the declaration alone did not solve the layout. An
+ earlier background function had already exposed the relevant
+ identifier bucket, and that experience became useful in
+ the reconstruction of Supervisor::LoadDat.
+
+ Selecting the known backing identifier for the real file-size + variable put it and the character buffer in the required homes. + The buffer remained its genuine 128 bytes. All 151 bytes of the + function and its 15 relocations matched. +
++ That is narrow knowledge. A general model has little reason to + arrive with a detailed repertoire of one old compiler's local + allocation behavior. A specialist could encounter it repeatedly. + It could learn which source variations are worth testing first. +
++ In many of these final matching problems, we already understand + the behavior. The remaining question is which ordinary source form + this compiler turns into these instructions. That gives + specialization a specific job: propose plausible forms, then let + the target decide which survive. +
++ There are limits to how narrowly we can define the repertoire. + TH08 uses VC7. TH095 and TH09 use VC7.1. TH04 uses Turbo C++ in a + segmented DOS environment. Its + player reconstruction + needed another set of source-lifetime and linker-order hypotheses. + Shared authorship gives us a prior; each target tells us whether + that prior applies. +
+Where should that experience live?
++ Before training anything, there is an obvious place to look: the + repository. In the + previous essay, I wrote + about it becoming the project's working memory. Compiler lessons + belong there too. +
++ An agent can retrieve a known result. It can inspect the source + that produced it and read which alternatives failed. That changes + its starting point without changing its weights. +
++ A TH09 + GPT-6-Luna-max checkpoint + makes this worth taking seriously. It closed a 373-byte + enemy-spawn function using an ordinary loop with branch-local + failure handling. TH08's corresponding source family supplied the + hypothesis. TH09's own evidence established its bounds and calls, + and two cold builds confirmed the complete match. +
++ “The model did not know that compiler quirk” and “the model + could not use it” are different problems. + Supplying a fact may solve the first. Generating a useful source + candidate with that fact may still need practice. Choosing what to + try after a failed compile is another skill again. +
++ I would want a smaller general model with good retrieval in any + comparison. Otherwise, we might credit fine-tuning for an + advantage that came from giving one model a better library of + experience. +
+Learning the next useful experiment
++ In agentic reconstruction, the model proposes an explanation. The + program and its checks determine whether that explanation + survives. A failed candidate can be useful when it helps + distinguish the remaining possibilities. +
++ This makes the temporary-vector example especially interesting to + me. The final source contains the successful arrangement. It does + not contain everything the search taught us. +
+| Experiment | +What its result taught us | +
|---|---|
| Use a typed position assignment | ++ The copy behavior improves, but the compiled source still + differs. + | +
| Consume the returned vectors directly | ++ The function reaches the target size. Temporary storage + remains wrong. + | +
| Group the real operations into one full expression | +The returned values remain alive together. | +
| Keep the real collision workspace separate | +The complete function matches in a cold build. | +
+ Selected steps from the recorded Type18/24 search. The closing + session tested seven candidates; this table groups the lessons. +
++ A useful training record would preserve the proposed change and + the result. It would also preserve why the result justified the + next experiment. An ineffective variation is part of that + experience, especially if a later agent would otherwise repeat it. +
++ I would want the model to learn how to use feedback. + It should recognize when a change has no effect. It should notice + when a new mismatch points to a different cause. It should be able + to leave an unproductive source representation and inspect the + evidence again. +
++ That also means keeping autonomy. The agent may need another + caller, a compiler experiment or an earlier project's source. A + rigid sequence of required rewrites would prevent some of the + decisions we want it to learn. +
+Why RE is interesting for reinforcement learning
++ A bounded reconstruction problem has an attractive property. With + the target and compiler pinned, many experiments produce quick, + repeatable feedback. Compile a candidate. Inspect what differs. + Decide what to try next. +
++ Reinforcement learning could use that feedback to train the + decisions an agent makes during reconstruction. There is a + concrete result to optimize: a candidate accepted by a defined + check. We can also observe how much work it took to get there. +
++ This direction already has research behind it. + LLM4Decompile + studies models specialized for decompilation. + AutoDecompiler + trains multi-turn refinement using compiler and execution + feedback. Those results make the idea worth investigating. Their + functional recovery tasks differ from matching code emitted by a + particular historical compiler. +
++ The experiment I have in mind would train a policy for that + narrower search. Given what we have learned so far, which source + hypothesis is worth testing next? +
++ Other parts of RE are harder to package into a cheap loop. We may + still be establishing what a function owns. We may need a new + runtime measurement. In TH08, a + later collision divergence led back to a one-ULP numerical + difference + and the old x87 behavior. The productive hypothesis there was + partly about what to measure. +
++ That is another plausible place for a broad model's strength. + First make the problem checkable. Then ask how much a learned + search policy can do with that check. +
+When the oracle is wrong
++ An oracle is the reference or checker we use to accept a result. + Its answer can be repeatable while its coverage is incomplete. +
++ TH08 already gave us an example. Some reconstructed functions had + passed their exact comparisons with wrong floating constants. The + instruction check accounted for relocated addresses, but checking + the literal payload depended on optional explicit attestation. + That left references whose values were never checked. +
+
+ A
+ wider audit
+ examined all 1,548 floating-literal references and corrected
+ twelve stale rows across five accepted functions. In one case, the
+ bullet-spread increment was 1.0f where the target
+ required 32.0f.
+
+ A learner rewarded by the old check could learn to produce the + wrong reconstruction. + That is a consequence worth considering when we design an RL + environment. The observed problem here was an oracle defect; we + had not trained an RL model against it. +
++ Functional checks need the same attention. + Differential-fuzzing research on LLM decompilers + has found behavior differences beyond the supplied tests. An + accepted test result tells us something about the cases the test + actually exercises. +
++ Before using a comparator as a reward, I would give it deliberate + failures to reject. The final acceptance check should stay outside + the candidate's editable source. A byte array copied from the + target may satisfy a naive score while teaching us nothing about + reconstructing the source. +
++ Improving the learner and improving its oracle belong to the + same research problem. + Greater ability to optimize a check makes understanding that check + more important. +
+A comparison worth running
++ I would start with a fixed VC7.1 task family. A strong general + model would be one baseline. A smaller general model would be + another. Then I would give the smaller model the relevant + repository knowledge and compare that with supervised + specialization, followed by an RL-trained version. +
++ Each would get the same target evidence and tools. The + training-specific comparisons would use the same base model and + available knowledge. I would repeat the trials, since one lucky + candidate is a poor description of a search policy. +
++ The starting point matters. Reconstructing a function from its + instructions and closing the last four differences in maintained + source are two useful tasks. I would measure both separately. A + specialist might be excellent at the second while a general model + does much of the work needed to reach it. +
++ The test set also needs care. Related Touhou games share engine + code. Holding out a game's name would not be enough if nearly + identical functions had appeared in training. I would hold out + source families and keep their accepted answers out of both + training and retrieval. +
++ For the outcome, I would count complete, independently validated + matches. One view would give every model the same candidate or + compile budget. Another would measure time and total cost to an + accepted result, including compilation. Training costs would need + their own accounting too. +
++ A cheaper proposal can become an expensive workflow if it causes + more uninformative compiles. A slower model may pay for itself by + finding the right experiment sooner. Those are outcomes we can + measure. +
++ I would also be cautious with a reward for “getting closer.” One + TH09 + replay-save candidate + already had the target's 1,825-byte length while 467 byte + positions differed. A shorter mismatch list can also move abruptly + when a branch encoding changes. Diagnostics help choose an + experiment; the full acceptance check decides whether it worked. +
++ That experiment could produce an answer with conditions. A + specialist might win within a familiar compiler family. A general + model might win when the toolchain changes. Retrieval might + account for much of the apparent benefit of training. Each result + would tell us something useful about where to invest next. +
+What kind of agent does this suggest?
++ Perhaps the useful arrangement is a broad agent with access to a + specialist. It can decide when the task has become a familiar + compiler search, and ask the specialist for candidates. It can + still inspect the target or question the measurement when the + results stop making sense. +
++ That would make specialization a tool the agent can choose. It + would preserve the freedom that made these investigations + productive in the first place. +
++ In the previous essay, I described an industrial shift in how + reconstruction work can be organized. This question follows from + it. Once we can run many more experiments, we need to understand + which decisions require broad judgment and which improve through + repeated experience with a particular craft. +
++ The strongest general model has earned its place in my workflow. + It has opened problems I had left unresolved. What I want to find + out next is how much of that work requires its breadth, and how + much could be handled by a model that has learned this particular + kind of reconstruction. +
++ The repositories give us a way to test the question. + That is where I would like to take it next. +
+Evidence and model attribution
++ The examples describe historical reconstruction checks recorded + in public commits. Their byte counts apply to the stated + functions, not whole-game runtime correctness. Source and + matching records were reviewed on 13 October 2026. +
+
+ The TH09 label gpt-dots identifies GPT-6-Astra.
+ gpt-web changed with the model used at the time, so
+ that label alone does not identify an individual run. Explicit
+ model names in the other cited commits establish their recorded
+ attribution; the histories do not contain uniform per-run cost
+ or effort measurements.
+
-
+
- + TH09 temporary lifetimes: the + earlier context review + and + complete cold reconstruction. + +
- + TH09 Front callback: + non-exact diagnostic regions, + Astra's new graph + and its + accepted recompile replay. + +
- + Compiler knowledge: + TH095 identifier-bucket reuse, + TH04 player source and linker order + and + Luna's TH09 enemy-spawn closure. + +
- + Checks and diagnostics: + TH08 floating-literal audit, + TH08 numerical investigation + and + TH09 replay-save reconstruction. + +
+ The proposed specialist and RL comparisons are future + experiments. The project histories motivate them; no trained + specialist was evaluated for this essay. +
+