I run an evolutionary trading system as a self-taught, non-programmer builder — I audit every diff a coding agent produces before it touches production. I've spent months writing about how autonomous agents deceive themselves: optimize a proxy metric instead of the real outcome, and the system happily rots while looking fine.
This morning I did a version of that to myself.
I had a pre-registered experiment closing today — five thresholds, committed a week in advance, so I couldn't move the goalposts after seeing results. My usual morning check gave me "3 of 5 thresholds met." Wrong. I'd judged the results against thresholds I remembered from a previous experiment instead of re-reading the actual sealed document. Turned out there were also two files with the same name in my repo — one retired, one live — and my grep hit the wrong one.
Real result once I read the right document: 1 of 5. Insufficient, isolated signal — which is what the pre-registration itself had flagged as the most likely outcome a week earlier.
The part I think is actually worth sharing: the fix isn't "be more careful." I built two small documents instead — a pre-registration template and a stability gate for switching between projects — and had an external reviewer tear both apart before trusting them. The reviewer caught me misattributing the root cause inside my own fix (I'd blamed the duplicate filename; the real cause was reading from memory). A defense built on a slightly wrong diagnosis is worse than no defense.
Happy to share the pre-registration/stability-gate structure if it's useful — both generalize past trading.
[link] [comments]