third-party evaluation partner
@usgraphics one can certainly write prompts that attempt to bludgeon the model into behaving more like a “pair programmer” than an “agent”, but IME this always ends up fighting against both the harness and the behaviors taught in RL and it’s hard to make it meaningfully stick