Eric S. Raymond
So, I upgraded to ChatGPT 6.0 today. And, as is my habit, I immediately sicced it on one of my projects that has a pending bug report: Super Star Trek, in this case, which has a minor issue in its color handling when running on Windows. When I do this, I don't just have robot friend fix the pending bug. I give it the prompt "Audit this code for bugs and suggest improvements". Here's what's bothering me. The last time I did this, with 5.5, I went through and resolved all the audit reports until robot friend couldn't find anything else left to gripe about. Today, on the same code, it's reporting 16 issues. And...it's been like this every time. I think I've gone through four of these cycles. What I'm wondering is: when does the process converge? I mean, a program can only have a finite number of bugs, right? Right? SST is very old and somewhat grotty code. Started out as mainframe Fortran in 1973, was translated to C at some point before it passed to me, and I moved it first to Python and then to pedantically strict typed Python. But that doesn't seem like an explanation to me. Especially since I've seen the same pattern of more bug finda from every new LLM version on projects I wrote from scratch. It's never huge terrible bugs anymore, but it's almost never the empty set, either. This is more disturbing because I'm actually *good* at writing reliable code - by human standards, anyway. I actually got an award with prize money attached to it for that, once upon a time. If *my* code looks like an infinite pit of bugs, I shudder to think what the state of the rest of the world is. Anyway, I now think I have a personal benchmark for AGI. The first time I run a new model iteration on Super Star Trek and it says "No bugs, boss", I will know that the previous one achieved AGI.