Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

One of the best things you can do is start by having it do unit test coverage for existing behavior. A refactor with no tests breaks things pretty much no matter who does it, because they don't know what the right behavior is.


While I could generally agree, in this specific instance if the AI were “thinking” correctly it should have found the mistake. I admit it was a difficult problem though (solving it required creativity).

To be more precise, the prompt actually pointed to where there could be issues, and the issue, which was exactly of the kind that was pointed at, was not found.


There are a lot of factors in "should have found" which my recommendation improves.

If you told it to write unit test coverage, you would have covered more of the codebase. That reduces the size of context necessary for the next mistake finding investigation - it'll see it's already covered a lot of the paths. Then you say "Go look for issues" (or whatever you asked it to do) and it'll be able to think more deeply about what's left over.

What specific model were you using, at what effort? How big was your context window?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: