> Open internet with cheating detection => cheating is widespread, 20-36% of tasks are flagged for the stronger models, with source code lookup accounting for the majority of the violations.
Therefore:
> blocking internet access entirely is the appropriate default for ProgramBench
The fact that your Anthropic coding assistant has a tendency to search on the Internet code to be inserted into your program may count for an additional copyright violation (besides the possibility of reproducing recognizable fragments of its training data).
(I do not agree that copyright, at least in its current form, should be applicable to computer programs, but it is weird that the same companies who try to exploit copyrights against others also insist on the use of coding assistants that are a workaround against copyright laws, which is the main reason why they can increase programming productivity, because they may cut and paste code that you are not allowed to copy yourself.)
The output of a coding assistant cannot be copyrighted, but it may contain code from which the copyright has been removed and which is used in a manner incompatible with the original license.
Even the more permissive licenses, like BSD, MIT, etc., forbid the removal of the copyright notice when the code is reused.
While this may also happen with the source programs used for training, I was not aware about the behavior described in TFA for the Anthropic agents, which may search the Internet for source code applicable to the problem that must be solved. It seems even more likely that such code will not be used as allowed by its license.
Therefore:
> blocking internet access entirely is the appropriate default for ProgramBench
The fact that your Anthropic coding assistant has a tendency to search on the Internet code to be inserted into your program may count for an additional copyright violation (besides the possibility of reproducing recognizable fragments of its training data).
(I do not agree that copyright, at least in its current form, should be applicable to computer programs, but it is weird that the same companies who try to exploit copyrights against others also insist on the use of coding assistants that are a workaround against copyright laws, which is the main reason why they can increase programming productivity, because they may cut and paste code that you are not allowed to copy yourself.)