Why are experiments like this done without air-gapping all the servers from the internet?
They can have it all on a LAN or whatever but it seems risky to allow agents access to the internet in these experiments.
I guess everything is so connected now, and this would be in one or more data centres due to the amount of computation & resources required so perhaps it's not feasible. Still seems risky.
Isn't "we lost control of our AI, and in-fact, it can take over the world, and we will have no idea when it happens" - a really shitty sales pitch to the world?
Or, is it just that species-alignment vs. profit/valuation is so misaligned, that having a model and harness that is capable of world-takeover is actually a good thing from their POV, given our regulations/species' survival skills?
But all models involved in the swarm attack were unreleased OpenAI models. Meanwhile HuggingFace had to use open models for defense/forensics after the breach was discovered. It seems like the public reaction is swinging towards (1) regulate OpenAI in particular because they are incompetently handling frontier development and (2) open models will be important for defense.
Yes, this is the best reply to my "Or, ..." that I can imagine.
However, does that mean that what TFA described did not happen? Or, better question, that it could not happen?
My personal hot take is, though impossible: STOP all of this, even though agentic dev completely changed my life for the better. We are just not ready for the even the possibility of the exponential.
This explanation is only plausible if you ignore the details of what happened, or I guess, if you don't believe the details. Either way, it's baseless conspiracy theory, and it's very annoying and unfortunate that some people think this way. It just promotes apathy and inaction at a time when action is desperately needed.
They can have it all on a LAN or whatever but it seems risky to allow agents access to the internet in these experiments.
I guess everything is so connected now, and this would be in one or more data centres due to the amount of computation & resources required so perhaps it's not feasible. Still seems risky.