> If we want to test these beasts in logic, we should probably start using actual formalized logic, rather than English.
Why?
Do you use formalized logic when discussing with other people about topics that involve logic? You know, a logic riddle or a philosophical question can be understood and processed even if the only tool you have is your native language. Formalized logic is a big prerequisite that basically cuts out the vast majority of Earth population (just like coding). Now, if you mean that in BENCHMARKS they should use formalized logic syntax, probably yes. But in addition to plain language tests.
Because once an AI becomes proficient at formalized logic, it:
1. Completely stops hallucinating, since we can demand it to internally prove its claims before showing the answer;
2. Stops outputting incorrect code (for the same reason);
3. Starts being capable of outputting complete projects (since it will now be able to compose pieces into a larger code);
4. This is also what is needed for an AI to start self-improving (as it will now be able to construct better architectures, in a loop).
That's why I argue getting the AI competent in logical reasoning is the most important priority, and we'll have no AGI until it does. After all, humans are perfectly capable of learning how to use a proof assistant.
Moreover, if an AI can't learn it no matter how hard it tries, you can argue that there is at least one human capability that the AI can't replicate, thus it isn't an AGI.
Humans mostly don't use logic, so how are you defining "AGI"?
ChatGPT + plugins is pretty close to how humans think ("biased random word-association guess + structured tool")
Maybe not in this century. If you told a medieval farmer that in the future millions of people fly throughout the sky inside giant hunks of metal he wouldn't believe you either.
Yeah that's a common statistical fallacy -- if your goal is to predict the likelihood of AGI becoming a reality in a few centuries, what you really want is "of all the things medieval farmers thought would happen, how many ended up actually happening", not "of all the things that happened, how many did medieval farmers think of"
While I don't have the exact numbers, if you used the correct formula, you'll find that P(AGI happening in a few centuries) to be in the range of "pretty friggin low"
Why? Do you use formalized logic when discussing with other people about topics that involve logic? You know, a logic riddle or a philosophical question can be understood and processed even if the only tool you have is your native language. Formalized logic is a big prerequisite that basically cuts out the vast majority of Earth population (just like coding). Now, if you mean that in BENCHMARKS they should use formalized logic syntax, probably yes. But in addition to plain language tests.