Generative AI Doesn't Have a Coherent Understanding of the World, MIT Researchers Find (mit.edu) 33
Long-time Slashdot reader Geoffrey.landis writes: Despite its impressive output, a recent study from MIT suggests generative AI doesn't have a coherent understanding of the world. While the best-performing large language models have surprising capabilities that make it seem like the models are implicitly learning some general truths about the world, that isn't necessarily the case. The recent paper showed that Large Language Models and game-playing AI implicitly model the world, but the models are flawed and incomplete.
An example study showed that a popular type of generative AI model accurately provided turn-by-turn driving directions in New York City, without having formed an accurate internal map of the city. Though the model can still navigate effectively, when the researchers closed some streets and added detours, its performance plummeted. And when they dug deeper, the researchers found that the New York maps the model implicitly generated had many nonexistent streets curving between the grid and connecting far away intersections.
An example study showed that a popular type of generative AI model accurately provided turn-by-turn driving directions in New York City, without having formed an accurate internal map of the city. Though the model can still navigate effectively, when the researchers closed some streets and added detours, its performance plummeted. And when they dug deeper, the researchers found that the New York maps the model implicitly generated had many nonexistent streets curving between the grid and connecting far away intersections.
No kidding (Score:5, Insightful)
Seriously, did we need a MIT study? (Score:3)
Seriously, did we need a MIT study to know that?
Re: (Score:2)
Re: (Score:1)
And if it was MIT, why didn't they first try it out on a map of Boston [app.goo.gl]?
Probably because the city of NY (Manhattan island especially) is much simpler road-wise than Boston.
Re: (Score:2)
Apparently. There are a surprising number of people, even in the field, who have what I can only describe as religious beliefs about emergence. It's disturbing.
understanding? (Score:5, Insightful)
More anthropomorphizing neural networks. They don't have "understanding" at all, much less "coherent" understanding.
Results by (Score:2)
...CaptObviousGPT
Very few experts ever claimed it had common sense-like reasoning, and those who did usually added caveats to their claims.
And this is different from humans? (Score:2, Funny)
I have met lots of people who don't have a coherent understanding of the world. This week I watched them ... oh, never mind.
Re: (Score:1)
The expectation we have that LLMs should somehow be perfect is an expectation we don't tend to apply to humans.
If you ask a human a complex question it is likely the human will have a less than 100% understanding of the issue, and will give a less than 100% correct answer. But that's ok. We can continue the conversation, challenging the dubious parts, and working towards the truth.
Perhaps as this sort of AI progresses, systems like ChatGPT might become better at recognising their own weak areas of knowledge
Re: (Score:2)
You seem to think that LLMs are like science fiction robots. They are not.
Combine with a logic engine & rule base (Score:2)
I'm wondering if it would be possible to hook it up to the likes of Cyc, a logic engine and common-sense-rules-of-life database. The engine could find the best match between the language model (text) and Cyc models, weighting to favor shorter candidates (smallest logic graph). Generating candidate Cyc models from language models may first require a big training session itself.
I just smell value in Cyc's knowledge-base, there's nothing on Earth comparable (except smaller clones). Wish I could by stock in it
Typo Corrections: (Score:2)
Corrections:
"weighing to favor shorter candidates" [No JD jokes, please]
"Wish I could buy stock in it"
(Bumped the damned Submit too early)
Re: (Score:3)
Re: (Score:2)
Indeed. With all the investing going into increasingly questionable AI projects you'd think somebody with money would zig when everyone else is zagging to try bagging a missed solution branch/category.
Reminds me of the Shuji Nakamura story on the invention of a practical blue LED. Red and green LED's were already commercial viable. Blue was the missing "primary light color" in order to mix to get the full rainbow. Many big co's spent a lot of R&D on blue, but kept failing. Their blue LED's were just way
Re: Combine with a logic engine & rule base (Score:2)
Re: (Score:2)
Shuji's a true underdog Nerd Hero.
So true. His story should be required reading throughout the curricula of science and tech courses.
Re: (Score:2)
Re: Combine with a logic engine & rule base (Score:2)
Cool approach (Score:4, Interesting)
'These results show that transformers can perform surprisingly well at certain tasks without understanding the rules. If scientists want to build LLMs that can capture accurate world models, they need to take a different approach, the researchers say.'
The key thing here is they don't understand the rules. For example, an AI model might make legal chess moves every time, but if you modified the chess board [wikipedia.org] then it would suddenly make illegal moves with the knight. With current AI technology, you would try to "fix" this by including as many possible different chess boards as possible, but that's not how humans think. We know the rules of the knight and recognize that in a new situation, changing the board doesn't change the way the knight moves (but it might). And if you wanted to clarify,, you could ask someone, "Do all the pieces still move the same on this new board?", but that is what these researchers did (modified the map of NY with a detour), and it really confused the model.
:)
It is of course obvious that current LLMs do not have human intelligence because they are not Turing complete, but to understand what that means you'd need to have an internal understanding and mental model of what Turing machines are, and LLMs don't have that.
Re:Cool approach (Score:4, Interesting)
These things are really good at sussing out patterns in (seemingly) random data. It's an extremely useful feature/ability.
But of course they're lost when the rules and patterns change because they mastered the initial patterns already. They have no ability to think and realize, "oh, this is a new thing" because they don't ever "realize" anything. Change the chess board, the fake-I fails.
I assume the hallucinations come from finding patterns that really weren't there but only appeared to be based on previous training.
All I got from this silly MIT study was some poor bastards stuck in the world of publish or perish who were drinking and partying too much right up to their publish deadline then put out some bullshit to keep their careers alive for another season.
Re: (Score:2)
But of course they're lost when the rules and patterns change because they mastered the initial patterns already.
That's part of it, but also they have no ability to recognize what changes are "important" and what are not. Change the color of the chess piece from white to ivory and it won't recognize it (unless it has ivory in its training set).
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
It is of course obvious that current LLMs do not have human intelligence because they are not Turing complete, but to understand what that means you'd need to have an internal understanding and mental model of what Turing machines are, and LLMs don't have that. :)
You are correct they don't have human intelligence yet wrong about the reason why.
"Memory Augmented Large Language Models are Computationally Universal"
https://arxiv.org/pdf/2301.045... [arxiv.org]
The key thing here is they don't understand the rules. For example, an AI model might make legal chess moves every time, but if you modified the chess board then it would suddenly make illegal moves with the knight. With current AI technology, you would try to "fix" this by including as many possible different chess boards as possible,
Have you tried asking the AI to convert to a normal chess board before moving?
but that's not how humans think.
LLMs obviously don't work like humans.
Re: (Score:2)
The question is, what is going on inside the model to make it hallucinate?
So-called 'hallucinations' are not errors or mistakes, they are a natural and expected result of the how these models function.
The key thing here is they don't understand the rules.
They don't understand anything. That's not how they work. They don't operate on facts and concepts, they operate on statistical relationships between tokens.
Did someone think they did? (Score:1)
New discovery. (Score:2)
Anyone who thought this was even possible doesn't have a coherent understanding of the world.
Artificial Intelligence is NOT Actual Intelligence (Score:2)
as humans that grow up in the world, we hardly have a coherent understanding of the world, let alone an LLM that is trained on huge data sets and forms patterns to mimic a form of intelligence.
Sounds like the people that wasted their time on this don't have a coherent understanding of the world either. Marketing wank is just that, were they expecting actual Intelligence?
The horrors of being wrong (Score:2)
This is all rather interesting. People create systems inspired by how brains work then they turn around and get all upset it isn't perfect and criticize the system for its failure to magically compile and execute some kind of robust model of how the world works that would enable it to always generate infallible predictions.
On one hand we have people who either hate with a passion or dismiss LLMs outright as cut and paste machines which don't even deserve to be called AI. On the other hand we have people r
Turing Test (Score:2)
LLM's will have achieved true intelligence when they answer the question with <Maine_accent>Ya' can't get thea' from hea'</Maine_accent>.