Recent research from Apple suggests that models that got a high score on the GSM8K dataset may not be as intelligent as they seem.

Large Language Models (LLMs) have been widely praised for their seemingly impressive reasoning abilities. Models from companies like OpenAI, Google, and Meta are often showcased as powerful tools capable of solving complex problems, with tests like the GSM8K dataset being a popular benchmark to measure their reasoning skills.

Yet, Apple’s research is set to change the so-called trustworthy system.

What is GSM8K dataset?

The GSM8K dataset (Grade School Math 8K) is a benchmark used to evaluate the problem-solving and reasoning abilities of Large Language Models (LLMs). It consists of over 8,000 grade-school level math word problems, which typically require arithmetic, logical reasoning, and multi-step problem-solving skills to arrive at the correct answer.

The GSM8K dataset consists of:

• Grade school-level math: The problems are designed to mimic the type of questions a student in grades 1-8 might encounter, such as basic arithmetic, geometry, algebra, and logical puzzles.
• Word problems: Each question is presented in a word problem format, requiring the model to interpret the problem, identify the relevant numbers and operations, and solve the equation.
• Used for LLM evaluation: The dataset is often used as a test to see how well language models like OpenAI’s GPT, Google’s models, or Meta’s LLaMA can handle reasoning tasks beyond mere text prediction.
• Multi-step reasoning: The problems require multiple steps to solve, testing the model’s ability to track complex sequences of reasoning, rather than simply producing a single-step answer.

The GSM8K dataset has become a popular tool to assess whether LLMs can reason logically and solve real-world problems. However, there is concern that many AI models perform well on this dataset through pattern matching rather than true reasoning, as they might have been exposed to similar problems during training.

The GSM8K dataset’s limitations of LLMs

Apple researchers argue that this success may be more about sophisticated pattern matching than genuine logical reasoning. Since the GSM8K dataset is so commonly used, there’s a risk of data contamination—meaning that many LLMs may have already seen these problems during training, inflating their apparent intelligence.

To address this, Apple developed a new benchmark called GSM-Symbolic. This test retains the core reasoning elements of the GSM8K dataset but introduces changes like different names, numbers, and complexity, along with irrelevant information.

The results? Every LLM tested, including models like OpenAI’s GPT-4 and Meta’s Llama 3, saw a significant drop in performance when faced with this new challenge. This suggests that LLMs struggle with true reasoning when variables are altered, further questioning their actual problem-solving skills.

Why do LLMs struggle?

The study by Apple sheds light on a critical flaw in LLMs: They are excellent at detecting patterns in the training data but lack true logical reasoning. For example, when math problems included irrelevant details, such as the size of kiwis in a fruit-picking scenario, many LLMs subtracted that irrelevant detail from the equation, demonstrating a failure to discern which information was necessary to solve the problem.

In tests with the GSM8K dataset, LLMs like OpenAI’s models performed better than their open-source counterparts, but the drop in accuracy when irrelevant information was added suggests that these systems are far from achieving genuine intelligence. This has profound implications for the future development of AI, showing that while LLMs may mimic intelligence, they still struggle to truly understand context.

Smarter AI or just better at seeming smart?

Apple’s research underscores the limitations of relying on benchmarks like the GSM8K dataset to assess AI intelligence. While these tests can measure pattern recognition, they don’t always capture the nuances of true logical reasoning. The introduction of the GSM-Symbolic benchmark provides a more rigorous test of an AI’s ability to handle unfamiliar variables and irrelevant information—skills essential for real-world problem-solving.

Sam Altman, CEO of OpenAI, has even acknowledged these challenges, referring to current LLMs as “incredibly dumb” despite their impressive outward appearance in an exclusive interview with MIT Technology Review. The real test for future LLMs will be their ability to go beyond pattern recognition and develop more robust problem-solving abilities.

The findings from Apple’s study offer a sobering perspective on the current state of LLMs. While models trained on datasets like GSM8K may perform well in controlled environments, their reasoning abilities falter when tested on more complex, real-world problems. This highlights the importance of further research and development to ensure that AI models move beyond surface-level intelligence and develop true logical reasoning skills.

For now, it’s crucial to temper the excitement surrounding AI with healthy skepticism, focusing on safer, smarter AI systems that can handle more than just pattern recognition.

Image credits: DC Studio/Freepik