AI large language models can give an illusion of intelligence but in fact it is inherently limited in high-level reasoning
AI large language models can give an illusion of intelligence but in fact the technology is inherently limited in high-level reasoning
The grade school math 8k suite has become a popular benchmark for various AI large language models , such as. The suite contains 8,500 problems like the one above, divided into problems to train a LLM and then the real problems to be solved. The latest LLM from ChatGPT’s OpenAI, the GPT-4o model, has scored 92.5 per cent on the GSM8K suite while Google’s LLM Gemini 1.5 Pro scored 91.7 per cent.
Even more intriguingly, dropping or adding additional clauses had a significant impact on the performance of the LLMs. For example, removing the clause specifying a call price reduction after 10 minutes in the test problem above, or adding a new clause giving a 5 per cent discount for calls costing more than $10, frequently caused a variation in the accuracy of the results.
The researchers concluded: “Ultimately, our work underscores significant limitations in the ability of LLMs to perform genuine mathematical reasoning. The high variance in LLM performance on different versions of the same question, their substantial drop in performance with a minor increase in difficulty, and their sensitivity to inconsequential information indicate that their reasoning is fragile. It may resemble sophisticated pattern matching more than true logical reasoning.
Apple Chatgpt Open-Ai Artificial-Intelligence
Ireland Latest News, Ireland Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Kathleen Watkins, writer, broadcaster and wife of Gay Byrne dies aged 90Much loved broadcaster, writer and musician Kathleen Watkins has passed away surrounded by family, just over five years since her husband , RTE legend Gay Byrne, died
Read more »
Kathleen Watkins, broadcaster and author, has died aged 90Watkins, who was the first continuity announcer on RTÉ, was married to Late Late Show host Gay Byrne
Read more »
Deposit Return Scheme enjoys 90% return rate at MusgravesOn Newstalk Breakfast Musgraves spokesman Owen Keogh confirmed the schemes success in their Centra and SuperValu stores.
Read more »
Tributes to 90 year-old hero who died saving woman's life in car park crashCal O'Keeffe, 90, was killed when he was struck by a car in a supermarket car park in Fermoy, Co Cork, on Friday morning
Read more »
Tears as mourners hear how 90 year-old hero died saving woman's lifeFather of four and retired teacher and farmer, Cal O’Keeffe, was pronounced dead in the Supervalu car park on Courthouse Road shortly after 11.15am last Friday
Read more »
Man (90) killed pushing woman out of way of car remembered as ‘great people person’Cal O’Keeffe was pronounced dead in SuperValu car park, with woman in her 60s rushed to hospital
Read more »