The study showed that Google’s Gemini Pro is inferior to OpenAI’s GPT-3.5 Turbo
A study conducted by Carnegie Mellon University and BerriAI found that Google’s Gemini Pro language model was less efficient than the GPT-3 model.5 Turbo from OpenAI in most of the test tasks, writes VentureBeat.
The test included multiple-choice questions in various subject areas, as well as logic tasks.
Gemini Pro showed weaker results, especially in areas related to math, programming, and processing complex queries. At the same time, this model has shown some advantages in tasks related to word sorting, symbols, and text translation.
However, despite these individual successes, Gemini Pro remains behind GPT-3.5 Turbo in terms of overall performance. Responding to the results of the study, a Google representative pointed out the superiority of the Gemini Pro in other internal tests of the company and expressed expectations that the next model, the Gemini Ultra, scheduled for release in 2024, will outperform current models, including the GPT-4.
