Limits, uses and challenges of generative AI

In the first part of the series, we covered the basics of LLMs. We saw that language models are AIs trained on huge text databases to interpret human language and generate coherent responses. There are 6 key stages in building an LLM: data collection and processing, initial configuration, intensive training, verification and improvement. We also covered important concepts such as ‘tokens’, which are units of text used to feed the language model, and ‘fine-tuning’, which involves re-training an already-trained LLM to incorporate additional information. We also took the time to deconstruct a number of preconceived ideas:

1. Contrary to popular belief, an LLM like ChatGPT does not learn from the questions asked by users.
2. Hallucinations cannot be avoided due to its probability-based architecture, lack of deep understanding and the limitations of training data.

For more details, we invite you to (re)read part one: Discovering the fundamentals of LLM!

The limits of LLMs

Once a generative AI model has been trained, it becomes static and can no longer be modified, except through the process of fine-tuning, a specific adjustment to adapt the LLM’s responses. To maintain the relevance of its information, ChatGPT, for example, is connected to the Internet and carries out searches in real time via Bing. This enables it to enrich its context and provide up-to-date answers, without requiring a complete overhaul of its initial training.

On the other hand, LLMs do not always respond reliably. A test conducted on StackOverflow revealed that ChatGPT was wrong 54% of the time when answering programming questions. In addition, a study carried out by the University of Hong Kong in 2023 found that 36% of the answers generated by LLMs were incorrect. This shows that, although these models perform well for certain tasks, they cannot always be considered reliable sources, particularly for advanced or complex queries.

Current uses of LLMs

What you can do with an LLM VS what is useful

 

We can really distinguish between what is POSSIBLE to do with LLMs VS what is USEFUL. Do we really need to ask ChatGPT for the weather forecast when a dedicated application can do the job? It’s possible, but is it useful?

We have identified 6 cases where the use of an LLM adds considerable value in several areas:

• Personal assistants: Large language models power tools such as Microsoft Copilot and Siri, which help users answer questions, manage daily tasks and access information quickly and efficiently.
• Automatic summaries: These models can summarise long texts, which is particularly useful for processing legal documents, studies or reports in sectors such as finance, research or health.
• Machine translation: Increasingly powerful, LLMs are used for machine translation, facilitating communications on an international scale.
• Legal assistant: Larges Languages Models are used to assist legal professionals in analysing and drafting complex legal documents, researching case law and checking compliance. These models are also used to provide quick answers to common legal questions.
• Virtual therapists (medicine, psychology, etc.) : These can also be used in the field of mental and physical health, as virtual assistants for therapists or as support for patients. They can be used to carry out initial analyses, suggest stress management exercises or provide treatment reminders.
• Code writing: Tools like GitHub Copilot enable developers to write code faster and improve productivity by providing intelligent suggestions.Although LLMs can generate a variety of texts, summarise lengthy information or even produce advice, the usefulness of an LLM lies in its suitability for practical purposes rather than solely in the scope of its capabilities.

Although LLMs can generate a variety of texts, summarise lengthy information or even produce advice, the usefulness of an LLM lies in its suitability for practical purposes rather than solely in the scope of its capabilities.

Have the consequences of AI on employment been studied?

These technological innovations are redefining many professions, sometimes giving rise to concerns that they may be replacing individuals. Questions are being asked about their limits and their true effectiveness in certain contexts. Some companies are sounding the alarm about the impact of AI on specialised professions. Translation companies*, for example, fear that automation will compromise the accuracy and nuance of their work. Lawyers* are also raising concerns about the reliability of the legal answers generated, and therapists* are worried about the limited empathy of AI models. Indeed, in development, studies* show that the code produced by LLMs often remains at a beginner’s level, still requiring human intervention for complex tasks.

Are LLMs being put to good use?

Increasingly present in businesses, the use of generative AI is not always essential or effective. While attractive for automating tasks, generating content, or improving customer service, LLMs can be overkill for some uses, especially if advanced analysis or personalisation is not critical. Many decision-makers fear that they will miss out on the technological shift* towards AI, but for the best ethical use of these technologies, it is important to assess their real impact, by comparing their advantages with less costly and resource-intensive solutions.

Models are effective for specific tasks, but they are not a universal solution. For certain needs, simpler, targeted tools are often better suited, less costly and quicker to deploy.

 

LLM challenges

Although LLMs offer undeniable advantages, they face 4 major constraints to their smooth operation: technical, ecological, economic and legal.

Technical constraints:

LLMs impose significant technical constraints, due to their need for colossal computing power to operate efficiently. For example, Microsoft plans to acquire 1.8 million GPU chips by 2024* to support these intensive computing requirements.

Training data must be plentiful and meet certain criteria:

• They must be qualitative* and come from reliable sources (Wikipedia, press, scientific articles, blogs, etc.).
• Respect copyright* and personal data*.
• Be as unbiased as possible*.
• Multilingual*: Llama 3, for example, has been trained on only 5% of non-English languages.
• Non-poisoned*: the source data was not manipulated before the models were trained.

Given the scarcity of data available online, a phenomenon known as the ‘data wall’ is emerging. This refers to the difficulty of accessing sufficiently varied, complete and high-quality data sets to train language models effectively. Instead of ‘feeding’ on personalised, unbiased data, free from manipulation, language models risk being fed more and more on synthetic data produced by other LLMs, which can be detrimental to their accuracy and quality.

Ecological:

Recent advances in language models raise important ecological questions, particularly with regard to their energy consumption. According to projections, by 2030, the energy required to run these systems could account for up to 25%* of the total energy supply in the United States, an amount equivalent to that consumed by an entire country*. To cope with this explosion in energy demand, companies like Microsoft are looking at innovative solutions, including the construction of small-scale nuclear reactors*. Discussions are also emerging around harnessing nuclear fusion* to power these technologies, as indicated by reports of OpenAI’s interest in acquiring fusion energy from Helion.

At the same time, the water consumption required to generate and operate LLMs is also a major concern. In 2022, Microsoft consumed nearly 6.4 billion litres of water, a volume equivalent to around 8,700 Olympic swimming pools*. Each generation of content from a Large Language Model requires millions of litres of water, which represents around half a litre of water per interaction with a service like ChatGPT*. These figures illustrate not only the ecological footprint of these technologies, but also the urgent need to find sustainable solutions to minimise their environmental impact.

Economic :

Beyond the technical and ecological challenges, LLMs also involve high economic costs. Electricity consumption has risen sharply*, largely due to the growing needs associated with AI, resulting in higher energy costs for businesses and consumers.

On the other hand, IT resources are also important. For example, the daily operation of ChatGPT requires around $700,000 a day in server resources*, a cost that makes it difficult to make these tools profitable on a large scale. This figure highlights the importance of a robust and expensive infrastructure to ensure a reliable service.

The growing shortage of engineers specialising in AI* further accentuates these economic challenges. With salaries rising to attract talent*, companies must navigate an increasingly competitive labour market, making the implementation and development of these technologies even more expensive. Taken together, these factors contribute to a complex economic dynamic that requires particular attention from players in the sector.

 

Legal :

LLMs also raise legal issues, particularly in terms of personal data protection. On the one hand, compliance with the General Data Protection Regulation (GDPR) is crucial, especially as the majority of generative AI providers are based in the US*. These models may have been trained on personal data*, and the prompts we submit may also be used to refine their learning*. This raises questions about the use of language models with sensitive data and the need to guarantee user confidentiality.

On the other hand, the AI ACT*, which came into force on 1 August 2024, aims to ensure that artificial intelligence systems respect the fundamental rights of European citizens*. This text imposes requirements such as maximum transparency, the updating of technical documents, compliance with European copyright law, and the obligation to provide summaries of the content used for training. It also requires tests to be carried out to guarantee the security of the systems deployed. These measures reflect a growing trend, with similar legislative frameworks being prepared in other countries*, underlining the importance of a consistent regulatory approach to the use of LLMs on a global scale.

In addition, companies must ensure that the data collected and used to train models complies with these legal requirements, or face sanctions.

In conclusion, Large Language Models have transformed our approach to AI and have become indispensable tools in a variety of sectors. However, it is important to recognise their limitations and address the challenges associated with their use. Whether it’s overcoming energy constraints, managing biases or improving the reliability of answers, LLMs require constant attention and careful management if they are to be used to best effect. For example, some departments have equipped themselves with LLMs for tasks where simpler, more cost-effective tools might suffice. This quest for rapid modernisation is also fuelled by the need to improve the user experience, sometimes to the detriment of a full assessment of the usefulness or reliability of language models. To maximise their use, it is therefore important for companies to carefully assess the specific needs of their business and ensure that LLMs are tailored to the intended use cases, taking into account their limitations and their environmental and economic cost.

 

 

👉 Stay tuned for the rest of the ‘Generative AI Focus’ series:

• Part 3: What future for LLMs? We analyse the LLM market, the outlook for the future and our positioning as experts

 

 

(Monitoring report by Mathieu Changeat)