Artificial Intelligence trends and challenges in recent years - Literature review

2.1 Short-term trends (within 1 year)

The rise of generative AI and large language models

2023 was the year of the breakthrough of generative AI, especially in the area of language models available to the public (stateof.ai). OpenAI’s GPT-4 model and similar LLMs outperformed their predecessors in many tasks, achieving human-level performance on certain tests and exams (stateof.ai). In parallel, diffusion models revolutionized image generation, enabling the creation of artificially generated image and multimedia content of unprecedented quality. The widespread availability of generative AI tools has led to 79% of people having some form of exposure to these technologies and 22% using them regularly in their work, according to a global survey in 2023.¹ On the enterprise side, adoption has also been dramatic: a third of companies have already implemented generative AI in at least one business function, meaning that 60% of companies using AI are using generative tools.¹ This wave is not just being felt in the technology sector - the initial hype around generative AI is encouraging experimentation everywhere from education to healthcare.

Investment surge

In 2023, startups specializing in generative AI reported record levels of venture capital investment. According to one report, more than USD 18 billion has been invested worldwide in generative AI applications (stateof.ai). At the same time, the market structure is also changing alongside the big tech companies developing the strongest models (OpenAI, Google, Meta, etc.), there is a growing demand and drive to create open-source alternatives. This is driven both by the need for transparency and control, and by the recognition that the performance of powerful models may soon face data limitations - the available human-derived data cannot be extended indefinitely, so new methods are needed to scale further (stateof.ai).

Widespread democratization

Among the short-term trends, the democratization of AI should be highlighted, i.e. AI tools are becoming increasingly accessible to users without technical expertise. In 2023, several low-code/no-code AI platforms will emerge, offering AI functionality in a graphical interface using pre-trained models.² This will allow smaller organizations or even individual users to take advantage of AI - for example, to create data analytics applications - without having to acquire deep machine learning knowledge. This phenomenon contributes to bringing AI capabilities to the “masses”, partly addressing the bottleneck caused by the lack of AI experts. Bernard Marr² argues that from 2023, ‘AI from the armchair’ will become a reality, as anyone can create AI algorithms on simple interfaces, reducing the AI skills gap in organizations.²

The impact of AI in different sectors

In the short term, we see that the advance of AI is having an immediate impact in certain areas. For example, marketing, sales and customer service are the areas where generative AI tools are spreading fast - these are the functions where companies can create value the fastest.¹⁰ A recent survey found that marketing and product development departments are leading the adoption of generative AI, consistent with the fact that these areas are estimated to be the most likely to benefit from AI.¹ In addition, there have also been spectacular breakthroughs in scientific research in the short term: AI has been successfully applied in drug discovery (design of new molecules), healthcare (image diagnostics) and other life science applications (stateof.ai). These initial results confirm that AI is having a multidisciplinary impact in the short term, stimulating innovation in several areas.

The appearance of regulatory and safety reflections

The explosive growth of AI in the short term raised awareness among lawmakers and society about the potential downsides of technology. In 2023, the media was also abuzz with the potential dangers of AI - the public was exposed to the phenomenon of hallucinations (misinformation generated by models), the threat of deep-fake videos and voices, and the fact that AI decisions are often opaque “black boxes”. This has led to increased interest in AI security worldwide: professional discussions have been launched on how to measure and improve the reliability of models (stateof.ai). At the governmental level, there has been an unprecedented speed of response: the European Union has started intensive work to finalize the Artificial Intelligence Regulation, while several countries (e.g. US, UK) have issued recommendations and frameworks for the ethical and safe use of AI. However, this regulatory activity is also divided: the international community is not yet unified in its approach, with different governments emphasizing different principles (stateof.ai). In the short term, the picture is therefore mixed: there is great excitement about the potential of AI, but at the same time there is a discourse and first concrete steps (e.g. voluntary commitments, ex-ante guidelines) to mitigate risks.

2.2 Medium-term trends (1–5 years)

Widespread business embeddedness

Integration of AI technologies is expected to deepen and spread across most industries in the next 1–5 years. While in the short term many companies have introduced AI on an experimental basis, in the medium-term AI transformation will become part of everyday business operations. According to McKinsey’s annual survey,¹ “AI high-performer” companies (where at least 20% of operating profit is driven by AI) are already adopting AI in more business areas and are focusing less on cost reduction and more on creating new revenue streams. This trend could become more general: companies are increasingly realizing that AI is not just for automation, but can be the basis for new products, services and business models. In the medium term, the use of AI will deepen in areas such as research and development (product design optimization, simulations), logistics and supply chain management (predictive models), or HR (workforce planning, talent identification.¹ A recent survey shows that companies that have successfully adopted AI have typically implemented the technology in 4 or more business functions, while laggards have typically only tried it in 1 or 2 areas. This gap is expected to widen as leading firms implement complex, enterprise-wide AI strategies.

Labour market impacts of MI and retraining

In the medium term, MI is leading to an increasing labor market transformation. Most analysts expect technology to transform rather than eliminate jobs. Based on surveys on the future of work in 2023, the majority of companies predict that they will reskill more workers than they lay off as a result of AI: almost 40% say that more than a fifth of workers will need new skills in the coming years, while only 8% expect their workforce to be reduced by more than 20% as a result of AI.¹ Another analysis, by the World Economic Forum (WEF), predicts that by 2030, AI and automation could create 170 million new jobs globally, while eliminating 92 million old ones, so overall job growth, but with a significant reallocation.³ These figures imply that in the medium term, retraining and upskilling programmes will be valued. New professions will emerge, for example, ‘prompt engineering’ (expert design of AI inputs) is already a sought-after skill: in 2023, ~7% of firms using AI hired a specialist specifically for a prompt engineering position.¹ The democratization of AI will also help labor market adaptation - easy-to-use AI tools will enable a wide range of workers to expand their skill sets and incorporate AI into their daily tasks.

Technological developments and new research directions

The medium-term development of AI is expected to be characterized by the emergence of new research foci, in addition to the refinement of the current main paradigms. Further development of transformer-based language and image models will continue - e.g. even larger context window models, improved memory and multi-modality (ability to handle text, image, sound simultaneously) - while the community is already looking for ways to increase efficiency. The trend is towards achieving similar performance from less data and with smaller model sizes (e.g. fine-tuning for specific tasks, using specialized models in a particular domain instead of gigantic general models). In the medium term, we are likely to witness a diversification of the AI ecosystem: cloud giants (Google, Microsoft, Amazon) will integrate AI services into their platforms, while smaller players will offer customized AI solutions built on open models. In addition, rapid progress can be expected in areas such as the explainability of neural networks (XAI), human-robot collaboration (humanoid robots, further development of self-driving vehicles), or specialized AI in healthcare (e.g. drug design AIs capable of generating new molecules). Experts predict that by 2030, AI-based diagnostics could be routine in healthcare, detecting serious diseases at an early stage, even in the home environment. Laying the foundations for this will be an intensive research effort over the next 5 years.

Competition and geopolitics around MI

In the medium term, global competition in the development and use of AI will continue to intensify. The current leadership of the United States is being challenged: China is mobilizing huge resources in AI research and is rapidly catching up in publications and innovation (hai.stanford.edu). According to the Stanford University AI Index report, as early as 2023, China’s share of AI research and investment was already rising sharply, approaching or surpassing the US in some areas (hai.stanford.edu). In parallel, the EU is seeking to lead in a regulatory and coordinating role, although there are also significant research centers in Europe. The next 5 years may determine whether some form of international cooperation in the regulation and development of AI will emerge, or whether competition will lead to divergent standards and ethical norms. The risk of geopolitical rivalry is that some countries will use AI for strategic advantage (e.g. advanced cyber weapons, surveillance systems), which could trigger a new type of arms race in digital space. At the same time, an optimistic scenario in the medium-term trends is the vision of a concerted effort: possible international agreements on certain limits to AI development (as in the case of nuclear weapons), for potentially catastrophic AI systems.

2.3 Long-term trends (5–10 years)

Potential for general AI and superintelligence

Looking ahead over the next 5–10 years, one of the most exciting (but also most uncertain) questions is whether the breakthrough of Artificial General Intelligence (AGI) can be achieved. By AGI, we mean machine intelligence that operates at least at human levels of cognition, and possibly even surpasses the full range of human cognitive abilities. Although many experts believe that AGI is unlikely to be achieved before 2030, others do not rule out that discoveries could be made in this decade that could catalyze AGI development. In the long term, we should therefore expect a trend towards AI systems with increasingly generalized capabilities: a single system will be able to perform a wide range of tasks (language understanding, vision, decision making, planning) with minimal human intervention. If this happens, it could bring about a paradigm shift in society and the economy. It is important to stress, however, that there are considerable uncertainty and professional disagreement about the prospects for AGI - making it a scenario rather than a certain trend.

The pervasive presence of the AI

A more realistic long-term expectation is that AI systems will become a core infrastructure in almost all sectors and walks of life within 5–10 years. By around 2030, it is predicted that most companies (~86% according to the WEF) will have integrated some form of AI into their processes.³ Work will be transformed: routine tasks will be automated, while human work will focus on collaboration with machines and higher-level creative, strategic tasks. AI assistants, personalized AI helpers that support everyday tasks (even by proactively making suggestions and taking action in the background), could proliferate. There could also be leaps and bounds in robotics: the number of intelligent robots (manufacturing, logistics, service robots) operating in the physical world could multiply in the next decade. A bold prediction is that by 2030, hundreds of thousands of humanoid robots will be present in everyday environments worldwide, signaling the exit of digital intelligence from virtual space into physical reality.⁷ While this latter prediction may be over-optimistic, trends suggest that the convergence of cyber-physical systems (IoT devices, robots, autonomous vehicles) and AI will accelerate in the long term.

Socio-economic restructuring and new challenges

The long-term impact of AI goes far beyond the technological arena. It is expected to boost economic productivity - one estimate suggests that AI could raise productivity in advanced economies by 0.5–0.9 percentage points per year by the late 2020s⁴ - but the distribution of benefits will be critical. There may be a challenge that the value created by AI is concentrated in a few technology giants or in certain countries, increasing economic inequalities. At the same time, the structure of the labor market is fundamentally changing: many traditional professions may disappear or shrink, while new professions and industries may grow up around AI. In the long term, social adjustment will be needed, for example from the education system (to prepare young people for the AI collaboration) to social safety nets (if the transition leads to temporary unemployment in certain sectors). Policy measures such as retraining of workers with public support or even the introduction of unconditional basic income will be needed if job polarization increases.

The relationship between AI and humans, ethical dimensions

In the long term, a new set of norms for human-AI coexistence is emerging. One element of this will be the extent to which people accept and trust machine decisions (e.g. whether to accept medical therapy proposed by AI or to insist on a human doctor). The issue of trust and trustworthiness will remain key. If advances in technology are not matched by advances in transparency and accountability, a crisis of trust may develop people will be afraid to use AI in critical areas. On the other hand, if ethical and accountable AI systems can be developed, this will greatly help acceptance. In the long term, there is also the question of how we will view advanced AI in legal terms - for example, will there be a need for new forms of liability or legal personality (e.g. when an autonomous AI system causes harm, who is liable?). The EU AI Act and the first generation of other regulations will probably need to be amended in the light of experience to keep pace with new challenges.

Existential risks

Finally, the debate on existential risk is one that could become a real issue in the long term. In 2023, hundreds of prominent experts and business leaders have warned that the highly advanced AI systems of the future could threaten the very existence of humanity if we do not set appropriate limits. According to one statement, “mitigating the risk of extinction from AI should be a global priority - like pandemics and nuclear war”.⁴ This is not yet a consensus view, but it illustrates that long-term AI scenarios may require a global governance mechanism to ensure that the development of AI systems is in the interest of humanity and does not spiral out of control. In the long term, one can imagine, for example, an international AI oversight body or convention to regulate the research and application of frontier AI.⁸

3.1 Ethical and societal challenges

Bias and discrimination

AI systems often learn based on past data and thus run the risk of reproducing or amplifying biases in the data. Several studies have shown, for example, that image recognition algorithms are inaccurate in recognizing the faces of certain ethnic groups,¹⁰ or that AI systems for recruitment may discriminate against certain candidates based on past data e.g. Petruska¹⁰ draws attention to a similar problem in the context of religious issues. Such biased decision-making is a serious ethical problem, as it violates the principle of equal treatment and can undermine trust in AI. The challenge is both technical - how to ensure fair (equitable) models, what data to use - and legal and ethical. One of the main reasons for the EU AI Regulation is precisely that existing legislation is not sufficient to address the specific challenges (such as opacity and discrimination) caused by AI.¹¹ For high-risk AI systems, the AI Act sets strict data quality and risk management requirements, inter alia precisely to minimize discriminatory outputs.

Lack of transparency and explanations

The billions of parameter neural networks of modern deep learning models make decisions with virtually indecipherable internal logic and are therefore often “black box” in nature. This causes problems in any situation where it is important to understand why AI has arrived at a particular result. For example, in the case of an AI-supported medical diagnostic system, it is essential for both the doctor and the patient to know what the system is recommending a particular therapy based on. Similarly, in the case of credit scoring AI, it may be expected to explain why it has rejected someone. Currently, however, most models cannot do this - this is the explainability gap. In 2023, there are increasing efforts to address this problem: both in academic research (XAI - Explainable AI trend) and in industry, it is recognized that the key to trust in AI systems is better transparency.¹² Companies are also increasingly demanding at least partial control and audit of their algorithms. As part of ethical behavior, organizations implementing AI will be expected to provide some mechanism for explanation or human oversight at critical decision points. Regulation also has a role to play in addressing this challenge: the EU AI Act imposes transparency obligations in certain cases - e.g. to indicate to users when a chatbot is not human but AI,¹² or to ensure that the creators of generative models can identify their content as AI-generated (e.g. in the form of watermarks or metadata). These requirements aim to narrow the transparency gap, but the technical and operational details of their implementation remain challenging.

Social trust and acceptance

One of the ethical challenges is how the public perceives artificial intelligence. Trust is based in part on the two factors mentioned above (correct and explainable operation), but other factors also play a role. For example, the question of accountability: if an AI system makes a mistake, who is responsible? Until there is a clear and fair answer to this question, people are likely to be more distrustful of such systems. Transparent communication with the public about the capabilities and limitations of AI is also important. Currently, there is a lot of misunderstanding and exaggerated expectations surrounding AI stories in the media, which can distort expectations (e.g., many people believe that today’s AI “understands” meaning, when in fact it is largely a matter of statistical pattern recognition). The challenge is therefore to establish a kind of social dialogue on AI: how to ensure that the public has a realistic picture and uses the technology responsibly (e.g., not blindly trusting GPS directions or ChatGPT responses without checking them critically). Governments and civil society organizations also have a responsibility to help develop guidelines and ethical standards (e.g., voluntary AI codes of ethics for developers).

Data protection and privacy

Another ethical and legal challenge is that artificial intelligence systems use huge amounts of data, often including personal data. Generative language models, for example, are trained on billions of texts found on the internet, which inevitably include private information. This raises questions of consent and ownership: have people given permission for their data to be used to train an AI model? How can compliance with data protection rules (e.g., GDPR) be ensured during the development of AI systems? In 2023, there was a spectacular example when the Italian data protection authority temporarily blocked ChatGPT in the country, citing data protection concerns, highlighting the tension between privacy protection and AI development. To address this challenge, developers need to look for solutions such as data anonymization, using only authorized data, or new techniques (e.g., federated learning, where data stays in-house and only the parameters of the learned models are aggregated). On the user side, it is important to raise data awareness: what information do we share with AI systems (for example, it is not advisable to enter confidential company data into a public chatbot, as it could potentially learn from it and leak it). Although the AI Act does not explicitly address the processing of personal data – there are separate EU regulations (GDPR, ePrivacy) — but it indirectly affects the topic by banning certain AI applications that could seriously violate privacy, such as mass biometric surveillance in public spaces with real-time facial recognition, which the AI Act refers to as a prohibited practice.¹² This also shows that the challenges of data protection and AI ethics are closely linked.

3.2 Technological and operational challenges

Accuracy and reliability

Despite the impressive performance of today’s AI models, they are far from flawless. Large language models can be prone to hallucination—generating statements that appear completely plausible but are factually false. Sensor-based AI (e.g., sensors in self-driving cars) sometimes fail in unexpected situations. All of this points to a lack of robustness in the models. The challenge lies in ensuring that systems perform consistently well in the diverse conditions of the real world and that it is possible to assess when a model cannot be trusted. Currently, a significant proportion of companies are not prepared to manage the risks associated with AI: a 2023 survey found that only 21% of organizations have internal policies for the use of generative AI, and the majority do not actively mitigate the risks associated with AI deployment. Specifically, the most frequently cited risk in generative AI was inaccuracy (incorrect or fabricated information), but only ~32% of companies are taking steps to mitigate this.¹ Even more worrying is that the vulnerability of AI systems to cyberattacks is also poorly addressed: while half of companies sought to protect their AI models from cyber traps in 2022, only 38% said they did so in 2023.¹ This declining attention suggests that, amid rapid deployment, ensuring reliability may take a back seat—which poses a serious technical challenge for professionals. One solution could be to integrate MLOps (Machine Learning Operations) practices into companies’ daily routines: continuous model monitoring, real-time alerts in case of anomalies, regular retraining and version tracking of models. Current high-performing companies also report that the challenges of managing models and tools (e.g., monitoring model performance in a live environment) are the biggest challenges for them, rather than the strategic acceptance of AI, while less mature companies still face fundamental issues (lack of AI strategy and resources). This suggests that companies need to grow into the issue of technological reliability: operating systems running AI require the same care as any other critical IT system.

Data quality and data availability

The saying goes: “garbage in, garbage out”. If the data used to train models is noisy, incomplete or biased, the model’s performance will be poor. In the real world, however, the data available is often inaccurately labeled or unrepresentative. Another challenge is that many companies store their information in data silos that are not prepared for AI purposes (e.g., unstructured text or simply legal barriers to data use). The fact that poor or false data reduces the security level of AI applications also calls for increased scrutiny of training data.¹³ Industry experience shows that a large part of MI projects (up to 70–80%) are spent on cleaning and preparing data before any models are built.¹³ This requires significant resource investment and is often a bottleneck in the project. Meanwhile, in certain critical areas—such as rare disease research or specialized industrial quality control—there is little data available, which hinders the applicability of AI. One response to this challenge is the generation of synthetic data (which is made possible by generative AI): that is, the creation of artificial data that has similar statistical properties to real data but can be produced in unlimited quantities.¹⁴ Overall, the challenge of data quality in AI is complex: it has technological (data management tools, ETL processes), organizational (data management culture), and often legal (data sharing possibilities) aspects.

Scalability and costs

Although cloud infrastructures offer flexible resources in theory, running and storing large AI models can be extremely costly. Training a model with tens of billions of parameters can cost millions of dollars. Even during runtime—for example, the computing capacity behind ChatGPT—responding to each query incurs significant costs. This makes it challenging for many organizations to operate AI solutions in an economically sustainable manner. The cost is not only monetary but also in terms of energy, which has implications for environmental sustainability. The scalability challenge includes ensuring that systems can handle increasing loads (e.g., if many users suddenly start using our AI service) and how to integrate AI with existing legacy systems in a large enterprise environment. The technological response to scalability issues is to compress models so that they run on fewer resources, and the emergence of edge AI, where some of the computation is performed by devices on the edge network instead of sending everything to the cloud. Overall, the challenge is how to make AI scalable massively and cheaply without compromising performance.

Adversarial attacks and robustness

The vulnerability of AI systems to malicious manipulation poses a technical challenge. Research has shown that image recognition networks, for example, can be fooled by adversarial patterns—tiny, imperceptible disturbances added to the input data that cause the model to classify objects incorrectly. Similarly, language models can also be confused by special inputs or prompts that cause the model to produce prohibited outputs (the phenomenon known as prompt injection). These are not just theoretical exercises: attacks against AI systems are now considered a cyber security threat.¹⁵ Increasing robustness – i.e., ensuring that the model does not react drastically to small perturbations – is an active area of research. There are defensive techniques, but the cat-and-mouse game between attackers and defenders can also be observed here. The AI Act also addresses this issue: it requires high levels of cybersecurity and robustness for high-risk AI systems.¹⁶ However, putting this into practice is not trivial—how do we validate that a model is sufficiently robust? What metrics should be introduced for this? There are no uniform answers yet, but industry standards are likely to emerge in the coming years.

System integration and change management

For many organizations, the challenge of AI lies not in the development of models per se, but in how they fit into existing business processes. Introducing AI often requires process rethinking: workflows may need to be reorganized, new roles introduced (e.g., a person in the loop to oversee AI decisions), or customer service channels modified (e.g., chatbot vs. human agent ratio). This requires organizational change management, which can be difficult in companies with entrenched operating practices.

Internal resistance

A common obstacle is internal resistance to AI: employees fear for their jobs or are simply distrustful of the new tool. Developing the right AI strategy is also a challenge for senior management—many companies lack a clear AI vision, resulting in ad hoc, siloed solutions. According to McKinsey data, less mature AI users most often cite the lack of an AI strategy and insufficient resources as the main obstacles.¹ So, the challenge is not just technological, but also managerial: how to scale and “operationalize” AI at the enterprise level. The best practice here is to start with small-scale pilots, learn from them, and then gradually roll out the best solutions. Management support and vision are important, as is the involvement of stakeholders (e.g., training end users in the use of AI tools and incorporating their feedback). Overall, the success of technology depends largely on whether it can be successfully implemented at the human and process levels, which is at least as challenging as writing the algorithm itself.

Cybersecurity challenges

The emergence of AI systems expands the toolkit of both attackers and defenders, creating new threat vectors and defense opportunities. On the one hand, as mentioned above, the vulnerability of models is a challenge: adversarial attacks, data manipulation, and attempts by cybercriminals to steal or compromise AI models all pose new security risks.¹⁶ On the other hand, AI itself can become a weapon: malicious actors can use it to automate attacks. Looking ahead to 2024, experts expect AI-driven cyberattacks to become more common, e.g. AI-generated, personalized phishing emails that are more deceptive than their human-written counterparts.⁶ Tools for this purpose have already appeared on the darknet, such as the infamous WormGPT, a language model specifically fine-tuned for attackers that can be used, for example, to generate malicious code.

At the same time, AI can also help on the defensive side: AI-based intrusion detection systems, anomaly detection in network traffic, automated incident management—all areas where machine learning offers significant added value. The challenge, however, is for defenders to keep pace with attackers’ use of AI. According to a 2024 global survey, 56% of executives believe that emerging technologies such as AI are more beneficial to attackers, and only ~9% believe that they will benefit from defense.¹⁷ This highlights that the security community is currently wary of the increased attack surface posed by AI, e.g. through the many vulnerable IoT devices and new threats such as deepfake-based social engineering or the future cryptographic capabilities of quantum computers (which could become a reality in 5–10 years).¹⁷

3.3 Economic and environmental sustainability challenges

Costs and business returns

While we mentioned in the medium-term trends that AI has the potential to increase productivity, in the short term, many initiatives are hampered by uncertainty about returns. Companies are spending significant amounts on AI development, but it is not always clear in advance what the ROI will be. It is common for pilot projects to be launched that either fail to scale up or do not fit well into the business strategy, thus becoming money pits. This is the phenomenon of “POC hell” – many Proofs of Concept, few production results. The challenge here is to find the right use cases and business-professional collaboration: AI experts and business decision-makers need to identify tasks where AI really brings a competitive advantage and measurable results (cost reduction or revenue growth). Without this, there is a risk that the AI hype will run out of steam and lead to another “AI winter” because executives are disappointed with the results. According to Gartner’s famous Hype Cycle, generative AI is at the peak of excessive expectations in 2023 and is likely to fall through the trough of disillusionment before reaching a realistic productivity phase. The reason often lies in exaggerated promises and a lack of adequate business preparation.

AI skills shortage and organizational capabilities

The human resources required to implement AI may continue to be a bottleneck. Although democratization is improving the situation, complex projects still require data scientists, machine learning engineers, and MLOps experts. Tech layoffs in recent years have eased labor market pressures somewhat (more professionals have become available), but it remains difficult to find the right people for certain key roles.¹ There is a particularly high demand for people with combined knowledge who understand both business and AI. The challenge here is training and knowledge transfer: how to retrain traditional IT or domain experts to become experts with AI competencies, and how to attract and retain talent. Companies often struggle with the scaling of AI projects because they may have two or three data scientists in a team, but when dozens of projects are launched, there are not enough people to handle them all. This can be helped using the aforementioned no-code tools and partnerships (e.g., with universities and startups). Knowledge sharing within the organization will also be an important part of building an AI ecosystem, e.g., through internal AI communities, training programs, and mentoring.

Environmental impact and sustainability

The rapid development of artificial intelligence comes with a significant environmental footprint. Training large models requires enormous computing capacity, which means a lot of electricity consumption. If this energy comes from fossil fuels, carbon emissions are also substantial. For example, one estimate suggests that training the OpenAI GPT-3 model once resulted in more than 500 tons of CO2 emissions¹⁸ – roughly equivalent to the emissions of 500 transatlantic flights. Even in the case of Bloom, an open model, ~25 tons of CO2 were generated during training, which increased to ~50 tons over its entire life cycle (including hardware production and operation).¹⁹

These figures highlight the fact that AI’s energy hunger is a serious challenge in the fight against climate change. Large tech companies are striving to power their data centers with renewable energy and set carbon-neutral targets, but as AI becomes more widespread, the emissions of the entire ICT sector are increasing. According to some estimates, in 2020, the IT sector (including AI) accounted for ~2–3% of global emissions, which is almost the same as the share of aviation (Kirkpatrick, 2023). If this continues to rise, it will be problematic.

The challenge is twofold: making AI more efficient (algorithms, increasing hardware efficiency) and relying on renewable energy sources for operation. On the technology front, there are already efforts to “green” models (e.g., optimized training algorithms, more energy-efficient chip architectures). Transparency is also important: currently, developers of large models are reluctant to publish their exact energy consumption, citing trade secrets, which makes it difficult for researchers to measure efficiency. It has been suggested that regulations could require model developers to report their carbon footprint, but this has not yet been included in the final text of the AI Act.¹⁹ In any case, in the long term, the AI industry will also have to adapt to sustainability requirements—this is a challenge, but also an opportunity for innovation.

4.1 Risk classification and obligations

The AI Act introduces a risk-based approach with four risk levels⁹:

The AI Act therefore strikes a balance based on the above “pyramid”: it cuts off the most risky (human rights or life-threatening) AI applications, strictly controls high-risk ones, and leaves the rest alone. This differentiated approach is novel in technology regulation and has been the subject of much debate as to what should fall into which category. The definitions have been refined in the final text (e.g., generative models have been given special mention in the “restricted” category with transparency obligations).

4.2 Expected impacts and challenges arising from the AI Act

The challenge of innovation vs. regulation

The AI Act raises the question of how to provide protection without stifling innovation. Many consider the European approach to be strict—for example, the US prefers loose, guideline-based rules, while China takes a content censorship approach. Within the EU, startups and smaller developers fear that the high-risk AI classification will place too much administrative and financial burden on them, which could put them at a competitive disadvantage. For example, a medical AI startup must comply with CE certification as well as the AI Act (dual compliance), which could slow its entry into the market. At the same time, larger companies have welcomed the regulation because it creates a uniform framework in Europe, so they only must comply with one set of national rules instead of 27. The regulation also introduces a European AI Authority to coordinate implementation and the development of technical standards.²⁰ The EU’s aim is to use this reliability framework to gain a competitive advantage: as the first in the world to regulate this area, it hopes that the European “trustworthy AI” brand will become a mark of quality and a global standard (just as the GDPR has become the benchmark in many countries). However, this is still questionable – in the short term, there may be some uncertainty among developers.

Compliance and costs

Developers of high-risk AI systems will need serious preparation to apply the AI Act. Standardization is expected to make things easier for them: for example, ISO standards are being developed for AI management systems, and AI-specific audit frameworks will be added to the existing ISO 27001 (information security) and ISO 9001 (quality management) standards. Companies will need to establish internal processes to verify the legal compliance of their AI systems, and many will appoint AI compliance officers. As with the General Data Protection Regulation, violations of the AI Act will be subject to very high fines (up to 6% of global turnover), so larger companies will certainly take it seriously. The main challenge will be documentation and risk management: developers will have to compile documents about their models that they are not typically accustomed to writing, e.g., describing how representative the dataset is, what possibilities for bias exist and how they have been addressed, the exact cases for which the model is and is not suitable, the built-in control mechanisms, etc. This is essentially the creation of a technical report on the models. It is advisable to think about this during development (built-in compliance) rather than trying to retrieve the information afterwards. New tools and platforms may help here (e.g., AI model audit trail software).

Boundaries and new questions

The AI Act has not been able to resolve all issues completely. For example, in the case of generative general-purpose models (foundation models), a compromise has been reached: the law does not prohibit them (although there were calls to classify very large models such as GPT-4 as high risk), but it does impose a few requirements: transparency (labeling) and safety assurance by the developer. The latter is vague but could cover things such as security procedures for fine-tuning models, filtering dangerous outputs, and the legal clarity of training data. The Future Society, for example, has made a separate proposal for the regulation of GPAI (general purpose AI) models, calling for even stricter requirements for very powerful models (e.g., regulatory approval above a certain size),¹⁴ but this proposal was not fully incorporated into the text – for the time being, the milder obligations remain in place. This means that large models (such as GPT-4) can continue to be used in Europe, but service providers will have to register them and provide content labeling, for example. However, if, for example, a bank uses this model for internal decision support, then the banking application will be the high-risk system to which all regulations apply. Thus, responsibility is partially shifted to the user, not just the model creator.

Enforcement

The AI Act stipulates that if someone suffers harm due to an AI system, they should have the right to complain and seek redress. However, in practice, mechanisms need to be put in place for this – for example, supervisory authorities (at Member State level) need to be prepared to investigate AI incidents. This is a new area, such as, for example, if a self-driving car causes an accident, in addition to the traffic accident investigator, the AI supervisory authority may also have a role to play in investigating whether the system complied with the regulations.

DORA and NIS2 connections

In addition to the AI Act, the regulatory environment also includes DORA (2022), NIS2 (2022) and other horizontal legislation (e.g. the Digital Services Act, which regulates the handling of illegal content on platforms, including deepfakes). These rules interact with each other. For example, under NIS2, a hospital is required to guarantee the security of its IT systems – if it uses AI for diagnostics, it is subject to both NIS2 and the AI Act. This results in a complex compliance matrix. Ideally, the EU aims to ensure that these rules are aligned and do not impose conflicting requirements. For example, the risk assessment required under DORA may also apply to AI systems, which overlaps with the risk management requirements of the AI Act – here, the two are likely to be integrated in the financial sector, and the requirements of both pieces of legislation can be fulfilled in a single process (e.g. a bank prepares an AI risk report that complies with the AI Act and submits it as part of its ICT risk report under DORA).

Global impact

The AI Act is likely to be a globally impactful regulation, as any company wishing to sell or operate an AI system in the EU will have to comply with it (like the GDPR). For this reason, multinational companies often decide to follow the strictest rules everywhere so that they do not have to maintain different versions. Thus, the provisions of the AI Act – high data quality, human oversight, transparency, etc. – could even become a de facto global standard. At the same time, there is competition: in October 2023, the US issued an Executive Order on the safe development of AI, which uses different tools but also encourages risk management.²⁰ International forums (G7, OECD, UN) are also working on soft law recommendations. DORA and NIS2 have less direct impact outside the EU, but group-level application is likely in the financial world, for example, due to the European subsidiaries of large global banks.⁹

Overall, the regulatory landscape suggests that AI development will increasingly take place in a regulated environment, especially for critical applications. This is a new challenge for developers and operators, but also an important step towards ensuring responsible AI. The AI Act and related legislation send the message that AI cannot be a partisan project: it must be subject to the same safety, ethical, and legal standards as any other technology before it is widely deployed.