United States

AI has much to learn from other areas in cognitive science, and
can in turn contribute much to them.&lt;br&gt;&lt;br&gt;
**Main Takeaways**
* Cognitive Science is a multidisciplinary field that was inspired by AI’s
exploration of the hypothesis of computation as a scientific language for
understanding cognition.
* Some continued interactions between AI and other areas in cognitive science
have yielded valuable insights and systems, notably cognitive architecture.
* Expanding these interactions could yield important advances for both fields. &lt;br&gt;

**Context &amp; History**&lt;br&gt;&lt;br&gt;
AI was the first field founded on
the intellectual hypothesis that
computation could become a scientific
language for understanding the nature
of intelligence, no matter what the
substrate. Cognitive Science was the
second, a multidisciplinary gathering
of researchers in AI, psychology,
linguistics, neuroscience, anthropology,
and other disciplines. Computational
ideas from AI were highly influential
in early cognitive science. However,
over time, AI has drifted apart from the
rest of cognitive science, for a variety
of reasons [3]. We believe that there
are now important benefits to be
gained from rebuilding those bridges
and exploring how progress in AI can
help understand human cognition (and
animal cognition more broadly) and
how progress in other areas of cognitive
science can help us build better AI
systems. In some cases, this will be
learning how to achieve in software the
kinds of cognitive capabilities organisms
have, and in other cases, deliberately
choosing to be different in ways that
complement human cognition so that
human-AI teams are more productive.&lt;br&gt;&lt;br&gt;
**Current State &amp; Trends**&lt;br&gt;&lt;br&gt;
Cognitive Science is broad, so we focus
on three areas where research is likely
to be synergistic with AI.&lt;br&gt;&lt;br&gt;
**Human-like learning and reasoning.**
Many animals learn, but humans are
pre-eminent learners and reasoners
in many ways. A surprising amount
of human learning is, in machine
learning terms, incremental, continual,
and data-efficient, often producing
articulable models (e.g. Gentner
&amp; Maravilla, 2018). While today’s industrial knowledge graphs reach into
the tens of billions of facts, they lack the
expressiveness of human conceptual
structure [4]. Today’s reasoning systems,
like SAT solvers and model checkers,
are often superhuman in the size
of the problems they address and
the complexity of the solutions they
generate [2]. But today’s AI reasoning
systems cannot reason robustly with
incomplete and partially incorrect
domain theories, nor can they reason
from large bodies of experience as
people do.&lt;br&gt;&lt;br&gt;
**Cognitive architectures** are systems
that explore hypotheses about the fixed
structures that define the processes
and representations used for cognition
[7]. They are used to investigate how
to build AI systems that do real-time
integration of perception, cognition, and
motor control across many tasks, and to
better understand human intelligence.
For example, cognitive architectures
have been used to simulate findings
(and make predictions) from cognitive
psychology and cognitive neuroscience
(e.g. [1,8]). While every cognitive
architecture involves multiple
processes and representations, they
vary considerably in the subset of
human cognition they explore and the
granularity of assumptions made.&lt;br&gt;&lt;br&gt;
**Social agents.** One of the signature
properties of humans is that we
construct and live in a world of
collaboration where we learn about
each other and culturally-specific social
norms through interaction with others.
Progress in understanding how to build
social agents is essential to building
AI systems that live in our world as
collaborators and partners [9]. Social
AI is often developed independently
of findings and theories from social
science and learns social behavior quite differently from how people acquire social skills.&lt;br&gt;&lt;br&gt;
**Research Challenges**&lt;br&gt;&lt;br&gt;
Progress on these challenges will
lead to more adaptable AI systems
and reduce the computational and
environmental burdens of our systems,
better understand human cognition,
including social cognition, and provide
better tools for thought. &lt;br&gt;&lt;br&gt;
**Human-like Learning and Reasoning**&lt;br&gt;&lt;br&gt;
1. How can we develop human-like
incremental, data-efficient learning
methods that can produce articulable
models?
2. Develop formal ontologies that
span the range of human conceptual
structures, both concerning abstract
concepts and sensory-motor grounded
concepts.
3. How can AI systems robustly reason
with incomplete and partially incorrect
domain theories, and use experience in
reasoning, with human-scale bodies of
knowledge?&lt;br&gt;&lt;br&gt;
**Cognitive Architectures**&lt;br&gt;&lt;br&gt;
1. Expanding the higher-level cognitive
capabilities of cognitive architectures
to include the dynamical integration
of the full range of human capabilities
in response to task demands: diverse
forms of reasoning, metacognition,
online, lifelong continual learning across
modalities and knowledge types, and
engaging in ongoing human interaction
(e.g.[6]). These capabilities will require
learning and reasoning over models of
the physical world, abstractions, and
other agents using symbolic relational
and modality-specific representations of the current, future, and past
situations.&lt;br&gt;&lt;br&gt;
2. Exploring the integration of
foundation models within cognitive
architectures, including sources for
knowledge and to interpret/generate
natural modalities (e.g. [10]). Can
the incremental learning capabilities
exhibited by cognitive architectures
overcome the limitations of stale
information in foundation models?&lt;br&gt;&lt;br&gt;
3. Developing a comprehensive
benchmark task suite to evaluate the
breadth and integration of human
cognitive capabilities in end-to-end
performance as described above.
The tasks should be diverse and
broad to ensure robust assessments.
Additionally, the tasks should
be diagnostic, isolating cognitive
capabilities and their interactions to
provide insights into specific strengths
and weaknesses. &lt;br&gt;&lt;br&gt;
**Social Agents**&lt;br&gt;&lt;br&gt;
1. Facilitate learning through interaction:
The current generation of AI systems
learn by passively observing social
behavior rather than participating
in social behavior (analogous to the
distinction between decision theory
and game theory). In contrast, people
continuously co-construct behavior by mutually adapting to each other. At best, AI systems simulate interaction
by training on frozen simulated users
(e.g., RLHF), but this fails to account
for mutual adaptation. Thus, we need
research into ways to support (or
simulate) interactive learning at scale.&lt;br&gt;&lt;br&gt;
2. Facilitate Privacy-preserving methods
to acquire social data: Human social
cues (face, voice) typically reveal the
identity of the social actor. Interpreting
social cues requires acquiring intrusive
situational information (e.g., the
meaning of a smile depends on not
just the face of the target person but
who else is in the situation, what they
are doing, the nature of the physical
environment, etc.). The ability to collect
this information is wisely restricted
by law (e.g., the EU’s AI act). Yet this
dramatically restricts the ability to
acquire data and deploy applications.
How do we create algorithms that
identify socially meaningful information
while proving that no future algorithm
could recover privacy/anonymity violating information from what is
stored? Developing methods that can
collect yet provably de-identify social
data is crucial for the advancement of
social agents.&lt;br&gt;&lt;br&gt;
3. Developing interactional
benchmarks: Given that AI aspires
to build systems with general social
capabilities, we need a reliable way
to measure and assess if new models
are improvements. This includes
characterizing potential for bias, issues
of value alignment, whether the model
is willing to engage in deception, etc.
People now propose ad hoc collections
of tasks, but research is needed to
develop a comprehensive taxonomy
of tasks and measures. In contrast, the
social sciences have developed theorybased ontologies for characterizing
social situations. Research is needed to
translate these findings into systematic
and comprehensive benchmarks
of human social and interactional
behavior.&lt;br&gt;&lt;br&gt;
**Community Opinion**&lt;br&gt;&lt;br&gt;
Engagement with other areas of cognitive science varies across
the survey respondents, with 30% responding to the questions in this
section. Among those who responded, in terms of influence on their research, 18% said always, 32% said usually, 32% said sometimes. Only 2.8% said never and 12% said rarely. Thus 82% are influenced to a reasonable degree by research in other areas of cognitive science. Which other areas provide the most influence? Our respondents report psychology (82%), Neuroscience (44%), Linguistics (40%), and Anthropology (22%), with 13% mentioning other fields, Philosophy
being the most common. In terms of what issues are most relevant, a
broad set of creative responses were produced, some quite creative, e.g.
studying how minds completely different to our own might function.&lt;br&gt;&lt;br&gt;
1. Anderson, J. R. (2007). How can the human mind exist in the physical universe? New York, NY: Oxford University Press.
2. Biere, A., Heule, M., et al. (2021) Handbook of Satisfiability, 2nd Edition, IOS Press
3. Forbus, K. (2010). AI and Cognitive Science: The Past and Next 30 years. Topics in Cognitive Science, 2(3), p. 346-356, https://doi.org/10.1111/j.1756-8765.2010.01083.x
4. Forbus, K. (2021). Evaluating revolutions in artificial intelligence from a human perspective. In OECD, AI and the Future of Skills, Volume 1: Capabilities and Assessments, OECD
Publishing, Paris. DOI:https://doi.org/10.1787/004710fe-en
5. Gentner, D. &amp; Maravilla, F. (2018). Analogical reasoning. L. J. Ball &amp; V. A. Thompson (eds.) International Handbook of Thinking &amp; Reasoning (pp. 186-203). NY, NY: Psychology Press.
6. Gluck, K. &amp; Laird, J. (2019) Interactive Task Learning: Humans, Robots, and Agents Acquiring New Tasks through Natural Interactions. MIT Press.
7. Kotseruba, I., &amp; Tsotsos, J. K. (2020). 40 years of cognitive architectures: Core cognitive abilities and practical applications. Artificial Intelligence Review, 53(1), 17–94. https://doi.
org/10.1007/s10462-018-9646-y
8. Laird, J. (2012) The Soar Cognitive Architecture, MIT Press.
9. Lugrin, Birgit; Pelachaud, Catherine; Traum, David (Ed.) (2021). The Handbook on Socially Interactive Agents, pp. 433–462, ACM, New York, NY, USA, 2021, ISBN: 978-1-4503-8720-0.
10. Sumers, T. R., Yao, S., Narasimhan, K., &amp; Griffiths, T. L. (2023). Cognitive architectures for language agents. Transactions on Machine Learning Research, arXiv preprint arXiv:2309.02427.

The Future of AI Research - Webinar Series

AI &amp; Cognitive Science

The AAAI 2025 presidential panel on the future of AI research aims to help all AI stakeholders navigate the recent significant transformations in AI capabilities, as well as AI research methodologies, environments, and communities. It includes 17 chapters, each covering one topic related to AI research, and sketching its history, current trends and open challenges. The study has been conducted by 25 AI researchers and supported by 15 additional contributors and 475 respondents to a community survey.

AI has much to learn from other areas in cognitive science, and
can in turn contribute much to them. 
**Main Takeaways**
* Cognitive Science is a multidisciplinary field that was inspired by AI’s
exploration of the hypothesis of computation as a scientific language for
understanding cognition.
* Some continued interactions between AI and other areas in cognitive science
have yielded valuable insights and systems, notably cognitive architecture.
* Expanding these interactions could yield important advances for both fields. 

**Context & History** 
AI was the first field founded on
the intellectual hypothesis that
computation could become a scientific
language for understanding the nature
of intelligence, no matter what the
substrate. Cognitive Science was the
second, a multidisciplinary gathering
of researchers in AI, psychology,
linguistics, neuroscience, anthropology,
and other disciplines. Computational
ideas from AI were highly influential
in early cognitive science. However,
over time, AI has drifted apart from the
rest of cognitive science, for a variety
of reasons [3]. We believe that there
are now important benefits to be
gained from rebuilding those bridges
and exploring how progress in AI can
help understand human cognition (and
animal cognition more broadly) and
how progress in other areas of cognitive
science can help us build better AI
systems. In some cases, this will be
learning how to achieve in software the
kinds of cognitive capabilities organisms
have, and in other cases, deliberately
choosing to be different in ways that
complement human cognition so that
human-AI teams are more productive. 
**Current State & Trends** 
Cognitive Science is broad, so we focus
on three areas where research is likely
to be synergistic with AI. 
**Human-like learning and reasoning.**
Many animals learn, but humans are
pre-eminent learners and reasoners
in many ways. A surprising amount
of human learning is, in machine
learning terms, incremental, continual,
and data-efficient, often producing
articulable models (e.g. Gentner
& Maravilla, 2018). While today’s industrial knowledge graphs reach into
the tens of billions of facts, they lack the
expressiveness of human conceptual
structure [4]. Today’s reasoning systems,
like SAT solvers and model checkers,
are often superhuman in the size
of the problems they address and
the complexity of the solutions they
generate [2]. But today’s AI reasoning
systems cannot reason robustly with
incomplete and partially incorrect
domain theories, nor can they reason
from large bodies of experience as
people do. 
**Cognitive architectures** are systems
that explore hypotheses about the fixed
structures that define the processes
and representations used for cognition
[7]. They are used to investigate how
to build AI systems that do real-time
integration of perception, cognition, and
motor control across many tasks, and to
better understand human intelligence.
For example, cognitive architectures
have been used to simulate findings
(and make predictions) from cognitive
psychology and cognitive neuroscience
(e.g. [1,8]). While every cognitive
architecture involves multiple
processes and representations, they
vary considerably in the subset of
human cognition they explore and the
granularity of assumptions made. 
**Social agents.** One of the signature
properties of humans is that we
construct and live in a world of
collaboration where we learn about
each other and culturally-specific social
norms through interaction with others.
Progress in understanding how to build
social agents is essential to building
AI systems that live in our world as
collaborators and partners [9]. Social
AI is often developed independently
of findings and theories from social
science and learns social behavior quite differently from how people acquire social skills. 
**Research Challenges** 
Progress on these challenges will
lead to more adaptable AI systems
and reduce the computational and
environmental burdens of our systems,
better understand human cognition,
including social cognition, and provide
better tools for thought. 
**Human-like Learning and Reasoning** 
1. How can we develop human-like
incremental, data-efficient learning
methods that can produce articulable
models?
2. Develop formal ontologies that
span the range of human conceptual
structures, both concerning abstract
concepts and sensory-motor grounded
concepts.
3. How can AI systems robustly reason
with incomplete and partially incorrect
domain theories, and use experience in
reasoning, with human-scale bodies of
knowledge? 
**Cognitive Architectures** 
1. Expanding the higher-level cognitive
capabilities of cognitive architectures
to include the dynamical integration
of the full range of human capabilities
in response to task demands: diverse
forms of reasoning, metacognition,
online, lifelong continual learning across
modalities and knowledge types, and
engaging in ongoing human interaction
(e.g.[6]). These capabilities will require
learning and reasoning over models of
the physical world, abstractions, and
other agents using symbolic relational
and modality-specific representations of the current, future, and past
situations. 
2. Exploring the integration of
foundation models within cognitive
architectures, including sources for
knowledge and to interpret/generate
natural modalities (e.g. [10]). Can
the incremental learning capabilities
exhibited by cognitive architectures
overcome the limitations of stale
information in foundation models? 
3. Developing a comprehensive
benchmark task suite to evaluate the
breadth and integration of human
cognitive capabilities in end-to-end
performance as described above.
The tasks should be diverse and
broad to ensure robust assessments.
Additionally, the tasks should
be diagnostic, isolating cognitive
capabilities and their interactions to
provide insights into specific strengths
and weaknesses. 
**Social Agents** 
1. Facilitate learning through interaction:
The current generation of AI systems
learn by passively observing social
behavior rather than participating
in social behavior (analogous to the
distinction between decision theory
and game theory). In contrast, people
continuously co-construct behavior by mutually adapting to each other. At best, AI systems simulate interaction
by training on frozen simulated users
(e.g., RLHF), but this fails to account
for mutual adaptation. Thus, we need
research into ways to support (or
simulate) interactive learning at scale. 
2. Facilitate Privacy-preserving methods
to acquire social data: Human social
cues (face, voice) typically reveal the
identity of the social actor. Interpreting
social cues requires acquiring intrusive
situational information (e.g., the
meaning of a smile depends on not
just the face of the target person but
who else is in the situation, what they
are doing, the nature of the physical
environment, etc.). The ability to collect
this information is wisely restricted
by law (e.g., the EU’s AI act). Yet this
dramatically restricts the ability to
acquire data and deploy applications.
How do we create algorithms that
identify socially meaningful information
while proving that no future algorithm
could recover privacy/anonymity violating information from what is
stored? Developing methods that can
collect yet provably de-identify social
data is crucial for the advancement of
social agents. 
3. Developing interactional
benchmarks: Given that AI aspires
to build systems with general social
capabilities, we need a reliable way
to measure and assess if new models
are improvements. This includes
characterizing potential for bias, issues
of value alignment, whether the model
is willing to engage in deception, etc.
People now propose ad hoc collections
of tasks, but research is needed to
develop a comprehensive taxonomy
of tasks and measures. In contrast, the
social sciences have developed theorybased ontologies for characterizing
social situations. Research is needed to
translate these findings into systematic
and comprehensive benchmarks
of human social and interactional
behavior. 
**Community Opinion** 
Engagement with other areas of cognitive science varies across
the survey respondents, with 30% responding to the questions in this
section. Among those who responded, in terms of influence on their research, 18% said always, 32% said usually, 32% said sometimes. Only 2.8% said never and 12% said rarely. Thus 82% are influenced to a reasonable degree by research in other areas of cognitive science. Which other areas provide the most influence? Our respondents report psychology (82%), Neuroscience (44%), Linguistics (40%), and Anthropology (22%), with 13% mentioning other fields, Philosophy
being the most common. In terms of what issues are most relevant, a
broad set of creative responses were produced, some quite creative, e.g.
studying how minds completely different to our own might function. 
1. Anderson, J. R. (2007). How can the human mind exist in the physical universe? New York, NY: Oxford University Press.
2. Biere, A., Heule, M., et al. (2021) Handbook of Satisfiability, 2nd Edition, IOS Press
3. Forbus, K. (2010). AI and Cognitive Science: The Past and Next 30 years. Topics in Cognitive Science, 2(3), p. 346-356, https://doi.org/10.1111/j.1756-8765.2010.01083.x
4. Forbus, K. (2021). Evaluating revolutions in artificial intelligence from a human perspective. In OECD, AI and the Future of Skills, Volume 1: Capabilities and Assessments, OECD
Publishing, Paris. DOI:https://doi.org/10.1787/004710fe-en
5. Gentner, D. & Maravilla, F. (2018). Analogical reasoning. L. J. Ball & V. A. Thompson (eds.) International Handbook of Thinking & Reasoning (pp. 186-203). NY, NY: Psychology Press.
6. Gluck, K. & Laird, J. (2019) Interactive Task Learning: Humans, Robots, and Agents Acquiring New Tasks through Natural Interactions. MIT Press.
7. Kotseruba, I., & Tsotsos, J. K. (2020). 40 years of cognitive architectures: Core cognitive abilities and practical applications. Artificial Intelligence Review, 53(1), 17–94. https://doi.
org/10.1007/s10462-018-9646-y
8. Laird, J. (2012) The Soar Cognitive Architecture, MIT Press.
9. Lugrin, Birgit; Pelachaud, Catherine; Traum, David (Ed.) (2021). The Handbook on Socially Interactive Agents, pp. 433–462, ACM, New York, NY, USA, 2021, ISBN: 978-1-4503-8720-0.
10. Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023). Cognitive architectures for language agents. Transactions on Machine Learning Research, arXiv preprint arXiv:2309.02427.

AI & Cognitive Science

webinar

AI for social good is a subdiscipline of AI research where
measurable societal impact, particularly for vulnerable and
under-resourced groups, is a primary objective, focusing on
areas that have historically lacked sufficient AI research and
development. 
**Main Takeaways**
* **Ethical and Impact-Driven AI Development** – AI4SG projects have grown significantly in the past decade spurred by advancements in AI and ML, and they have prioritized ethical considerations, fairness, and societal benefits, ensuring
solutions address real-world problems in a responsible manner.
* **Interdisciplinary Collaboration is Crucial** – Successful AI4SG initiatives require strong partnerships between AI researchers, domain experts, policymakers, and local communities to ensure relevance and long-term sustainability.
* **Scalability and Sustainability Challenges** – While AI4SG has demonstrated significant potential, maintaining and scaling solutions in resource-constrained environments remains a key challenge. 

**Context & History** 
 “AI for Social Impact” (AI4SI / AI4SG)
has emerged as a distinct sub-discipline
within AI, characterized by its focus on
measurable societal impact, particularly
for vulnerable and underserved
populations. Unlike traditional AI
research, which often prioritizes
methodological advancements, AI4SI
places direct social impact as a primary
objective. It addresses problems that
have historically lacked sufficient
attention in AI research, aiming to
bridge the gap between AI capabilities
and real-world societal challenges such
as poverty, agriculture, public health,
and environmental conservation. The
goal is to create impactful solutions
tailored to real-world problems, often
in resource-constrained environments. 
AI4SI research necessitates deep
engagement with domain experts
and community members to identify
relevant problems, design effective
interventions, and rigorously evaluate
their impact. This interdisciplinary
approach draws insights from fields like
human-computer interaction, public
health, and social work, emphasizing
the importance of understanding
and addressing the specific needs
of targeted communities. These
projects require a balance of technical
innovation, ethical considerations, and
practical feasibility. 
The “AI for Social Good” workshop
organized by the White House Office
of Science and Technology Policy in
2016 may be credited as the single
event that sparked a significant
interest in this topic [1]. It ended up
unifying diverse efforts focused on
social impact under one umbrella
emerging field. This surge in interest is
attributed to several key factors. First,
the remarkable advancements in AI
technologies, including deep learning,
natural language processing, and
reinforcement learning, have provided
powerful tools applicable to a wide
range of social issues. The availability of
increased computing power and large
datasets has further accelerated this
progress, enabling the development of
sophisticated AI models. 
Secondly, the establishment of
government and industry-supported
funding programs, dedicated
workshops, conferences, and special
tracks within major AI conferences has
increased awareness and attracted
researchers to AI4SG[2,3]. This
increasing academic focus has led to a
substantial rise in publications related
to AI4SI, demonstrating the field’s
growing maturity[4]. 
**Current State & Trends** 
Advancing interdisciplinary
collaboration is becoming the norm in
AI for Social Good work. This necessity
stems from the complex nature of
societal challenges, which often require
insights from diverse fields. Close
partnerships with domain experts,
local practitioners, and policymakers
ensure that AI solutions are not only
technically sound but also relevant
and effective in the real world. This
collaborative approach fosters a shared
understanding of the problem and
enables the development of solutions
that are tailored to the specific needs of
the communities they serve. 
Emphasizing ethical AI is also critical,
with fairness, transparency, and privacy
as paramount considerations. Major
concerns, such as bias in data collection
and the unintended consequences of
AI deployment, must be addressed
from the outset to ensure that AI
systems align with societal values and
avoid harm. AI4SG projects, by their
very nature, work with vulnerable
populations and sensitive data, making
ethical considerations even more
crucial. By proactively addressing
potential ethical issues, researchers
can build trust with communities and
ensure that AI is used for good, rather
than exacerbating existing inequalities. 
Several emerging opportunities are
shaping the future of AI4SG. Firstly,
leveraging cloud-based platforms for
scalable AI deployments, allows for
wider reach and impact. Cloud-based
solutions enable the deployment of AI
tools to remote or resource-constrained
areas, democratizing access to AI
technologies. Secondly, incorporating
explainability and transparency in AI
solutions is crucial for gaining trust
from practitioners and beneficiaries,
particularly in high-stakes domains
like disaster response and public
health. When AI systems can explain
their reasoning and decision-making
processes, they are more likely to be
accepted and used effectively. Thirdly,
emphasizing localized AI4SI solutions
that can be sustainably maintained by
end users fosters long-term impact
and community ownership. By
empowering local communities
to manage and maintain AI tools,
AI4SG projects can ensure that their
benefits continue long after the initial
development phase. Finally, exploiting
available foundation models presents
a significant opportunity to accelerate
the development and deployment of
AI4SG applications. These pre-trained
models can serve as a starting point
for developing AI solutions for specific
social problems, reducing the time and
resources required for development [5]. 
**Research Challenges** 
A significant challenge facing AI for
Social Good research revolves around
the design of AI systems that are
not only technically effective but
also deeply contextually relevant
within social impact settings. This
requires a nuanced understanding
of the specific needs, cultural
sensitivities, and practical constraints
of the communities being served.
Researchers must move beyond purely
algorithmic considerations and engage
in participatory design processes that
prioritize the voices and experiences of
end-users, ensuring that AI solutions
are truly aligned with their real-world
needs and challenges. 
Another critical hurdle lies in
overcoming the limitations of data. AI
for Social Good projects frequently
grapple with scarce, low-quality, or
biased data, which can significantly
impact the performance and fairness
of AI models. Developing robust
data collection strategies, employing
techniques for data augmentation
and bias mitigation, and exploring
alternative data sources are essential
for building reliable and equitable AI
systems. This also requires a careful
consideration of the cultural context in
which data is gathered, ensuring that AI
data collection methods are adapted
to local practices, rather than imposing
external standards. 
Ensuring the sustainability and
scalability of AI deployments in
resource-constrained environments
presents a further complex challenge.
Beyond the initial prototype phase,
the long-term viability of AI solutions
depends on their ability to be
maintained and scaled by organizations
with limited resources, such as NGOs
and government agencies. This
necessitates the development of
sustainable software architectures, the
creation of user-friendly interfaces,
and the provision of adequate training
and support for local stakeholders.
Moreover, funding these efforts, often
because of the lack of commercial
viability, is in itself a major concern. 
Additionally, robust evaluation
frameworks are crucial for assessing the
impact of AI solutions in field settings
and building stakeholder trust. These
frameworks must go beyond traditional
performance metrics and incorporate
measures of social impact, user
satisfaction, and ethical considerations.
The accessibility community’s mantra,
“Nothing about us without us,” [6]
serves as a powerful reminder of the
importance of involving stakeholders
in all stages of the evaluation process.
Furthermore, we must be vigilant
against the potential for corporate
ethics-washing or green-washing,
ensuring that AI initiatives are genuinely
driven by a commitment to social good,
rather than serving as mere public
relations exercises. 
Finally, there is currently also a gap
between traditional AI education
and the specific skills required for
impactful social work. Standard
AI curricula primarily focus on
algorithm design and analysis, often
emphasizing theoretical concepts and
performance on benchmark datasets.
This approach, while essential for
advancing core AI methodologies,
leaves students ill-equipped to address
the complex, real-world problems
that AI4SG tackles. Effective AI4SG
research demands a broader skillset,
extending beyond purely technical
expertise. It requires the ability to
collaborate effectively with domain
experts, such as public health officials,
environmental scientists, or social
workers, and to engage meaningfully
with community members whose lives
are directly affected by the technology.
Understanding the nuanced socioeconomic and cultural contexts of social challenges, and translating
technical advancements into practical,
user-centered interventions, are crucial
competencies. 
**Community Opinion** 
The AAAI Community Survey on
the Future of AI research, with 475
responses, explored various aspects
of AI, particularly its application to
social good. A significant portion
of respondents (119) engaged with
questions related to AI for social good. 
Regarding the relevance of AI for
Social Good, a majority of the 119
respondents who answered this
question found it relevant or very
relevant to their research. Specifically,
33.61% considered it very relevant,
26.89% relevant, and 21.01% somewhat
relevant. When asked about barriers
to integrating AI into social impact
projects, respondents cited several
challenges. The most significant
barrier, with 47.06% of respondents
selecting it, was “Testing the solution
in the field.” Other substantial barriers
included “Scaling up the solution”
(38.66%), “Connecting to non-profit or
government organizations” (32.77%),
“Problem definition” (36.13%), and
“Assessing AI readiness” (30.25%).
“Sustainability of the business model”
was also a concern for 42.86% of
respondents. 
The survey also explored crucial
resources for scaling AI-driven solutions
for social impact. “Money” and
“Data” were identified as extremely
important by a large majority of
respondents (the exact percentages
are cut off in the provided snippets).
Other important resources included
“Government-companies partnerships,”
“Government-university partnerships,”
and “Technical support.” Finally,
regarding measuring the success of
AI interventions in addressing social
challenges, “Improvement outcome”
was the most frequently cited metric,
with 47.90% of the 119 respondents
choosing it. “Sustainable results” were
also considered important by 37.82%,
and “Adoption rate” by 48.74%. 

1. United Nations. (2015). Transforming our world: the 2030 Agenda for Sustainable Development. https://sdgs.un.org/2030agenda
2. White House Office of Science and Technology Policy Workshop on AI for Social Good 2016. https://cra.org/ccc/events/ai-social-good/
3. AAAI Conference Call for the Special Track on AI for Social Impact, 2024 https://aaai.org/aaai-24-conference/call-for-the-special-track-on-ai-for-social-impact/
4. IJCAI conference Call For Papers And Projects: Multi-Year Track On AI And Social Good https://2025.ijcai.org/call-for-papers-and-projects-multi-year-track-on-ai-and-social-goodspecial-track/
5. Shi, Z., Wang, C., Fang, F. “AI for social good: A survey”, 2020. https://arxiv.org/abs/2001.01818
6. United Nations Division for Social Inclusive Development “AI for Good Impact Report” https://aiforgood.itu.int/newsroom/publications-and-reports/
7. Zhao, Y., Boehmer, N., Taneja, A., Tambe, M. Towards Foundation-model-based Multiagent System to Accelerate AI for Social Impact, AAMAS 2025
8. Charlton, J. I. Nothing About Us Without Us: Disability Oppression and Empowerment. University of California Press, 1998 http://www.jstor.org/stable/10.1525/j.ctt1pnqn9

AI for Social Good

AI & Sustainability

Artificial Intelligence (AI) is revolutionizing scientific discovery
by accelerating the entire research cycle from knowledge
extraction and hypothesis generation to automation of
experimentation and verification at an unprecedented speed. 
**Main Takeaways**
* Progress of AI for Scientific Discovery will accelerate the pace of scientific
discovery in unprecedented ways and transform the way we do science.
* Highly automated AI systems for scientific discovery are emerging: While their capabilities are limited, they can execute the entire scientific discovery cycle with minimal human intervention for defined tasks.
* AI’s role in scientific discovery raises new challenges in ethics, collaboration, and reliability, requiring interdisciplinary efforts to address them. 

**Context & History** 
Scientific discovery has historically
been driven by human ingenuity, but
Artificial Intelligence (AI) is emerging as
a pivotal tool in reshaping this process
and accelerating it. 
Early AI systems, such as DENDRAL,
designed in the 1960s, were among the
first to automate hypothesis generation
and problem-solving in organic
chemistry [1]. Similarly, systems like
EURISKO explored heuristic learning
and adaptation, demonstrating the
potential of AI in creative problemsolving [2]. 
These pioneering efforts laid the
groundwork for AI’s role in science
by showcasing its ability to process
datasets and formulate hypotheses. 
With the progress of technology for
comprehensive and high-precision
measurements, scientists are
overwhelmed by the massive data
generated and the complexity of
systems behind it. Technologies to
discover patterns from massive data
that can be linked to novel hypotheses
to be tested by experiments are
essential for scientific research. AI for
Scientific Discovery is a much-needed
research area and is expected to drive
practice of science in a data-driven
approach. 
There is a spectrum of approaches in
how AI transforms scientific discovery.
First, we envision increasingly powerful
AI tools that will assist human scientists
to make discoveries faster and enable
them to tackle even more challenging
problems. The alternative approach is
to develop integrated AI and robotics
systems aiming at performing the
entire cycle of scientific research
with minimum interventions from
human scientists. The middle ground
is to position AI as a collaborator
with scientists that can interactively
solve scientific problems. AI for
Scientific Discovery embraces a wider
spectrum of relationship between AI
and scientists, and rapid progress is
expected in all ranges of this spectrum
that will fundamentally transform the
way we do science. A series of papers,
reports and workshops concluded
that AI for Science is one of the most
important research areas in coming
years[3–6]. 
**Current State & Trends** 
**1. From Tools to AI Collaborators and
Autonomous AI Scientists** 
AI’s capability in supporting scientific discovery has expanded dramatically,
with systems like AlphaFold2 from DeepMind achieving groundbreaking
success in structural biology[7,8]. AlphaFold2 solved the decadesold problem of protein folding, revolutionizing molecular biology
and enabling applications in drug discovery and biomedicine. AlphaFold2
represents the most successful case of AI as a tool for scientific discovery
that transforms biomedicine and biochemistry research and resulted
in a 2024 Nobel Prize in Chemistry. An increasing number of AI/robotics
systems have been developed to be effective tools for scientists. Beyond
biology, numerous AI tools have been developed for chemistry[9–11], material science[12], mathematics[13–15] and many other scientific fields to
accelerate scientific discovery. For example, the Ramanujan Machine is
an early effort to generate conjectures on fundamental constants that can be a tool for mathematicians undertaking a specific task in mathematics[13]. AI and robotics integrated systems have been developed that perform a specific type of chemistry experiment automatically[16,17]. 
The other end of the spectrum is
to develop highly autonomous AI/
robotics systems to perform scientific
discovery without (or with minimum)
human interventions. The Adam and
Eve systems, developed by Ross King,
represent the early effort on the other
side of the spectrum where scientific
discovery occurs without human-inthe-loop[18,19]. These ‘robot scientists’
not only generate hypotheses but also
design and execute experiments to
test them. For example, Eve identified
potential drugs for malaria through
automated high-throughput screening,
showcasing AI’s ability to iterate
scientific cycles autonomously[20]. The
Nobel Turing Challenge represents the
extreme end of the challenge gearing
toward highly autonomous AI and
robotics systems with the capability for
high impact scientific discovery[21,22].
Proposed as a grand vision, it aims to
develop AI systems capable of scientific
discoveries on par with Nobel Prizewinning work. This challenge entails an
instance of the Feigenbaum test - Can
I replicate the best human expert in
defined domains? - that is a variation
of the Turing test[23]. These systems
would not merely assist researchers
but act as autonomous entities capable
of proposing, testing, and refining
theories. 
**2. Impact on Science** 
AI’s integration into scientific workflows
heralds a paradigm shift:
* **Accelerated Discovery:** By
automating an entire cycle of scientific discovery, AI can reduce
the time required to achieve
breakthroughs, enabling a rapid
expansion of knowledge.
* **Enhanced Collaboration:** AI systems
like AlphaFold demonstrate how
interdisciplinary approaches—
combining AI, biology, and
physics—can tackle long standing
challenges. Eventually, a network of
collaboration may be formed among
AI systems enabling extensive
exchange of ideas and data at
the scale not possible by human
scientists.3. Exploration
* **Beyond Human Intuition:** AI’s ability to exhaustively explore hypothesis spaces enables discoveries that might elude human researchers
constrained by cognitive and methodological biases.
* **Transformation in Data Handling:**
An AI-centric approach dramatically changes the way researchers handle
experimental data. In the AI-centric approach all data is important,
not just the subset that strongly supports the expected outcome, but
also data that are not consistent with the expectation because most, if not
all, data need to be provided to AI system training for better hypothesis
generation and prediction. 

**3. Social and Ethical Implications** 
AI-driven science will likely have profound societal impacts:
* **Healthcare Transformation:** With
AI accelerating drug discovery and
personalized medicine, millions of
lives could be saved or improved.
* **Environmental and Climate Change:**
Rapid progress of material science,
chemistry, and environmental
sciences may provide us with
discoveries that can be used to
mitigate climate change and improve
the state of the environment.
* **Ethical Considerations:** The
autonomy of AI systems raises
questions about accountability,
credit for discoveries, and the
potential displacement of human
researchers. There are concerns
that hazardous materials may be
designed and produced with highly
autonomous AI scientists. Ethical
and safety measures shall be taken
to prevent malevolent uses of
powerful synthesis abilities with
materials and biology[24]. Proper
measures must be taken to prevent
such uses. 

**Research Challenges** 
1. Communication issues: A major
challenge may be to be able to
understand and communicate with
human scientists because accumulation
of knowledge and communications with
peers mostly takes the form of natural
languages with ample ambiguities,
analogies, and often with cultural
context. Of particular importance:
* **Commonsense knowledge:**
Professional knowledge is grounded
in everyday knowledge. Construction
of a shared broad conceptual
knowledge base of the world, to
ground reasoning and models and
provide grist for analogies.
* **Collaboration:** Science is often
a team sport, being able to work
effectively with others (human or AI)
will be important.
* **Communication:** Scientists talk,
draw, and use multiple forms of
interactive media. AIs for science
will need to be able to read and
understand the scientific literature
and communicate well with human
partners.
* **Models of scientific reasoning and
approaches** to developing modeling
and inference machinery that can
augment human cognition for
scientific discovery[25]. 

2. Defining Hypothesis Space: Science
is an open-ended problem: Unlike
most games, such as chess or the game
of GO, the structure and scale of the
problem space is not obvious and likely
to be unbounded and could be with
very high dimensionality. Extracting
or acquiring knowledge and properly
placing it within the space of scientific
knowledge is a non-trivial problem.
Similarly, the process of hypothesis
generation faces the problem of
identifying the dimensionality and scale
of hypothesis space it should work on. 
3. Inaccuracies, noise, and
reproducibility of data: Data in science
can be very noisy, inaccurate and
not reproducible in some fields. In
biology, inaccuracy and noise in data
is considered to be inevitable due to
artifacts introduced in experiments and
with intrinsic variability of experimental
samples. Studies revealed that a
significant proportion of data reported
in publications cannot be reproduced
properly in biomedical research[26,
27]. This may pose problems for the
quality of hypotheses generated or their
verification process at the early stage of
the research. 
**Community Opinion** 
In the community opinion survey, 32%
of those responding consider this is
somewhat relevant. The area of study
that AI to be most useful is ranked as
(1) Biology (47%), (2) Physics (14%), (3)
Chemistry (12%), with 26% responding
for other domains. In the question of if
an AI system can ever make discovery
worthy of the Nobel Prize, only 13% of
those responding said never, with 25%
saying no idea. 11% thought it might
happen in the 2020s, and 45% thought
it might happen by the 2050s. 

1. Lindsay R, Buchanan B, Feigenbaum E, Lederberg J. DENDRAL: A Case Study of the First Expert System for Scientific Hypothesis Formation. Artif Intell. 1993;61: 209–261.
2. Lenat D, Brown J. Why AM and EURISKO appear to work. Artif Intell. 1984;23: 269–294.
3. Wang H, Fu T, Du Y, Gao W, Huang K, Liu Z, et al. Scientific discovery in the age of artificial intelligence. Nature. 2023;620: 47–60.
4. Science and Engineering Capacity Development Unit, Computer Science and Telecommunications Board, Policy and Global Affairs, Division on Engineering and Physical Sciences,
National Academies of Sciences, Engineering, and Medicine. AI for scientific discovery: Proceedings of a workshop. Pool R, editor. Washington, D.C.: National Academies Press; 2024.
doi:10.17226/27457
5. Gil Y, Greaves M, Hendler J, Hirsh H. Artificial Intelligence. Amplify scientific discovery with artificial intelligence. Science. 2014;346: 171–172.
6. President’s Council of Advisors on Science and Technology. PCAST: Report to the president on supercharging research: Harnessing artificial intelligence to meet global challenges.
Office of Scientific and Technical Information (OSTI); 2024 Apr. doi:10.2172/2481685
7. Senior AW, Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, et al. Improved protein structure prediction using potentials from deep learning. Nature. 2020;577: 706–710.
8. Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O, et al. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596: 583–589.
9. Peplow M. ChatGPT for chemistry: AI and robots join forces to build new materials. Nature. 2023 [cited 30 Nov 2024]. doi:10.1038/d41586-023-03745-5
10. Dias AL, Rodrigues T. Large language models direct automated chemistry laboratory. Nature. 2023;624: 530–531.
11. Boiko DA, MacKnight R, Kline B, Gomes G. Autonomous chemical research with large language models. Nature. 2023;624: 570–578.
12. Manica M, Born J, Cadow J, Christofidellis D, Dave A, Clarke D, et al. Accelerating material design with the generative toolkit for scientific discovery. Npj Comput Mater. 2023;9: 1–6
13. Raayoni G, Gottlieb S, Manor Y, Pisha G, Harris Y, Mendlovic U, et al. Generating conjectures on fundamental constants with the Ramanujan Machine. Nature. 2021;590: 67–73.
14. Davies A, Veličković P, Buesing L, Blackwell S, Zheng D, Tomašev N, et al. Advancing mathematics by guiding human intuition with AI. Nature. 2021;600: 70–74.
15. He Y-H. AI-driven research in pure mathematics and theoretical physics. Nature Reviews Physics. 2024; 1–8.
16. Coley CW, ThomasIII DA, Lummiss JAM, Jaworski JN, Breen CP, Schultz V, et al. A robotic platform for flow synthesis of organic compounds informed by AI planning. Science. 2019;365.
doi:10.1126/science.aax1566
17. Burger B, Maffettone PM, Gusev VV, Aitchison CM, Bai Y, Wang X, et al. A mobile robotic chemist. Nature. 2020;583: 237–241.
18. King RD, Rowland J, Oliver SG, Young M, Aubrey W, Byrne E, et al. The Automation of Science. Science. 2009;324: 85–89.
19. King RD, Whelan KE, Jones FM, Reiser PG, Bryant CH, Muggleton SH, et al. Functional genomic hypothesis generation and experimentation by a robot scientist. Nature. 2004;427:
247–252.
20.Williams K, Bilsland E, Sparkes A, Aubrey W, Young M, Soldatova LN, et al. Cheaper faster drug development validated by the repositioning of drugs against neglected tropical diseases.
J R Soc Interface. 2015;12: 20141289.
21. Kitano H. Nobel Turing Challenge: creating the engine for scientific discovery. npj Systems Biology and Applications. 2021;7: 29.
22.Kitano - 2016 - Artificial Intelligence to Win the Nobel Prize and Beyond Creating the Engine for Scientific Discovery, AI Magazine, 37:1 2016.
23.Feigenbaum, Edward A. (2003). “Some challenges and grand challenges for computational intelligence”. Journal of the ACM. 50 (1): 32–40. doi:10.1145/602382.602400. S2CID 15379263
24.Wittmann BJ, Alexanian T, Bartling C, Beal J, Clore A, Diggans J, et al. Toward AI-resilient screening of nucleic acid synthesis orders: Process, results, and recommendations. bioRxiv.
2024. p. 2024.12.02.626439. doi:10.1101/2024.12.02.626439
25.Hope, T, Downey, D., Weld, D.S., Etzioni, O. and Horvitz, E. (2023). A Computational Inflection for Scientific Discovery. Communications of the ACM 66, 8 (August 2023), 62–73. https://
doi.org/10.1145/3576896
26.Prinz F, Schlange T, Asadullah K (August 2011). “Believe it or not: how much can we rely on published data on potential drug targets?”. Nature Reviews. Drug Discovery. 10 (9): 712.
doi:10.1038/nrd3439-c1. PMID 21892149.
27. Begley CG, Ellis LM (March 2012). “Drug development: Raise standards for preclinical cancer research”. Nature (Comment article). 483 (7391): 531–533. Bibcode:2012Natur.483..531B.
doi:10.1038/483531a

AI for Scientific Discovery

Artificial General Intelligence (AGI)

How should we challenge exaggerated claims about AI’s
capabilities and set realistic expectations?

**Main Takeaways**

* Over the last 70 years, against a background of constant delivery of new and important technologies, many AI innovations have generated excessive hype.
* Like other technologies these hype trends have followed the general Gartner Hype Cycle characterization.
* The current Generative AI Hype Cycle is the first introduction to AI for perhaps the majority of people in the world and they do not have the tools to gauge the validity of many claims 

**Context & History**

Artificial intelligence, or AI, is the field
that studies the synthesis and analysis
of computational agents that act
intelligently [6]. AI has gone through
hype cycles multiple times since the
1956 workshop that established the
name AI and set the course for early
computer science departments to
include AI as a major component of
their research and teaching. All hype
bubbles eventually burst, as the
essence of hype is that it is beyond
reality. Over the decades this has led to
AI winters where funding has dried up
for all of AI or for specific aspects of AI
such as neural networks or robotics.

A study published 2017 on trends with
the public perception of AI over a 30-
year period found that discussion about
AI had sharply increased since 2009
and that discussions in the public press
had been consistently more optimistic
than pessimistic [4]. The study also
found that hopes about AI applications
of AI in healthcare and education
were increasing over time. Another
finding was that concerns were growing
about loss of control of AI, ethical
implications, and the negative impact of
AI on work. 
Perhaps the difference in recent years
with prior periods is that hype has
gone beyond the pages of academic
conferences, conference papers, and
scientific magazines, out into both the
mainstream media, and social media.
AI and Artificial Intelligence have
become common words that nontechnical people have heard about
and a common subject for leaders
of almost all countries to talk about.
Governments, for the first time, have AI
policies. 
One of the problems is that AI is
actually a wide-reaching term that can
be used in many different ways. But
now in common parlance it is used
as if it refers to a single thing. In their
2024 book [5] Narayanan and Kapoor
likened it to the language of transport
having only one noun, ‘vehicle’, say, to
refer to bicycles, skate boards, nuclear
submarines, rockets, automobiles, 18
wheeled trucks, container ships, etc.
It is impossible to say almost anything
about ‘vehicles’ and their capabilities
in those circumstances, as anything
one says will be true for only a small
fraction of all ‘vehicles’. This lack of
distinction compounds the problem
of hype, as particular statements get
overgeneralized. 
The hype also sets expectations for
ordinary people. Many are fearful that
they will lose their jobs to AI in the
short term. Social scientists then work
to solve labor disruptions, e.g., for
displaced truck drivers [6], based on
predictions about AI (and in this case,
self-driving trucks) and its adoption that
turn out to be wildly optimistic. There
are no deployed self-driving trucks in
the predicted time frame. 
Hype in response to a technology
trigger is not restricted to AI. Indeed
the business intelligence company
Gartner, has deliberately made
a practice of using a graphical
representation of hype levels through
five stages that are common for many
technologies: (1) technology trigger,
(2) peak of inflated expectations, (3)
trough of disillusionment, (4) slope
of enlightenment, and (5) plateau
of productivity. They have used this
framework to track many technologies,
including quantum computers, block
chain, autonomous vehicles, nanotechnology, etc. In November 2024 they
[1] estimated that hype for Generative
AI had just passed its peak and was on
the downswing. 
The question for AI professionals is
how to respond to this hype, how to
question it, and how to help others
understand what is hubris, while
maintaining their own intellectual
modesty and probity. This is hard to do
in the middle of outsized claims about
one’s own field, and often it is up to
future historians to carefully dissect
past scientific arguments. 
Historian Thomas Haigh has tried to
do such a dissection, almost in real
time, in a recent series of articles in the
Communications of the ACM. In [2] he
gives a post-mortem on the impact of
over-hype in AI that resulted in what is
known as the AI-winter in the 1980s. His
one line summary is: “Fallout from an
exploding bubble of hype triggered the
real AI Winter in the late 1980s.” In [3] he
makes a comparison between the hype
of today and of those earlier times.
He summarizes this particular opinion
piece with the line: “From engines of
logic to engines of bullshit?” 
**Research Challenges** 
Many of us who have worked in AI for
decades face the challenge of trying to
remain honest brokers when we see
that many of the public statements of
people quite new to the field are out of
line with reality. 
The big question is whether, given
the dynamics of social media and the
search for clicks, professional opinions
and peer reviewed research papers
have any impact on dampening the
overclaims and the ways they distort
common understanding of where AI is,
and what is its potential in one year, five
years, ten years, etc. 
If we are currently left out of the
conversations how can we change that? 
**Community Opinion** 
The Community Survey gives
perspectives on the reactions to the AI
Perception vs Reality theme. First, the
results of the survey are summarized
here. 36% of the survey respondents
chose to answer the questions for this
theme. This is the summary breakdown
of the responses to each question: 
**1. How relevant is this Theme for your
own research?** 72% of respondents
said it was somewhat relevant (24%),
relevant (29%) or very relevant (19%). 
**2. The current perception of AI
capabilities matches the reality of AI
research and development.** 79% of
respondents disagreed (47%) or strongly
disagreed (32%). 
**3. In what way is the mismatch
hindering AI research?** 90% of
respondents agreed that it is hindering
research: 74% agreeing that the
directions of AI research are driven by
the hype, 12% saying that theoretical AI
research is suffering as a result, and 4%
saying that less students are interested
in academic research. 
**4. Should there be a communitydriven initiative to counter the hype
by fact-checking claims about AI?** 78%
yes; 51% agree and 27% strongly agree. 
**5. Should there be a communitydriven initiative to organize public
debates on AI perception vs reality,
with video recordings to be made
available to all?** 74% yes; 46% agree
and 28% strongly agree. 
**6. Should there be a communitydriven initiative to build and maintain
a repository of predictions about
future AI’s capabilities, to be checked
regularly for validating their accuracy?**
59% yes; 40% agree and 29% strongly
agree. 
**7. Should there be a communitydriven initiative to educate the public
(including the press and the VCs) about
the diversity of AI techniques and
research areas?** 87% yes; 45% agree
and 42% strongly agree. 
**8. Should there be a communitydriven initiative to develop a method
to produce an annual rating of the
maturity of the AI technology for
several tasks?** 61% yes; 42% agree and
19% strongly agree. 
Since the respondents to this theme
are self-selected (about a third of all
respondents), that bias must be kept
in mind. Of those who responded,
a strong and consistent (though not
completely monolithic) portion felt that
the current perception of AI capabilities
was overblown, that it had a real impact
on the field, and that the field should
find a way to educate people about the
realities.

1. Chandrasekaran, A. [2024]. What’s Driving the Hype Cycle for Generative AI, 2024. https://www.gartner.com/en/articles/hype-cycle-for-genai
2. Haigh, T. [2024]. How the AI Boom Went Bust. Vol. 67 No. 2, pp 22-26.
3. Haigh, T. [2025]. Artificial Intelligence Then and Now. Vol. 68 No. 2, pp 24-29.
4. Fast, E. and Horvitz, E. [2017]. Long-term trends in the public perception of artificial intelligence, AAAI’17: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence,
February 4, 2017, pp 963-969. https://ojs.aaai.org/index.php/AAAI/article/view/10635
5. Narayanan, A. and Kapoor, S. [2024]. AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference. Princeton University Press.
6. Poole, D. L. and Mackworth, A. K. [2023]. Artificial Intelligence: Foundations of Computational Agents, 3rd edition. Cambridge University Press.
7. Wang, S., Mack, E. A., Van Fossen, J. A., Medwid, M., Cotten, S. R., Chang, C. H., Mann, J., Miller, S. R., Savolainen, P.T. and Baker, N. [2023]. Assessing alternative occupations for truck
drivers in an emerging era of autonomous vehicles. Transportation Research Interdisciplinary Perspectives, Vol 19. May. https://doi.org/10.1016/j.trip.2023.100793

AI Perception vs. Reality

It is important to encourage and support research on a
variety of AI paradigms, old and new. This includes diverse
methodologies (beyond just neural networks) both new and
old, interdisciplinary collaboration, and consideration of
societal implications. 
**Main Takeaways**
* Historically, the field of AI has simultaneously encompassed many different methodologies and research paradigms.
* There is a risk that the current convergence of the field towards focusing on neural approaches could impede innovation.
* We call for active support of research on classical (non-neural) approaches, as well as research that combines neural approaches with other approaches, and that integrates various paradigms into more complete cognitive architectures. And we especially encourage support of creative investigations of completely new paradigms that may be the key to overcoming the limitations of existing paradigms. 

**Context & History** 
There’s a long history in the field of AI
of separate subcommunities deeply
pursuing different approaches to
replicating intelligence in computers.
Sometimes they have been organized
around approaches, such as Planning,
Evolutionary Computation, Constraint
Satisfaction or Combinatorial Search.
Sometimes they have been organized
around applications, such as Computer
Vision, Natural Language Processing, or
Robotics. 
While there are usually some areas
that are more “in fashion” than
others, a few notable area disputes
notwithstanding, the community has
generally been good about tolerating,
and even encouraging, a diversity of
approaches. Indeed, it could be argued
that the current flourishing of neural networks-based generative AI is a result
of this tolerance. Neural networks were
introduced and studied even before
the term “Artificial Intelligence” was
coined in the 1950s, and there was a lot
of research on the topic in the 1960s.
After they didn’t live up to the level of
hype about them at the time, there
was a time when the majority of the
field considered “connectionism” to
be a dead end. But a subcommunity
persisted, and eventually got their day
in the sun (to say the least). 
**Current State & Trends** 
However, there is now a risk that this
tradition of diversity will be lost. It is
likely that one or more of the currently
unfashionable research areas could
eventually also see its day in the sun.
But we do not currently know which
ones. Due to the current dominance
of neural approaches, many of the
other approaches are losing steam,
or even being redefined as no longer
AI (A recent article in IEEE Spectrum
implied in its headline that classical
search is not AI: https://spectrum.
ieee.org/chip-design-ai). Indeed, as a
community, we seem to be in danger of
discouraging newcomers from pursuing
any alternatives. 
We think that would be a mistake. On
the contrary, for the long-term health of
the field, it is important that we find a
way to support the brave souls who are
resisting jumping on the bandwagon,
even though their papers may be less
likely to be accepted, and/or may
accumulate fewer citations. Some of
those papers may nonetheless end up
being immensely impactful. And even
(or especially) for people focussing their
attention on neural networks, we find it
important that they are knowledgeable
about alternative paradigms so as not
to impede innovation by needing to
“reinvent the wheel”. 
This isn’t to say that we condone
ignoring progress in neural-networkbased generative AI. It may be the
biggest revolution our field has seen,
and deserved an enormous amount of
attention. Just not all the attention. 
**Research Challenges** 
We predict that some of the future
breakthroughs will come from other
areas, either on their own, or in
combination with neural and other
classical methods. 
For example, investigations of the
planning capabilities of Large Language
Models find that they are really
unable to reason and plan effectively
[Valmeekam et al., 2023; Valmeekam
et al., 2024]. It may be that some
form of symbolic reasoning system
is required to work with the LLM to
produce sensible plans. Neurosymbolic
approaches are moving in that
direction. And, similarly, conformal
prediction [Angelopoulos and Bates,
2023] is an effort to inject probabilistic
reasoning into neural models. 
We call on the AI community to
complement its deep focus on
the capabilities and limitations
of neural approaches by actively
supporting research on classical
(non-neural) approaches such as
search, optimization, constraint
satisfaction, and causal reasoning,
as well as research that combines
neural approaches with symbolic and
probabilistic approaches, and that
integrates various paradigms into more
complete cognitive architectures. And
we especially encourage support of
creative investigations of completely
new paradigms that may be the key to
overcoming the limitations of existing
paradigms. 
This support could come in the
form of workshops devoted to such
investigations, and should also include
directing some funding towards
these priorities. We should especially
seek ways to encourage and support
researchers, old and new, who are
interested in pursuing new ideas
from new perspectives, as well as
opportunities for intersecting various
new and existing approaches, even
though they are likely to struggle to get
traction at first. 
**Community Opinion** 
57% of those surveyed, or a total of 176 respondents, chose to answer
questions pertaining to this theme. We begin by reporting the questions asked and the breakdown of responses. 
**1. How relevant is this Theme for your own research?** 92% of respondents said it was somewhat relevant (23%), relevant (37%) or very relevant (32%). 
**2. Do you think neural approaches
alone are sufficient to achieve general
purpose AI agents that match or
surpass human intelligence in all
ways?** 16% of respondents said yes,
with the remainder saying no.. 
**3. What percentage of AI research should be devoted to combining
neural approaches with other approaches?** 94% said at least 25%, including those who said 25% (31% of respondents), 50% (35%) or greater than 50% (28%). One respondent answered 0%. 
**4. What percentage of AI research
should be devoted to purely nonneural approaches?** 86% said at least
25%, including those who said 25% (37% of respondents), 50% (38%) or greater than 50% (11%). 3% (6 respondents) answered 0%. 
**5. What paradigms beyond neural networks do you think deserve the most research attention currently?**
This open-ended question received 3 responses as follows.
* “We need to understand whether the brain is quantum.”
* “Classic approaches to AI that focus on high-level cognition, rely on
structured representations, take a systems-level approach, incorporate
ideas from psychology, and aim for theoretical insight rather than
winning competitions.”
* “interdisciplinary collaboration and consideration of ethical and
societal implications are extremely important.” 

As for all the survey categories, it is important to keep in mind that the
respondents to this theme are self selected (a little more than half of all
respondents). Of these respondents, most indicated resonance with the
main message of this theme, namely that it is important to invest to some
extent in non-neural research. On the other hand, it wouldn’t be surprising if many of those who chose not to answer these questions feel differently. 

1. Angelopoulos, Anastasios N. and Stephen Bates, “Conformal Prediction: A Gentle Introduction”, Foundations and Trends in Machine Learning: Vol. 16: No. 4, pp 494-591. http://dx.doi.
org/10.1561/2200000101 (2023)
2. Valmeekam, Karthik, et al. “On the planning abilities of large language models (a critical investigation with a proposed benchmark).” arXiv preprint arXiv:2302.06706 (2023).
3. Valmeekam, Karthik, Kaya Stechly, and Subbarao Kambhampati. “LLMs Still Can’t Plan; Can LRMs? A Preliminary Evaluation of OpenAI’s o1 on PlanBench.” arXiv preprint
arXiv:2409.13373 (2024).

Diversity of AI Research Approaches

Expanding AI research to include diverse perspectives and
expertise from outside the core AI research 
**Main Takeaways**
* Incorporating perspectives of social scientists, ethicists, and policymakers, to ensure responsible and ethical development and deployment of AI technologies
* AI is not just an engineering discipline but a societal force that reshapes governance, culture, economy, and ethics, requiring holistic approaches for responsible development and deployment.
* ‘Intelligent support’ and tools that facilitate seamless collaboration between AI researchers and experts from diverse domains, ensuring ethical, explainable, and application-specific AI solutions. 

**Context & History** 
‘Research beyond the AI research
community’ emphasizes the
importance of expanding AI research
to include diverse perspectives and
expertise from outside the core AI
research community. There are three
directions to this expansion: First, we
must include perspectives of a.o. social
scientists, ethicists, digital humanities,
critical data/ communication / media
studies, STS, and other disciplines,
and policymakers, to ensure
responsible and ethical development
and deployment of AI technologies.
Second, researchers and practitioners
in disciplines that increasingly rely
on AI (e.g., biology, law, business,
neuroscience, cognitive science) may
also influence how we think about AI
itself. Finally, as multidisciplinary efforts
increase we can provide ‘intelligent
support’ and tools for the interaction. 
The societal, ethical, legal, and cultural
challenges posed by AI highlight
the necessity of a multidisciplinary
approach to its development and
deployment [1,2]. Issues such as
biased decisions, privacy breaches,
governance, accountability, and
inclusivity demand solutions
that extend beyond the scope of
engineering. Addressing these
challenges requires the integration of
perspectives from humanities, social
sciences, and other fields to ensure
that AI systems are aligned with human
rights, societal values, and global equity. 
This broader approach to AI emphasizes
that its development and impact
cannot be compartmentalized as
solely technical advancements or
applications within specific fields, i.e.
AI is no longer purely an engineering
discipline. Instead, it requires a holistic
understanding of AI as a transformative
force that reshapes societal structures,
cultural norms, economic models,
and ethical frameworks. Such an
approach considers AI not merely as a
tool for individual disciplines but as an
integrated phenomenon that influences
and is influenced by diverse societal
factors. By involving diverse expertise,
including ethicists, legal experts,
social scientists, and policymakers, we
can develop governance frameworks
and accountability mechanisms to
address concerns like bias, inequality,
and unintended societal impacts.
Efforts to enhance inclusivity and
diversity in AI design can also mitigate
risks and maximize the benefits of AI
applications. 
Additionally, “intelligent support” for
these efforts refers to a combination
of tools, frameworks, and methods
that facilitate effective integration of
AI into diverse fields and collaboration
between AI researchers and experts in
other fields. 
**Current State & Trends** 
* Social scientists and ethicists
are increasingly involved in
developing guidelines on how AI
developers should handle personal
data, preventing inappropriate
surveillance, and ensuring the
responsible use of AI. Also
governance frameworks such as the
EU’s AI Act [3] are being actively
discussed in order to ensure that AI
development is aligned with human
rights, justice and societal needs.
* AI has become a key tool for
healthcare [4], law [5], business [6],
etc., which increasingly influences
AI research. For example, AI
applications in fields demand
highly accurate, explainable, and
interpretable AI systems due to the
importance of accountability and
transparency. This has spurred more
research in explainable AI (XAI) as
well as AI algorithms tailored for
certain applications, such as cancer
analysis [7].
* Intelligent support’ and tools for
the interaction between AI & other
disciplines: So far non-AI tools such
as GitHub, Kaggle, and research
repositories have been actively used
by the communities. We expect
that intelligent capabilities may be
able to promote more productive
interactions. 

**Research Challenges** 
In formulating future research, here is a
set of core pillars that we need to take
into account:
* **Societal Dynamics:** AI alters the
fabric of human interaction, labour
markets, and societal governance.
Understanding how automation,
algorithmic decision-making,
and AI-driven systems impact
democracy, social justice, and
equality is critical to harness its
potential without exacerbating
existing disparities.
* **Ethical Integration:** Ethics must be
embedded into AI from its inception.
This includes navigating dilemmas
around privacy, accountability, and
fairness. Ethical considerations
must go beyond technical checks to
involve diverse cultural,
philosophical, and societal inputs
to create systems that are globally
adaptable and contextually relevant.
* **Legal and Regulatory Frameworks:**
We need to address the need for regulation and governance, including
reevaluation of existing regulations
for intellectual property, liability,
and human rights. A collaborative,
interdisciplinary effort is essential
to craft governance models that
increase trust and safety, safeguard
public interests and contribute to
increased responsible innovation.
Cultural Adaptation and
Diversity: As AI technologies
become pervasive across cultures
and geographies, they must
adapt to diverse social norms,
languages, and traditions. Ensuring
cultural sensitivity in AI design
and implementation promotes
inclusivity and equitable access to
technological benefits.
* **Education and Public Awareness:**
The broad impact of AI requires a
rethinking of education systems to
prepare future generations for an AIinfused world. This includes fostering
a transdisciplinary understanding of
AI among engineers, social scientists,
policymakers, and the general public
to build informed, empowered
societies.
* **Environmental Sustainability:** The
energy demands of AI development
and deployment highlight its
environmental footprint. A broad
approach to AI considers how it can contribute to sustainability efforts,
from optimizing resource allocation
to advancing climate solutions, while
minimizing ecological harm. 

The inclusion of other disciplines is
more than developing AI technologies
to understanding and guiding their
interaction with society, but also to
ensure that their influence is equitable,
inclusive, and beneficial across all
domains of human activity. 
In supporting these efforts and
promoting interaction between AI and
other disciplines, future research topics
can also include the following: 
* **User friendly AI development
platforms:** These can enable people
from various disciplines to develop
AI solutions or train a deep learning
model without much programming
experience. The platform can also
guide ethical development of AI.
Intelligent user interfaces and
visualization tools can allow people
to understand AI model outputs and
gain insights.
* **Domain Specific AI tools:** AI systems
tailored to the particular needs of
a field, such as medical diagnosis,
cyber security, law, etc. We look
forward to intelligent capabilities
that understand the needs of the
field workers, and provide seamless support for domain-specific
workflows and reasoning processes,
which should be much more
powerful than current tools such as
domain-specific pretrained models
or knowledge-based systems
* **AI enabled Collaboration Tools:**
Intelligent environments that assist
AI researchers and experts in other
fields to work together. 

**Community Opinion** 
We began by asking the community which activities they considered most important. The top three priorities were: 1) Promoting interdisciplinary collaboration in AI research 2)
Developing AI solutions for specific application areas, such as healthcare, law, and business, and 3) Enhancing public understanding of AI’s impact. The next tier of priorities included:
4) Integrating AI education into noncomputer science disciplines, 5) Establishing accountability and liability standards, and 6) Addressing societal and cultural impacts of AI through diverse disciplinary perspectives and promoting inclusivity in AI design.
While activities such as optimizing resource allocation and developing governance models received slightly lower ratings, the differences were not significant. Overall, the community
expressed a strong interest in pursuing a broad range of initiatives. 
When asked about key areas of focus, an overwhelming 88% of respondents identified healthcare as the top priority. This was followed by climate (50%), education (45%), and biology
(43%). Other areas of interest included manufacturing (38%), business (19%), law (18%), and finance (15%). The respondents listed the following areas as additional candidates: agriculture,
transportation, brain diseases, physics, elder care, disaster response, media analytics, and cybersecurity. 
Regarding tools to support interdisciplinary collaboration, the
highest-rated option was user-friendly AI development platforms (65%), followed by AI-enabled collaboration tools (57%) and domain-specific AI tools (53%). Beyond technological support,
the community suggested the following actions for fostering collaboration: educational support, incentives to
encourage cross-disciplinary work, formation of interdisciplinary research groups, and sustainable funding for long-term projects. As multidisciplinary efforts become increasingly important
to AI research, the community seemed to emphasize the urgent need for stronger support from diverse perspectives. 
Approximately one-third of all participants responded to this theme, with the majority (around 90%) identifying AI as their primary field. Although input from other disciplines was limited, the survey suggests a strong interest within our community
in integrating diverse perspectives into research and enhanced support for interdisciplinary collaboration. 

1. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed, Bias and Fairness in Large Language
Models: A Survey. Computational Linguistics, 50(3):1097–1179, 2024.
2. Youngsik Yun and Jihie Kim. CIC: A framework for Cultually-aware Image Captioning, International Joint Conference on Artificial Intelligence, 2024.
3. The EU Artificial Intelligence Act, Regulation (EU) 2024/1689,
https://artificialintelligenceact.eu/
4. AI in health care, nature collection 22, 2024. https://www.nature.com/collections/dbfcjjigbi
5. D.W. Kite-Jackson2023 Artificial Intelligence (AI) TechReport, American Bar Association, 2023.
6. McKinsey & Company, The state of AI in early 2024: Gen AI adoption spikes and starts to generate value, 2024.
7. Joseph D. Janizek, Ayse B. Dincer, Safiye Celik, Hugh Chen, William Chen, Kamila Naxerova, and Su-In Lee. Uncovering expression signatures of synergistic drug responses via
ensembles of explainable machine-learning models. Nature Biomedical Engineering 7, 811–829, 2023. https://www.nature.com/articles/s41551-023-01034-0

Research Beyond the AI Research Community

State-of-the art AI is now largely driven by the private sector,
and universities struggle to compete: they need to find a role in
the new era of “big AI”. 
**Main Takeaways**
* The centre of gravity of AI research now lies behind the closed doors of big tech companies.
* Universities cannot compete with big tech companies with respect to the resources – data, compute, and salaries – that are being mobilised by the private sector.
* Universities struggle to retain AI faculty, and struggle to persuade AI graduates to remain in academia.
* The challenge is therefore now to find a role for academia (and publicly funded research) in the new era of “big AI”. 
 

**Context & History** 
While there has always been industrial interest in AI, for much of the history of the field, progress was driven largely by academia. The AI Turing award winners (Feigenbaum, McCarthy, Minsky, Newell, Pearl, Reddy, Simon) were university-based, and the key
concepts of both symbolic and neural AI were all developed within academia. The deep learning Turing award winners
(Bengio, Hinton, LeCun) all did their Laureate work in universities. But over the past decade, the centre of gravity in cutting edge AI has shifted decisively. In particular, primary work on generative AI is now being carried out largely within the private sector, and
a small number of big tech companies in particular. These companies have the data and compute resources available
to train large scale state of the art AI systems, and moreover are able to offer salaries that universities could not compete with. 
**Current State & Trends** 
These changes raise a number of issues for universities. 
First, the demand for AI research talent has led to an exodus of AI talent from universities to the private sector. In one
notably early case, Uber hired an entire research lab of roboticists from CMU [1]. Even the world’s leading universities are unable to offer packages to compete with the startling salaries offered by
big tech AI labs; nor can they muster the associated research resources that such companies routinely offer. The upshot has been a hollowing-out of many university AI groups, with
many faculty on seemingly permanent leave, or on part-time contracts that leave them largely disconnected from the conventional university business. Hiring AI faculty has been extremely challenging for more than a decade: PhD graduates prefer to move directly from their studies into the private
sector. 
At the same time the nature of AI research has changed dramatically. At the turn of the century the dominant paradigm with (for example) the AAAI community was conventionally scientific, drawing from mathematics (definition-lemma-theorem-proof). By contrast, the methodology of deep learning and its associated areas, which dominates at the time of writing, is
predominantly that of engineering. 
As a consequence the demand within
universities for AI courses in particular,
and computing courses more generally,
has skyrocketed, and this, coupled with
the issues we refer to above, has placed
intense pressure on departments,
whose staff struggle to cope with the
level of demand. Some ivy league
computing departments report that
while they have been given what
amounts to a blank cheque for hiring
new faculty in high demand areas,
they simply find it impossible to hire
enough high-quality faculty. In some
departments, this situation has reached
crisis levels, with faculty reporting stress
and other mental health issues as a
consequence. 
There is a need for all students
(regardless of their discipline) to
become literate in AI, the use of AI
tools is changing education and the
way subject are taught, and at the same
time students in AI must be educated
in a more multi-disciplinary context to
be aware of the ethical, legal, societal
and economic implications of AI. This
in turn places further demands on
universities, who need to rethink how
they teach and assess students when
those students have powerful general
purpose AI tools available. 
One irony of the present situation
is that a key role of universities
historically has been to provide a talent
pipeline to satiate the demands of the
technology world for technical talent.
But universities struggle to achieve this
simply because they don’t have the
capacity – because of the hollowingout of university departments by those same companies. 
At the same time, the nature of cutting
edge AI research means that universities
are simply unable to carry out work
that competes with big private sector AI
labs. A recent estimate from Meta put
the cost of building their latest GPTclass models at ~ $440 million. That is several orders of magnitude outside
the scope of all but a tiny number of
the world’s very richest universities;
and most nation states would struggle
to put assemble such a package for a
national level initiative: in 2023, the UK
contemplated a sovereign AI capability,
and one option considered was building
a “BritGPT”. The proposal foundered
at an early stage because of (i) cost, (ii)
risk, and (iii) being seen to compete
with the private sector in an area where
innovation (and development costs)
have been led by the private sector
[2]. Recent developments such as
that around the Chinese open-source
DeepSeek Large Language model may
offer an opportunity in this regard as
it could make AI available at lower
costs – although it presents challenges
of its own with respect to privacy and
security. 
The objectives of the private sector are different from those of universities.
The private sector is driven mainly by
profit, whereas universities aspire to
contribute to society through research
and education. These goals can be
in competition with each other. One
effect is that results from the private
sector are not always fully available
for inspection or evaluation. Also,
the high costs of the experiments
implies that results are not always
reproducible. Academics have a role to
play in providing independent advice
and interpretations of these results
and their consequences. The private
sector focuses more on the short term,
and universities and society more on a
longer term perspective.
These observation together with the
massive costs of generative AI research
has also motivated calls for large public-funded
initiatives, for instance, the plan supported by European Commission President Ursula von der Leyen for a CERN for AI. This vision is supported and was actually initiated by a large
number of AI researchers in Europe gathered in the CAIRNE network. It is modelled after the renowned CERN for
Nuclear Research in Geneva, and should serve as an alternative and attractive research environment to big tech companies, and should solve problems of public interest. Numerous countries
have also developed their national strategies and funding programs for AI. 
**Research Challenges** 
* How should universities respond to the new era of “Big AI”?
* What form of AI research can universities/public sector research
most usefully engage in – what should be the AI research agenda of universities going forward?
* How should universities respond to the challenges of hiring and retaining AI faculty and students?
* How can publicly funded universities best work with big tech AI
companies? 

**Community Opinion** 
Most respondents (approx. 75%)
agreed the universities have difficulty
recruiting in AI, and agreed that
universities have difficulty in engaging
in resource intensive AI research (80%).
Respondents felt that offering better
salaries and investing in better compute
resources were ways of attracting
people; offering joint appointments
and providing other benefits were also
seen as possibilities. There was less
agreement on whether universities
needed to refocus their research
priorities, although it seems that
respondents felt that focussing on
theoretical AI and multidisciplinary
AI were areas where universities
could be competitive. Public sector
funding for large scale compute
was seen as attractive (70%). There
was overwhelming agreement that
academia was very relevant to the
future of AI research. 
1. https://www.fastcompany.com/3046902/carnegie-mellon-in-a-crisis-after-uber-poached-40-of-its-researchers
2. https://lordslibrary.parliament.uk/large-language-models-and-generative-ai-house-of-lords-communications-and-digital-committee-report/

Role of Academia

Geopolitical Aspects & Implications of AI

AI evaluation is the process of assessing the performance, reliability, and safety of AI systems. 
**Main Takeaways** 
* AI systems introduce unique evaluation challenges that extend far beyond the scope of standard software validation and verification methods. 
* Current approaches to evaluation focus on benchmark-driven testing, e.g., of the quality of (generative) models, with insufficient attention paid to other critical factors such as usability, transparency, and adherence to ethical guidelines. 
* New insights and methods for evaluating AI systems are needed to provide the assurance for trustworthy, wide-scale deployments. 

**Context & History** 
The recent advances in AI have spurred tremendous innovation in potential applications for the technology. However, many organizations are hesitant to deploy AI systems due to risks that include reputational damage from generative AI hallucinations, leakage of proprietary data, and lack of assurance that legal and ethical guardrails will be enforced. 
Empirical methods have long played a role in AI research (e.g., [1]). Indeed, the research community has developed a robust body of metrics and methods for evaluating individual AI algorithms that has enabled the field to quantify performance and track progress. In contrast, less attention has been paid to evaluating AI systems and their deployment in real-world settings, including their usage by non-AI experts. 
AI systems introduce unique evaluation challenges that extend far beyond the scope of standard software validation and verification methods. The generality, complexity, and breadth of AI capabilities makes it impossible to test them exhaustively, requiring new thinking as to what constitutes sufficiency in testing. Run-time adaptivity and the evolution of learned models can change system behavior on the fly, introducing the need for continuous monitoring and validation. Many AI systems are designed to be used interactively, making collaborative usage and its impact on humans an important consideration. 
**Current State & Trends** 
Current practice for evaluating generative AI systems focuses on model-level testing relative to a growing body of benchmarks. Some benchmarks seek to measure general capabilities (e.g., GLUE [2], ARC-AGI [3], MMLU [4]) while others address particular types of reasoning and knowledge (e.g., MATH for mathematics [5], GPQA for logic [6], HumanEval for coding [7]). Benchmark-driven testing provides valuable insights into capabilities and shortcomings, as well as a principled means to evaluate progress over time. Benchmarks are used as proxies for AI capabilities but have an inherent contextualization that does not necessarily generalize well to new domains. Furthermore, benchmark-based testing is insufficient for ensuring successful deployment given the lack of attention to expected usage in real-world settings and aspects of human use. Benchmark-driven evaluation further raises issues related to overfitting and contamination of test data with training data. As embodied in Goodhart’s law, *“When a measure becomes a target, it ceases to be a good measure”*. 
Evaluating AI systems is inherently complex, especially if these systems are broadly applicable and capable of learning after deployment, requiring a balanced approach that is clear and transparent while avoiding overfitting to specific metrics at the expense of broader reliability, fairness, and real-world applicability. System-level evaluation, when done, explores representative use cases rather than seeking to be comprehensive. Redteaming serves as a complementary method through the use of adversarial interactions to identify misalignment with desired behavior models. 
Moving forward, AI evaluation needs to consider multiple dimensions of a system’s performance. Most evaluation efforts focus on capability, i.e., producing correct answers or behaviors in response to queries or tasking, for the scope of problems over which the system is expected to operate. Meeting capability requirements is essential for use; however, other aspects of performance must be considered. 
*Usability* is another critical dimension for evaluation. A principal factor of usability is transparency, meaning that mechanisms are provided that enable users to understand the basis for system actions and responses. Usability further requires directability, meaning that users can control and modify the behavior of the system to meet current and specialized needs (now often referred to as “alignment”). For AI systems being deployed to aid humans, evaluation must necessarily consider whether the technology ultimately improves combined humansystem performance. 
*Adherence* to legal requirements and ethical guidelines constitutes another important dimension for evaluation. Increasingly, geo-political entities are introducing legislation to restrict what and how AI systems will be allowed to operate within their jurisdictions, requiring validation that performance will stay within defined guardrails. Both government and commercial organizations have ethical and financial motivations to ensure that their use of AI is fair and unbiased. To support these goals, various trustworthy AI assessment frameworks have been developed to guide organizations in evaluating AI systems for fairness, transparency, robustness, and compliance with ethical standards. Notable frameworks include the EU Trustworthy AI Assessment Framework, the NIST AI Risk Management Framework, and the ISO/IEC 42001:2023 AI Management System Standard. 
AI systems introduce multiple operational issues related to their deployment. Privacy is a main concern: protecting personal or corporate information within a model from being leaked. AI systems have become attack surfaces themselves, with adversaries seeking to exfiltrate data or model weights, or to bias responses for purposes at odds with the model’s creators. Resource consumption and the cost for both training and deployment are additional considerations in evaluating overall performance of an AI system. 
These various factors must be weighed together, including fairness, robustness, interpretability, and compliance with evolving regulations. A comprehensive evaluation framework must balance these diverse considerations, ensuring AI systems are secure, efficient, and aligned with ethical and legal standards. 
**Research Challenges** 
There is a need for a science of evaluation for AI systems that will inject additional rigor into the evaluation process. This science will build on existing metrics and methodologies but incorporate new approaches that will increase confidence in our ability to deploy AI systems in missioncritical settings (e.g., [8] for evaluating Retrieval-Augmented Generation systems). Frameworks for auditing and reproducibility will be important to ensure the reliability and robustness of results [9]; as well, more attention should be paid to education within the field on proper empirical methodology. Below are key challenges for advancing our understanding of how to conduct effective evaluations for AI systems.
* Develop a better understanding how to monitor and assess AI systems that are deployed over extended periods of time, especially for those that evolve their behavior. 
* Develop frameworks for evaluating the safety of agentic AI systems that can take actions in the world. 
* Create methods to provide increased transparency into machine learning models. 
* Develop evaluation methodologies that directly address human engagement with AI capabilities (as was the case with the Turing test). 
* Understand the trade-offs between different dimensions of evaluation, such as whether increased transparency justifies higher costs, or adherence to guardrails outweighs potential impacts on privacy, or how other crossdimensional considerations might influence overall outcomes.
 
**Community Opinion** 
The responses to the community survey show that there is significant concern regarding the state of practice for evaluating AI systems. More specifically, 75% of the respondents either agreed or strongly agreed with the statement “The lack of rigor in evaluating AI systems is impeding AI research progress.” Only 8% of respondents disagreed or strongly disagreed, with 17% neither agreeing nor disagreeing. These results reinforce the need for the community to devote more attention to the question of evaluation, including creating new methods that align better with emerging AI approaches and capabilities. 
Given the responses to the first question, it is interesting that only 58% of respondents agreed or strongly agreed with the statement *“Organizations will be reluctant to deploy AI systems without more compelling evaluation methods.”* Approximately 17% disagreed or strongly disagreed with this statement while 25% neither agreed nor disagreed. If one assumes that the lack of rigor for AI research transfers to a lack of rigor for AI applications, then the responses to these two statements expose a concern that AI applications are being rushed into use without suitable assessments having been conducted to validate them. 
For the question *“What percentage of time do you spend on evaluation compared to other aspects of your work on AI?”* the results show 90% of respondents spend more than 10% of their time on evaluation and 30% spend more than 30% of their time. This clearly indicates that respondents take evaluation seriously and devote significant effort towards it. While the prioritization of evaluation is commendable, the results would also seem to indicate that evaluation is a significant burden, raising the question of what measures could be taken to reduce the effort that it requires. Potential actions might include promoting an increased focus on establishing best practices and guidelines for evaluation practices, increased sharing of datasets, and furthering the current trend of community-developed benchmarks. 
The most widely selected response to the question* “Which of the following presents the biggest challenge to evaluating AI systems?”* was a lack of suitable evaluation methodologies (40%), followed by the black-box nature of systems (26%), and the cost/ time required to conduct evaluations (18%). These results underscore the need for the community to evolve approaches to evaluation that align better with current techniques and broader deployment settings. 
1. Cohen, P.R. (1995). Empirical Methods for Artificial Intelligence. MIT Press. 2. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.R. (2018). GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. BlackboxNLP@ EMNLP. 3. Chollet, F. (2019). On the Measure of Intelligence. ArXiv, abs/1911.01547. 4. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D.X., and Steinhardt, J. (2020). Measuring Massive Multitask Language Understanding. ArXiv, abs/2009.03300. 5. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D.X., and Steinhardt, J. (2021). Measuring Mathematical Problem Solving With the MATH Dataset. ArXiv, abs/2103.03874. 6. Rein, D., Hou, B.L., Stickland, A.C., Petty, J., Pang, R., Dirani, J., Michael, J., and Bowman, S.R. (2023). GPQA: A Graduate-Level Google-Proof Q&A Benchmark. ArXiv, abs/2311.12022. 7. Chen, M. et al. (2021). Evaluating Large Language Models Trained on Code. ArXiv abs/2107.03374 8. Shahul, E., James, J., Anke, L.E., and Schockaert, S. (2023). RAGAs: Automated Evaluation of Retrieval Augmented Generation. Conference of the European Chapter of the Association for Computational Linguistics. 9. Gundersen, O.E., Helmert, M., and Hoos, H. (2024). Improving Reproducibility in AI Research: Four Mechanisms Adopted by JAIR. J. Artif. Intell. Res. 81, 1019-1041.

Content not yet available

Next from The Future of AI Research - Webinar Series

AI for Social Good

Similar lecture

AI Evaluation