Archive for Programming – Page 3

What is vibe coding? A computer scientist explains what it means to have AI write computer code − and what risks that can entail

By Chetan Jaiswal, Quinnipiac University 

Whether you’re streaming a show, paying bills online or sending an email, each of these actions relies on computer programs that run behind the scenes. The process of writing computer programs is known as coding. Until recently, most computer code was written, at least originally, by human beings. But with the advent of generative artificial intelligence, that has begun to change.

Now, just as you can ask ChatGPT to spin up a recipe for a favorite dish or write a sonnet in the style of Lord Byron, you can now ask generative AI tools to write computer code for you. Andrej Karpathy, an OpenAI co-founder who previously led AI efforts at Tesla, recently termed this “vibe coding.”

For complete beginners or nontechnical dreamers, writing code based on vibes – feelings rather than explicitly defined information – could feel like a superpower. You don’t need to master programming languages or complex data structures. A simple natural language prompt will do the trick.

How it works

Vibe coding leans on standard patterns of technical language, which AI systems use to piece together original code from their training data. Any beginner can use an AI assistant such as GitHub Copilot or Cursor Chat, put in a few prompts, and let the system get to work. Here’s an example:

“Create a lively and interactive visual experience that reacts to music, user interaction or real-time data. Your animation should include smooth transitions and colorful and lively visuals with an engaging flow in the experience. The animation should feel organic and responsive to the music, user interaction or live data and facilitate an experience that is immersive and captivating. Complete this project using JavaScript or React, and allow for easy customization to set the mood for other experiences.”

But AI tools do this without any real grasp of specific rules, edge cases or security requirements for the software in question. This is a far cry from the processes behind developing production-grade software, which must balance trade-offs between product requirements, speed, scalability, sustainability and security. Skilled engineers write and review the code, run tests and establish safety barriers before going live.

But while the lack of a structured process saves time and lowers the skills required to code, there are trade-offs. With vibe coding, most of these stress-testing practices go out the window, leaving systems vulnerable to malicious attacks and leaks of personal data.

And there’s no easy fix: If you don’t understand every – or any – line of code that your AI agent writes, you can’t repair the code when it breaks. Or worse, as some experts have pointed out, you won’t notice when it’s silently failing.

The AI itself is not equipped to carry out this analysis either. It recognizes what “working” code usually looks like, but it cannot necessarily diagnose or fix deeper problems that the code might cause or exacerbate.

IBM computer scientist Martin Keen explains the difference between AI programming and traditional programming.

Why it matters

Vibe coding could be just a flash-in-the-pan phenomenon that will fizzle before long, but it may also find deeper applications with seasoned programmers. The practice could help skilled software engineers and developers more quickly turn an idea into a viable prototype. It could also enable novice programmers or even amateur coders to experience the power of AI, perhaps motivating them to pursue the discipline more deeply.

Vibe coding also may signal a shift that could make natural language a more viable tool for developing some computer programs. If so, it would echo early website editing systems known as WYSIWYG editors that promised designers “what you see is what you get,” or “drag-and-drop” website builders that made it easy for anyone with basic computer skills to launch a blog.

For now, I don’t believe that vibe coding will replace experienced software engineers, developers or computer scientists. The discipline and the art are much more nuanced than what AI can handle, and the risks of passing off “vibe code” as legitimate software are too great.

But as AI models improve and become more adept at incorporating context and accounting for risk, practices like vibe coding might cause the boundary between AI and human programmer to blur further.The Conversation

About the Author:

Chetan Jaiswal, Associate Professor of Computer Science, Quinnipiac University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

 

AI literacy: What it is, what it isn’t, who needs it and why it’s hard to define

By Daniel S. Schiff, Purdue University; Arne Bewersdorff, Technical University of Munich, and Marie Hornberger, Technical University of Munich 

It is “the policy of the United States to promote AI literacy and proficiency among Americans,” reads an executive order President Donald Trump issued on April 23, 2025. The executive order, titled Advancing Artificial Intelligence Education for American Youth, signals that advancing AI literacy is now an official national priority.

This raises a series of important questions: What exactly is AI literacy, who needs it, and how do you go about building it thoughtfully and responsibly?

The implications of AI literacy, or lack thereof, are far-reaching. They extend beyond national ambitions to remain “a global leader in this technological revolution” or even prepare an “AI-skilled workforce,” as the executive order states. Without basic literacy, citizens and consumers are not well equipped to understand the algorithmic platforms and decisions that affect so many domains of their lives: government services, privacy, lending, health care, news recommendations and more. And the lack of AI literacy risks ceding important aspects of society’s future to a handful of multinational companies.

How, then, can institutions help people understand and use – or resist – AI as individuals, workers, parents, innovators, job seekers, students, employers and citizens? We are a policy scientist and two educational researchers who study AI literacy, and we explore these issues in our research.

What AI literacy is and isn’t

At its foundation, AI literacy includes a mix of knowledge, skills and attitudes that are technical, social and ethical in nature. According to one prominent definition, AI literacy refers to “a set of competencies that enables individuals to critically evaluate AI technologies; communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the workplace.”

AI literacy is not simply programming or the mechanics of neural networks, and it is certainly not just prompt engineering – that is, the act of carefully writing prompts for chatbots. Vibe coding, or using AI to write software code, might be fun and important, but restricting the definition of literacy to the newest trend or the latest need of employers won’t cover the bases in the long term. And while a single master definition may not be needed, or even desirable, too much variation makes it tricky to decide on organizational, educational or policy strategies.

Who needs AI literacy? Everyone, including the employees and students using it, and the citizens grappling with its growing impacts. Every sector and sphere of society is now involved with AI, even if this isn’t always easy for people to see.

Exactly how much literacy everyone needs and how to get there is a much tougher question. Are a few quick HR training sessions enough, or do we need to embed AI across K-12 curricula and deliver university micro credentials and hands-on workshops? There is much that researchers don’t know, which leads to the need to measure AI literacy and the effectiveness of different training approaches.

Ethics is an important aspect of AI literacy.

Measuring AI literacy

While there is a growing and bipartisan consensus that AI literacy matters, there’s much less consensus on how to actually understand people’s AI literacy levels. Researchers have focused on different aspects, such as technical or ethical skills, or on different populations – for example, business managers and students – or even on subdomains like generative AI.

A recent review study identified more than a dozen questionnaires designed to measure AI literacy, the vast majority of which rely on self-reported responses to questions and statements such as “I feel confident about using AI.” There’s also a lack of testing to see whether these questionnaires work well for people from different cultural backgrounds.

Moreover, the rise of generative AI has exposed gaps and challenges: Is it possible to create a stable way to measure AI literacy when AI is itself so dynamic?

In our research collaboration, we’ve tried to help address some of these problems. In particular, we’ve focused on creating objective knowledge assessments, such as multiple-choice surveys tested with thorough statistical analyses to ensure that they accurately measure AI literacy. We’ve so far tested a multiple-choice survey in the U.S., U.K. and Germany and found that it works consistently and fairly across these three countries.

There’s a lot more work to do to create reliable and feasible testing approaches. But going forward, just asking people to self-report their AI literacy probably isn’t enough to understand where different groups of people are and what supports they need.

Approaches to building AI literacy

Governments, universities and industry are trying to advance AI literacy.

Finland launched the Elements of AI series in 2018 with the hope of educating its general public on AI. Estonia’s AI Leap initiative partners with Anthropic and OpenAI to provide access to AI tools for tens of thousands of students and thousands of teachers. And China is now requiring at least eight hours of AI education annually as early as elementary school, which goes a step beyond the new U.S. executive order. On the university level, Purdue University and the University of Pennsylvania have launched new master’s in AI programs, targeting future AI leaders.

Despite these efforts, these initiatives face an unclear and evolving understanding of AI literacy. They also face challenges to measuring effectiveness and minimal knowledge on what teaching approaches actually work. And there are long-standing issues with respect to equity − for example, reaching schools, communities, segments of the population and businesses that are stretched or under-resourced.

Next moves on AI literacy

Based on our research, experience as educators and collaboration with policymakers and technology companies, we think a few steps might be prudent.

Building AI literacy starts with recognizing it’s not just about tech: People also need to grasp the social and ethical sides of the technology. To see whether we’re getting there, we researchers and educators should use clear, reliable tests that track progress for different age groups and communities. Universities and companies can try out new teaching ideas first, then share what works through an independent hub. Educators, meanwhile, need proper training and resources, not just additional curricula, to bring AI into the classroom. And because opportunity isn’t spread evenly, partnerships that reach under-resourced schools and neighborhoods are essential so everyone can benefit.

Critically, achieving widespread AI literacy may be even harder than building digital and media literacy, so getting there will require serious investment – not cuts – to education and research.

There is widespread consensus that AI literacy is important, whether to boost AI trust and adoption or to empower citizens to challenge AI or shape its future. As with AI itself, we believe it’s important to approach AI literacy carefully, avoiding hype or an overly technical focus. The right approach can prepare students to become “active and responsible participants in the workforce of the future” and empower Americans to “thrive in an increasingly digital society,” as the AI literacy executive order calls for.

The Conversation will be hosting a free webinar on practical and safe use of AI with our tech editor and an AI expert on June 24 at 2pm ET/11am PT. Sign up to get your questions answered.The Conversation

About the Author:

Daniel S. Schiff, Assistant Professor of Political Science, Purdue University; Arne Bewersdorff, Post Doctoral Researcher in Educational Sciences, Technical University of Munich, and Marie Hornberger, Research Associate at the School of Social Sciences and Technology, Technical University of Munich

This article is republished from The Conversation under a Creative Commons license. Read the original article.

 

Will AI take your job? The answer could hinge on the 4 S’s of the technology’s advantages over humans

By Bruce Schneier, Harvard Kennedy School and Nathan Sanders, Harvard University 

If you’ve worried that AI might take your job, deprive you of your livelihood, or maybe even replace your role in society, it probably feels good to see the latest AI tools fail spectacularly. If AI recommends glue as a pizza topping, then you’re safe for another day.

But the fact remains that AI already has definite advantages over even the most skilled humans, and knowing where these advantages arise — and where they don’t — will be key to adapting to the AI-infused workforce.

AI will often not be as effective as a human doing the same job. It won’t always know more or be more accurate. And it definitely won’t always be fairer or more reliable. But it may still be used whenever it has an advantage over humans in one of four dimensions: speed, scale, scope and sophistication. Understanding these dimensions is the key to understanding AI-human replacement.

Speed

First, speed. There are tasks that humans are perfectly good at but are not nearly as fast as AI. One example is restoring or upscaling images: taking pixelated, noisy or blurry images and making a crisper and higher-resolution version. Humans are good at this; given the right digital tools and enough time, they can fill in fine details. But they are too slow to efficiently process large images or videos.

AI models can do the job blazingly fast, a capability with important industrial applications. AI-based software is used to enhance satellite and remote sensing data, to compress video files, to make video games run better with cheaper hardware and less energy, to help robots make the right movements, and to model turbulence to help build better internal combustion engines.

Real-time performance matters in these cases, and the speed of AI is necessary to enable them.

Scale

The second dimension of AI’s advantage over humans is scale. AI will increasingly be used in tasks that humans can do well in one place at a time, but that AI can do in millions of places simultaneously. A familiar example is ad targeting and personalization. Human marketers can collect data and predict what types of people will respond to certain advertisements. This capability is important commercially; advertising is a trillion-dollar market globally.

AI models can do this for every single product, TV show, website and internet user. This is how the modern ad-tech industry works. Real-time bidding markets price the display ads that appear alongside the websites you visit, and advertisers use AI models to decide when they want to pay that price – thousands of times per second.

Scope

Next, scope. AI can be advantageous when it does more things than any one person could, even when a human might do better at any one of those tasks. Generative AI systems such as ChatGPT can engage in conversation on any topic, write an essay espousing any position, create poetry in any style and language, write computer code in any programming language, and more. These models may not be superior to skilled humans at any one of these things, but no single human could outperform top-tier generative models across them all.

It’s the combination of these competencies that generates value. Employers often struggle to find people with talents in disciplines such as software development and data science who also have strong prior knowledge of the employer’s domain. Organizations are likely to continue to rely on human specialists to write the best code and the best persuasive text, but they will increasingly be satisfied with AI when they just need a passable version of either.

How AI is affecting the job market.

Sophistication

Finally, sophistication. AIs can consider more factors in their decisions than humans can, and this can endow them with superhuman performance on specialized tasks. Computers have long been used to keep track of a multiplicity of factors that compound and interact in ways more complex than a human could trace. The 1990s chess-playing computer systems such as Deep Blue succeeded by thinking a dozen or more moves ahead.

Modern AI systems use a radically different approach: Deep learning systems built from many-layered neural networks take account of complex interactions – often many billions – among many factors. Neural networks now power the best chess-playing models and most other AI systems.

Chess is not the only domain where eschewing conventional rules and formal logic in favor of highly sophisticated and inscrutable systems has generated progress. The stunning advance of AlphaFold2, the AI model of structural biology whose creators Demis Hassabis and John Jumper were recognized with the Nobel Prize in chemistry in 2024, is another example.

This breakthrough replaced traditional physics-based systems for predicting how sequences of amino acids would fold into three-dimensional shapes with a 93 million-parameter model, even though it doesn’t account for physical laws. That lack of real-world grounding is not desirable: No one likes the enigmatic nature of these AI systems, and scientists are eager to understand better how they work.

But the sophistication of AI is providing value to scientists, and its use across scientific fields has grown exponentially in recent years.

Context matters

Those are the four dimensions where AI can excel over humans. Accuracy still matters. You wouldn’t want to use an AI that makes graphics look glitchy or targets ads randomly – yet accuracy isn’t the differentiator. The AI doesn’t need superhuman accuracy. It’s enough for AI to be merely good and fast, or adequate and scalable. Increasing scope often comes with an accuracy penalty, because AI can generalize poorly to truly novel tasks. The 4 S’s are sometimes at odds. With a given amount of computing power, you generally have to trade off scale for sophistication.

Even more interestingly, when an AI takes over a human task, the task can change. Sometimes the AI is just doing things differently. Other times, AI starts doing different things. These changes bring new opportunities and new risks.

For example, high-frequency trading isn’t just computers trading stocks faster; it’s a fundamentally different kind of trading that enables entirely new strategies, tactics and associated risks. Likewise, AI has developed more sophisticated strategies for the games of chess and Go. And the scale of AI chatbots has changed the nature of propaganda by allowing artificial voices to overwhelm human speech.

It is this “phase shift,” when changes in degree may transform into changes in kind, where AI’s impacts to society are likely to be most keenly felt. All of this points to the places that AI can have a positive impact. When a system has a bottleneck related to speed, scale, scope or sophistication, or when one of these factors poses a real barrier to being able to accomplish a goal, it makes sense to think about how AI could help.

Equally, when speed, scale, scope and sophistication are not primary barriers, it makes less sense to use AI. This is why AI auto-suggest features for short communications such as text messages can feel so annoying. They offer little speed advantage and no benefit from sophistication, while sacrificing the sincerity of human communication.

Many deployments of customer service chatbots also fail this test, which may explain their unpopularity. Companies invest in them because of their scalability, and yet the bots often become a barrier to support rather than a speedy or sophisticated problem solver.

Where the advantage lies

Keep this in mind when you encounter a new application for AI or consider AI as a replacement for or an augmentation to a human process. Looking for bottlenecks in speed, scale, scope and sophistication provides a framework for understanding where AI provides value, and equally where the unique capabilities of the human species give us an enduring advantage.The Conversation

About the Author:

Bruce Schneier, Adjunct Lecturer in Public Policy, Harvard Kennedy School and Nathan Sanders, Affiliate, Berkman Klein Center for Internet & Society, Harvard University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

 

Weaponized storytelling: How AI is helping researchers sniff out disinformation campaigns

By Mark Finlayson, Florida International University and Azwad Anjum Islam, Florida International University 

It is not often that cold, hard facts determine what people care most about and what they believe. Instead, it is the power and familiarity of a well-told story that reigns supreme. Whether it’s a heartfelt anecdote, a personal testimony or a meme echoing familiar cultural narratives, stories tend to stick with us, move us and shape our beliefs.

This characteristic of storytelling is precisely what can make it so dangerous when wielded by the wrong hands. For decades, foreign adversaries have used narrative tactics in efforts to manipulate public opinion in the United States. Social media platforms have brought new complexity and amplification to these campaigns. The phenomenon garnered ample public scrutiny after evidence emerged of Russian entities exerting influence over election-related material on Facebook in the lead-up to the 2016 election.

While artificial intelligence is exacerbating the problem, it is at the same time becoming one of the most powerful defenses against such manipulations. Researchers have been using machine learning techniques to analyze disinformation content.

At the Cognition, Narrative and Culture Lab at Florida International University, we are building AI tools to help detect disinformation campaigns that employ tools of narrative persuasion. We are training AI to go beyond surface-level language analysis to understand narrative structures, trace personas and timelines and decode cultural references.

Disinformation vs. misinformation

In July 2024, the Department of Justice disrupted a Kremlin-backed operation that used nearly a thousand fake social media accounts to spread false narratives. These weren’t isolated incidents. They were part of an organized campaign, powered in part by AI.

Disinformation differs crucially from misinformation. While misinformation is simply false or inaccurate information – getting facts wrong – disinformation is intentionally fabricated and shared specifically to mislead and manipulate. A recent illustration of this came in October 2024, when a video purporting to show a Pennsylvania election worker tearing up mail-in ballots marked for Donald Trump swept platforms such as X and Facebook.

Within days, the FBI traced the clip to a Russian influence outfit, but not before it racked up millions of views. This example vividly demonstrates how foreign influence campaigns artificially manufacture and amplify fabricated stories to manipulate U.S. politics and stoke divisions among Americans.

Humans are wired to process the world through stories. From childhood, we grow up hearing stories, telling them and using them to make sense of complex information. Narratives don’t just help people remember – they help us feel. They foster emotional connections and shape our interpretations of social and political events.

Stories have profound effects on human beliefs and behavior.

This makes them especially powerful tools for persuasion – and, consequently, for spreading disinformation. A compelling narrative can override skepticism and sway opinion more effectively than a flood of statistics. For example, a story about rescuing a sea turtle with a plastic straw in its nose often does more to raise concern about plastic pollution than volumes of environmental data.

Usernames, cultural context and narrative time

Using AI tools to piece together a picture of the narrator of a story, the timeline for how they tell it and cultural details specific to where the story takes place can help identify when a story doesn’t add up.

Narratives are not confined to the content users share – they also extend to the personas users construct to tell them. Even a social media handle can carry persuasive signals. We have developed a system that analyzes usernames to infer demographic and identity traits such as name, gender, location, sentiment and even personality, when such cues are embedded in the handle. This work, presented in 2024 at the International Conference on Web and Social Media, highlights how even a brief string of characters can signal how users want to be perceived by their audience.

For example, a user attempting to appear as a credible journalist might choose a handle like @JamesBurnsNYT rather than something more casual like @JimB_NYC. Both may suggest a male user from New York, but one carries the weight of institutional credibility. Disinformation campaigns often exploit these perceptions by crafting handles that mimic authentic voices or affiliations.

Although a handle alone cannot confirm whether an account is genuine, it plays an important role in assessing overall authenticity. By interpreting usernames as part of the broader narrative an account presents, AI systems can better evaluate whether an identity is manufactured to gain trust, blend into a target community or amplify persuasive content. This kind of semantic interpretation contributes to a more holistic approach to disinformation detection – one that considers not just what is said but who appears to be saying it and why.

Also, stories don’t always unfold chronologically. A social media thread might open with a shocking event, flash back to earlier moments and skip over key details in between.

Humans handle this effortlessly – we’re used to fragmented storytelling. But for AI, determining a sequence of events based on a narrative account remains a major challenge.

Our lab is also developing methods for timeline extraction, teaching AI to identify events, understand their sequence and map how they relate to one another, even when a story is told in nonlinear fashion.

Objects and symbols often carry different meanings in different cultures, and without cultural awareness, AI systems risk misinterpreting the narratives they analyze. Foreign adversaries can exploit cultural nuances to craft messages that resonate more deeply with specific audiences, enhancing the persuasive power of disinformation.

Consider the following sentence: “The woman in the white dress was filled with joy.” In a Western context, the phrase evokes a happy image. But in parts of Asia, where white symbolizes mourning or death, it could feel unsettling or even offensive.

In order to use AI to detect disinformation that weaponizes symbols, sentiments and storytelling within targeted communities, it’s critical to give AI this sort of cultural literacy. In our research, we’ve found that training AI on diverse cultural narratives improves its sensitivity to such distinctions.

Who benefits from narrative-aware AI?

Narrative-aware AI tools can help intelligence analysts quickly identify orchestrated influence campaigns or emotionally charged storylines that are spreading unusually fast. They might use AI tools to process large volumes of social media posts in order to map persuasive narrative arcs, identify near-identical storylines and flag coordinated timing of social media activity. Intelligence services could then use countermeasures in real time.

In addition, crisis-response agencies could swiftly identify harmful narratives, such as false emergency claims during natural disasters. Social media platforms could use these tools to efficiently route high-risk content for human review without unnecessary censorship. Researchers and educators could also benefit by tracking how a story evolves across communities, making narrative analysis more rigorous and shareable.

Ordinary users can also benefit from these technologies. The AI tools could flag social media posts in real time as possible disinformation, allowing readers to be skeptical of suspect stories, thus counteracting falsehoods before they take root.

As AI takes on a greater role in monitoring and interpreting online content, its ability to understand storytelling beyond just traditional semantic analysis has become essential. To this end, we are building systems to uncover hidden patterns, decode cultural signals and trace narrative timelines to reveal how disinformation takes hold.The Conversation

About the Author:

Mark Finlayson, Associate Professor of Computer Science, Florida International University and Azwad Anjum Islam, Ph.D. Student in Computing and Information Sciences, Florida International University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

 

Can you upload a human mind into a computer? A neuroscientist ponders what’s possible

By Dobromir Rahnev, Georgia Institute of Technology 

Curious Kids is a series for children of all ages. If you have a question you’d like an expert to answer, send it to [email protected].


Is it possible to upload the consciousness of your mind into a computer? – Amreen, age 15, New Delhi, India


The concept, cool yet maybe a little creepy, is known as mind uploading. Think of it as a way to create a copy of your brain, a transmission of your mind and consciousness into a computer. There you would live digitally, perhaps forever. You’d have an awareness of yourself, you’d retain your memories and still feel like you. But you wouldn’t have a body.

Within that simulated environment, you could do anything you do in real life – eating, driving a car, playing sports. You could also do things impossible in the real world, like walking through walls, flying like a bird or traveling to other planets. The only limit is what science can realistically simulate.

Doable? Theoretically, mind uploading should be possible. Still, you may wonder how it could happen. After all, researchers have barely begun to understand the brain.

Yet science has a track record of turning theoretical possibilities into reality. Just because a concept seems terribly, unimaginably difficult doesn’t mean it’s impossible. Consider that science took humankind to the Moon, sequenced the human genome and eradicated smallpox. Those things too were once considered unlikely.

As a brain scientist who studies perception,
I fully expect mind uploading to one day be a reality. But as of today, we’re nowhere close.

Living in a laptop

The brain is often regarded as the most complex object in the known universe. Replicating all that complexity will be extraordinarily difficult.

One requirement: The uploaded brain needs the same inputs it always had. In other words, the external world must be available to it. Even cloistered inside a computer, you would still need a simulation of your senses, a reproduction of the ability to see, hear, smell, touch, feel – as well as move, blink, detect your heart rate, set your circadian rhythm and do thousands of other things.

But why is that? Couldn’t you just exist in a pure mental bubble, inside the computer without sensory input?

Depriving people of their senses, like putting them in total darkness, or in a room without sound, is known as sensory deprivation, and it’s regarded as a form of torture. People who have trouble sensing their bodily signals – thirst, hunger, pain, an itch – often have mental health challenges.

That’s why for mind uploading to work, the simulation of your senses and the digital environment you’re in must be exceptionally accurate. Even minor distortions could have serious mental consequences.

For now, researchers don’t have the computing power, much less the scientific knowledge, to perform such simulations.

New and updated scanning technology is a necessity.

Scanning billions of pinheads

The first task for a successful mind upload: Scanning, then mapping the complete 3D structure of the human brain. This requires the equivalent of an extraordinarily sophisticated MRI machine that could detail the brain in an advanced way. At the moment, scientists are only at the very early stages of brain mapping – which includes the entire brain of a fly and tiny portions of a mouse brain.

In a few decades, a complete map of the human brain may be possible. Yet even capturing the identities of all 86 billion neurons, all smaller than a pinhead, plus their trillions of connections, still isn’t enough. Uploading this information by itself into a computer won’t accomplish much. That’s because each neuron constantly adjusts its functioning, and that has to be modeled, too.

It’s hard to know how many levels down researchers must go to make the simulated brain work. Is it enough to stop at the molecular level? Right now, no one knows.

Technological immortality comes with significant ethical concerns.

2045? 2145? Or later?

Knowing how the brain computes things might provide a shortcut. That would let researchers simulate only the essential parts of the brain, and not all biological idiosyncrasies. It’s easier to manufacture a new car knowing how a car works, compared to attempting to scan and replicate an existing car without any knowledge of its inner workings.

However, this approach requires that scientists figure out how the brain creates thoughts – how collections of thousands to millions of neurons come together to perform the computations that make the human mind come alive. It’s hard to express how very far we are from this.

Here’s another way: Replace the 86 billion real neurons with artificial ones, one at a time. That approach would make mind uploading much easier. Right now, though, scientists can’t replace even a single real neuron with an artificial one.

But keep in mind the pace of technology is accelerating exponentially. It’s reasonable to expect spectacular improvements in computing power and artificial intelligence in the coming decades.

One other thing is certain: Mind uploading will certainly have no problem finding funding. Many billionaires appear glad to part with lots of their money for a shot at living forever.

Although the challenges are enormous and the path forward uncertain, I believe that one day, mind uploading will be a reality. The most optimistic forecasts pinpoint the year 2045, only 20 years from now. Others say the end of this century.

But in my mind, both of these predictions are probably too optimistic. I would be shocked if mind uploading works in the next 100 years. But it might happen in 200 – which means the first person to live forever could be born in your lifetime.


Hello, curious kids! Do you have a question you’d like an expert to answer? Ask an adult to send your question to [email protected]. Please tell us your name, age and the city where you live.

And since curiosity has no age limit – adults, let us know what you’re wondering, too. We won’t be able to answer every question, but we will do our best.The Conversation

Dobromir Rahnev, Associate Professor of Psychology, Georgia Institute of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Do photons wear out? An astrophysicist explains light’s ability to travel vast cosmic distances without losing energy

By Jarred Roberts, University of California, San Diego 

My telescope, set up for astrophotography in my light-polluted San Diego backyard, was pointed at a galaxy unfathomably far from Earth. My wife, Cristina, walked up just as the first space photo streamed to my tablet. It sparkled on the screen in front of us.

“That’s the Pinwheel galaxy,” I said. The name is derived from its shape – albeit this pinwheel contains about a trillion stars.

The light from the Pinwheel traveled for 25 million years across the universe – about 150 quintillion miles – to get to my telescope.

My wife wondered: “Doesn’t light get tired during such a long journey?”

Her curiosity triggered a thought-provoking conversation about light. Ultimately, why doesn’t light wear out and lose energy over time?

Let’s talk about light

I am an astrophysicist, and one of the first things I learned in my studies is how light often behaves in ways that defy our intuitions.

A photo of outer space that shows a galaxy shaped like a pinwheel.
The author’s photo of the Pinwheel galaxy.
Jarred Roberts

Light is electromagnetic radiation: basically, an electric wave and a magnetic wave coupled together and traveling through space-time. It has no mass. That point is critical because the mass of an object, whether a speck of dust or a spaceship, limits the top speed it can travel through space.

But because light is massless, it’s able to reach the maximum speed limit in a vacuum – about 186,000 miles (300,000 kilometers) per second, or almost 6 trillion miles per year (9.6 trillion kilometers). Nothing traveling through space is faster. To put that into perspective: In the time it takes you to blink your eyes, a particle of light travels around the circumference of the Earth more than twice.

As incredibly fast as that is, space is incredibly spread out. Light from the Sun, which is 93 million miles (about 150 million kilometers) from Earth, takes just over eight minutes to reach us. In other words, the sunlight you see is eight minutes old.

Alpha Centauri, the nearest star to us after the Sun, is 26 trillion miles away (about 41 trillion kilometers). So by the time you see it in the night sky, its light is just over four years old. Or, as astronomers say, it’s four light years away.

Imagine – a trip around the world at the speed of light.

With those enormous distances in mind, consider Cristina’s question: How can light travel across the universe and not slowly lose energy?

Actually, some light does lose energy. This happens when it bounces off something, such as interstellar dust, and is scattered about.

But most light just goes and goes, without colliding with anything. This is almost always the case because space is mostly empty – nothingness. So there’s nothing in the way.

When light travels unimpeded, it loses no energy. It can maintain that 186,000-mile-per-second speed forever.

It’s about time

Here’s another concept: Picture yourself as an astronaut on board the International Space Station. You’re orbiting at 17,000 miles (about 27,000 kilometers) per hour. Compared with someone on Earth, your wristwatch will tick 0.01 seconds slower over one year.

That’s an example of time dilation – time moving at different speeds under different conditions. If you’re moving really fast, or close to a large gravitational field, your clock will tick more slowly than someone moving slower than you, or who is further from a large gravitational field. To say it succinctly, time is relative.

An astronaut floats weightless aboard the International Space Station.
Even astronauts aboard the International Space Station experience time dilation, although the effect is extremely small.
NASA

Now consider that light is inextricably connected to time.
Picture sitting on a photon, a fundamental particle of light; here, you’d experience maximum time dilation. Everyone on Earth would clock you at the speed of light, but from your reference frame, time would completely stop.

That’s because the “clocks” measuring time are in two different places going vastly different speeds: the photon moving at the speed of light, and the comparatively slowpoke speed of Earth going around the Sun.

What’s more, when you’re traveling at or close to the speed of light, the distance between where you are and where you’re going gets shorter. That is, space itself becomes more compact in the direction of motion – so the faster you can go, the shorter your journey has to be. In other words, for the photon, space gets squished.

Which brings us back to my picture of the Pinwheel galaxy. From the photon’s perspective, a star within the galaxy emitted it, and then a single pixel in my backyard camera absorbed it, at exactly the same time. Because space is squished, to the photon the journey was infinitely fast and infinitely short, a tiny fraction of a second.

But from our perspective on Earth, the photon left the galaxy 25 million years ago and traveled 25 million light years across space until it landed on my tablet in my backyard.

And there, on a cool spring night, its stunning image inspired a delightful conversation between a nerdy scientist and his curious wife.The Conversation

About the Author:

Jarred Roberts, Project Scientist, University of California, San Diego

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Challenges to high-performance computing threaten US innovation

By Jack Dongarra, University of Tennessee 

High-performance computing, or HPC for short, might sound like something only scientists use in secret labs, but it’s actually one of the most important technologies in the world today. From predicting the weather to finding new medicines and even training artificial intelligence, high-performance computing systems help solve problems that are too hard or too big for regular computers.

This technology has helped make huge discoveries in science and engineering over the past 40 years. But now, high-performance computing is at a turning point, and the choices the government, researchers and the technology industry make today could affect the future of innovation, national security and global leadership.

High-performance computing systems are basically superpowerful computers made up of thousands or even millions of processors working together at the same time. They also use advanced memory and storage systems to move and save huge amounts of data quickly.


Oak Ridge National Laboratory’s Frontier supercomputer is one of the world’s fastest.
Oak Ridge Leadership Computing Facility, CC BY

With all this power, high-performance computing systems can run extremely detailed simulations and calculations. For example, they can simulate how a new drug interacts with the human body, or how a hurricane might move across the ocean. They’re also used in fields such as automotive design, energy production and space exploration.

Lately, high-performance computing has become even more important because of artificial intelligence. AI models, especially the ones used for things such as voice recognition and self-driving cars, require enormous amounts of computing power to train. High-performance computing systems are well suited for this job. As a result, AI and high-performance computing are now working closely together, pushing each other forward.

Lawrence Livermore National Laboratory’s supercomputer El Capitan is currently the world’s fastest.

I’m a computer scientist with a long career working in high-performance computing. I’ve observed that high-performance computing systems are under more pressure than ever, with higher demands on the systems for speed, data and energy. At the same time, I see that high-performance computing faces some serious technical problems.

Technical challenges

One big challenge for high-performance computing is the gap between how fast processors are and how well memory systems can keep up with the processors’ output. Imagine having a superfast car but being stuck in traffic – it doesn’t help to have speed if the road can’t handle it. In the same way, high-performance computing processors often have to wait around because memory systems can’t send data quickly enough. This makes the whole system less efficient.

Another problem is energy use. Today’s supercomputers use a huge amount of electricity, sometimes as much as a small town. That’s expensive and not very good for the environment. In the past, as computer parts got smaller, they also used less power. But that trend, called Dennard scaling, stopped in the mid-2000s. Now, making computers more powerful usually means they use more energy too. To fix this, researchers are looking for new ways to design both the hardware and the software of high-performance computing systems.

There’s also a problem with the kinds of chips being made. The chip industry is mainly focused on AI, which works fine with lower-precision math like 16-bit or 8-bit numbers. But many scientific applications still need 64-bit precision to be accurate. The greater the bit count, the more digits to the right of the decimal point a chip can process, hence the greater precision. If chip companies stop making the parts that scientists need, then it could become harder to do important research.

This report discusses how trends in semiconductor manufacturing and commercial priorities may diverge from the needs of the scientific computing community, and how a lack of tailored hardware could hinder progress in research.

One solution might be to build custom chips for high-performance computing, but that’s expensive and complicated. Still, researchers are exploring new designs, including chiplets – small chips that can be combined like Lego bricks – to make high-precision processors more affordable.

A global race

Globally, many countries are investing heavily in high-performance computing. Europe has the EuroHPC program, which is building supercomputers in places such as Finland and Italy. Their goal is to reduce dependence on foreign technology and take the lead in areas such as climate modeling and personalized medicine. Japan built the Fugaku supercomputer, which supports both academic research and industrial work. China has also made major advances, using homegrown technology to build some of the world’s fastest computers. All of these countries’ governments understand that high-performance computing is key to their national security, economic strength and scientific leadership.

The U.S.-China supercomputer rivalry explained.

The United States, which has been a leader in high-performance computing for decades, recently completed the Department of Energy’s Exascale Computing Project. This project created computers that can perform a billion billion operations per second. That’s an incredible achievement. But even with that success, the U.S. still doesn’t have a clear, long-term plan for what comes next. Other countries are moving quickly, and without a national strategy, the U.S. risks falling behind.

I believe that a U.S. national strategy should include funding new machines and training for people to use them. It would also include partnerships with universities, national labs and private companies. Most importantly, the plan would focus not just on hardware but also on the software and algorithms that make high-performance computing useful.

Hopeful signs

One exciting area for the future is quantum computing. This is a completely new way of doing computation based on the laws of physics at the atomic level. Quantum computers could someday solve problems that are impossible for regular computers. But they are still in the early stages and are likely to complement rather than replace traditional high-performance computing systems. That’s why it’s important to keep investing in both kinds of computing.

The good news is that some steps have already been taken. The CHIPS and Science Act, passed in 2022, provides funding to expand chip manufacturing in the U.S. It also created an office to help turn scientific research into real-world products. The task force Vision for American Science and Technology, launched on Feb. 25, 2025, and led by American Association for the Advancement of Science CEO Sudip Parikh, aims to marshal nonprofits, academia and industry to help guide the government’s decisions. Private companies are also spending billions of dollars on data centers and AI infrastructure.

All of these are positive signs, but they don’t fully solve the problem of how to support high-performance computing in the long run. In addition to short-term funding and infrastructure investments, this means:

  • Long-term federal investment in high-performance computing R&D, including advanced hardware, software and energy-efficient architectures.
  • Procurement and deployment of leadership-class computing systems at national labs and universities.
  • Workforce development, including training in parallel programming, numerical methods and AI-HPC integration.
  • Hardware road map alignment, ensuring commercial chip development remains compatible with the needs of scientific and engineering applications.
  • Sustainable funding models that prevent boom-and-bust cycles tied to one-off milestones or geopolitical urgency.
  • Public-private collaboration to bridge gaps between academic research, industry innovation and national security needs.

High-performance computing is more than just fast computers. It’s the foundation of scientific discovery, economic growth and national security. With other countries pushing forward, the U.S. is under pressure to come up with a clear, coordinated plan. That means investing in new hardware, developing smarter software, training a skilled workforce and building partnerships between government, industry and academia. If the U.S. does that, the country can make sure high-performance computing continues to power innovation for decades to come.The Conversation

About the Author:

Jack Dongarra, Emeritus Professor of Computer Science, University of Tennessee

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Why predicting battery performance is like forecasting traffic − and how researchers are making progress

By Emmanuel Olugbade, Missouri University of Science and Technology 

Lithium-ion batteries are quietly powering large parts of the world, including electric vehicles and smartphones. They have revolutionized how people store and use energy. But as these batteries become more central to daily life, they bring more attention to the challenges of managing them and the energy they store safely, efficiently and intelligently.

I’m a mechanical engineer who studies these nearly ubiquitous batteries. They have been around for decades, yet researchers like me are still trying to fully understand how these batteries behave – especially when they are working hard.

Batteries may seem simple, but they are as complicated as the real-world uses people devise for them.

The big picture

At their core, lithium-ion batteries rely on the movement of charged particles, called ions, of the element lithium between two electric poles, or electrodes. The lithium ions move from the positive electrode to the negative one through a conductive substance called an electrolyte, which can be a solid or a liquid.

The basics of how a lithium-ion battery works.

How much energy these batteries store and how well they work depends on a tangle of factors, including the temperature, physical structure of the battery and how the materials age over time.

Around the world, researchers are trying to answer questions about each of these factors individually and in concert with each other. Some research focuses on improving lifespan and calculating how batteries degrade over time. Other projects are tackling safety under extreme conditions, such as fast-charging use in extreme climates – either hot or cold. Many are exploring entirely new materials that could make batteries cheaper, longer-lasting or safer. And a significant group – including me – are working with computer simulations to improve real-time battery monitoring.

Real‑time monitoring in your electric vehicle’s battery system functions like a health check: It tracks voltage, current and temperature to estimate how much energy remains so you won’t be stranded with a dead battery.

But it’s difficult to precisely measure how well each of the energy cells within the battery is performing as they age or as the weather changes from cold in winter to hotter in summer. So the battery management system uses a computer simulation to estimate those factors. When combined with real-time monitoring, the system can prevent overusing the battery, balance charging speed with long-term health, avoid power failures and keep performance high. But there are a lot of variables.

The traffic analogy

One of the best ways to understand this challenge is to think about city traffic.

Let’s say you want to drive across town and need to determine whether your car has enough charge to travel the best route. If your navigation simulator accounted for every stoplight, every construction zone and every vehicle on the road, it would give you a very accurate answer. But it might take an hour to run, by which time the circumstances would have changed and the answer would likely be wrong. That’s not helpful if you’re trying to make a decision right now.

A simpler model might assume that every road is clear and every car is moving at the speed limit. That simulation delivers a result instantly – but its results are very inaccurate when traffic is heavy or a road is closed. It doesn’t capture the reality of rush hour.

While you’re driving, the battery management system would do a similar set of calculations to see how much charge is available for the rest of the trip. It would look at the battery’s temperature, how old it is and how much energy the car is asking for, like when going up a steep hill or accelerating quickly to keep up with other cars. But like the navigation simulations, it has to strike a balance between being extremely accurate and giving you useful information before your battery runs out in the middle of your trip.

The most accurate models, which simulate every chemical reaction inside the battery, are too slow for real-time use. The faster models simplify things so much that they miss key behaviors – especially under stress, such as fast charging or sudden bursts of energy use.

How researchers are bridging the gap

This trade-off between speed and accuracy is at the heart of battery modeling research today. Scientists and engineers are exploring many ways to solve it.

Some are rewriting modeling software to make the physics calculations more efficient, reducing complexity without losing the key details. Others, like me, are turning to machine learning – training computers to recognize patterns in data and make fast, accurate predictions without having to solve every underlying equation.

In my recent work, I used a high-accuracy battery simulator – one of the ones that’s really accurate but very slow – to generate a massive amount of data about how a battery functions when charging and discharging. I used that data to train a machine learning algorithm called XGBoost, which is particularly good at finding patterns in data.

Then I used software to pair the XGBoost system with a simple, fast-running battery model that captures the basic physics but can miss finer details. The simpler model puts out an initial set of results, and the XGBoost element fine-tunes those to make corrections on the fly, especially when the battery is under strain.

The result is a hybrid model that is able to respond both quickly and accurately to changes in driving conditions. A driver who floors the accelerator with just the simple model wouldn’t get enough energy; a more detailed model would give the right amount of energy only after it finished all its calculations. My hybrid model delivers a rapid boost of energy without delays.

Other teams are working on similar hybrid approaches, blending physics and artificial intelligence in creative ways. Some are even building digital twinsreal-time virtual replicas of physical batteries – to offer sophisticated simulations that update constantly as conditions change.

What’s next

Battery research is moving quickly, and the field is already seeing signs of change. Models are becoming more reliable across a wider range of conditions. Engineers are using real-time monitoring to extend battery life, prevent overheating and improve energy efficiency. Machine learning lets researchers train battery management systems to optimize performance for specific applications, such as high power demands in electric vehicles, daily cycles of home electricity use, short power bursts for drones, or long-duration requirements for building-scale battery systems.

And there’s more to come: Researchers are working to include other important factors into their battery models, such as heat generation and mechanical stress.

Some teams are taking hybrid models and compiling their software into lightweight code that runs on microcontrollers inside battery hardware. In practice, that means each battery pack carries its own brain on-board, calculating state-of-charge, estimating aging and tracking thermal or mechanical stress in near-real time. By embedding the model in the device’s electronics, the pack can autonomously adjust its charging and discharging strategy on the fly, making every battery smarter, safer and more efficient.

As the energy landscape evolves – with more electric vehicles on the road, more renewable energy sources feeding into the grid, and more people relying on batteries in daily life – the ability to understand what a battery is doing in real time becomes more critical than ever.The Conversation

About the Author:

Emmanuel Olugbade, Ph.D. Candidate in Mechanical Engineering, Missouri University of Science and Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

 

Popular AIs head-to-head: OpenAI beats DeepSeek on sentence-level reasoning

By Manas Gaur, University of Maryland, Baltimore County 

ChatGPT and other AI chatbots based on large language models are known to occasionally make things up, including scientific and legal citations. It turns out that measuring how accurate an AI model’s citations are is a good way of assessing the model’s reasoning abilities.

An AI model “reasons” by breaking down a query into steps and working through them in order. Think of how you learned to solve math word problems in school.

Ideally, to generate citations an AI model would understand the key concepts in a document, generate a ranked list of relevant papers to cite, and provide convincing reasoning for how each suggested paper supports the corresponding text. It would highlight specific connections between the text and the cited research, clarifying why each source matters.

The question is, can today’s models be trusted to make these connections and provide clear reasoning that justifies their source choices? The answer goes beyond citation accuracy to address how useful and accurate large language models are for any information retrieval purpose.

I’m a computer scientist. My colleagues − researchers from the AI Institute at the University of South Carolina, Ohio State University and University of Maryland Baltimore County − and I have developed the Reasons benchmark to test how well large language models can automatically generate research citations and provide understandable reasoning.

We used the benchmark to compare the performance of two popular AI reasoning models, DeepSeek’s R1 and OpenAI’s o1. Though DeepSeek made headlines with its stunning efficiency and cost-effectiveness, the Chinese upstart has a way to go to match OpenAI’s reasoning performance.

Sentence specific

The accuracy of citations has a lot to do with whether the AI model is reasoning about information at the sentence level rather than paragraph or document level. Paragraph-level and document-level citations can be thought of as throwing a large chunk of information into a large language model and asking it to provide many citations.

In this process, the large language model overgeneralizes and misinterprets individual sentences. The user ends up with citations that explain the whole paragraph or document, not the relatively fine-grained information in the sentence.

Further, reasoning suffers when you ask the large language model to read through an entire document. These models mostly rely on memorizing patterns that they typically are better at finding at the beginning and end of longer texts than in the middle. This makes it difficult for them to fully understand all the important information throughout a long document.

Large language models get confused because paragraphs and documents hold a lot of information, which affects citation generation and the reasoning process. Consequently, reasoning from large language models over paragraphs and documents becomes more like summarizing or paraphrasing.

The Reasons benchmark addresses this weakness by examining large language models’ citation generation and reasoning.

How DeepSeek R1 and OpenAI o1 compare generally on logic problems.

Testing citations and reasoning

Following the release of DeepSeek R1 in January 2025, we wanted to examine its accuracy in generating citations and its quality of reasoning and compare it with OpenAI’s o1 model. We created a paragraph that had sentences from different sources, gave the models individual sentences from this paragraph, and asked for citations and reasoning.

To start our test, we developed a small test bed of about 4,100 research articles around four key topics that are related to human brains and computer science: neurons and cognition, human-computer interaction, databases and artificial intelligence. We evaluated the models using two measures: F-1 score, which measures how accurate the provided citation is, and hallucination rate, which measures how sound the model’s reasoning is − that is, how often it produces an inaccurate or misleading response.

Our testing revealed significant performance differences between OpenAI o1 and DeepSeek R1 across different scientific domains. OpenAI’s o1 did well connecting information between different subjects, such as understanding how research on neurons and cognition connects to human-computer interaction and then to concepts in artificial intelligence, while remaining accurate. Its performance metrics consistently outpaced DeepSeek R1’s across all evaluation categories, especially in reducing hallucinations and successfully completing assigned tasks.

OpenAI o1 was better at combining ideas semantically, whereas R1 focused on making sure it generated a response for every attribution task, which in turn increased hallucination during reasoning. OpenAI o1 had a hallucination rate of approximately 35% compared with DeepSeek R1’s rate of nearly 85% in the attribution-based reasoning task.

In terms of accuracy and linguistic competence, OpenAI o1 scored about 0.65 on the F-1 test, which means it was right about 65% of the time when answering questions. It also scored about 0.70 on the BLEU test, which measures how well a language model writes in natural language. These are pretty good scores.

DeepSeek R1 scored lower, with about 0.35 on the F-1 test, meaning it was right about 35% of the time. However, its BLEU score was only about 0.2, which means its writing wasn’t as natural-sounding as OpenAI’s o1. This shows that o1 was better at presenting that information in clear, natural language.

OpenAI holds the advantage

On other benchmarks, DeepSeek R1 performs on par with OpenAI o1 on math, coding and scientific reasoning tasks. But the substantial difference on our benchmark suggests that o1 provides more reliable information, while R1 struggles with factual consistency.

Though we included other models in our comprehensive testing, the performance gap between o1 and R1 specifically highlights the current competitive landscape in AI development, with OpenAI’s offering maintaining a significant advantage in reasoning and knowledge integration capabilities.

These results suggest that OpenAI still has a leg up when it comes to source attribution and reasoning, possibly due to the nature and volume of the data it was trained on. The company recently announced its deep research tool, which can create reports with citations, ask follow-up questions and provide reasoning for the generated response.

The jury is still out on the tool’s value for researchers, but the caveat remains for everyone: Double-check all citations an AI gives you.The Conversation

About the Author:

Manas Gaur, Assistant Professor of Computer Science and Electrical Engineering, University of Maryland, Baltimore County

This article is republished from The Conversation under a Creative Commons license. Read the original article.

 

What is reinforcement learning? An AI researcher explains a key method of teaching machines – and how it relates to training your dog

By Ambuj Tewari, University of Michigan 

Understanding intelligence and creating intelligent machines are grand scientific challenges of our times. The ability to learn from experience is a cornerstone of intelligence for machines and living beings alike.

In a remarkably prescient 1948 report, Alan Turing – the father of modern computer science – proposed the construction of machines that display intelligent behavior. He also discussed the “education” of such machines “by means of rewards and punishments.”

Turing’s ideas ultimately led to the development of reinforcement learning, a branch of artificial intelligence. Reinforcement learning designs intelligent agents by training them to maximize rewards as they interact with their environment.

As a machine learning researcher, I find it fitting that reinforcement learning pioneers Andrew Barto and Richard Sutton were awarded the 2024 ACM Turing Award.

What is reinforcement learning?

Animal trainers know that animal behavior can be influenced by rewarding desirable behaviors. A dog trainer gives the dog a treat when it does a trick correctly. This reinforces the behavior, and the dog is more likely to do the trick correctly the next time. Reinforcement learning borrowed this insight from animal psychology.

But reinforcement learning is about training computational agents, not animals. The agent can be a software agent like a chess-playing program. But the agent can also be an embodied entity like a robot learning to do household chores. Similarly, the environment of an agent can be virtual, like the chessboard or the designed world in a video game. But it can also be a house where a robot is working.

Just like animals, an agent can perceive aspects of its environment and take actions. A chess-playing agent can access the chessboard configuration and make moves. A robot can sense its surroundings with cameras and microphones. It can use its motors to move about in the physical world.

Agents also have goals that their human designers program into them. A chess-playing agent’s goal is to win the game. A robot’s goal might be to assist its human owner with household chores.

The reinforcement learning problem in AI is how to design agents that achieve their goals by perceiving and acting in their environments. Reinforcement learning makes a bold claim: All goals can be achieved by designing a numerical signal, called the reward, and having the agent maximize the total sum of rewards it receives.

Reinforcement learning from human feedback is key to keeping AIs aligned with human goals and values.

Researchers do not know if this claim is actually true, because of the wide variety of possible goals. Therefore, it is often referred to as the reward hypothesis.

Sometimes it is easy to pick a reward signal corresponding to a goal. For a chess-playing agent, the reward can be +1 for a win, 0 for a draw, and -1 for a loss. It is less clear how to design a reward signal for a helpful household robotic assistant. Nevertheless, the list of applications where reinforcement learning researchers have been able to design good reward signals is growing.

A big success of reinforcement learning was in the board game Go. Researchers thought that Go was much harder than chess for machines to master. The company DeepMind, now Google DeepMind, used reinforcement learning to create AlphaGo. AlphaGo defeated top Go player Lee Sedol in a five-match game in 2016.

A more recent example is the use of reinforcement learning to make chatbots such as ChatGPT more helpful. Reinforcement learning is also being used to improve the reasoning capabilities of chatbots.

Reinforcement learning’s origins

However, none of these successes could have been foreseen in the 1980s. That is when Barto and his then-Ph.D. student Sutton proposed reinforcement learning as a general problem-solving framework. They drew inspiration not only from animal psychology but also from the field of control theory, the use of feedback to influence a system’s behavior, and optimization, a branch of mathematics that studies how to select the best choice among a range of available options. They provided the research community with mathematical foundations that have stood the test of time. They also created algorithms that have now become standard tools in the field.

It is a rare advantage for a field when pioneers take the time to write a textbook. Shining examples like “The Nature of the Chemical Bond” by Linus Pauling and “The Art of Computer Programming” by Donald E. Knuth are memorable because they are few and far between. Sutton and Barto’s “Reinforcement Learning: An Introduction” was first published in 1998. A second edition came out in 2018. Their book has influenced a generation of researchers and has been cited more than 75,000 times.

Reinforcement learning has also had an unexpected impact on neuroscience. The neurotransmitter dopamine plays a key role in reward-driven behaviors in humans and animals. Researchers have used specific algorithms developed in reinforcement learning to explain experimental findings in people and animals’ dopamine system.

Barto and Sutton’s foundational work, vision and advocacy have helped reinforcement learning grow. Their work has inspired a large body of research, made an impact on real-world applications, and attracted huge investments by tech companies. Reinforcement learning researchers, I’m sure, will continue to see further ahead by standing on their shoulders.The Conversation

About the Author:

Ambuj Tewari, Professor of Statistics, University of Michigan

This article is republished from The Conversation under a Creative Commons license. Read the original article.