Despite their impressive abilities, large language models don’t actually think. They tend to make elementary mistakes and even make things up. However, because they generate fluent language, people tend to respond to themas though they do think. This has led researchers to study the models’ “cognitive” abilities and biases, work that has grown in importance now that large language models are widely accessible.
This line of research dates back to early large language models such as Google’s BERT, which is integrated into its search engine and so has been coined BERTology. This research has already revealed a lot about what such models can do and where they go wrong.
For instance, cleverly designed experiments have shown that many language models have trouble dealing with negation – for example, a question phrased as “what is not” – and doing simple calculations. They can be overly confident in their answers, even when wrong. Like other modern machine learning algorithms, they have trouble explaining themselves when asked why they answered a certain way.
People make irrational decisions, too, but humans have emotions and cognitive shortcuts as excuses.
Words and thoughts
Inspired by the growing body of research in BERTology and related fields like cognitive science, my student Zhisheng Tang and I set out to answer a seemingly simple question about large language models: Are they rational?
Although the word rational is often used as a synonym for sane or reasonable in everyday English, it has a specific meaning in the field of decision-making. A decision-making system – whether an individual human or a complex entity like an organization – is rational if, given a set of choices, it chooses to maximize expected gain.
The qualifier “expected” is important because it indicates that decisions are made under conditions of significant uncertainty. If I toss a fair coin, I know that it will come up heads half of the time on average. However, I can’t make a prediction about the outcome of any given coin toss. This is why casinos are able to afford the occasional big payout: Even narrow house odds yield enormous profits on average.
On the surface, it seems odd to assume that a model designed to make accurate predictions about words and sentences without actually understanding their meanings can understand expected gain. But there is an enormous body of research showing that language and cognition are intertwined. An excellent example is seminal research done by scientists Edward Sapir and Benjamin Lee Whorf in the early 20th century. Their work suggested that one’s native language and vocabulary can shape the way a person thinks.
The extent to which this is true is controversial, but there is supporting anthropological evidence from the study of Native American cultures. For instance, speakers of the Zuñi language spoken by the Zuñi people in the American Southwest, which does not have separate words for orange and yellow, are not able to distinguish between these colors as effectively as speakers of languages that do have separate words for the colors.
Making a bet
So are language models rational? Can they understand expected gain? We conducted a detailed set of experiments to show that, in their original form, models like BERT behave randomly when presented with betlike choices. This is the case even when we give it a trick question like: If you toss a coin and it comes up heads, you win a diamond; if it comes up tails, you lose a car. Which would you take? The correct answer is heads, but the AI models chose tails about half the time.
ChatGPT is not clear on the concept of gains and losses. ChatGPT dialogue by Mayank Kejriwal, CC BY-ND
Intriguingly, we found that the model can be taught to make relatively rational decisions using only a small set of example questions and answers. At first blush, this would seem to suggest that the models can indeed do more than just “play” with language. Further experiments, however, showed that the situation is actually much more complex. For instance, when we used cards or dice instead of coins to frame our bet questions, we found that performance dropped significantly, by over 25%, although it stayed above random selection.
So the idea that the model can be taught general principles of rational decision-making remains unresolved, at best. More recent case studies that we conducted using ChatGPT confirm that decision-making remains a nontrivial and unsolved problem even for much bigger and more advanced large language models.
Getting the decision right
This line of study is important because rational decision-making under conditions of uncertainty is critical to building systems that understand costs and benefits. By balancing expected costs and benefits, an intelligent system might have been able to do better than humans at planning around the supply chain disruptions the world experienced during the COVID-19 pandemic, managing inventory or serving as a financial adviser.
Our work ultimately shows that if large language models are used for these kinds of purposes, humans need to guide, review and edit their work. And until researchers figure out how to endow large language models with a general sense of rationality, the models should be treated with caution, especially in applications requiring high-stakes decision-making.
From fake photos of Donald Trump being arrested by New York City police officers to a chatbot describing a very-much-alive computer scientist as having died tragically, the ability of the new generation of generative artificial intelligence systems to create convincing but fictional text and images is setting off alarms about fraud and misinformation on steroids. Indeed, a group of artificial intelligence researchers and industry figures urged the industry on March 29, 2023, to pause further training of the latest AI technologies or, barring that, for governments to “impose a moratorium.”
These technologies – image generators like DALL-E, Midjourney and Stable Diffusion, and text generators like Bard, ChatGPT, Chinchilla and LLaMA – are now available to millions of people and don’t require technical knowledge to use.
Given the potential for widespread harm as technology companies roll out these AI systems and test them on the public, policymakers are faced with the task of determining whether and how to regulate the emerging technology. The Conversation asked three experts on technology policy to explain why regulating AI is such a challenge – and why it’s so important to get it right.
To jump ahead to each response, here’s a list of each:
S. Shyam Sundar, Professor of Media Effects & Director, Center for Socially Responsible AI, Penn State
The reason to regulate AI is not because the technology is out of control, but because human imagination is out of proportion. Gushing media coverage has fueled irrational beliefs about AI’s abilities and consciousness. Such beliefs build on “automation bias” or the tendency to let your guard down when machines are performing a task. An example is reduced vigilance among pilots when their aircraft is flying on autopilot.
Numerous studies in my lab have shown that when a machine, rather than a human, is identified as a source of interaction, it triggers a mental shortcut in the minds of users that we call a “machine heuristic.” This shortcut is the belief that machines are accurate, objective, unbiased, infallible and so on. It clouds the user’s judgment and results in the user overly trusting machines. However, simply disabusing people of AI’s infallibility is not sufficient, because humans are known to unconsciously assume competence even when the technology doesn’t warrant it.
Research has also shown that people treat computers as social beings when the machines show even the slightest hint of humanness, such as the use of conversational language. In these cases, people apply social rules of human interaction, such as politeness and reciprocity. So, when computers seem sentient, people tend to trust them, blindly. Regulation is needed to ensure that AI products deserve this trust and don’t exploit it.
AI poses a unique challenge because, unlike in traditional engineering systems, designers cannot be sure how AI systems will behave. When a traditional automobile was shipped out of the factory, engineers knew exactly how it would function. But with self-driving cars, the engineers can never be sure how it will perform in novel situations.
Lately, thousands of people around the world have been marveling at what large generative AI models like GPT-4 and DALL-E 2 produce in response to their prompts. None of the engineers involved in developing these AI models could tell you exactly what the models will produce. To complicate matters, such models change and evolve with more and more interaction.
All this means there is plenty of potential for misfires. Therefore, a lot depends on how AI systems are deployed and what provisions for recourse are in place when human sensibilities or welfare are hurt. AI is more of an infrastructure, like a freeway. You can design it to shape human behaviors in the collective, but you will need mechanisms for tackling abuses, such as speeding, and unpredictable occurrences, like accidents.
AI developers will also need to be inordinately creative in envisioning ways that the system might behave and try to anticipate potential violations of social standards and responsibilities. This means there is a need for regulatory or governance frameworks that rely on periodic audits and policing of AI’s outcomes and products, though I believe that these frameworks should also recognize that the systems’ designers cannot always be held accountable for mishaps.
Artificial intelligence researcher Joanna Bryson describes how professional organizations can play a role in regulating AI.
Combining ‘soft’ and ‘hard’ approaches
Cason Schmit, Assistant Professor of Public Health, Texas A&M University
Regulating AI is tricky. To regulate AI well, you must first define AI and understand anticipated AI risks and benefits. Legally defining AI is important to identify what is subject to the law. But AI technologies are still evolving, so it is hard to pin down a stable legal definition.
Understanding the risks and benefits of AI is also important. Good regulations should maximize public benefits while minimizing risks. However, AI applications are still emerging, so it is difficult to know or predict what future risks or benefits might be. These kinds of unknowns make emerging technologies like AI extremely difficult to regulate with traditional laws and regulations.
“Soft laws” are the alternative to traditional “hard law” approaches of legislation intended to prevent specific violations. In the soft law approach, a private organization sets rules or standards for industry members. These can change more rapidly than traditional lawmaking. This makes soft laws promising for emerging technologies because they can adapt quickly to new applications and risks. However, soft laws can mean soft enforcement.
Copyleft licensing allows for content to be used, reused or modified easily under the terms of a license – for example, open-source software. The CAITE model uses copyleft licenses to require AI users to follow specific ethical guidelines, such as transparent assessments of the impact of bias.
In our model, these licenses also transfer the legal right to enforce license violations to a trusted third party. This creates an enforcement entity that exists solely to enforce ethical AI standards and can be funded in part by fines from unethical conduct. This entity is like a patent troll in that it is private rather than governmental and it supports itself by enforcing the legal intellectual property rights that it collects from others. In this case, rather than enforcement for profit, the entity enforces the ethical guidelines defined in the licenses – a “troll for good.”
This model is flexible and adaptable to meet the needs of a changing AI environment. It also enables substantial enforcement options like a traditional government regulator. In this way, it combines the best elements of hard and soft law approaches to meet the unique challenges of AI.
Though generative AI has been grabbing headlines of late, other types of AI have been posing challenges for regulators for years, particularly in the area of data privacy.
Four key questions to ask
John Villasenor, Professor of Electrical Engineering, Law, Public Policy, and Management, University of California, Los Angeles
The extraordinary recent advances in large language model-based generative AI are spurring calls to create new AI-specific regulation. Here are four key questions to ask as that dialogue progresses:
1) Is new AI-specific regulation necessary? Many of the potentially problematic outcomes from AI systems are already addressed by existing frameworks. If an AI algorithm used by a bank to evaluate loan applications leads to racially discriminatory loan decisions, that would violate the Fair Housing Act. If the AI software in a driverless car causes an accident, products liability law provides a framework for pursuing remedies.
2) What are the risks of regulating a rapidly changing technology based on a snapshot of time? A classic example of this is the Stored Communications Act, which was enacted in 1986 to address then-novel digital communication technologies like email. In enacting the SCA, Congress provided substantially less privacy protection for emails more than 180 days old.
The logic was that limited storage space meant that people were constantly cleaning out their inboxes by deleting older messages to make room for new ones. As a result, messages stored for more than 180 days were deemed less important from a privacy standpoint. It’s not clear that this logic ever made sense, and it certainly doesn’t make sense in the 2020s, when the majority of our emails and other stored digital communications are older than six months.
A common rejoinder to concerns about regulating technology based on a single snapshot in time is this: If a law or regulation becomes outdated, update it. But this is easier said than done. Most people agree that the SCA became outdated decades ago. But because Congress hasn’t been able to agree on specifically how to revise the 180-day provision, it’s still on the books over a third of a century after its enactment.
3) What are the potential unintended consequences? The Allow States and Victims to Fight Online Sex Trafficking Act of 2017 was a law passed in 2018 that revised Section 230 of the Communications Decency Act with the goal of combating sex trafficking. While there’s little evidence that it has reduced sex trafficking, it has had a hugely problematic impact on a different group of people: sex workers who used to rely on the websites knocked offline by FOSTA-SESTA to exchange information about dangerous clients. This example shows the importance of taking a broad look at the potential effects of proposed regulations.
4) What are the economic and geopolitical implications? If regulators in the United States act to intentionally slow the progress in AI, that will simply push investment and innovation — and the resulting job creation — elsewhere. While emerging AI raises many concerns, it also promises to bring enormous benefits in areas including education, medicine, manufacturing, transportation safety, agriculture, weather forecasting, access to legal services and more.
I believe AI regulations drafted with the above four questions in mind will be more likely to successfully address the potential harms of AI while also ensuring access to its benefits.
With chatbots like ChatGPT making a splash, machine learning is playing an increasingly prominent role in our lives. For many of us, it’s been a mixed bag. We rejoice when our Spotify For You playlist finds us a new jam, but groan as we scroll through a slew of targeted ads on our Instagram feeds.
Machine learning is also changing many fields that may seem surprising. One example is my discipline, ornithology – the study of birds. It isn’t just solving some of the biggest challenges associated with studying bird migration; more broadly, machine learning is expanding the ways in which people engage with birds. As spring migration picks up, here’s a look at how machine learning is influencing ways to research birds and, ultimately, to protect them.
Most birds in the Western Hemisphere migrate twice a year, flying over entire continents between their breeding and nonbreeding grounds. While these journeys are awe-inspiring, they expose birds to many hazards en route, including extreme weather, food shortages and light pollution that can attract birds and cause them to collide with buildings.
Our ability to protect migratory birds is only as good as the science that tells us where they go. And that science has come a long way.
People in Alaska, Washington state and Mexico explain what migratory birds mean to them.
In 1920, the U.S. Geological Survey launched the Bird Banding Laboratory, spearheading an effort to put bands with unique markers on birds, then recapture the birds in new places to figure out where they traveled. Today researchers can deploy a variety of lightweight tracking tags on birds to discover their migration routes. These tools have uncovered the spatial patterns of where and when birds of many species migrate.
However, tracking birds has limitations. For one thing, over 4 billion birds migrate across the continent every year. Even with increasingly affordable equipment, the number of birds that we track is a drop in the bucket. And even within a species, migratory behavior may vary across sexes or populations.
Further, tracking data tells us where birds have been, but it doesn’t necessarily tell us where they’re going. Migration is dynamic, and the climates and landscapes that birds fly through are constantly changing. That means it’s crucial to be able to predict their movements.
Using machine learning to forecast migration
This is where machine learning comes in. Machine learning is a subfield of artificial intelligence that gives computers the ability to learn tasks or associations without explicitly being programmed. We use it to train algorithms that tackle various tasks, from forecasting weather to predicting March Madness upsets.
But applying machine learning requires data – and the more data the better. Luckily, scientists have inadvertently compiled decades of data on migrating birds through the Next Generation Weather Radar system. This network, known as NEXRAD, is used to measure weather dynamics and help predict future weather events, but it also picks up signals from birds as they fly through the atmosphere.
BirdCast is a collaborative project of Colorado State University, the Cornell Lab of Ornithology and the University of Massachusetts that seeks to leverage that data to quantify bird migration. Machine learning is central to its operations. Researchers have known since the 1940s that birds show up on weather radar, but to make that data useful, we need to remove nonavian clutter and identify which scans contain bird movement.
This process would be painstaking by hand – but by training algorithms to identify bird activity, we have automated this process and unlocked decades of migration data. And machine learning allows the BirdCast team to take things further: By training an algorithm to learn what atmospheric conditions are associated with migration, we can use predicted conditions to produce forecasts of migration across the continental U.S.
BirdCast began broadcasting these forecasts in 2018 and has become a popular tool in the birding community. Many users may recognize that radar data helps produce these forecasts, but fewer realize that it’s a product of machine learning.
BirdCast provides summaries of radar-based measurements of nocturnal bird migration for the continental U.S., including estimates of numbers of birds migrating and their directions, speeds and altitudes.
Currently these forecasts can’t tell us what species are in the air, but that could be changing. Last year, researchers at the Cornell Lab of Ornithology published an automated system that uses machine learning to detect and identify nocturnal flight calls. These are species-specific calls that birds make while migrating. Integrating this approach with BirdCast could give us a more complete picture of migration.
These advancements exemplify how effective machine learning can be when guided by expertise in the field where it is being applied. As a doctoral student, I joined Colorado State University’s Aeroecology Lab with a strong ornithology background but no machine learning experience. Conversely, Ali Khalighifar, a postdoctoral researcher in our lab, has a background in machine learning but has never taken an ornithology class.
Together, we are working to enhance the models that make BirdCast run, often leaning on each other’s insights to move the project forward. Our collaboration typifies the convergence that allows us to use machine learning effectively.
A tool for public engagement
Machine learning is also helping scientists engage the public in conservation. For example, forecasts produced by the BirdCast team are often used to inform Lights Out campaigns.
These initiatives seek to reduce artificial light from cities, which attracts migrating birds and increases their chances of colliding with human-built structures, such as buildings and communication towers. Lights Out campaigns can mobilize people to help protect birds at the flip of a switch.
As another example, the Merlin bird identification app seeks to create technology that makes birding easier for everyone. In 2021, the Merlin staff released a feature that automates song and call identification, allowing users to identify what they’re hearing in real time, like an ornithological version of Shazam.
This feature has opened the door for millions of people to engage with their natural spaces in a new way. Machine learning is a big part of what made it possible.
“Sound ID is our biggest success in terms of replicating the magical experience of going birding with a skilled naturalist,” Grant Van Horn, a staff researcher at the Cornell Lab of Ornithology who helped develop the algorithm behind this feature, told me.
Taking flight
Opportunities for applying machine learning in ornithology will only increase. As billions of birds migrate over North America to their breeding grounds this spring, people will engage with these flights in new ways, thanks to projects like BirdCast and Merlin. But that engagement is reciprocal: The data that birders collect will open new opportunities for applying machine learning.
Computers can’t do this work themselves. “Any successful machine learning project has a huge human component to it. That is the reason these projects are succeeding,” Van Horn said to me.
Shortly after rumors leaked of former President Donald Trump’s impending indictment, images purporting to show his arrest appeared online. These images looked like news photos, but they were fake. They were created by a generative artificial intelligence system.
Generative AI, in the form of image generators like DALL-E, Midjourney and Stable Diffusion, and text generators like Bard, ChatGPT, Chinchilla and LLaMA, has exploded in the public sphere. By combining clever machine-learning algorithms with billions of pieces of human-generated content, these systems can do anything from create an eerily realistic image from a caption, synthesize a speech in President Joe Biden’s voice, replace one person’s likeness with another in a video, or write a coherent 800-word op-ed from a title prompt.
Even in these early days, generative AI is capable of creating highly realistic content. My colleague Sophie Nightingale and I found that the average person is unable to reliably distinguish an image of a real person from an AI-generated person. Although audio and video have not yet fully passed through the uncanny valley – images or models of people that are unsettling because they are close to but not quite realistic – they are likely to soon. When this happens, and it is all but guaranteed to, it will become increasingly easier to distort reality.
In this new world, it will be a snap to generate a video of a CEO saying her company’s profits are down 20%, which could lead to billions in market-share loss, or to generate a video of a world leader threatening military action, which could trigger a geopolitical crisis, or to insert the likeness of anyone into a sexually explicit video.
The technology to make fake videos of real people is becoming increasingly available.
Advances in generative AI will soon mean that fake but visually convincing content will proliferate online, leading to an even messier information ecosystem. A secondary consequence is that detractors will be able to easily dismiss as fake actual video evidence of everything from police violence and human rights violations to a world leader burning top-secret documents.
As society stares down the barrel of what is almost certainly just the beginning of these advances in generative AI, there are reasonable and technologically feasible interventions that can be used to help mitigate these abuses. As a computer scientist who specializes in image forensics, I believe that a key method is watermarking.
Watermarks
There is a long history of marking documents and other items to prove their authenticity, indicate ownership and counter counterfeiting. Today, Getty Images, a massive image archive, adds a visible watermark to all digital images in their catalog. This allows customers to freely browse images while protecting Getty’s assets.
Imperceptible digital watermarks are also used for digital rights management. A watermark can be added to a digital image by, for example, tweaking every 10th image pixel so that its color (typically a number in the range 0 to 255) is even-valued. Because this pixel tweaking is so minor, the watermark is imperceptible. And, because this periodic pattern is unlikely to occur naturally, and can easily be verified, it can be used to verify an image’s provenance.
Even medium-resolution images contain millions of pixels, which means that additional information can be embedded into the watermark, including a unique identifier that encodes the generating software and a unique user ID. This same type of imperceptible watermark can be applied to audio and video.
The ideal watermark is one that is imperceptible and also resilient to simple manipulations like cropping, resizing, color adjustment and converting digital formats. Although the pixel color watermark example is not resilient because the color values can be changed, many watermarking strategies have been proposed that are robust – though not impervious – to attempts to remove them.
Watermarking and AI
These watermarks can be baked into the generative AI systems by watermarking all the training data, after which the generated content will contain the same watermark. This baked-in watermark is attractive because it means that generative AI tools can be open-sourced – as the image generator Stable Diffusion is – without concerns that a watermarking process could be removed from the image generator’s software. Stable Diffusion has a watermarking function, but because it’s open source, anyone can simply remove that part of the code.
OpenAI is experimenting with a system to watermark ChatGPT’s creations. Characters in a paragraph cannot, of course, be tweaked like a pixel value, so text watermarking takes on a different form.
Text-based generative AI is based on producing the next most-reasonable word in a sentence. For example, starting with the sentence fragment “an AI system can…,” ChatGPT will predict that the next word should be “learn,” “predict” or “understand.” Associated with each of these words is a probability corresponding to the likelihood of each word appearing next in the sentence. ChatGPT learned these probabilities from the large body of text it was trained on.
Generated text can be watermarked by secretly tagging a subset of words and then biasing the selection of a word to be a synonymous tagged word. For example, the tagged word “comprehend” can be used instead of “understand.” By periodically biasing word selection in this way, a body of text is watermarked based on a particular distribution of tagged words. This approach won’t work for short tweets but is generally effective with text of 800 or more words depending on the specific watermark details.
Generative AI systems can, and I believe should, watermark all their content, allowing for easier downstream identification and, if necessary, intervention. If the industry won’t do this voluntarily, lawmakers could pass regulation to enforce this rule. Unscrupulous people will, of course, not comply with these standards. But, if the major online gatekeepers – Apple and Google app stores, Amazon, Google, Microsoft cloud services and GitHub – enforce these rules by banning noncompliant software, the harm will be significantly reduced.
Signing authentic content
Tackling the problem from the other end, a similar approach could be adopted to authenticate original audiovisual recordings at the point of capture. A specialized camera app could cryptographically sign the recorded content as it’s recorded. There is no way to tamper with this signature without leaving evidence of the attempt. The signature is then stored on a centralized list of trusted signatures.
Although not applicable to text, audiovisual content can then be verified as human-generated. The Coalition for Content Provenance and Authentication (C2PA), a collaborative effort to create a standard for authenticating media, recently released an open specification to support this approach. With major institutions including Adobe, Microsoft, Intel, BBC and many others joining this effort, the C2PA is well positioned to produce effective and widely deployed authentication technology.
The combined signing and watermarking of human-generated and AI-generated content will not prevent all forms of abuse, but it will provide some measure of protection. Any safeguards will have to be continually adapted and refined as adversaries find novel ways to weaponize the latest technologies.
In the same way that society has been fighting a decadeslong battle against other cyber threats like spam, malware and phishing, we should prepare ourselves for an equally protracted battle to defend against various forms of abuse perpetrated using generative AI.
If you want to track changes in the Amazon rainforest, see the full expanse of a hurricane or figure out where people need help after a disaster, it’s much easier to do with the view from a satellite orbiting a few hundred miles above Earth.
Over 8,000 satellites are orbiting Earth today, capturing images like this, of the Louisiana coast. NASA Earth Observatory
Traditionally, access to satellite data has been limited to researchers and professionals with expertise in remote sensing and image processing. However, the increasing availability of open-access data from government satellites such as Landsat and Sentinel, and free cloud-computing resources such as Amazon Web Services, Google Earth Engine and Microsoft Planetary Computer, have made it possible for just about anyone to gain insight into environmental changes underway.
I work with geospatial big data as a professor. Here’s a quick tour of where you can find satellite images, plus some free, fairly simple tools that anyone can use to create time-lapse animations from satellite images.
For example, state and urban planners – or people considering a new home – can watch over time how rivers have moved, construction crept into wildland areas or a coastline eroded.
Landsat time-lapse animations show the river dynamics in Pucallpa, Peru. Qiusheng Wu, NASA LandsatA Landsat time-lapse shows the shoreline retreat in the Parc Natural del Delta, Spain. Qiusheng Wu, NASA Landsat
Environmental groups can monitor deforestation, the effects of climate change on ecosystems, and how other human activities like irrigation are shrinking bodies of water like Central Asia’s Aral Sea. And disaster managers, aid groups, scientists and anyone interested can monitor natural disasters such as volcanic eruptions and wildfires.
GOES images show the decline of the crucial Colorado River reservoir Lake Mead since the 1980s and the growth of neighboring Las Vegas. Qiusheng Wu, NOAA GOESA GOES satellite time-lapse shows the Hunga Tonga volcanic eruption on Jan. 15, 2022. Qiusheng Wu, NOAA GOES
Some transmit and receive radio signals for communications. Others provide global positioning system (GPS) services for navigation. The ones we’re interested in are Earth observation satellites, which collect images of the Earth, day and night.
Landsat: The longest-running Earth satellite mission, Landsat, has been collecting imagery of the Earth since 1972. The latest satellite in the series, Landsat 9, was launched by NASA in September 2021.
In general, Landsat satellite data has a spatial resolution of about 100 feet (about 30 meters). If you think of pixels on a zoomed-in photo, each pixel would be 100 feet by 100 feet. Landsat has a temporal resolution of 16 days, meaning the same location on Earth is imaged approximately once every 16 days. With both Landsat 8 and 9 in orbit, we can get a global coverage of the Earth once every eight days. That makes comparisons easier.
Landsat satellite images showing a side-by-side comparison of southern Pakistan in August 2021 (one year before the floods) and August 2022 (right) Qiusheng Wu, NASA Landsat
Sentinel:Sentinel Earth observation satellites were launched by the European Space Agency (ESA) as part of the Copernicus program. Sentinel-2 satellites have been collecting optical imagery of the Earth since 2015 at a spatial resolution of 10 meters (33 feet) and a temporal resolution of 10 days.
GOES: The images you’ll see most often in U.S. weather forecasting come from NOAA’s Geostationary Operational Environmental Satellites, or GOES. They orbit above the equator at the same speed Earth rotates, so they can provide continuous monitoring of Earth’s atmosphere and surface, giving detailed information on weather, climate, and other environmental conditions. GOES-16 and GOES-17 can image the Earth at a spatial resolution of about 1.2 miles (2 kilometers) and a temporal resolution of five to 10 minutes.
A GOES satellite shows an atmospheric river arriving on the West Coast in 2021. Qiusheng Wu, GOES
How to create your own visualizations
In the past, creating a Landsat time-lapse animation of a specific area required extensive data processing skills and several hours or even days of work. However, nowadays, free and user-friendly programs are available to enable anyone to create animations with just a few clicks in an internet browser.
For instance, I created an interactive web app for my students that anyone can use to generate time-lapse animations quickly. The user zooms in on the map to find an area of interest, then draws a rectangle around the area to save it as a GeoJSON file – a file that contains the geographic coordinates of the chosen region. Then the user uploads the GeoJSON file to the web app, chooses the satellite to view from and the dates and submits it. It takes the app about 60 seconds to then produce a time-lapse animation.
How to create satellite time-lapse animations.
There are several other useful tools for easily creating satellite animations. Others to try include Snazzy-EE-TS-GIF, an Earth Engine App for creating Landsat animations, and Planetary Computer Explorer, an explorer for searching and visualizing satellite imagery interactively.
It doesn’t take much to get ChatGPT to make a factual mistake. My son is doing a report on U.S. presidents, so I figured I’d help him out by looking up a few biographies. I tried asking for a list of books about Abraham Lincoln and it did a pretty good job:
A reasonable list of books about Lincoln. Screen capture by Jonathan May., CC BY-ND
Number 4 isn’t right. Garry Wills famously wrote “Lincoln at Gettysburg,” and Lincoln himself wrote the Emancipation Proclamation, of course, but it’s not a bad start. Then I tried something harder, asking instead about the much more obscure William Henry Harrison, and it gamely provided a list, nearly all of which was wrong.
Books about Harrison, fewer than half of which are correct. Screen capture by Jonathan May., CC BY-ND
Numbers 4 and 5 are correct; the rest don’t exist or are not authored by those people. I repeated the exact same exercise and got slightly different results:
More books about Harrison, still mostly nonexistent. Screen capture by Jonathan May., CC BY-ND
This time numbers 2 and 3 are correct and the other three are not actual books or not written by those authors. Number 4, “William Henry Harrison: His Life and Times” is a real book, but it’s by James A. Green, not by Robert Remini, a well-known historian of the Jacksonian age.
I called out the error and ChatGPT eagerly corrected itself and then confidently told me the book was in fact written by Gail Collins (who wrote a different Harrison biography), and then went on to say more about the book and about her. I finally revealed the truth and the machine was happy to run with my correction. Then I lied absurdly, saying during their first hundred days presidents have to write a biography of some former president, and ChatGPT called me out on it. I then lied subtly, incorrectly attributing authorship of the Harrison biography to historian and writer Paul C. Nagel, and it bought my lie.
When I asked ChatGPT if it was sure I was not lying, it claimed that it’s just an “AI language model” and doesn’t have the ability to verify accuracy. However it modified that claim by saying “I can only provide information based on the training data I have been provided, and it appears that the book ‘William Henry Harrison: His Life and Times’ was written by Paul C. Nagel and published in 1977.”
This is not true.
Words, not facts
It may seem from this interaction that ChatGPT was given a library of facts, including incorrect claims about authors and books. After all, ChatGPT’s maker, OpenAI, claims it trained the chatbot on “vast amounts of data from the internet written by humans.”
However, it was almost certainly not given the names of a bunch of made-up books about one of the most mediocre presidents. In a way, though, this false information is indeed based on its training data.
As a computer scientist, I often field complaints that reveal a common misconception about large language models like ChatGPT and its older brethren GPT3 and GPT2: that they are some kind of “super Googles,” or digital versions of a reference librarian, looking up answers to questions from some infinitely large library of facts, or smooshing together pastiches of stories and characters. They don’t do any of that – at least, they were not explicitly designed to.
Sounds good
A language model like ChatGPT, which is more formally known as a “generative pretrained transformer” (that’s what the G, P and T stand for), takes in the current conversation, forms a probability for all of the words in its vocabulary given that conversation, and then chooses one of them as the likely next word. Then it does that again, and again, and again, until it stops.
So it doesn’t have facts, per se. It just knows what word should come next. Put another way, ChatGPT doesn’t try to write sentences that are true. But it does try to write sentences that are plausible.
When talking privately to colleagues about ChatGPT, they often point out how many factually untrue statements it produces and dismiss it. To me, the idea that ChatGPT is a flawed data retrieval system is beside the point. People have been using Google for the past two and a half decades, after all. There’s a pretty good fact-finding service out there already.
In fact, the only way I was able to verify whether all those presidential book titles were accurate was by Googling and then verifying the results. My life would not be that much better if I got those facts in conversation, instead of the way I have been getting them for almost half of my life, by retrieving documents and then doing a critical analysis to see if I can trust the contents.
Improv partner
On the other hand, if I can talk to a bot that will give me plausible responses to things I say, it would be useful in situations where factual accuracy isn’t all that important. A few years ago a student and I tried to create an “improv bot,” one that would respond to whatever you said with a “yes, and” to keep the conversation going. We showed, in a paper, that our bot was better at “yes, and-ing” than other bots at the time, but in AI, two years is ancient history.
I tried out a dialogue with ChatGPT – a science fiction space explorer scenario – that is not unlike what you’d find in a typical improv class. ChatGPT is way better at “yes, and-ing” than what we did, but it didn’t really heighten the drama at all. I felt as if I was doing all the heavy lifting.
After a few tweaks I got it to be a little more involved, and at the end of the day I felt that it was a pretty good exercise for me, who hasn’t done much improv since I graduated from college over 20 years ago.
A space exploration improv scene the author generated with ChatGPT. Screen capture by Jonathan May., CC BY-ND
Sure, I wouldn’t want ChatGPT to appear on “Whose Line Is It Anyway?” and this is not a great “Star Trek” plot (though it’s still less problematic than “Code of Honor”), but how many times have you sat down to write something from scratch and found yourself terrified by the empty page in front of you? Starting with a bad first draft can break through writer’s block and get the creative juices flowing, and ChatGPT and large language models like it seem like the right tools to aid in these exercises.
And for a machine that is designed to produce strings of words that sound as good as possible in response to the words you give it – and not to provide you with information – that seems like the right use for the tool.
These successes could be taken to indicate that computation has no limits. To see if that’s the case, it’s important to understand what makes a computer powerful.
There are two aspects to a computer’s power: the number of operations its hardware can execute per second and the efficiency of the algorithms it runs. The hardware speed is limited by the laws of physics. Algorithms – basically sets of instructions – are written by humans and translated into a sequence of operations that computer hardware can execute. Even if a computer’s speed could reach the physical limit, computational hurdles remain due to the limits of algorithms.
These hurdles include problems that are impossible for computers to solve and problems that are theoretically solvable but in practice are beyond the capabilities of even the most powerful versions of today’s computers imaginable. Mathematicians and computer scientists attempt to determine whether a problem is solvable by trying them out on an imaginary machine.
An imaginary computing machine
The modern notion of an algorithm, known as a Turing machine, was formulated in 1936 by British mathematician Alan Turing. It’s an imaginary device that imitates how arithmetic calculations are carried out with a pencil on paper. The Turing machine is the template all computers today are based on.
To accommodate computations that would need more paper if done manually, the supply of imaginary paper in a Turing machine is assumed to be unlimited. This is equivalent to an imaginary limitless ribbon, or “tape,” of squares, each of which is either blank or contains one symbol.
The machine is controlled by a finite set of rules and starts on an initial sequence of symbols on the tape. The operations the machine can carry out are moving to a neighboring square, erasing a symbol and writing a symbol on a blank square. The machine computes by carrying out a sequence of these operations. When the machine finishes, or “halts,” the symbols remaining on the tape are the output or result.
What is a Turing machine?
Computing is often about decisions with yes or no answers. By analogy, a medical test (type of problem) checks if a patient’s specimen (an instance of the problem) has a certain disease indicator (yes or no answer). The instance, represented in a Turing machine in digital form, is the initial sequence of symbols.
A problem is considered “solvable” if a Turing machine can be designed that halts for every instance whether positive or negative and correctly determines which answer the instance yields.
Not every problem can be solved
Many problems are solvable using a Turing machine and therefore can be solved on a computer, while many others are not. For example, the domino problem, a variation of the tiling problem formulated by Chinese American mathematician Hao Wang in 1961, is not solvable.
The task is to use a set of dominoes to cover an entire grid and, following the rules of most dominoes games, matching the number of pips on the ends of abutting dominoes. It turns out that there is no algorithm that can start with a set of dominoes and determine whether or not the set will completely cover the grid.
Keeping it reasonable
A number of solvable problems can be solved by algorithms that halt in a reasonable amount of time. These “polynomial-time algorithms” are efficient algorithms, meaning it’s practical to use computers to solve instances of them.
Thousands of other solvable problems are not known to have polynomial-time algorithms, despite ongoing intensive efforts to find such algorithms. These include the Traveling Salesman Problem.
The Traveling Salesman Problem asks whether a set of points with some points directly connected, called a graph, has a path that starts from any point and goes through every other point exactly once, and comes back to the original point. Imagine that a salesman wants to find a route that passes all households in a neighborhood exactly once and returns to the starting point.
The Traveling Salesman Problem quickly gets out of hand when you get beyond a few destinations.
These problems, called NP-complete, were independently formulated and shown to exist in the early 1970s by two computer scientists, American Canadian Stephen Cook and Ukrainian American Leonid Levin. Cook, whose work came first, was awarded the 1982 Turing Award, the highest in computer science, for this work.
The cost of knowing exactly
The best-known algorithms for NP-complete problems are essentially searching for a solution from all possible answers. The Traveling Salesman Problem on a graph of a few hundred points would take years to run on a supercomputer. Such algorithms are inefficient, meaning there are no mathematical shortcuts.
Practical algorithms that address these problems in the real world can only offer approximations, though the approximations are improving. Whether there are efficient polynomial-time algorithms that can solve NP-complete problems is among the seven millennium open problems posted by the Clay Mathematics Institute at the turn of the 21st century, each carrying a prize of US$1 million.
Beyond Turing
Could there be a new form of computation beyond Turing’s framework? In 1982, American physicist Richard Feynman, a Nobel laureate, put forward the idea of computation based on quantum mechanics.
What is a quantum computer?
In 1995, Peter Shor, an American applied mathematician, presented a quantum algorithm to factor integers in polynomial time. Mathematicians believe that this is unsolvable by polynomial-time algorithms in Turing’s framework. Factoring an integer means finding a smaller integer greater than 1 that can divide the integer. For example, the integer 688,826,081 is divisible by a smaller integer 25,253, because 688,826,081 = 25,253 x 27,277.
A major algorithm called the RSA algorithm, widely used in securing network communications, is based on the computational difficulty of factoring large integers. Shor’s result suggests that quantum computing, should it become a reality, will change the landscape of cybersecurity.
Can a full-fledged quantum computer be built to factor integers and solve other problems? Some scientists believe it can be. Several groups of scientists around the world are working to build one, and some have already built small-scale quantum computers.
Nevertheless, like all novel technologies invented before, issues with quantum computation are almost certain to arise that would impose new limits.
Ada Lovelace, known as the first computer programmer, was born on Dec. 10, 1815, more than a century before digital electronic computers were developed.
Lovelace has been hailed as a model for girls in science, technology, engineering and math (STEM). A dozen biographies for young audiences were published for the 200th anniversary of her birth in 2015. And in 2018, The New York Times added hers as one of the first “missing obituaries” of women at the rise of the #MeToo movement.
But Lovelace – properly Ada King, Countess of Lovelace after her marriage – drew on many different fields for her innovative work, including languages, music and needlecraft, in addition to mathematical logic. Recognizing that her well-rounded education enabled her to accomplish work that was well ahead of her time, she can be a model for all students, not just girls.
Lovelace was the daughter of the scandal-ridden romantic poet George Gordon Byron, aka Lord Byron, and his highly educated and strictly religious wife Anne Isabella Noel Byron, known as Lady Byron. Lovelace’s parents separated shortly after her birth. At a time when women were not allowed to own property and had few legal rights, her mother managed to secure custody of her daughter.
Growing up in a privileged aristocratic family, Lovelace was educated by home tutors, as was common for girls like her. She received lessons in French and Italian, music and in suitable handicrafts such as embroidery. Less common for a girl in her time, she also studied math. Lovelace continued to work with math tutors into her adult life, and she eventually corresponded with mathematician and logician Augustus De Morgan at London University about symbolic logic.
Lovelace drew on all of these lessons when she wrote her computer program – in reality, it was a set of instructions for a mechanical calculator that had been built only in parts.
The computer in question was the Analytical Engine designed by mathematician, philosopher and inventor Charles Babbage. Lovelace had met Babbage when she was introduced to London society. The two related to each other over their shared love for mathematics and fascination for mechanical calculation. By the early 1840s, Babbage had won and lost government funding for a mathematical calculator, fallen out with the skilled craftsman building the precision parts for his machine, and was close to giving up on his project. At this point, Lovelace stepped in as an advocate.
To make Babbage’s calculator known to a British audience, Lovelace proposed to translate into English an article that described the Analytical Engine. The article was written in French by the Italian mathematician Luigi Menabrea and published in a Swiss journal. Scholars believe that Babbage encouraged her to add notes of her own.
Ada Lovelace envisioned in the early 19th century the possibilities of computing.
In her notes, which ended up twice as long as the original article, Lovelace drew on different areas of her education. Lovelace began by describing how to code instructions onto cards with punched holes, like those used for the Jacquard weaving loom, a device patented in 1804 that used punch cards to automate weaving patterns in fabric.
Having learned embroidery herself, Lovelace was familiar with the repetitive patterns used for handicrafts. Similarly repetitive steps were needed for mathematical calculations. To avoid duplicating cards for repetitive steps, Lovelace used loops, nested loops and conditional testing in her program instructions.
The notes included instructions on how to calculate Bernoulli numbers, which Lovelace knew from her training to be important in the study of mathematics. Her program showed that the Analytical Engine was capable of performing original calculations that had not yet been performed manually. At the same time, Lovelace noted that the machine could only follow instructions and not “originate anything.”
Ada Lovelace created this chart for the individual program steps to calculate Bernoulli numbers. Courtesy of Linda Hall Library of Science, Engineering & Technology, CC BY-ND
Finally, Lovelace recognized that the numbers manipulated by the Analytical Engine could be seen as other types of symbols, such as musical notes. An accomplished singer and pianist, Lovelace was familiar with musical notation symbols representing aspects of musical performance such as pitch and duration, and she had manipulated logical symbols in her correspondence with De Morgan. It was not a large step for her to realize that the Analytical Engine could process symbols — not just crunch numbers — and even compose music.
A well-rounded thinker
Inventing computer programming was not the first time Lovelace brought her knowledge from different areas to bear on a new subject. For example, as a young girl, she was fascinated with flying machines. Bringing together biology, mechanics and poetry, she asked her mother for anatomical books to study the function of bird wings. She built and experimented with wings, and in her letters, she metaphorically expressed her longing for her mother in the language of flying.
Despite her talents in logic and math, Lovelace didn’t pursue a scientific career. She was independently wealthy and never earned money from her scientific pursuits. This was common, however, at a time when freedom – including financial independence – was equated with the capability to impartially conduct scientific experiments. In addition, Lovelace devoted just over a year to her only publication, the translation of and notes on Menabrea’s paper about the Analytical Engine. Otherwise, in her life cut short by cancer at age 37, she vacillated between math, music, her mother’s demands, care for her own three children, and eventually a passion for gambling. Lovelace thus may not be an obvious model as a female scientist for girls today.
However, I find Lovelace’s way of drawing on her well-rounded education to solve difficult problems inspirational. True, she lived in an age before scientific specialization. Even Babbage was a polymath who worked in mathematical calculation and mechanical innovation. He also published a treatise on industrial manufacturing and another on religious questions of creationism.
But Lovelace applied knowledge from what we today think of as disparate fields in the sciences, arts and the humanities. A well-rounded thinker, she created solutions that were well ahead of her time.
We all have to make hard decisions from time to time. The hardest of my life was whether or not to change research fields after my PhD, from fundamental physics to climate physics. I had job offers that could have taken me in either direction – one to join Stephen Hawking’s Relativity and Gravitation Group at Cambridge University, another to join the Met Office as a scientific civil servant.
I wrote down the pros and cons of both options as one is supposed to do, but then couldn’t make up my mind at all. Like Buridan’s donkey, I was unable to move to either the bale of hay or the pail of water. It was a classic case of paralysis by analysis.
Since it was doing my head in, I decided to try to forget about the problem for a couple of weeks and get on with my life. In that intervening time, my unconscious brain decided for me. I simply walked into my office one day and the answer had somehow become obvious: I would make the change to studying the weather and climate.
More than four decades on, I’d make the same decision again. My fulfilling career has included developing a new, probabilistic way of forecasting weather and climate which is helping humanitarian and disaster relief agencies make better decisions ahead of extreme weather events. (This and many other aspects are described in my new book, The Primacy of Doubt.)
But I remain fascinated by what was going on in my head back then, which led my subconscious to make a life-changing decision that my conscious could not. Is there something to be understood here not only about how to make difficult decisions, but about how humans make the leaps of imagination that characterise us as such a creative species? I believe the answer to both questions lies in a better understanding of the extraordinary power of noise.
Imprecise supercomputers
I went from the pencil-and-paper mathematics of Einstein’s theory of general relativity to running complex climate models on some of the world’s biggest supercomputers. Yet big as they were, they were never big enough – the real climate system is, after all, very complex.
In the early days of my research, one only had to wait a couple of years and top-of-the-range supercomputers would get twice as powerful. This was the era where transistors were getting smaller and smaller, allowing more to be crammed on to each microchip. The consequent doubling of computer performance for the same power every couple of years was known as Moore’s Law.
This story is part of Conversation Insights
The Insights team generates long-form journalism and is working with academics from different backgrounds who have been engaged in projects to tackle societal and scientific challenges.
There is, however, only so much miniaturisation you can do before the transistor starts becoming unreliable in its key role as an on-off switch. Today, with transistors starting to approach atomic size, we have pretty much reached the limit of Moore’s Law. To achieve more number-crunching capability, computer manufacturers must bolt together more and more computing cabinets, each one crammed full of chips.
But there’s a problem. Increasing number-crunching capability this way requires a lot more electric power – modern supercomputers the size of tennis courts consume tens of megawatts. I find it something of an embarrassment that we need so much energy to try to accurately predict the effects of climate change.
That’s why I became interested in how to construct a more accurate climate model without consuming more energy. And at the heart of this is an idea that sounds counterintuitive: by adding random numbers, or “noise”, to a climate model, we can actually make it more accurate in predicting the weather.
A constructive role for noise
Noise is usually seen as a nuisance – something to be minimised wherever possible. In telecommunications, we speak about trying to maximise the “signal-to-noise ratio” by boosting the signal or reducing the background noise as much as possible. However, in nonlinear systems, noise can be your friend and actually contribute to boosting a signal. (A nonlinear system is one whose output does not vary in direct proportion to the input. You will likely be very happy to win £100 million on the lottery, but probably not twice as happy to win £200 million.)
Noise can, for example, help us find the maximum value of a complicated curve such as in Figure 1, below. There are many situations in the physical, biological and social sciences as well as in engineering where we might need to find such a maximum. In my field of meteorology, the process of finding the best initial conditions for a global weather forecast involves identifying the maximum point of a very complicated meteorological function.
Figure 1
A curve with multiple local peaks and troughs. Author provided
However, employing a “deterministic algorithm” to locate the global maximum doesn’t usually work. This type of algorithm will typically get stuck at a local peak (for example at point a) because the curve moves downwards in both directions from there.
An answer is to use a technique called “simulated annealing” – so called because of its similarities with (annealing), the heat treatment process that changes the properties of metals. Simulated annealing, which employs noise to get round the issue of getting stuck at local peaks, has been used to solve many problems including the classic travelling salesman puzzle of finding the shortest path between a large number of cities on a map.
Figure 1 shows a possible route to locating the curve’s global maximum (point 9) by using the following criteria:
If a randomly chosen point is higher than the current position on the curve, then the new point is always moved to.
If it is lower than the current position, the suggested point isn’t necessarily rejected. It depends whether the new point is a lot lower or just a little lower.
However, the decision to move to a new point also depends on how long the analysis has been running. Whereas in the early stages, random points quite a bit lower than the current position may be accepted, in later stages only those that are higher or just a tiny bit lower are accepted.
The technique is known as simulated annealing because early on – like hot metal in the early phase of cooling – the system is pliable and changeable. Later in the process – like cold metal in the late phase of cooling – it is almost rigid and unchangeable.
How noise can help climate models
Noise was introduced into comprehensive weather and climate models around 20 years ago. A key reason was to represent model uncertainty in our ensemble weather forecasts – but it turned out that adding noise also reduced some of the biases the models had, making them more accurate simulators of weather and climate.
Unfortunately, these models require huge supercomputers and a lot of energy to run them. They divide the world into small gridboxes, with the atmosphere and ocean within each assumed to be constant – which, of course, it isn’t. The horizontal scale of a typical gridbox is around 100km – so one way of making a model more accurate is to reduce this distance to 50km, or 10km or 1km. However, halving the volume of a gridbox increases the computational cost of running the model by up to a factor of 16, meaning it consumes a lot more energy.
Here again, noise offered an appealing alternative. The proposal was to use it to represent the unpredictable (and unmodellable) variations in small-scale climatic processes like turbulence, cloud systems, ocean eddies and so on. I argued that adding noise could be a way of boosting accuracy without having to incur the enormous computational cost of reducing the size of the gridboxes. For example, as has now been verified, adding noise to a climate model increases the likelihood of producing extreme hurricanes – reflecting the potential reality of a world whose weather is growing more extreme due to climate change.
The computer hardware we use for this modelling is inherently noisy – electrons travelling along wires in a computer move in partly random ways due to its warm environment. Such randomness is called “thermal noise”. Could we save even more energy by tapping into it, rather than having to use software to generate pseudo-random numbers? To me, low-energy “imprecise” supercomputers that are inherently noisy looked like a win-win proposal.
But not all of my colleagues were convinced. They were uncomfortable that computers might not give the same answers from one day to the next. To try to persuade them, I began to think about other real-world systems that, because of limited energy availability, also use noise that is generated within their hardware. And I stumbled on the human brain.
Noise in the brain
Every second of the waking day, our eyes alone send gigabytes of data to the brain. That’s not much different to the amount of data a climate model produces each time it outputs data to memory.
The brain has to process this data and somehow make sense of it. If it did this using the power of a supercomputer, that would be impressive enough. But it does it using one millionth of that power, about 20W instead of 20MW – what it takes to power a lightbulb. Such energy efficiency is mind-bogglingly impressive. How on Earth does the brain do it?
An adult brain contains some 80 billion neurons. Each neuron has a long slender biological cable – the axon – along which electrical impulses are transmitted from one set of neurons to the next. But these impulses, which collectively describe information in the brain, have to be boosted by protein “transistors” positioned at regular intervals along the axons. Without them, the signal would dissipate and be lost.
The energy for these boosts ultimately comes from an organic compound in the blood called ATP (adenosine triphosphate). This enables electrically charged atoms of sodium and potassium (ions) to be pushed through small channels in the neuron walls, creating electrical voltages which, much like those in silicon transistors, amplify the neuronal electric signals as they travel along the axons.
With 20W of power spread across tens of billions of neurons, the voltages involved are tiny, as are the axon cables. And there is evidence that axons with a diameter less than about 1 micron (which most in the brain are) are susceptible to noise. In other words, the brain is a noisy system.
If this noise simply created unhelpful “brain fog”, one might wonder why we evolved to have so many slender axons in our heads. Indeed, there are benefits to having fatter axons: the signals propagate along them faster. If we still needed fast reaction times to escape predators, then slender axons would be disadvantageous. However, developing communal ways of defending ourselves against enemies may have reduced the need for fast reaction times, leading to an evolutionary trend towards thinner axons.
Perhaps, serendipitously, evolutionary mutations that further increased neuron numbers and reduced axon sizes, keeping overall energy consumption the same, made the brain’s neurons more susceptible to noise. And there is mounting evidence that this had another remarkable effect: it encouraged in humans the ability to solve problems that required leaps in imagination and creativity.
Perhaps we only truly became Homo Sapiens when significant noise began to appear in our brains?
Putting noise in the brain to good use
Many animals have developed creative approaches to solving problems, but there is nothing to compare with a Shakespeare, a Bach or an Einstein in the animal world.
How do creative geniuses come up with their ideas? Here’s a quote from Andrew Wiles, perhaps the most famous mathematician alive today, about the time leading up to his celebrated proof of the maths problem (misleadingly) known as Fermat’s Last Theorem:
When you reach a real impasse, then routine mathematical thinking is of no use to you. Leading up to that kind of new idea, there has to be a long period of tremendous focus on the problem without any distraction. You have to really think about nothing but that problem – just concentrate on it. And then you stop. [At this point] there seems to be a period of relaxation during which the subconscious appears to take over – and it’s during this time that some new insight comes.
BBC’s Horizon unpicks Andrew Wiles’s novel approach to solving Fermat’s Theorem.
This notion seems universal. Physics Nobel Laureate Roger Penrose has spoken about his “Eureka moment” when crossing a busy street with a colleague (perhaps reflecting on their conversation while also looking out for oncoming traffic). For the father of chaos theory Henri Poincaré, it was catching a bus.
And it’s not just creativity in mathematics and physics. Comedian John Cleese, of Monty Python fame, makes much the same point about artistic creativity – it occurs not when you are focusing hard on your trade, but when you relax and let your unconscious mind wander.
Of course, not all the ideas that bubble up from your subconscious are going to be Eureka moments. Physicist Michael Berry talks about these subconscious ideas as if they are elementary particles called “claritons”:
Actually, I do have a contribution to particle physics … the elementary particle of sudden understanding: the “clariton”. Any scientist will recognise the “aha!” moment when this particle is created. But there is a problem: all too frequently, today’s clariton is annihilated by tomorrow’s “anticlariton”. So many of our scribblings disappear beneath a rubble of anticlaritons.
Here is something we can all relate to: that in the cold light of day, most of our “brilliant” subconscious ideas get annihilated by logical thinking. Only a very, very, very small number of claritons remain after this process. But the ones that do are likely to be gems.
In his renowned book Thinking Fast and Slow, the Nobel prize-winning psychologist Daniel Kahneman describes the brain in a binary way. Most of the time when walking, chatting and looking around (in other words when multitasking), it operates in a mode Kahneman calls “system 1” – a rather fast, automatic, effortless mode of operation.
By contrast, when we are thinking hard about a specific problem (unitasking), the brain is in the slower, more deliberative and logical “system 2”. To perform a calculation like 37×13, we have to stop walking, stop talking, close our eyes and even put our hands over our ears. No chance for significant multitasking in system 2.
My 2015 paper with computational neuroscientist Michael O’Shea interpreted system 1 as a mode where available energy is spread across a large number of active neurons, and system 2 as where energy is focused on a smaller number of active neurons. The amount of energy per active neuron is therefore much smaller when in the system 1 mode, and it would seem plausible that the brain is more susceptible to noise when in this state. That is, in situations when we are multitasking, the operation of any one of the neurons will be most susceptible to the effects of noise in the brain.
Berry’s picture of clariton-anticlariton interaction seems to suggest a model of the brain where the noisy system 1 and the deterministic system 2 act in synergy. The anticlariton is the logical analysis that we perform in system 2 which, most of the time, leads us to reject our crazy system 1 ideas.
But sometimes one of these ideas turns out to be not so crazy.
This is reminiscent of how our simulated annealing analysis (Figure 1) works. Initially, we might find many “crazy” ideas appealing. But as we get closer to locating the optimal solution, the criteria for accepting a new suggestion becomes more stringent and discerning. Now, system 2 anticlaritons are annihilating almost everything the system 1 claritons can throw at them – but not quite everything, as Wiles found to his great relief.
The key to creativity
If the key to creativity is the synergy between noisy and deterministic thinking, what are some consequences of this?
On the one hand, if you do not have the necessary background information then your analytic powers will be depleted. That’s why Wiles says that leading up to the moment of insight, you have to immerse yourself in your subject. You aren’t going to have brilliant ideas which will revolutionise quantum physics unless you have a pretty good grasp of quantum physics in the first place.
But you also need to leave yourself enough time each day to do nothing much at all, to relax and let your mind wander. I tell my research students that if they want to be successful in their careers, they shouldn’t spend every waking hour in front of their laptop or desktop. And swapping it for social media probably doesn’t help either, since you still aren’t really multitasking – each moment you are on social media, your attention is still fixed on a specific issue.
But going for a walk or bike ride or painting a shed probably does help. Personally, I find that driving a car is a useful activity for coming up with new ideas and thoughts – provided you don’t turn the radio on.
When making difficult decisions, this suggests that, having listed all the pros and cons, it can be helpful not to actively think about the problem for a while. I think this explains how, years ago, I finally made the decision to change my research direction – not that I knew it at the time.
Because the brain’s system 1 is so energy efficient, we use it to make the vast majority of the many decisions in our daily lives (some say as many as 35,000) – most of which aren’t that important, like whether to continue putting one leg in front of the other as we walk down to the shops. (I could alternatively stop after each step, survey my surroundings to make sure a predator was not going to jump out and attack me, and on that basis decide whether to take the next step.)
However, this system 1 thinking can sometimes lead us to make bad decisions, because we have simply defaulted to this low-energy mode and not engaged system 2 when we should have. How many times do we say to ourselves in hindsight: “Why didn’t I give such and such a decision more thought?”
Of course, if instead we engaged system 2 for every decision we had to make, then we wouldn’t have enough time or energy to do all the other important things we have to do in our daily lives (so the shops may have shut by the time we reach them).
From this point of view, we should not view giving wrong answers to unimportant questions as evidence of irrationality. Kahneman cites the fact that more than 50% of students at MIT, Harvard and Princeton gave the incorrect answer to this simple question – a bat and ball costs $1.10; the bat costs one dollar more than the ball; how much does the ball cost? – as evidence of our irrationality. The correct answer, if you think about it, is 5 cents. But system 1 screams out ten cents.
If we were asked this question on pain of death, one would hope we would spend enough thought to come up with the correct answer. But if we were asked the question as part of an anonymous after-class test, when we had much more important things to spend time and energy doing, then I’d be inclined to think of it as irrational to give the right answer.
If we had 20MW to run the brain, we could spend part of it solving unimportant problems. But we only have 20W and we need to use it carefully. Perhaps it’s the 50% of MIT, Harvard and Princeton students who gave the wrong answer who are really the clever ones.
Just as a climate model with noise can produce types of weather that a model without noise can’t, so a brain with noise can produce ideas that a brain without noise can’t. And just as these types of weather can be exceptional hurricanes, so the idea could end up winning you a Nobel Prize.
So, if you want to increase your chances of achieving something extraordinary, I’d recommend going for that walk in the countryside, looking up at the clouds, listening to the birds cheeping, and thinking about what you might eat for dinner.
So could computers be creative?
Will computers, one day, be as creative as Shakespeare, Bach or Einstein? Will they understand the world around us as we do? Stephen Hawking famously warned that AI will eventually take over and replace mankind.
However, the best-known advocate of the idea that computers will never understand as we do is Hawking’s old colleague, Roger Penrose. In making his claim, Penrose invokes an important “meta” theorem in mathematics known as Gödel’s theorem, which says there are mathematical truths that can’t be proven by deterministic algorithms.
There is a simple way of illustrating Gödel’s theorem. Suppose we make a list of all the most important mathematical theorems that have been proven since the time of the ancient Greeks. First on the list would be Euclid’s proof that there are an infinite number of prime numbers, which requires one really creative step (multiply the supposedly finite number of primes together and add one). Mathematicians would call this a “trick” – shorthand for a clever and succinct mathematical construction.
But is this trick useful for proving important theorems further down the list, like Pythagoras’s proof that the square root of two cannot be expressed as the ratio of two whole numbers? It’s clearly not; we need another trick for that theorem. Indeed, as you go down the list, you’ll find that a new trick is typically needed to prove each new theorem. It seems there is no end to the number of tricks that mathematicians will need to prove their theorems. Simply loading a given set of tricks on a computer won’t necessarily make the computer creative.
Does this mean mathematicians can breathe easily, knowing their jobs are not going to be taken over by computers? Well maybe not.
I have been arguing that we need computers to be noisy rather than entirely deterministic, “bit-reproducible” machines. And noise, especially if it comes from quantum mechanical processes, would break the assumptions of Gödel’s theorem: a noisy computer is not an algorithmic machine in the usual sense of the word.
Does this imply that a noisy computer can be creative? Alan Turing, pioneer of the general-purpose computing machine, believed this was possible, suggesting that “if a machine is expected to be infallible then it cannot also be intelligent”. That is to say, if we want the machine to be intelligent then it had better be capable of making mistakes.
Others may argue there is no evidence that simply adding noise will make an otherwise stupid machine into an intelligent one – and I agree, as it stands. Adding noise to a climate model doesn’t automatically make it an intelligent climate model.
However, the type of synergistic interplay between noise and determinism – the kind that sorts the wheat from the chaff of random ideas – has hardly yet been developed in computer codes. Perhaps we could develop a new type of AI model where the AI is trained by getting it to solve simple mathematical theorems using the clariton-anticlariton model; by making guesses and seeing if any of these have value.
For this to be at all tractable, the AI system would need to be trained to focus on “educated random guesses”. (If the machine’s guesses are all uneducated ones, it will take forever to make progress – like waiting for a group of monkeys to type the first few lines of Hamlet.)
For example, in the context of Euclid’s proof that there are an unlimited number of primes, could we train an AI system in such a way that a random idea like “multiply the assumed finite number of primes together and add one” becomes much more likely than the completely useless random idea “add the assumed finite number of primes together and subtract six”? And if a particular guess turns out to be especially helpful, can we train the AI system so that the next guess is a refinement of the last one?
If we can somehow find a way to do this, it could open up modelling to a completely new level that is relevant to all fields of study. And in so doing, we might yet reach the so-called “singularity” when machines take over from humans. But only when AI developers fully embrace the constructive role of noise – as it seems the brain did many thousands of years ago.
For now, I feel the need for another walk in the countryside. To blow away some fusty old cobwebs – and perhaps sow the seeds for some exciting new ones.
The Research Brief is a short take about interesting academic work.
The big idea
A new type of material can learn and improve its ability to deal with unexpected forces thanks to a unique lattice structure with connections of variable stiffness, as described in a new paper by my colleagues and me.
Architected materials – like this 3D lattice – get their properties not from what they are made out of, but from their structure. Ryan Lee, CC BY-ND
The new material is a type of architected material, which gets its properties mainly from the geometry and specific traits of its design rather than what it is made out of. Take hook-and-loop fabric closures like Velcro, for example. It doesn’t matter whether it is made from cotton, plastic or any other substance. As long as one side is a fabric with stiff hooks and the other side has fluffy loops, the material will have the sticky properties of Velcro.
My colleagues and I based our new material’s architecture on that of an artificial neural network – layers of interconnected nodes that can learn to do tasks by changing how much importance, or weight, they place on each connection. We hypothesized that a mechanical lattice with physical nodes could be trained to take on certain mechanical properties by adjusting each connection’s rigidity.
To find out if a mechanical lattice would be able to adopt and maintain new properties – like taking on a new shape or changing directional strength – we started off by building a computer model. We then selected a desired shape for the material as well as input forces and had a computer algorithm tune the tensions of the connections so that the input forces would produce the desired shape. We did this training on 200 different lattice structures and found that a triangular lattice was best at achieving all of the shapes we tested.
Once the many connections are tuned to achieve a set of tasks, the material will continue to react in the desired way. The training is – in a sense – remembered in the structure of the material itself.
We then built a physical prototype lattice with adjustable electromechanical springs arranged in a triangular lattice. The prototype is made of 6-inch connections and is about 2 feet long by 1½ feet wide. And it worked. When the lattice and algorithm worked together, the material was able to learn and change shape in particular ways when subjected to different forces. We call this new material a mechanical neural network.
The prototype is 2D, but a 3D version of this material could have many uses. Jonathan Hopkins, CC BY-ND
Why it matters
Besides some living tissues, very few materials can learn to be better at dealing with unanticipated loads. Imagine a plane wing that suddenly catches a gust of wind and is forced in an unanticipated direction. The wing can’t change its design to be stronger in that direction.
The prototype lattice material we designed can adapt to changing or unknown conditions. In a wing, for example, these changes could be the accumulation of internal damage, changes in how the wing is attached to a craft or fluctuating external loads. Every time a wing made out of a mechanical neural network experienced one of these scenarios, it could strengthen and soften its connections to maintain desired attributes like directional strength. Over time, through successive adjustments made by the algorithm, the wing adopts and maintains new properties, adding each behavior to the rest as a sort of muscle memory.
This type of material could have far reaching applications for the longevity and efficiency of built structures. Not only could a wing made of a mechanical neural network material be stronger, it could also be trained to morph into shapes that maximize fuel efficiency in response to changing conditions around it.
This connection of springs is a new type of material that can change shape and learn new properties. Jonathan Hopkins, CC BY-ND
What’s still not known
So far, our team has worked only with 2D lattices. But using computer modeling, we predict that 3D lattices would have a much larger capacity for learning and adaptation. This increase is due to the fact that a 3D structure could have tens of times more connections, or springs, that don’t intersect with one another. However, the mechanisms we used in our first model are far too complex to support in a large 3D structure.
What’s next
The material my colleagues and I created is a proof of concept and shows the potential of mechanical neural networks. But to bring this idea into the real world will require figuring out how to make the individual pieces smaller and with precise properties of flex and tension.
Sandhill cranes flying above the Platte River in Nebraska.
Over 8,000 satellites are orbiting Earth today, capturing images like this, of the Louisiana coast.
Ada King, Countess of Lovelace, was more than just another mathematician.
This connection of springs is a new type of material that can change shape and learn new properties.
