A version of this article was published in TIME’s newsletter Into the Metaverse. Subscribe for a weekly guide to the future of the Internet. You can find past issues of the newsletter here.
There are two versions of a BMW factory in the medieval town of Regensburg, Germany. One is a physical plant that cranks out hundreds of thousands of cars a year. The other is a virtual 3-D replica, accessed by screen or VR headset, in which every surface and every bit of machinery looks exactly the same as in real life. Soon, whatever is happening in the physical factory will be reflected inside the virtual one in real time: frames being dipped in paint; doors being sealed onto hinges; avatars of workers carrying machinery to its next destination.
[time-brightcove not-tgx=”true”]
The latter factory is an example of a “digital twin”: an exact digital re-creation of an object or environment. The concept might at first seem like sci-fi babble or even a frivolous experiment: Why would you spend time and resources to create a digital version of something that already exists in the real world?
But digital twins are now proving invaluable across multiple industries, especially those that involve costly or scarce physical objects. Created by feeding video, images, blueprints or other data into advanced 3-D mapping software, digital twins are being used in medicine to replicate and study internal organs. They’ve propelled engineers to devise car and plane prototypes—including Air Force fighter jets—more quickly. They allow architects and urban planners to envision and then build skyscrapers and city blocks with clarity and precision.
And this year, digital twins began to break into the mainstream of manufacturing and research. In April, chipmaker Nvidia launched a version of its Omniverse 3-D simulation engine that allows businesses to build 3-D renderings of their own—including digital twins. Amazon Web Services announced a competing service, the IoT TwinMaker, in November. The digital-twin market already generated sales of more than $3 billion in 2020, according to the research firm Research and Markets, and tech executives leading digital-twin efforts say we’re still at the dawn of this technology.
Digital twins could have huge implications for training workers, for formulating complicated technical plans without having to waste physical resources—even for improving infrastructure and combatting climate change. “Health care, music, education, taking city kids on safari: it’s hard to imagine where digital twins won’t have an impact,” says Richard Kerris, Nvidia’s vice president of Omniverse development.
The need was always there. In the 1960s, NASA created physical replicas of spaceships and connected them to simulators so that if a crisis ensued on the actual vehicle hundreds of thousands of miles away, a team could workshop solutions on the ground. Dave Rhodes, the senior vice president of digital twins at Unity Technologies, a video-game and 3-D-platform company, says that digital-twin technology is only now being widely released because of several confluent factors, including the increased computing power of cloud-based systems, the spread of 5G networks, improvements in 3-D rendering and the remote work demands of COVID-19.
Digital twins can replicate real-world objects ranging in size from millimeters to miles. In Poland, a team of doctors and technologists is starting with one of the smallest objects imaginable: the human fetal heart. About 1 in 100 newborns has a congenital heart disease, which can be fatal if not treated. But studies have shown that more than half of those diseases go undetected. Sonography simulators are expensive and bulky, and most medical schools don’t include hands-on training. “Most likely you will encounter a congenital heart defect for the first time when you are already in your clinic,” Marcin Wiechec, an ob-gyn and associate professor at Jagiellonian University in Krakow, says.
So Wiechec and his team created Fetal Heart VR, which allows doctors to guide a probe across a belly-like dome in order to study normal and abnormal beating fetal hearts—re-created identically from real-life scans—through a VR headset. Wiechec has been using the app, created with Unity, to train students in Krakow. Jill Beithon, a retired sonographer and educator based in Fergus Falls, Minn., says the app could have enormous benefits for medical workers in areas with less access to resources or cutting-edge education centers. “The fetal heart is very intimidating—and the size of a quarter at 20 weeks, beating at 100 beats per minute. It takes additional training, and it’s not training you can easily find,” she says. “With the VR, you don’t have to go to expensive courses or try to find a mentor. This is going to replace hands-on experience.”
Digital-twin technology is being trialed across the medical landscape, for planning surgical procedures and exploring the heart risks of various drugs. In November, seven medical researchers from around the U.S., writing in the journal Nature Medicine, called for an increase in clinical studies of “cancer patient digital twins” to precisely track a patient’s physical state and adjust treatment accordingly. “[Digital twins] are poised to revolutionize how cancer and a host of other complex diseases are treated and managed,” the researchers wrote.
The automotive industry is also being transformed. Back in Regensburg, BMW can now test or tweak parts of the assembly line without having to move around heavy machinery; the company estimates the technology will cut the time it takes to plan out factory operations by at least 25%. A few months ago, factory managers created their first piece of new equipment inside the digital twin: a machine that puts door seals on a car frame. “Under the old-fashioned way, we would have had to draw it and build cardboard simulations, which is very time consuming. With COVID-19, we were very restricted in getting people on site,” says Frank Bachmann, the plant manager. “So with the digital twin we were able to work virtually, test it and have variations of the plan for more or less no money.” The machine was installed over the holidays—but before the break, Bachmann was able to show employees how to navigate their new workstations via the digital twin.
In Pittsburgh, Ding Zhao, an assistant professor of mechanical engineering at Carnegie Mellon, has been working with carmakers to use digital twins to improve the safety of self-driving vehicles. In his lab, he leverages vast quantities of data collected from real tests of self-driving cars to build complex digital-twin simulators. The simulations, he says, help predict how a car’s AI will react in dicey situations that could be dangerous and difficult to re-create IRL: when merging onto a dark snowy highway, for instance, or when jammed in between two trucks.
Crucially, digital twins also allow researchers to run crash-test simulations countless times without having to destroy cars or endanger real people. That means digital-twin technology is becoming essential to the development of self-driving cars. “Real-world testing is too expensive and sometimes not even effective,” Zhao says. Digital twins are also being used in other complex and potentially dangerous machines, from nuclear reactors in Idaho to wind turbines in Paris.
Others are deploying the technology at an even larger scale, to create digital twins of entire cities or even countries. This year, the Orlando Economic Partnership, a nonprofit community-development organization, announced it was partnering with Unity to build a digital twin of 40 sq. mi. of the Florida city. CEO Tim Giuliani hopes that the twin will eventually be used as a public resource and “backbone infrastructure,” allowing transportation experts to see how a rail system might impact the region, for utility companies to map out 5G networks and for ecologists to study the potential impacts of climate change. He estimates the project will cost $1 million to $2 million.
Of course, creating digital replicas at increasingly large scale raises questions about privacy and cybersecurity. Many of these digital twins are made possible by a multitude of sensors that track real-world data and movement. Workers at factories with digital twins may find their every movement followed; the hacker of a digital twin could gain frighteningly precise knowledge about a complex proprietary system. Zhao, at Carnegie Mellon, stresses the need for regulations. “Legislators and companies need to work together on this. You cannot just say, ‘I am a good company, I will never do evil things, just give me your data,’ ” he says.
Before the regulations arrive, technology companies are rolling full steam ahead. Many of them believe digital twins will gain importance with the rise of the metaverse, a collection of connected virtual worlds that increasingly impact—or even replace—what happens in the real world. Digital twin humans are coming too: the NFL and Amazon Web Services have created a “digital athlete” that will run infinite scenarios to better understand and treat football injuries.
BMW plans to take its digital-twin factory model to the world. The company is in the process of building a new plant in Hungary that is modeled completely in Nvidia’s Omniverse. But BMW could soon implement digital twins at all facilities. Bachmann, the plant director in Regensburg, says that the advantages of digital twins will only be fully realized when every factory is digitized in a standard way. “We need these processes of digital twins everywhere,” he says.
Subscribe to Into the Metaverse for a weekly guide to the future of the Internet.