Say hello to your digital twin! Deep tech experts in Israel can capture an extremely realistic 3D image of you using multiple cameras and AI.
Then they bring your twin to life, either doing what they actually filmed you doing: playing the guitar, taking a golf swing, jumping out of a plane, or something you didn’t do.
Your digital twin can be programmed to replicate the activities of a library full of things that other people have done while you sit at home and enjoy your coffee.
Yumfounded in Tel Aviv, is one of the companies that is called dimensional video, creating a 3D image with infinite possibilities.
The Internet is taking another leap into the future with “immersive experiences.” That means we won’t just see the character in two dimensions. When we watch an artist’s song, for example, we will move the mouse on our device to swing from left to right, to zoom in or out, to be low or high and to make the experience unique and personal.
Surround video is the technology that makes this possible. Yoom has been working on it since 2016 and says the AI that powers their system is one of the best because it’s learned from thousands of shots and has been fine-tuned and refined over the years.
It’s an expensive and complicated process, so the main customers are currently sports stars, entertainers, game developers and retailers.
As the “real” and virtual worlds increasingly merge; this is what Metaverse and Web3 are all about, the demand for immersive and interactive content will grow exponentially. Consumers expect a 3D experience, rather something that is flat and 2D.
“We allow people to create without any physical boundaries,” said Bonka Lim, Yoom’s Chief Marketing Officer.
“We’re working with a Canadian artist called Riel. He had a creative vision of making a video about his childhood and the struggles he faced.
“But he didn’t have the resources to film with the team in Europe and elsewhere. So we captured him in the studio and created the whole world that he had in his mind.
“We made a video about the vision he had as a creator. Our technology ultimately lowers the barriers to content creation because once we have your 3D model, we can create any environment you want.”
Riel was shot using a set of eight cameras, and his movements were processed using Yoom’s highly sophisticated neural networks, a set of algorithms that mimic the workings of the human brain to refine rendering, texture and lighting.
The images they shot resulted in a digital twin of him on an apocalyptic city street and a vivid house for his single “End It” in scenes that would otherwise have been impossible to create on a budget.
“Before this I knew nothing about dimensional capture,” he said. “I did not know this existed, nor did I understand it at first, as it was explained to me three times.
“I’m really surprised that it’s a camera with an algorithm that creates a 3D version of you. It took me a while to wrap my head around that. It recreates you, which I think is sick.”
The Yoom team also enlisted Dustin Brown to mark his departure LA Kings: The ice hockey team in the USA after almost two decades.
His digital twin appeared on a huge billboard in front of the team’s arena, appearing to have broken a window and holding aloft the Stanley Cup, the National Hockey League championship trophy. It was the result of capturing a 3D image of him in the studio, then enhancing it and adding effects.
Paul Oakenfold, the godfather of electronic music and a legendary Brit record producer and: trans DJ:, is another popular Yoom volume capture. He appears in PerfectoVerse, a 72-minute pay-per-view concert film and immersive experience that combines his lyrics and music with computer-generated visual wizardry.
“He came into the studio and we captured him and created a whole experience where people can actually see him perform, they can be with him and interact with the content. If you click, there are sparks and other effects,” says Lim.
The filming process can take as little as 30 minutes of actual filming, although a lot of technical work happens after that.
Lim, who was at Warner Bros. for 12 years, joined Yoom because he believes the technology it has developed represents the future of content creation and production.
“The magic is that once we have your digital twin, we can then fit you into any kind of digital environment; we can put you in a game, and we can put you in an AR (augmented reality) experience.
“If you look at the history of Internet content, we started with text, then we moved to image, then we moved to video. But videos are a very passive experience. Your only interaction with the video is play, pause, fast forward, rewind, share. And that’s it.
“I believe the future is more interactive content, so your experience will be different from mine. And this is possible thanks to the technology we have developed. As you interact with it, you become more immersed in the content, which means you are more engaged.”
