With live virtual set software, you can film from seemingly anywhere and quickly jump between locations without leaving the room. In this blog, we cover the basics of virtual set software, including green screen virtual sets.
A virtual set, AKA a virtual studio set, is an environment in a studio that allows teams to manipulate a digital background. This allows filmmakers and broadcasters to alter things like the location, weather, and time of day using software. Teams can place on-set actors anywhere!
Unlike green screen virtual sets, LED walls can offer more accurate lighting and reflections while also allowing the entire cast and crew to see the environment around them. Although they cost more up front, teams can spend less time in post manipulating scenes.
There are certain disadvantages of green screens, such as not providing realistic reflections and lighting from the background and not allowing the actors to see the environment. But, there are certain areas where green screen virtual sets shine.
Whether you are using a green screen or an LED wall, real-time software is key to allowing creatives to view the environment while filming, instead of waiting until post-production adds in the backgrounds. When using real-time software, version control ensures you are always using the correct versions of the virtual sets while filming.
The process for creating a virtual set depends on your needs and the scale of your project. Whether you use a green screen or LED wall, though, the first step is using digital content creation (DCC) tools to build a digital environment. Much like creating a game asset, making assets for a virtual set involves using 3D modeling programs like Maya, Blender, and zBrush and art programs like Photoshop and Substance Painter to create props and set pieces for the virtual environment. You might then bring those together into a game engine like Unreal (more on that below) to arrange and finalize the scene.
Once your virtual environment is built, you will set up the virtual camera within the game engine to sync with your physical camera, using a camera tracking system. This will ensure the perspective of the camera is coordinated with that of the environment displayed, so that the scene looks realistic.
A commonly used game engine for virtual sets is Epic's Unreal Engine. It was used for Ford v Ferrari, Westworld, The Mandalorian and more to create next-level images. Virtual sets can be created in Unreal Engine during previs and adjusted in real-time over the course of production, allowing teams to tweak scenes and virtual production images, even while on set.
The 3D assets can also be reused in post-production for full CG scenes, or in marketing images to promote the final film. Being able to share virtual set files helps unite cross-functional teams and teams spread around the world.
Unity has been used by game development studios for years. Recently, they are breaking into the virtual production space with the purchase of Weta Digital. This game engine offers similar capabilities as Unreal Engine and is behind CG animation for the Jungle Book and Blade Runner 2045.
When Entertainment Technology Center (ETC) started working on their short film Ripple Effect, they wanted to build a virtual set and leverage an LED wall. They brought together ICVR, Halon Entertainment, and other studios to create the visuals in Unreal Engine.
Are you prepared to elevate your broadcasts with virtual studios that captivate and immerse your audience in enthralling stories? Harness the power to break creative and technical barriers for productions ranging from live events to episodic TV and news broadcasts. With our cutting-edge and user-friendly tools, you can enhance storytelling and engage viewers across all demographics, transforming your content into unforgettable experiences.
Go beyond the limitations of physical sets. One virtual set means infinite shows. Whether to design from scratch or download from online libraries such as Unreal Engine Marketplace, there are many options for finding the right environment for your broadcast.
At a conventional virtual studio production, the background renderer, foreground renderer and chroma keyer are separate layers and sometimes pieces of hardware that require individual adjusting when a change is made.
Brainstorm has been leading the way for virtual sets for more than 25 years. Since its foundation in 1993 and with the creation of eStudio, Brainstorm has been at the edge of innovation in 3D and virtual studio technology, even pioneered the application of camera tracking to 3D real-time virtual set. Brainstorm technology can combine with other render engines like Unreal Engine to provide state-of-the-art hyper realistic scenes, that also integrate elements which are essential in broadcast operation, such as 3D data-driven motion graphics, charts and many other.
With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location.[4] OSVP can be viewed as an application of extended reality. OSVP contrasts with virtual studio technology, in which a green screen backdrop surrounds the set, and the virtual surroundings are composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like lens distortion, depth of field effects, bokeh and lens flare. This makes it a far more natural experience that more closely approximates location shooting, making the film-making process faster and more intuitive than can be achieved on a virtual set.[citation needed]
The virtual set can transform into remote locations, bringing the location to the Fox Sports studio instead of taking the studio to the location. Fox Sports will integrate its own custom production workflows, allowing content to be instantaneously populated/changed within the virtual set .
In addition, the facility is equipped with Zero Density studio software, including the most advanced keying features possible on a virtual set. The keying will make use of a 50- x 47-ft. green-screen area, while maintaining the ability to shoot 360 degrees.
While LED monitors are increasingly showing up in news studios large and small, in many cases replacing the green screen studios of old, make no mistake that virtual sets are advancing and, in tandem with augmented reality graphics, are changing the way stories are told on air.
While the addition of a virtual background can place studio presenters in any location in the world, augmented reality (AR) graphics are rendered in front of the on-air talent and help liven up the set in ways physical set pieces cannot. When used in combination with each other, as many media companies are now doing, the on-air effect can be captivating. AR systems are used to place virtual 3D graphic elements in the physical world.
These elements are arranged three dimensionally and attached to a live video signal, then output as a combination of live video and graphics. The technology takes the idea of virtual graphics from virtual sets but places them in the front of the presenter or in the foreground of a studio. A big benefit is that these graphical elements can then be used in tandem with a physical set, adding an extra layer of depth to the studio, or extending the studio space, without the need for a chromakey or green screen wall. The technology can even be used for outdoor studio environments, such as during live sporting events.
Virtual sets used to be complex to set up. The technology is now mature enough that people know how to work with them and they understand the issues involved in setting up a virtual environment. You need to look at the lighting of the set, the camera tracking, and the composition of the shot.
"Chroma keying and lighting are key elements within virtual sets," said Thierry Gonzalo, Senior Product Manager of AR/VR products at Vizrt. "The biggest challenge is combining the real world, that has a green background and talent, with the virtual world."
Addressing ease of use, Vizrt has introduced XR Set Express, a fully-equipped, virtual studio-in-a-box. It includes pre-configured and pre-calibrated features enabling quick setup and instantaneous use.
For the Winter Games this year, BBC Sport constructed a stunning virtual set with Viz Engine 4 and Unreal Engine to create video content for all coverage coming from Beijing. Designed with seven different views, the actual studio space was only 84 sq. meters (about 904 sq. ft.). BBC Sport was able to create a seamless virtual studio environment with easy operator control to bring fans unprecedented access to the Winter Games and it's been incredibly popular as well. BBC Sport has also used the virtual studio for the Australian Open, Golf, and various other sports shows.
Virtual sets rely on camera movement data that must be carefully tracked with a real-time rendering and compositing system to create these highly realistic environments. Camera tracking is critical in order to convey an accurate on-air impression of a presenter walking around and interacting with a virtual world. The tracking data from the camera and the rendering of the graphics are composited and output as a final image. Keyed graphics can be included on top if necessary.
df19127ead