From the Forbidden City to the Dunhuang Caves to the Great Wall of China, Tencent has supported a number of major sites listed under the UNESCO World Heritage Convention to digitize and preserve priceless cultural assets. As a result, more visitors can appreciate these wonders of the world through virtual online tours. The added benefit of having high-definition, 3D images of historic relics allows exhibition visitors to study intricate patterns and designs more clearly than in real life.

More good news is on the way for cultural buffs. Leveraging Tencent Video’s interactive mixed reality (MR) technology, our MR tourism solution will make it more affordable for museums, heritage landmarks and natural attractions to create their own compelling MR experiences. Put human interaction in the mix – whether real or digital – the new software development heralds a new era of opportunities in content creation and cultural tourism.

More for Less – Flexibly and Efficiently

Until recently, providing a high-quality virtual tourism experience meant excessively long hours on location to conduct proper image recording. It requires professional manpower to achieve satisfying results, leading to high cost. Tencent’s cloud-enabled MR cultural tourism solution significantly reduces the cost of producing immersive MR experiences compared to traditional on-site filming method.

Six key features of the new MR solution:

1. Video shooting from fixed camera position, mirrored virtually with real-time rendering for seamless imagery.

2. Portable QR-code green screen matting for realistic and large-scale reconstruction of scenery.

3. The solution recognizes movements and gestures, restores relationship between light and shadow, and supports natural interaction with virtual backgrounds and objects.

4. 360-degree panoramic backgrounds, then AI merges people with virtual scenes. 

5. AI matting technology creates scenes with real people and virtual backgrounds without the need for a green screen.

6. People from different locations can be rendered in virtual scenes and interact with each other.

Four Applications of Tencent’s MR Technology

Tencent's MR technology can revolutionize the production of traditional and virtual reality videos, as well as create immersive virtual live rooms. With the help of cutting-edge technologies such as 3D engines, cloud rendering, image-style tracking, and virtual-reality interaction, you can integrate people into virtual scenes seamlessly to enhance the viewing experience of videos and streamline their production. Our self-developed algorithm drastically reduces the deployment cost and complexity of live-broadcasting set-up, allowing both companies and individuals to achieve high-quality and cost-effective production.

1. Explore tourist attractions with your digital avatar

Whether it’s a country park or a historic landmark, visitors can choose to explore offline attractions at their own pace with a digital avatar acting as their tour guide. The motion-capture software will allow visitors to create their own tour guide. 

To activate the digital human guide, visitors simply scan a QR code at key locations around the attractions. They can then interact with it on a digital screen just as they would with a human tour guide, asking questions, seeking information, and obtaining directions. 

2. Travel anytime through livestreaming

Scenic attractions and heritage sites will be recorded around the clock and broadcast live in an uninterrupted stream via the cloud. A virtual tour guide will provide commentary at any time to online visitors. This feature can be used as a creative marketing tactic to promote the attraction as well as encourage people to visit the site in person.

3. Immersive experience with cultural sites and relics

Tourists can integrate themselves into the virtual world and interact with cultural relics and virtual humans using a camera or smartphone. By applying MR technology, AI and advanced background matting solutions, anything from a large-scale scene to an intricate artwork can be recreated without green screens. Visitors can get close to and examine the cultural relics in detail, giving new life to artifacts that are precious or fragile and are otherwise at risk of damage.

4. Put on a show starring a combination of real and digital humans 

Overcome spatial and resource limitations and host a launch event or celebration with digital avatars sharing the stage with a physical one. Tencent’s MR technology lets you create various backgrounds, whether it’s a television studio, conference hall, or outdoor stadium. Real-life presenters can then take to the stage and perform with their virtual avatar counterparts. 

A World of Possibilities 

Such simple yet professional technologies open a world of marketing possibilities for smaller museums and tourist venues. Whether it’s offering previews before visit or integrated virtual and real navigation, artwork exhibitions, and restoration of cultural relics and historic sites, the virtual world is increasingly fused with real life in innovative ways.