AI & Immersive Technologies

Augmented, Virtual, Mixed and Extended Realities

Background

Virtual Reality (VR) was first imagined in science fiction, and emerged in real life via an immersive film-viewing cabinet created in the 1950s. It wasn’t until 2010 that the Oculus Rift VR headset was first shown and by 2014 the company was purchased by Facebook for $2 billion – valued largely for its consumer gaming applications. Since then other major manufacturers have developed VR technology, including Sony and HTC and several hundred companies have been launched to develop content, applications and provide systems and services.

If you’re among the millions of those who’ve used their smartphones to chase Pokémon in the real world, then you already know how amazing augmented reality (AR) can be, although you may not be aware that it has moved beyond gaming and entertainment to wider applications such as advertising, retail, training – and more.

This technology continues to grow rapidly, and it could soon become bigger than VR. The reason for this is simple. Unlike VR, which requires expensive equipment, AR can be deployed on smartphones or tablets, which makes it more accessible and far cheaper. Plus, AR is more realistic. While VR puts the user into a completely different, immersive environment, AR uses our existing environment and enriches it with virtual objects.

Ultimately, immersive technologies are on the rise with new forms emerging as part of online and mobile games, cinematic experiences, exhibitions and events, and news media, which all aim to engage audiences and place them at the centre of the action or experience.

Definitions

There is a whole spectrum of immersive technologies out there. Before we progress further, let’s pause and ensure we’re all on the same page regarding what these terms actually mean.

  • Virtual reality (VR) immerses users in a fully artificial digital environment.
  • Augmented reality (AR) overlays virtual objects on the real-world environment.
  • Mixed reality (MR) not just overlays but anchors virtual objects to the real world i.e. blend the physical and digital worlds. This is sometimes referred to as XR which stands for Cross Reality or Extended Reality which HoloLens inventor Alex Kipman describes as – “the world of atoms and the world of bits”.

These immersive technologies are not about one application; rather they represent the evolution of personal and mobile computing disrupting and revolutionising the way humans interact with machines and one another.

Perhaps the most exciting potential for immersive technology is the impact on learning and experiential learning.

Few would argue that it’s much easier to learn something by “doing it yourself” (DIY) rather than by watching, reading, or being told what to do. Imagine a future where you can immerse yourself in various scenarios so that the experiences and actions within it feel naturally like your own – within a hybrid physical and digital world.

Immersive Technology Improves Efficiency

The UK Government is so convinced immersive technologies (AR, VR, MR) are the future that they have set aside significant investment as part of its Industrial Strategy Challenge Fund. The fund aims to enable business and researchers to work together in creating the technologies which will entertain and educate the audiences of the future.

It is expected that by pushing immersive technologies into the public consciousness, it will spark a wider uptake of the technology and shape the perception and behaviour towards virtual and augmented reality.

report commissioned by Immerse UK concluded that immersive technologies have the potential to help drive the UK economy. In the wake of Brexit, UK companies will look to AR/VR to overcome the productivity gap.

Advantages of VR

Virtual Reality environments have been used in learning simulators for years. They were not exactly originally conceived as games and their usage was more oriented to the adult public. Its evolution and the appearance of new immersive technologies, such as the Samsung Gear VR or Oculus Rift (vincular a páginas?) and other low cost solutions make VR more accessible to the public in general. It also makes possible the development of a wide variety of applications based on VR and with many different purposes, from leisure to educational, and for all ages.

Advantages of AR

If VR seems to be a powerful development tool for training or teaching, what happens if the information comes to you in a real world environment? The possibilities of AR applications are incredible because the knowledge can be offered in the space where the learning, training or coaching needs occur.

The real environment is augmented with information which can help the user learn or train for specific skills. In this case, AR is supporting the real world rather than replacing it with virtual environments. Its everyday usage is more often in games, tourism, leisure, training and education.

COVID19 and Immersive Technologies

In the pre-pandemic era, companies from across different industries including entertainment, healthcare, gaming, and manufacturing, etc. were already harnessing AR/VR technologies to reach customers and fuel growth. Tech giants including Facebook, Google, Samsung, HTC, etc. have been investing billions of dollars into developing AR/VR products and services for the past few years. 

However, the #covid19pandemic has given a significant impetus to the development and adoption of new and emerging technologies such as robotics, augmented reality (AR), virtual reality (VR), and mixed reality (MR).

As a result of government enforced lockdown across countries throughout the world, physical channels and in-person meetings have come to a grinding halt forcing all industries, namely, education, businesses, healthcare and entertainment to transition to digital on-line experiences. Many businesses have sadly been destroyed and our high streets and towns have emptied significantly.

AR/VR technologies aren’t just hype. They have proven real-world use cases.

Remote working

As more and more businesses are adapting to the new normal of work, i.e. remote working, the demand for AR and VR solutions is also surging to stay connected and boost productivity.

Facebook announced the availability of Oculus for Business, an enterprise solution for streamlining and expanding virtual reality in the workplace to help organisations meet the early demand for VR-powered training and collaboration to keep up with the tech-driven future of work.

Platforms and devices like Workplace, Portal, and Oculus were built for a time when the economic opportunity might no longer depend on geography, a time when what you do could matter more than where you are. That time starts now“, says Facebook CEO Mark Zuckerberg.

Product visualisation

In times of social distancing, online shopping is becoming more and more prevalent; supplemented by home delivery and courier services.

AR/VR technologies are helping businesses interact with their customers and allowing them to virtually see their products in great detail as in reality. The use of visualisation technology in online shopping reduces the time and physical effort of visiting “bricks and mortar” stores and improves customer satisfaction and subsequently consumer-brand interaction.

Education / Online learning

Around the world many educational institutions (schools, colleges, universities) have been forced to close temporarily. They have all transitioned to virtual learning to curb the spread of COVID-19 whilst preserving the educational experience for its students – some better than others. While this in itself deserves praise, societal inequalities have surfaced highlighting economic and class differences between schools, families and communities. 

Healthcare

Healthcare is one of the most promising sectors for the growth of AR/VR. These immersive technologies have allowed researchers and health experts to better analyse viruses or diseases like COVID-19.

Combined with Artificial Intelligence (AI), AR/VR technologies have aided in the COVID-19 drug discovery efforts by enabling scientists to gain insights into the molecular mechanics of the novel coronavirus in virtual reality.

This has also extended to telehealth services to coronavirus patients which has included remote medical care enabled delivered via VR headsets and VR therapy.

Virtual tourism

Travel and tourism businesses have arguably suffered the most due COVID-19 induced travel restrictions. According to United Nations’ WTO latest research, 83 percent of destinations in Europe have introduced complete closure of borders for international tourism. Amidst this challenge, Virtual Reality has a major role in making a real difference in the sector.

As the COVID 19 pandemic has limited travel and public gatherings, the Tower of David Museum has created a VR project that lets visitors immerse in Jerusalemʼs Old City through a transcendent stereoscopic 360 degrees Virtual Reality Documentary. The wide-ranging initiative, dubbed “The Holy City,” features immersive experiences of the Holy Fire ceremony at the Church of the Holy Sepulcher, the priestly blessing (“Birkat Kohanim”) at the Western Wall, and Ramadan prayers in Al-Aqsa Mosque compound.

Each one of us approached the different institutions in Jerusalem and [made clear] that they are not going to be misrepresented by anyone”, explained Nimrod Shanit, executive director of Holy City VR. “It was a really interesting way to work, kind of like an interfaith project”.

While VR may never replace in-person travels, it could offer time and cost-saving substitute to adapt to the COVID-19 induced ‘new normal’ within the travel and tourism industry and also help reduce the industry’s environmental and carbon footprint.

AR/VR post COVID-19

The rapid shift to remote work enabled by the internet and digital channels (such as Zoom and Teams) has also fuelled interest in immersive (and other emerging) technologies which will most likely continue even after the pandemic subsides.

No alt text provided for this image

We know that the road ahead is bumpy for all businesses, regardless of the industry. The bottom line is that attention-grabbing immersive technologies including AR/VR have huge potential to remotely enhance collaboration, educational and workplace productivity but given the economic downturn and the immature state of the technologies, it will take some more time before they go mainstream.

One thing is certain: AR/VR are no longer just about the technology; it’s about defining how we want to live in the real world with these new immersive technologies and how we will design experiences that are meaningful and can enrich humanity.

AI & Immersive Tech

Artificial Intelligence (AI) is the capability of a machine to imitate intelligent human behaviour. When coupled with AR/VR, AI has the capacity to take immersive experiences to another level.

For example: in a learning, training & skills development capacity, AI can replace numerous situations that occur randomly and learn from a students behaviour. As the student gets better, the system will present increasing difficult situations, personalising the education.

Implementation of AI for AR/VR is expected to offer more immersive technology which will be increasingly personalised. Based on previous behaviours within environments, AI has the potential to offer up relevant options to the user or predict required outcomes before they occur.

The trick with VR will be for AI to anticipate what the viewer wants to see and prepare as it streams out to their headset. Gamers are doing this now by starting to incorporate AI alongside canned player action responses. This way when a player does something out of the norm, a new reaction is “dreamed up” by the AI engine.

One of the more common implementations of AI and AR is managing and recognising real-world items in the context of an augmented world.

An example of this is the face filters used on apps like Snapchat. AI is used to determine the face and how it is orientated. The technology is also used to continuously track a person’s face and facial expressions.

Without the help of AI, AR would be unable to detect the proper orientation of a person’s face. Instead, the movement of your device would determine both the vertical and horizontal axis of your filter.

Perhaps one of the most exciting announcements this year is the debut of Microsoft’s Mesh technology a new collaborative mixed reality platform allowing users to experience a combination of AR and VR together; bringing people into your world, and vice versa.

Think of it a bit like the holographic messages we’ve seen in Star Wars and other science fiction stories.

Microsoft Technical Fellow, Alex Kipman, the man behind the HoloLens and Kinect, indicated that Mesh is powered by Azure and “all of its AI and compute capabilities, working seamlessly together whether companies are accessing resources in the cloud or at the edge of the network.

Likewise, Facebook has also been making headlines. The company is heavily investing in an augmented reality (AR) future. Today, Facebook Reality Labs (FRL) has given another teasing look into its vision, which revolves around a contextually-aware, AI-powered interface for its AR glasses.

This could work hand-in-hand with soft, wearable input systems like CTRL-Labs’ wristband – the company Facebook acquired in 2019 – where the AI would offer suggestions that you could say yes or no to. CEO Mark Zuckerberg said, “I think neural interfaces are going to be fundamentally intertwined with VR and AR work in terms of how the input works as well”.

Facebook also mentions that next week it’ll be unveiling more of its research to do with “wrist-based input combined with usable but limited contextualised AI”, followed later in the year by its work on all-day wearable devices and haptic gloves.

There are already a lot of companies trying to be the pathway to a global interconnected metaverse of VR and AR – powered by AI. Microsoft could be one of the first to make it all work, but how well Microsoft Mesh ends up playing with all of Facebook’s collaborative VR/AR tools, and eventually Google and Apple’s, is the big unknown.

In closing, a natural consequence of AI adoption is the increased automation of routine tasks to eradicate internal inefficiencies, increase operational efficiency and improve profitability.

Imagine for a moment .. the countless possibilities .. perhaps likened to a “sixth sense” beyond our “normal” human senses of :-

  1. Sight
  2. Hearing
  3. Touch
  4. Smell
  5. Taste

Nonetheless, AI and immersive technologies are all cool technologies that need to find practical uses. When combined, just like the invention of the internet and smartphones, it’s hard to predict where exactly this transformative new tech can take us. 

Concluding Thoughts

Today, we are at the beginning of a Fourth Industrial Revolution. Developments in genetics, artificial intelligence, robotics, nanotechnology, 3D printing and biotechnology, to name just a few, are all building on and amplifying one another. This will lay the foundation for a revolution more comprehensive and all-encompassing than anything we have ever seen.

Additionally, from enabling online learning to opening access to cultural events and experiences, applications of AR and VR can help us overcome the isolation of COVID-19 lockdowns.

Below are four key strategies to expand AR/VR initiatives across cultural institutions, schools and workplaces:

  1. Put a centralised governance model in place and build AR/VR awareness.
  2. Invest in upgrading talent to gear up for future adoption.
  3. Focus on the right use cases that provide lasting value and support employees.
  4. Prepare technology infrastructure to integrate AR/VR.

Seeing, hearing, and touching possible realities through the power of AR/VR can stir our collective willingness to welcome and activate positive change in a world that will be forever changed due to COVID19. Combined with AI, the possibilities appear to be endless.

Let’s make it our collective goal and commitment to design for the best of humanity.

2 thoughts on “AI & Immersive Technologies

  1. Pingback: AI & Immersive Tech (audio) – Salim's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s