SciComm Corner – How Virtual and Augmented Reality Could Aid Science Communication

Article written by Ingrid Fadelli

How Virtual and Augmented Reality Could Aid Science Communication

Virtual Reality (VR) and Augmented Reality (AR) are immersive technologies that allow users to experience media content in more engaging ways, as if it was playing out in their immediate surroundings. This is achieved through the use of smart headsets, glasses, and other wearable devices with LCD or OLED screens integrated in them, such as Oculus Rift, HTC Vive, Playstation VR, and Microsoft HoloLens.

VR technology allows users to visually experience an entirely new reality using headsets and other smart devices, while AR adds digital or virtual elements to one’s actual surroundings, by projecting images, graphics, and other media elements in specific areas. In other words, if VR completely replaces a user’s visual experience, AR ‘adds’ to it.

In recent years, VR and AR tools opened up new exciting possibilities for the entertainment and gaming industry, enabling the production of increasingly realistic and interactive media content. Although they have so far been primarily used for amusement or professional training purposes, these tools can have a wide range of interesting applications. 

Now that the COVID-19 pandemic is forcing countless people to work and study from home, for instance, VR and AR platforms could allow users to acquire new knowledge remotely in more engaging ways. They might be particularly useful for communicating scientific topics, which are often far easier to understand when they are explained through visual representations.

Learning science in a different dimension 

By allowing users to experience media in an immersive way, VR technology could be the perfect tool to communicate complex scientific discoveries and observations to a non-expert audience.

Instead of reading an article about a medical intervention and its effects on the body, for instance, one could watch a simulation of how it unfolds while wearing a VR headset. Similarly, VR headsets could allow users to immerse themselves in engaging content about how climate change is impacting the planet, as well as videos showing the unique behaviours of different animal species or explaining biological processes uncovered by researchers.

A few years ago, NASA uploaded a series of 360° videos that can be viewed using VR gear on YouTube, which are designed to take users on short tours in space and at NASA’s facilities. Many other research institutes and universities worldwide also created science-related content that can be viewed using VR headsets.

A further example of how VR was previously used to create immersive science-related experiences is ChimeraX, a molecular visualisation program compatible with Oculus Rift and HTC Vive developed by researchers at the University of California-San Francisco, which can be downloaded online for free. This unique program allows users to go on virtual tours of biological structures, for instance closely visualising proteins and observing how they are affected by specific pharmaceutical drugs.

In addition to allowing science communicators and researchers to approach scientific topics in more stimulating ways, VR technology could be used by planetariums, museums and other scientific institutions to offer immersive virtual tours or experiences while strict social distancing measures are in place.

Interactive scientific experiences

While VR technology can help to create immersive science-related media content for both expert and non-expert audiences, AR could be used to produce new interactive learning experiences.

AR tools are typically less advanced than VR gear in terms of hardware and software, but they can also be more user-friend and less invasive, as they allow users to remain in touch with their surrounding environment, merely adding virtual elements to it. As some people experience motion sickness when wearing VR headsets, AR tools can be a simpler and more approachable alternative. Many AR applications do not even require headsets or wearable devices, as they can be downloaded and accessed on smart phones or tablets.

AR technology is already being used to assist and train professionals in several real-world settings, including hospitals. For instance, before they perform medical procedures, some surgeons at Children’s Mercy in Kansas City are using HoloLens mixed reality smart glasses to view detailed CT scans of their patients’ hearts.

In addition, some museums are using AR to guide students and other visitors during tours, helping them to gain a better understanding of science-related topics. For instance, the National Museum of Singapore introduced an immersive and captivating installation called Story of the Forest, featuring 69 drawings from the William Farquhar Collection of Natural History Drawings presented as 3D animations, which visitors can interact with and learn from. This unique exhibition is designed to teach visitors about the flora and fauna in the Malay peninsula throughout the 19^th century.

Alternatively, some museums, including the Smithsonian Institution in Washington, created AR smartphone applications that enhance their visitors’ experience of different displays or collections. The Smithsonian created Skin and Bone, an app that allows users to look at the bones exhibited in the museum’s ‘Bone Hall’ in an interactive way, while also learning more about different animals, ranging from bats to giant sea cows, rattlesnakes, and other species.

As AR is generally more accessible than VR, it could be better suited for disseminating science-related information to large audiences in more engaging ways. A company called DEVAR, for instance, developed a range of toys and story books with AR elements, which allow children to learn basic facts about animals, coding, engineering, and other scientific topics in interactive ways. The DEVAR app can be downloaded and accessed on most smart devices, which means that children can simply use their parents’ smartphone or other devices to view AR elements of their books or toys.

Similarly, an assistant professor at USC’s Mark and Mary Stevens Neuroimaging and Informatics Institute, Tyler Ard, developed a smartphone application that uses AR to make scientific documents more engaging. This unique application, called Schol-AR, allows science communicators, educators and other interested parties to easily create dynamic elements for scientific articles, documents, or books, which can then be viewed by readers through the Schol-AR app.

Increasing the use of AR and VR

Despite its considerable potential for science communication, VR technology can be quite expensive, as high-quality headsets typically cost between £400 and £1000. Although there are also cheaper headsets (e.g., Google Cardboard) that allow users to replace the screens generally found inside VR headsets with their smartphones, these do not perform as well more expensive devices.

Due to their high price and limited use, so far VR tools have rarely been used for science communication. On the other hand, AR technology is far more accessible and user-friendly, which makes it easier to implement on a large scale.

With many science museums, schools, and universities worldwide still closed to limit the spread of COVID-19, this could be an ideal time to test the use of AR and VR technology for communicating scientific topics in more entertaining and captivating ways.