Elevate Your Web App With Augmented Reality
AR could be the next big thing, here’s an example of how you can use it to improve user experience
2020 was an exceptional year. The pandemic has profoundly changed our habits and the way we approach technology. From work, education to leisure and shopping, every area of our lives has been affected. In this new digital world, there are plenty of opportunities for us as developers to engage with our users and to provide innovative and exciting ways to experience the web. One of these is Augmented Reality, a technology that has seen unprecedented growth in the last few years, with a market that is worth approximately $15.3 billion, and is forecast to reach $72.8 billion by 2024.
What is AR?
Well, if you have ever tried an Instagram or Snapchat filter, or if you are a bit of a nerd like me, and you’ve been running around your town trying to catch your favourite Pokemon, there’s a high chance you already know what AR is about. To put it in a few words:
Augmented Reality is a layer of digital objects and information, rendered over the real world
What’s the difference between Augmented Reality and Virtual Reality, I hear you ask? AR allows the user to be in control of the environment, which has been digitally augmented or enhanced in some way. VR, on the other hand, is a fully immersive experience, where the real-world is replaced with a digitally simulated one.
…and why should we care?
First of all, AR can simplify user’s life. An example would be IKEA Place, an app that lets you virtually place furniture directly in your room.
Multiple retailers are now implementing apps that let users try their product online using AR technology (try this, for example). You can see how powerful AR can be in this case. The user no longer has to guess how a specific product would look before buying it, and this builds trust and confidence, which translates to higher conversion rates.
Another important aspect of AR is that it immediately captures the user’s attention, making it more likely that he/she will spend more time interacting and engaging with your product.
AR for the web
As of February 2021, AR and VR are accessible from most modern browsers, thanks to technologies like Web Assembly and WebGL. These technologies are used to leverage some of the device’s features (Camera, GPS, Sensors) to render 3D objects and to create interactive experiences. At the moment, WebAR only offers a limited selection of the main features possible using native app AR, but it has the overwhelming benefit of being accessible by most devices, without having to buy expensive equipment. With an app, the size or data allowance for download and device type can discourage people to use AR, but WebAR makes it more accessible and doesn’t eat up people’s data through chunky downloads.
Note: At the time of this article, the new WebXR Device API specification is still in draft, although it has been partially implemented in Chrome 79. If you want to learn more about this technology, check out this MDN page, as well as these cool examples.
One of the most popular library that brings Augmented Reality to the web is AR.js (developed by Jerome Etienne). This library uses JSARToolkit for the image tracking, and can display 3D content with either Three.js or A-Frame.
It supports the following types of augmented reality:
- Marker-based AR, which uses patterns like QR codes or other custom designs to trigger an augmented experience. When a marker in the physical world is recognized by a device’s camera, the digital content is placed on top of it.
- Image tracking, similar to marker-based, it makes it possible to use 2D images to display content, instead of the classic marker.
- Location-based AR, ties digital content and the experience it creates to a specific place. The objects are mapped out so that when a user’s location matches the predetermined spot it is displayed on the screen.
Use case example
For the purpose of this article, I’ve created a simple project with AR.js and Vue JS, to showcase some AR capabilities in a real-world use case.
In Tui Musement, we provide our customers with thousands of experiences and excursions everywhere in the world. When users buy an experience on one of our platforms, they receive a ticket, which they may need to print to show it to the provider of that experience.
One possible way to improve our customer experience, and to add a “wow factor” to our products, would be to use Augmented Reality to show some dynamic information to our customer, directly on their ticket. The possibilities are endless, but to keep things simple I’ve chosen to show some simple hard-coded information to tell our customer how to reach the meeting point. Obviously in a real-world we would have pulled this data from an API, but this is just a starting point. You can easily expand this project with all the functionality you need.
The logic is very simple. We use a QR Code to redirect our customer to our WebAR app (you can also use a custom-generated QR code to pass some data to the app, like an order reference number, through a query string). The custom marker is then used to render the content on the voucher.
Project Setup
First of all, let’s create a new project using vue-cli:
vue create my-awesome-ar-projectNext, we need to install AR.js. Unfortunately, there’s no npm package available at the moment, so we need to import the library directly in our index.html. You can find AR.js and A-Frame build files here, or you can just use the CDN, as shown on the documentation.
The A-Frame version of AR.js uses custom HTML elements to define the scene and all of it’s components. We need to tell Vue to ignore those elements, otherwise we will get a bunch of warnings printed in the console. Here we are telling Vue to ignore all elements that start with “a-” (a-scene, a-entity, etc).
Basic Scene
In App.vue, we use <a-marker> to enable the marker tracking and render a 3D model (defined by the <a-entity> component) once the marker is found by the device’s camera. By default, you can use two marker presets, “hiro” and “kanji”, which look like this:
For this project, I wanted to use a custom marker with our Tui logo. To make it, I’ve used this online tool. Here’s what I came up with:
Once downloaded, you will find a .patt file. This is what we need to include in our project (Note: you will also need to specify type=”pattern” in order to make it work)
<a-marker type=”pattern” url=”path-to-patt-file">Display Content in the scene
At this point I’m ready to render some content in the scene. First thing first, I would like to show some information to tell our customer how to reach the meeting point, as well as the distance and the travel time.
Obviously this data is fake, but you can see how easy it could be at this point to implement an API call in order to fetch some dynamic data.
This is the component I’ve used to display the information:
Note: the units of measurement for dimensions (width, height) and position are directly proportional to the dimension of the marker in the real world. For example, if you print a marker that is 10x10cm, and you add a component like this one:
<a-plane width=”2" height=”1" />You will obtain a rectangle that (in real-world dimensions) will be 20x10cm. This makes it pretty simple for us to understand how to position the elements on the page, since we know how big the marker will be once we print the ticket.
The last component I’d like to implement for this project, is a button that allows our user to open the directions directly on the Google Maps app. Thanks to the raycaster component of A-Frame, we can check for user interactions (like click and gestures) with all the objects displayed on the screen. This allows us to attach some event handler to our objects:
Here’s a demo of the final result:
Note: On some devices you could notice some issues with the click event being off-center. This is an open issue, you can check this link if you want to contribute.
Conclusion
I’ve had a lot of fun researching this technology for this article. I think it has a lot of potential and you should definitely check it out. As Tim Cook, Apple’s CEO, said:
I do think that a significant portion of the population of developed countries, and eventually all countries, will have AR experiences every day, almost like eating three meals a day. It will become that much a part of you.
If you want to learn more about AR.js, check out this great article by Nicolò Carpignoli.
Here, you can find the repo I’ve created for this article, as well as a link to the live example.
