Augmented Reality (AR) & Virtual Reality (VR)

Oh what a time to be alive ! As humans, we explore this world we live in, learning the unknowns, improving the factors associated with happy life while minimising the environmental catastrophes. However, it is impossible to do such acts without understanding the nature of those activities, since errors associated with them can be expensive. Gaming industry has taken this concept, a long time ago, to create interaction between the users and the gaming platform. For example, Mario allows the users to press a couple of keys from the keyboard, and associate these inputs to certain actions of Mario. In recent years, the way interaction between users and a computer has grown beyond imagination.

Augmented Reality (AR) allows the users to interact with virtual objects in the real world [1]. A computer or an embedded system determines the position and the orientation of the camera on it, renders and displays three-dimensional (3D) graphics on the real world via superimposition such that it appears as if it is from the viewpoint of the camera. Virtual Reality (VR), on the other hand, creates a complete virtual world to give a full immersive experience with the aid of an additional device, generally a head-mounted device [2]. As visual creatures, while we enjoy visual information, we are also sensitive to sound and touch, which are the building designs of virtual reality. While a computer computes the camera coordinates with a visual sensor and carefully engineered algorithm in AR, the position of the user’s eyes are located in VR. This allows the visual design to perform similar actions as the user such as turning head and/or moving arms. This is more convincing in the gaming world.

In robotics, we have used multiple simulation environments for testing and experimenting hypotheses. We have well-developed simulation platforms for navigation, localization and rendering. While they are physics-based systems in general, the virtual environments for VR are more like an animated movie or a gaming environment, where the art, creativity and essence of social interactions are valued more than mathematical functions. There are exciting technological developments in this arena such as Mixed Reality (MR) where both VR and AR are implemented [3] and Extended Reality (XR), the umbrella term in which either AR, VR and/or MR is implemented [4]. While robots will not be excited about these developments since they already have the ability to perform such tasks on their own, it is the collaboration between robotics and XR with an essence of artificial intelligence which is exciting in robotics.

Applications of this field is the foundation of the emerging future. Medical and education field can have strong developments with XR. Doctors can interact with the patients in a common visual language, dissect animals without harming animals and this knowledge can be transferable among hospitals and medical schools. Not just medical fields, but these can be used to understand chemical reactions and physical forces better. Scientists’ lives would become way more easier where an addition of dimension in their thought process is now an extended reality. 

Construction companies have started to use technological advancement including the use of robots for 3D printed houses. With XR, it is possible to visualise building projects more interactively. A computer-generated images of structures can be imposed onto reality, 2D drawings can be visualised in 3D space and these techniques can also guide construction workers in terms of safety and measurements. The latter can also be an advantage for industrial manufacturing workers where they have no reasons to check for manuals or crunch some numbers while their AR glasses can display it to them. 

XR future is glooming in the marketing sector of companies. Virtual dressing rooms, furniture shopping spaces, fitness in gym, interactive virtual shopping have taken the world by surprise in this pandemic era. While these technologies are still in their developing stage, it is only a matter of time before they get integrated into our life. AR is also blended with poetry and artistic multidimensional experience. 

Robotics is currently working on similar sectors [5] but fundamentally replacing hard manual work and finding niche ways of doing things better and artificial intelligence is understanding what patterns are available and generalising across different datasets for an application necessarily making human-robot interaction collaborative, it can go a step further allowing robots to design and implement systems that interact with people. In the future, each of us can have a robot just as we have a mobile phone which understands our basic needs and wants and creates a virtual world for entertainment, or better make the reality way more interactive.


  1. Azuma, R.T., 1997. A survey of augmented reality. Presence: teleoperators & virtual environments, 6(4), pp. 355-385.
  2. Burdea, G.C. and Coiffet, P., 2003. Virtual reality technology. John Wiley & Sons.
  3. Andrews, C., Southworth, M.K., Silva, J.N. and Silva, J.R., 2019. Extended reality in medical practice. Current treatment options in cardiovascular medicine, 21(4), pp. 1-12.
  4. Hoenig, W., Milanes, C., Scaria, L., Phan, T., Bolas, M. and Ayanian, N., 2015. Mixed reality for robotics. International Conference on Intelligent Robots and Systems pp. 5382-5387.
  5. Makhataeva, Z. and Varol, H.A., 2020. Augmented reality for robotics: A review. Robotics, 9(2), p. 21.
%d bloggers like this: