Challenges | Augmented Reality

Challenges that A U G M E N T E D R E A L I T Y Faces

1. Hardware of Smartphones

Computing Capabilities

To make AR effective and attractive, images must be rendered in real time. As users move their phones, the AR program must update quickly to reflect the changes. This is difficult as mobile phones have limited computing capability due to the hardware constraints. The rendering on such small devices might not be powerful enough to constantly update the device’s information(displays) in real time - hence the device, in layman terms, lags. Implementing augmented reality in an open environment like the outdoor proves to be a huge challenge as rendering images in an open air environment ( without a power-source and just a battery - and this leads to problems of battery life ) require high processing power and cached data storage capability that even current high-end smartphones cannot handle efficiently.

Camera Sensors

Smartphone camera sensors are poor under bad lighting conditions. Thus, this make AR rendering almost impossible at night. However, using better camera sensors and technology such as night-vision ( whereby infrared light is emitted by a diode and bounces back to the IR sensor ) will consume much more battery. If the current camera sensors in smartphones is to be upgraded, the problem of short battery life will escalate. We will as such need advanced batteries with greater capacitance per unit volume - and if they can store a very large charge, does this mean the battery will take forever to charge? Therefore, we are stuck between a rock and a hard place as we must balance out the hardware specifications, the battery capabilities and the size of the smartphone.

Geographical Sensors

Smartphones also have geographical sensors that help AR program locate our position. The problem with these sensors is that they are not very accurate. The moment we hold our phone slightly off from the optimal position, it introduces noise data into the system that cause the AR computing to be inaccurate. The reduction of noise represents a significant challenge in confronting AR hardware.

2. Software in Smartphones

Difficulty in integrating 3D with image

The 3D rendering world in a mobile platform is hard to mesh with the 2D imagery obtained from a camera. This creates a huge challenge in providing accurate AR. User experience will quickly deteriorate when accuracy is lost. The reason why it is so difficult to mesh is due to the nature of AR. AR must be able to constantly update this 3D rendered world ( in the smartphone )as the user shifts the camera around. This will require a sophisticated software with intricate rendering capabilities.

Common Platform VS. Independent Platform

Should we have AR on an independent or common platform? If we were to have AR on a common platform, like Android, it would promote the growth of AR greatly because developers all over the world are able to code on a common platform, thus, allowing them to build sophisticated software needed for AR.

However, many developers have voiced out against having a common platform because they feel that this would severely cringe the flexibility of their coding. Such developers say that they want to have the freedom to customize the AR platform to their specific program.

Some companies which may develop AR, such as Microsoft, may choose to keep the technology proprietary i.e. to themselves. This will entice customers to buy their smartphone since their smartphone has advanced AR software. In effect, applications for Microsoft may not be available to users of other Android smartphones - unless they pay a fee, perhaps. This is speculation.

Point of Interest (POI)

A problem we are facing is the overloading of POI in city area. POI are common points that the AR program locks onto to help its rendering of the virtual world. If we were to have a common AR platform, when the user opens his/her AR app and points it up to a street corner in a city, multiple information streaming into the app, such as location, nearby restaurants etc… will overload the app and cause significant problems. If each developer builds their own AR platform, the problem of overloaded POI will not exist because each program will only detect the POI that each developers have coded. However, this would require more manpower as the POI keeps changing and updating of information is definitely constantly needed. If information is not updated quickly enough, users might be frustrated because the data would be inaccurate, irrelevant and incorrect - impeding their individual autonomy ( e.g. their plans for the day ) .

3. General and Specific Challenges

Even if the aforementioned issues ( i.e. hardware and software issues ) are sorted out in smartphones, there are still some general challenges facing the implementation of AR - these challenges will apply broadly to AR technologies in general.

General challenges

(a) Cost - The cost of an AR-device might be prohibitively high,for say, the consumer( e.g. Google Glasses, which are meant for everyday use ) or the business ( e.g. AR when it is used for better and more efficient construction )

Even smartphones might see increased prices because more expensive hardware will need to be added in, as well as the high costs of (AR)application development funds pumped in. As a result, an iPhone in future that is optimised to run AR applications and in fact has a proprietary AR application from Apple will be marginally more expensive. However, due to economies of scale and free market forces, the prices may not even be so high. Even if prices are high at first, they will be much more inexpensive in future. As such, we feel that price is not going to be a major challenge.

(b) User friendliness - Would AR technology be user-friendly? Because AR involves the empowerment of businesses and individuals to fulfill a wider variety of complicated tasks ( and do it quicker ) and involves cutting-edge capabilities, it might be more difficult for AR developers to develop a user-friendly, intuitive application.

(c) Bugs - As AR devices and software involve more complicated functions and capabilities, there will definitely be bugs at the starting phase. However, these bugs will eventually be sorted out and reach an acceptable level. It is important that when a product is put on the marketplace it has as little bugs as possible. Why is this so? If there are too many bugs, then it will be a long time before the mass market ( as opposed to tech junkies who will always pick up whatever cool latest gadgets ) will pick up these products and services; the skeptics will take even longer to be convinced to purchase these products and services in light of the bad press.

(d) Safety - Related to the issue on bugs, safety issues must be ironed out before the products are put into the market. For example, Google Glasses are placed right in front of your eyes. It would be most unfortunate if poor manufacturing practices result in an explosion and lead to blindness in a consumer. Likewise for AR contact lens. Also, what if you are driving halfway and the Google Glasses displayed an opaque message, leading to an unfortuante accident? Apropos, certain AR applications and products would need a higher safety standard e.g. those used for surgery and other medical procedures. While form and functionality is very important, safety must always remain at the forefront. If there are any safety-related incidents, pick-up rate will be much slower. This is especially so, since AR is not a vital or necessary part of our lives as of now.

Specific challenges

The issues and challenges facing the implementation of AR is highly context-specific. We discussed the issues viz the AR-enabled smartphone above. Let us look at other specific challenges.

In the medical arena for instance, concerns might arise over the reliability of such devices, and the prohibitive cost of acquiring an AR system - medical devices are known to run into costs in the range of tens of thousands of dollars - this might be reflected in the surgical procedure cost. For instance, endovenous ablation using ultrasound (AR) visualisation costs much more than the traditional surgery because (1) a specialised machine is used (2) a technician is present to guide the surgeon during the surgery and (3) the surgeon might need to undergo training to use this device - there are benefits though : less scarring, shorter down-time and quicker recovery, etc - thus the patient has to weigh the benefits vs. the additional costs.

In the educational arena, how effective is an AR-supplemented pedagogy? AR is generally used to supplement and not replace traditional classroom teaching. Will teachers find this a 'waste of time and money' or would they find value in AR applications and technologies which might make students more keen and interested to learn? It is imperative for us to keep an open mind, as an open mind is in parallel with the idea of being a 'rising star'. Singapore, for example, is an industrial leader in education, having incorporated mutimedia and IT in our curriculum as early as the 1990s. As such, today we have one of the highest broadband penetrations in the world, above that of the States, as well as a very high computer literacy rate. In the field of Chemistry, it might enable undergraduate or pre-university students visualise molecules and their reactions much more effectively - being particularly helpful to visual learners.

In the military arena, how secure are AR devices? i.e. how prone are they to data interception from rogue forces? Wireless communications, while they may be encrypted, are notoriously insecure. Would they be prone to electronic jamming?

In the case of AR being used in vehicles, it is noted that Mercedez-Benz is coming up with revolutionary AR systems that can essentially transform cars. There will always be pertinent issues to sort out - for example, how can we make sure that only the relevant hand gestures are captured by the system ? Of course we do not want the car to accelerate or decelerate, causing an accident, if the driver scratches his head. If such complicated systems fail, who will take the liability for the injury to life and/or property? Certainly the manufacturers and AR-component manufacturers would not want inordinate liability to befall them. Given the possible cost disparity ( which might be not that bad, given economies of scale ), will there be a significant dichotomy between cars with AR and cars without AR?