Tomorrow's UI Today... if you want it.

 by John Gnotek, Senior User Experience Designer at eFlex Systems

Augmented Reality (AR) is one of the latest breakthrough technologies offering to change the way people view and interact with the world around them.  Unlike Virtual Reality (VR) which puts the user in an entirely different environment, AR augments the real world.  That is, it provides additional information about the existing environment.  And this can be very beneficial for user interfaces (UI) on the factory floor!

But first, what exactly is augmented reality? Wikipedia states, “Augmented reality is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data.”  We see it in use already.  Heads up displays and backup camera guidelines on automobiles. Football games on TV that overlays the current yard lines (graphic) on the field.  Google Maps.  Pokemon Go!

Or soon, imagine driving your vehicle through a city or town.  On your center-stack monitor you see the view in front of you.  Popups from various establishments state what they are: a seafood restaurant, a hotel, a nightclub, a parking lot, a corporate headquarters.  Tapping on one of the popups gives more detailed information: a menu and operating hours, parking lot rate and availability, hotel rates and vacancy, a corporate history.

Or while driving, project the view five miles to see what lies ahead, or 50 miles out.  Getting tired from driving? Ask Siri or your personal assistant of choice to display hotels within the next 20 miles.  “Show availability at that Red Roof Inn there in Fargo.”  “Show the Commander Suite.”  “Book it!”  “And make a reservation at that Golden Corral restaurant.”  The possibilities are endless.

Tim Cook, CEO at Apple, has a “secret” team numbering in the hundreds working to develop applications for both AR and VR.  And of the two technologies Cooks says, “My own view is that augmented reality is the larger of the two, probably by far, because this gives the capability for both of us to sit and be very present talking to each other, but also have other things visually for both of us to see," Cook said, "Maybe it's something we're talking about, maybe it's someone else here that is not here, present, but could be made to appear to be present with us. So there's a lot of really cool things there."

AR may just be Apple’s “next big idea.”  “I regard it as a big idea like the smartphone,” Cook states.  Cook goes on to say, “There are things to discover before technology is good enough for the mainstream.”  But that’s not to say AR isn’t available now.  It is here… if you want it.  That is, this technology is a very feasible solution for numerous scenarios and applications—but it's not yet mainstream.  There isn't a box you can pull off the shelf and apply to your situation, but current software development can do a lot with this budding technology already.

Back to the factory floor… imagine wearing AR glasses and inspecting parts as they come through an assembly line station.  Focus the crosshairs within your glasses to the barcode or serial number on a part to see if it’s the correct one—or, view its entire build history if need be.  Scan to the next part—it’s been contained.  Pull it.

 Scan inventory to get a part count.

Focus on a robot or a machine—see its machine status, or maintenance schedule and history.  

Ping a station to view its manufacturing cycle time average or OEE.  Compare it to the next machine.  Do root cause analysis’.  Do instant analysis. 

 Use AR for guided work instructions to perform various tasks, or to learn a new machine or process.

Create process simulations, do line rebalancing.   Allow notifications, alerts and warnings to display.   Look in on the heat treatment station and read actual temperatures with statuses.  Endless possibilities.

In short, imagine much of the manufacturing and process data that typically demands a computer or mobile device and an MES application to view, right within… eyesight.  This may not be mainstream yet, but some big companies are working hard on making it so… the next big thing.  As mentioned, Apple.  Microsoft is also very heavily invested in this with their HoloLens technology.  Volvo, Microsoft's first automotive AR partner, uses HoloLens in several different ways, as part of their engineering workflow, marketing their cars to consumers, as well as for work instructions and a training tool on the plant floor.  And Daqri smart helmets and glasses are already quite amazing.

But, what is the return-on-investment?  All depends on how AR may benefit your particular needs, and how you view ROI. The benefits are many, such as:

• Efficiency - It's a lot more efficient and quick to look at a machine or a part and have pertinent information appear before your eyes than find a part, go back to a desk, do a serial number search to lookup the same info.  Likewise it's more efficient for a repair person constantly on the move to walk past machines to garner machine status and maintenance history—and address problems immediately than to lookup same information from an office somewhere [else], or even on a display at the station.
• Training/Task Instructions - Much more intuitive to have training and instructions right in front of you, overlaid on that which you are working—including any needed details, motions or animations—whether you are learning a new machine, process, or ever-changing task instructions.  The end of greasy manuals and portable whiteboards littered with paper instructions.
• Hands-free - Real-time data to workers whose efficiency and safety depends on keeping their hands free.
• Monitoring the plant - Plant managers, project managers, line leaders at a glance can all monitor work progress in real-time through AR markers or beacons on equipment.
• Decision-making - Improved decision-making—on the fly.
• Visual alerts - Get immediately-noticeable alerts, warnings, alarms and messages.
• Real-time data - For when real-time information is crucial to getting a job done.
• Geo-targeting - Easily locate anything anywhere on the plant floor.
• Real world augmentation - What you view is the actual world around you... with lots of additional, useful data.
• Forward thinking - AR conveys innovation and sets its adherents as forward-thinking companies.

AR technology, though still in its infancy, does allow these ideas discussed and much more possibilities—right now… if you want it.  Even if just on your iPad.