What role AR and VR can play in the industrial sector?

"We've got the software we need, but the hardware has some catching up to do!"

Apple's Vision Pro announcement has sparked a renewed interest in the world of augmented reality (AR) and virtual reality (VR). Looking through the lens of industrial environments, these technologies have already made significant contributions in manufacturing, maintenance, and training processes. In this article, we’ll examine key areas in which they hold potentially transformative power for the industrial sector with input from Hubs’ in-house engineers as well as other industry experts.

A primer on augmented and virtual reality.
Augmented reality, often abbreviated as AR, is a technology that superimposes digital information or virtual objects, such as images, videos, or 3D models, onto the real world. AR enables users to interact with and manipulate this digital content in real-time, creating immersive experiences through devices like smartphones, tablets, or specialized AR glasses.

On the other hand, virtual reality, known as VR, is a computer-generated simulation of a three-dimensional environment that users can interact with and explore. VR typically involves wearing a headset that displays virtual content and may include additional peripherals like hand controllers or haptic feedback devices.

How can you use AR and VR in the industrial sector?
In the industrial sector, AR is employed to enhance worker productivity and accuracy by overlaying digital instructions, real-time data, and visual cues onto machinery or workstations. This aids in assembly, quality control, maintenance, and troubleshooting, reducing errors and downtime.

VR, on the other hand, offers immersive simulations for training purposes, allowing workers to practice complex procedures or hazardous scenarios in a safe virtual environment. This helps them to acquire new skills, mitigate risks and save costs.

Overall, AR and VR empower the industrial sector with innovative solutions that optimize productivity, efficiency, and safety. Here are five examples of applications.

Factory layout planning.

AR and VR can revolutionize factory layout planning by providing immersive and interactive experiences of a factory before it is constructed. With AR, designers can overlay digital models onto physical spaces, allowing them to visualize how different equipment and layouts will fit within the factory environment. This can help identify potential design flaws or bottlenecks before construction begins, saving time and reducing cost. XYZ Reality uses AR to ensure precision in prefabricated construction (right).

VR, meanwhile, can enable stakeholders to virtually explore and interact with the planned layout, providing a realistic sense of scale and perspective. This allows for easy modifications and optimizations, facilitating collaborative decision-making among architects, engineers, and factory managers.

Operator training simulation.
AR and VR can also offer valuable training simulations for operators. With AR, operators can make use of real-time, contextual information overlaid on physical equipment, guiding them through complex procedures and enhancing their understanding of equipment operation.

VR can create immersive environments in which operators can practice handling various scenarios in a safe and controlled manner. By replicating real-world situations, trainees can gain hands-on experience, improve their skills, and learn how to respond to different challenges.

Real-time AR instructions.
Through wearable AR devices or mobile applications, workers can receive contextual information overlaid on their field of view, such as step-by-step instructions, safety guidelines, or equipment specifications.

Real-time AR instructions can be particularly beneficial in complex assembly processes, maintenance tasks, or quality control procedures, where precision and accuracy are extremely important. Companies like Airbus have already integrated Microsoft’s mixed reality Hololens into its factory processes.

Remote maintenance.
AR and VR can transform remote maintenance by enabling technicians to collaborate with experts from anywhere in the world.

With AR, technicians at the site can wear smart glasses or use mobile devices to stream live video to remote experts. Experts can then overlay virtual instructions or annotations onto the technician's view, guiding them through the troubleshooting or repair process in real time. This significantly reduces travel costs and downtime, as experts can diagnose and provide guidance without physically being present. German brand Oculavis (right)has demonstrated such capabilities over the last couple of years.

Safety alerts and evacuation.
AR and VR technologies can play crucial roles in safety alerts and evacuation procedures. AR can provide real-time safety alerts and warnings by overlaying visual cues or instructions onto the user's field of view. For example, in hazardous environments, AR can highlight potential dangers, mark safe pathways, or display emergency protocols.

VR can simulate emergency scenarios, allowing users to experience and practice evacuation procedures in a realistic virtual environment. This enables individuals to familiarize themselves with emergency routes, equipment locations, and evacuation protocols, leading to better preparedness and faster response times during actual emergencies. Additionally, VR simulations can help identify potential flaws or bottlenecks in evacuation plans.

What’s next for AR and VR in the industrial sector?
For further insights on where AR/VR currently stands in the industrial context, and to identify the steps needed before it could play a more significant role, Hubs reached out to experts in design and manufacturing. Here's what they had to tell us.

According to Charles Goin of Virginia-based firm Goindesign, VR and AR are where Building Information Modeling (BIM) and Virtual Design and Construction (VDC) in the architectural engineering and construction market were 10 years ago. "We've got the software we need, but the hardware has some catching up to do," he says. In his opinion, AR seems to hold the edge over VR, until VR becomes photorealistic, which again is going to be hardware-driven. The biggest moves in that field were, of course, made by Meta, HTC, Valve, and now Apple. Given that others paved the way but didn't quite strike gold, it will be interesting to see where Apple takes it.

Robert Jones, Manufacturing Engineer at Nissha Medical Technologies, also sees the huge potential of AR technology in the manufacturing space in training operators as well as remote maintenance. "In our facility, we use both manual and semi-automated processes to manufacture safe and functional medical devices. Training operators for these tasks can be time-consuming and resource-intensive. An AR-based program could transform this, offering real-time work instructions for efficient assembly. Paired with AI, it could even provide real-time alerts and corrective actions."

The areas listed above are only the tip of the iceberg when it comes to AR and VR in the industrial sector. Because both technologies are constantly evolving and improving, new applications and use cases are often introduced, resulting in increasingly sophisticated ways of interacting with industrial environments.

As AR advances, for example, the technology may become more compact and wearable, with devices like smart glasses becoming more lightweight and comfortable. This will make AR more accessible and practical for workers, allowing for hands-free operation and better integration into existing workflows.

---

@HubsMFG @AppleVisionPro @xyz_reality @HoloLens @Protolabs #Manufacturing #Automation

Immersive technology explored.

The newest issue of the Mouser Electronics's Methods technology and solutions journal has been published. Immersive Technology, the third issue of volume four, offers a collection of articles that explore emerging technologies, including augmented reality (AR), extended reality (XR), mixed reality (MR), and virtual reality (VR), that are integral to living, working and studying in new sensory environments. 

“Immersive technologies like VR and AR are top of mind as innovative companies seek to redefine the way we collaborate, communicate, and spend time together,” says Kevin Hess, Mouser Electronics’ Senior Vice President of Marketing. “The newest issue of Methods goes beyond the hype to explain some of the most important technologies underpinning these new applications, and what they mean for the future of work and entertainment.”

The Immersive Technology issue features contributions from leading subjective matter experts in the fields of robotics, VR and immersive collaboration. The issue offers multiple articles on the technological advances and engineering challenges shaping new applications for new sensory applications, in addition to pieces on 3D sound and product creation.

In addition to the Methods technology and solutions journal, Mouser offers a broad range of technical resources for design engineers and buyers, including blogs and e-books. Mouser’s content hub features exclusive design resources, white papers, videos, and product information, allowing design engineers to break new ground in product development and innovation.

@MouserElecEU @Publite #PAuto #AugmentedReality

CAD Envelope Geometry for Performance and IP Protection.

The latest version of CoreTechnologie's 3D_Evolution Simplifier software now offers special applications and interfaces for Virtual Reality (VR), Augmented Reality (AR) and Digital Factory. Clever features reduce large assemblies by up to 97 percent.

Two-step Simplification Process
With the new application, it is possible to further simplify the lightweight solid envelope geometry CAD models created by the Simplifier process in a second step, which is an polygon reduction function for tessellated formats, especially for VR/AR as well as factory planning software tools. In the first step, the inner geometry is removed and a closed CAD solid envelope geometry is automatically generated. This is followed by an automatic deletion of interior components and an optional filtering of solids with small volumes. For the generation of exact CAD models, the optimum is reached after this first step.

To convert the models into tessellated formats such as VRML, FBX, OBG or JT, the next step achieves a predefined percentage reduction of the polygons within seconds using the new polygon reduction function. This extreme reduction in the number of elements to be rendered achieves a much higher performance and fluid rendering, for example, when animating very large assemblies.

The Simplifier automatically generates lightweight CAD envelope geometries for optimal performance in all downstream applications as well as during further processing in the CAD system.

Know-how Protection and optimum Data Quality
The envelopes generated with Simplifier also protect the know-how and enable secure data exchange with development partners, customers and suppliers. The tool processes over 40 different formats, including CATIA, XT, CREO,NX, Solidworks as well as DGN, IFC, JT and STEP without access to a CAD license and ensures optimum quality of import models through automatic checking and correction functions.

@CoreTechnologie #PAuto #Robotics

Augmented reality in packaging.

Harpak-ULMA announced that the company is extending its platform capabilities to support Augmented Reality on its Rockwell Automation-enabled packaging solutions. The company has entered Beta-phase testing and is targeting commercial availability for its Augmented Reality (AR) option next year. AR represents a leap forward for staff knowledge transfer — employing digital visual overlays through a combination of both handheld and hands-free devices. It supersedes traditional approaches to operator training and maintenance of industrial assets.  

AR presents a powerful solution for producers seeking to alleviate the well-known upskilling issues associated with high labor turnover, productivity, and error proofing. According to a study by the Manufacturing Institute and Deloitte, by 2025, almost 3.5 million manufacturing jobs will become available, of which 2 million will remain unfilled. AR can help producers bridge the skills gap at the root of this major disconnect. By 2025, three out of every four workers will be millennials who will be driving new workplace expectations. The Journal of Organizational Learning and Leadership reveals that millennials’ innovation and exploration traits are best enhanced through active learning (pdf).

While AR has not yet been widely employed in packaging markets, other industrial experiences indicate that real-time “active-learning” dramatically accelerates front-line staff learning curves and reduces error. For example, Boeing workers completed intricate wire harnesses assemblies using AR-based work instructions, cutting assembly time by 25%, and reducing error rates to nearly zero. Staff with little or no formal training can be rapidly up-skilled to perform nonrepetitive complex activities — the kind of work inherent in many unfilled manufacturing positions today. When a machine malfunctions or a tool change is required, AR guides the operator or maintenance worker visually step by step through task execution.

In 2018, Harpak-ULMA launched its strategy to produce smart, connected platforms that deliver packaging as an integral part of the manufacturing digital thread. These platforms enable emerging Internet of Things (IoT) software applications (such as AR) that leverage production data to digitally transform traditional plant asset maintenance and operational processes. However, simply enabling the utilization of such applications is insufficient to help producers realize value. The combination of skill sets required to develop AR content is not only atypical of producer staffing models – they’re in short supply altogether. 

Harpak-ULMA’s Innovation Engineer, Alexander Ouellet, leads the company’s AR development effort. He explains, “AR requires new roles such as UX or UI designers, graphic designers to create assets, and 3D modelers who understand how to turn an engineering data set into production-ready, user-facing graphics. You also need IoT software architects to define AR experience data flows to ensure experience scalability. We quickly understood that it wasn’t enough to simply enable AR on our platforms – our customers want more than a toolkit. That’s why we are building out fully contextualized, solution-specific AR content. When our customers opt for AR as part of their purchase, we intend to provide a library of experiences for tool changes, maintenance, expert capture of training processes, and related device recommendations. We do the heavy lifting so customers can deploy a complete AR experience along with asset commissioning. We also anticipate that customers may request custom AR content, which will be accommodated as well.” 

Ouellet noted he will present the opportunities and challenges associated with deploying AR solutions as part of the upcoming PackExpo Innovation stage on November 9th at 2 pm Central Standard Time (CST).

Harpak-ULMA utilizes applications such as FactoryTalk Innovation Suite, powered by PTC and PTC’s popular ThingWorx IoT solution platform and Vuforia AR platform. All leverage a Rockwell Automation controls platform to optimize IoT data availability and scalability. Vuforia enables the rich, contextual digital overlay users demand for a true AR experience. Kevin Roach, CEO of Harpak-ULMA, notes that “While we believe our vision and role is critical to accelerating digital transformation of producer packaging processes, ‘it takes a village’ to bring these disruptive technologies to market today. Rockwell Automation and PTC are both major, well-known vendors in our markets. Together we will continue to build out compelling value that’s easy for our customers to implement and use.”

Jim Heppelmann, PTC’s president and CEO, offered additional insight, “Succeeding in our new normal demands that we bring digital to the 75% of workers – 2.7 billion globally – who are part of the front-line workforce. Knowledge workers have tools like Microsoft Office, Zoom, and others that work great on a computer screen – but not on the plant floor. Front-line workers need a way to collaborate, publish, and share ‘how-to’ digital data and content mapped onto the real world rather than a computer screen – and that’s the very definition of augmented reality. AR is essentially the front-line equivalent of those tools on which knowledge workers rely so heavily.”

Blake Moret, Chairman and CEO of Rockwell Automation, also commented, “Manufacturers around the world are seeking integrated approaches to digital connectivity and automation controls to unlock unprecedented operational productivity, bring their products to market faster, and increase asset utilization while lowering risk. Our connected enterprise vision embraces practical approaches that make these concepts real. Including working collaboratively, as we have in this case with Harpak-ULMA and PTC, helps more producers leverage innovative technologies to deliver tangible improvements in production operations. Together, we’re combining our strengths to bring a more connected enterprise to life.”

@HarpakULMA @ROKAutomation #Packaging #PAuto #AR 

Analytics and live remote assistance to improve productivity and safety.

Emerson Integrates Augmented Reality into Plantweb™ Optics Software, Enhancing Remote Collaboration and Workforce Effectiveness

A not uncommon scenario. Most of the workforce has retired, leaving more than 40% with less than 5 years experience. They have a corporate training of course but most of the training still occurs on the job.  Augmented reality (AR) thus becomes the augmented brain of this retired workforce who are no longer here…Ideas, knowledge, and methods they used can be put into the system and extracted out when and where a problem arises. 

A large plant will have many different systems on a site and difficulty in accessing personnel or information. Again AR can be of inestimable value in such a situation in guiding maintenance engineers to the site of a problem.

In the food & drink and other sectors where hygienic practices are of primary importance, ready and easy access to information or procedures is essential. AR is ready and able to provide this at a click.

These are just three possible applications where augmented reality can be of assistance in ensuring the smooth running of a plant in the 21st century.

Emerson have introduced augmented reality (AR) technology to its Plantweb™ Optics asset performance platform, delivering enhanced access to real-time diagnostics and analytics, as well as live remote assistance, to industrial plant workers responsible for maintaining and optimizing plant equipment. With AR technology integrated into Plantweb Optics, companies can improve productivity, collaboration and operational performance, without being limited by shortages of skilled workers or travel restrictions.

“Successful digital transformation programs that lead to Top Quartile performance have people and work practices as a key focus. Adopting innovative technology like augmented reality and institutionalizing best practices enable workers to add more value than ever to operational and business performance,” said Stuart Harris, group president for Emerson’s digital transformation business. “With these new Plantweb Optics technologies, customers can experience significant improvements in equipment reliability and the safety of their facilities.”

Plantweb Optics leverages artificial intelligence, machine learning analytics and data contextualization to provide real-time visibility into plant reliability and operational performance. Unlike standalone AR solutions that require custom engineering, AR is integrated into Plantweb Optics, providing immediate access to a wealth of data and translating into easier, less costly implementation and a faster return on investment. For use by manufacturers in the life sciences, food and beverage, chemical, metals and mining, power and water, pulp and paper and energy industries, Plantweb Optics is part of Emerson’s award-winning Plantweb™ digital ecosystem of technologies, software and services.

Augmented reality for Plantweb Optics transforms the way field technicians accomplish complex tasks through enhanced situational awareness, live remote assistance and analytics delivered in context of the plant. As a field technician walks an industrial plant with a mobile device, Plantweb Optics uses spatial computing technology to map assets and provide technicians with critical maintenance information relevant to their location. Plantweb Optics overlays real-time analytics, equipment health status and technical support documentation on their field of view, so technicians can safely resolve issues sooner.

The augmented and virtual reality market is projected to grow annually at 40% from 2017 through 2025, according to multiple research reports. Much of this spending will come from manufacturers around the globe using AR technology to help upskill their workforce for digitalized operations.

With live remote assistance, field technicians can be virtually shadowed by experts, either on-site or off-site, from Emerson, their own company or another service provider. Experts can talk, type or augment the technician’s mobile display with graphics to guide the next action. Live remote assistance enables technicians and experts to collaborate for safe troubleshooting and repairs, regardless of location and without travel costs. Live remote assistance sessions, best practices and notes from experienced engineers and step-by-step troubleshooting procedures can be logged into a knowledge library for use by all engineers at a site. The knowledge library is a resource for companies to standardize procedures and ensure engineers of all experience levels understand an asset’s history and are using best practices for safe, efficient operations.

@Emerson_News @EMR_Automation @HHC_Lewis #PAuto #Augmented reality

Displays for augmented reality (AR) glasses.

MICLEDI Microdisplays, an imec spin-off, has raised €4,5M seed capital from imec.xpand, with participation of PMV and FIDIMEC. The funding will be used to develop microLED displays for next generation Augmented Reality (AR) glasses.

MICLEDI’s vision is to enable AR for everyday personal use - smart glasses that are small, lightweight, with long battery life, and at reasonable cost. To make this happen, MICLEDI is developing the world smallest and brightest displays. The key innovation behind MICLEDI is the new integration technology for microLED on 300mm wafers developed in collaboration with imec.

L-r: Dr. Soeren Steudel, co-founder and CTO; Dr. Alexander Mityashin, co-founder and COO; Seán Lord, CEO

MICLEDI Microdisplays was founded by Dr. Soeren Steudel and Dr. Alexander Mityashin, both researchers from imec with a deep know-how in the R&D field and display development. They are joined by Seán Lord, an experienced executive in the semiconductor industry, as CEO.

“AR glasses may replace our smartphones in the future and display technology is a key enabler for such a transition. Today’s display technologies cannot fulfil the specifications needed for next generation AR glasses. At MICLEDI we are tackling this challenge and have developed displays that are 100x brighter than commercial alternatives” - explains Soeren Steudel, co-founder of MICLEDI.

In order to implement the vision that future consumer AR devices will be powered by a tiny display developed in Leuven, the company has raised a 4.5M euro seed investment from imec.xpand, PMV, and FIDIMEC.

“We are pleased to back a unique team. We look forward to working with them and to enable technology that will be at the core of future AR devices and will change how digital information is presented to consumers” – says Cyril Vančura, Partner at imec.xpand.

MICLEDI’s will develop its microLED technology and first prototypes on imec’s 300mm pilot-line infrastructure.

“Hardware development takes courage, capital and time. We are proud of the entrepreneurship of our engineers to bring promising technology developments originating from our R&D, to the market” says Luc Van den hove, president and CEO at imec. “Based on an original concept launched, now more than 2 years ago, by the MICLEDI team, imec together with imec.xpand incubated this into a promising technology base and exciting business case. It leverages imecs extensive R&D in organic displays, 300mm integration and wafer-scale optics technologies.”

“MICLEDI’s new generation displays for AR glasses fits in PMV’s strategy of supporting early stage technology with a largescale international potential, hence strengthening Flander’s position as a top region.” – Roald Borré, Head of Equity Investments at PMV.

#Research @imec_int