Augmented Reality: Focusing on Photonics in Industry 4.0

— Industry 4.0 (or 4 th industrial revolution) facilitates horizontal and vertical digital information flow along value chains up to the end-customer and is highly relevant in a broad variety of industries. Augmented reality (AR) is a key technology in Industry 4.0, which connects the virtual and real-world environments using such digital information flows. In doing so, the technology relies upon the systems that includes hardware and software components. Particularly, optics and photonics are of much importance in the display and processing of information in these systems. However, a particular challenge is that the AR-based systems have not been adopted in the industry as much as other technologies even after several decades of their existence. Based on review of academic literature, an industrial survey and experiments conducted in the industry, this article aims to identify success factors and challenges of AR systems and metrics of photonic components that can form the basis of an AR* framework for photonics-based system design for future research


I. INTRODUCTION
UGMENTED reality (AR) is one of the hot topics in Industry 4.0.New opportunities are offered in nextgeneration electronics by displaying information through transparent screens, like AR devices, smart surgical glasses, and smart windows [1].AR has found broad applications like manufacturing and assembly [2]- [4], medical and care [5], transportation [6], maintenance [7], infrastructure and construction [8], and education and training [9].As classified by the European Union, AR is one of the main technologies that will drive the smart factory development [10].
Industry 4.0 (or 4 th industrial revolution) concept has been associated with 20+ technologies, which are relevant for large as well as small and medium sized enterprises (see Fig. 1) [11].AR is a key technology of Industry 4.0, in which virtual content is generated through computer and overlaid on real world environment.Software and hardware are key building blocks of AR [4].Inside the hardware, electronic based components and also photonic (light technology or light) based components, devices and systems play essential roles [2], [12], [13].
The industrial digitalization drive was already gearing up when COVID-19 pandemic started in late 2019.The response to the ongoing COVID-19 pandemic has also highlighted the importance of digitalization in all walks of life including the industry.Safe distancing, flexibility and agility are only some key requirements, which have been tackled through use of immersive technologies (AR/virtual reality (VR)/mixed reality (MR)), robotics (particularly human-machine collaboration), digital twins, big data and AI to name a few.[5].
The use of immersive technologies, in particular, has been changing the way industries operate, e.g. by taking out the paper-based work instructions which may contribute to cost savings, achieving sustainability targets and also providing changeability if the work instructions change for example [14].
However, the immersive technologies are still only beginning to be adopted even after several decades since their inception.The adoption of the immersive technologies face challenges mainly related to the hardware, software and organizational aspects [3], [4].
The light-and light-based technologies are critical in how the information is displayed and processed in AR [12], [15]- [23].This is particularly valid for the visualization technology of head-mounted devices (HMDs).Therefore, the photonics is of vital importance in AR system design.
The aim of this paper is to highlight challenges of AR systems and metrics of photonic components that can form the basis of a new AR* framework for photonics-based system design for future research.This was achieved through review of academic literature, an industrial survey, industrial experiments, and further analysis of the results that are presented in this article for the first time.
The rest of this article is organized as following.Section II of the article is concerned about AR, in general terms, and its relevance or role in Industry 4.0.Industrial applications of AR are reviewed in Section III.Section IV is focused on photonics in AR. Discussion is presented in section V, which discusses the challenges as well as presents the new AR* framework.Finally, conclusions are presented in section VI.

II. RELEVANCE OF AR IN INDUSTRY 4.0
After highlighting the integral part that AR plays within Industry 4.0, the central aspects influenced by photonics are described within this section.

A. What is Industry 4.0?
Industry 4.0 and other similar terms like Industrial Internet, or Internet 2025 are broad concepts which are built around the horizontal value network integration, the vertical integration within a value creation system, and the end-to-end integration across lifecycle of products and services [24].The vision of concepts like Industry 4.0 is to build cyber-physical systems (CPS).These systems aim to integrate the physical and the digital world with an uninterrupted flow of information towards independent subsystems.Thereby, enabling increased flexibility, autonomy and adaptability [25].
These concepts are powered by digital technologies, fueling a fourth industrial revolution, therefore the term Industry 4.0 was coined [24].Those digital technologies include, for example, cloud computing, additive manufacturing, blockchain, internet of things (IoT) soft-and hardware, 5G, Cobots, intelligent automation, AR and others [11].
While the term fourth industrial revolution might indicate otherwise, it does not only extend towards industrial manufacturing and how we produce things, like the previous three industrial revolutions.Instead, the concepts used extend up to the end-user, changing the customer experience with the product and the interaction between businesses and their customers [26].This not only includes physical products, but also explicitly services provided either in the context of a physical product, or the service industry in general [27].
Hence, one central goal of Industry 4.0 is not to eliminate humans from the value chain, but to better integrate humans into the complex systems built and to more flexibly integrate end-to-end value chains .While the progress in adopting these concepts is often attributed to large companies, Industry 4.0 is highly relevant for SMEs as well [11].Thus, in economic terms, Industry 4.0 technologies need to be scalable within SMEs as well as large enterprises.

B. What is AR?
While AR is often perceived as a technology only invented in the recent past, the cornerstones of the concept, the technology and the terminology were laid over 50 years ago [28].In general, AR describes the situation where a digital layer of information is optically superimposed with the real world.The user is able to interact with the information in a variety of different ways, depending on the system used.
While the formal definitions of AR vary, a widely used classification framework was proposed by Milgram et al. [29].This so-called reality-virtuality continuum defines two extremes: the reality without virtual augmentation and the virtuality without content from the real world, which is also called virtual reality (VR).Along this continuum, content from the real world and virtual elements are mixed to a varying degree.This so-called mixed reality (MR) incorporates AR and augmented virtuality (AV).The transition between AR and AV is not clearly defined along the continuum.
The term AV is usually used when enriching the virtual world by real world content.Here, the information presented to the user is principally virtual.This is for example the case, when VR glasses are used to display information altered by visual input captured by imaging sensors.
AR describes the situation where virtual information is optically overlayed over the real world.The information can either pre presented in different ways.Contextualized information is adapted to the real world, e.g., through the location of the user or depth information.In addition, information can be displayed through monocular devices or in "3D" stereoscopic binocular devices.Some researchers relate to AR only when the information is presented in 3D [12].
If an interaction between a digital object mapped within the physical space through sensory input is possible, then this is sometimes also called MR [30].While Milgram et.al [29] sees MR as the overarching terminology, MR is often also seen as extension of AR with the following distinction: • AR focuses on the real world and overlays a separate layer of information.The layer of digital information does not interact, but only extends the real world.
• MR also provides a layer of digital information but enables the interaction between digital information and the real world, including the user of the system.This enables the anchoring of 3D objects in the real world and interacting with it through gestures within the physical boundaries of the real world.This article is based upon recent research presented in [2]- [4], [14].A broader definition of AR has been utilized, which incorporates 2D and 3D solutions and the possibility to interact with contextualized information.However, the photonics community highlights, that the challenges provided by the high requirements of binocular devices providing contextualized 3D information are significantly higher [31], [32].
The architecture of AR systems entails a variety of subsystems.The function of these systems and their interaction in relation to data flow is shown in Fig. 2 and described in further detail below.
1) User Interface (UI) This sub-system enables the user to interact with the AR system.The most prominent technologies used are gesture recognition by hand tracking [33], directional tracking of the gaze [34], and speech recognition [35].A selection of prominent AR systems, described in more detail later in Table 1, are also controlled predominantly by these modes.
Fig. 2: Sub-systems of an AR system and their respective interaction towards the end-user (based on [2], [12]- [13]).Red-marked (1.User Interface, 2. Visualization Technology and 5. Sensor Systems) are the sub-systems with the highest impact of photonics-related technology.

2) Visualization Technology
The visualization technology is the cornerstone of an AR system, as it is the primary way to present information towards the user.In general, four main technologies are available: handheld devices (HHDs), static screens, projectors or HMDs.

3) Tracking
In order to place digital objects within the framework provided by the real-world, the tracking system needs to provide boundary conditions as reference points for the graphic rendering engine.In general, two technologies can be identified: marker-based systems and marker-less systems.
Marker-based systems use physical markers.These AR markers are often QR codes and encode information themselves.However, any physical object which has a fixed location can be used as identifier [39].As soon as the object cannot be decoded through image processing, however, the spatial information is lost as well.Environmental conditions, like dust, lighting, or wear and tear impact the processability.
Thus, marker-less systems are increasingly used [40]- [41].They are requiring more sophisticated hardware for depth mapping through stereoscopic sensors but are not dependent on pre-defined spatial encoders.
4) Sensor System AR systems have a broad variety of sensors to gather information about the user and the environment.Optical sensors, accelerometers and gyroscopes are most commonly used.

5) Processing Unit
While the HoloLens 1 and HoloLens 2 are completely standalone devices, the Magic Leap utilizes a computing unit wired to the HMD.This unit is then intended to be carried on the belt of the user.

6) Software
The software for the use-case can be developed with a variety of software suites.As the advent of 3D data processing and modelling was primarily driven by the computer game industry, those platforms are also commonly used for VR and AR applications outside the entertainment industry.Fig. 3 shows an academic use case for an experiment conducted by the authors of this article [3].In this experiment, a HoloLens 1 was used.The 3D environment was created with Unity.The interaction by the scene was managed by the HoloToolkit provided by Microsoft.This HoloLens application then ran stand-alone on the HoloLens 1.While authoring AR content is still challenging and time consuming [42]- [44], the entertainment industry provides a suite of proven techniques and software to develop use-cases.

C. How is AR relevant in Industry 4.0?
Digital technologies are increasingly interwoven in all industries.The interaction between human and machine, however, has stayed the same since the advent of mouse and keyboard [45].AR is the enabler for seamless interaction between humans, their surroundings and digital data.All other emerging technologies, highlighted in the beginning of this section, facilitate more flexibility or adaptability in their respective domains.AR, however, is the only of those technologies fully focused on improving the interaction between physical humans and the digital cyber-world and thus inherently facilitating the emergence of CPS.
While intelligent automation is a key technology of Industry 4.0, this initiative does not focus on 'dark factories' without humans (i.e., fully automated).The goal is to empower employees within those CPS by providing real-time and contextualized information.The augmented operator paradigm [46] sees AR as the key link to merge the digital and physical worlds through human vision, as the highest throughput sensory data acquisition system.AR is classified by the European Union as one of the main technologies that will drive the smart factory development [10] by enabling the seamless human-machine interactions (HMI) through visualizing interactive and contextual information.In order to facilitate collaboration and interaction between humans and production systems based on digital data, researchers focus on utilizing AR in industrial applications.
All the arguments made above for industrial-focused applications can also be made for all roles within the valuechain and along the product-lifecycle up to the consumer.Thus, AR needs to be inherently human centered [47] to be accepted and implemented / used successfully [4].
In addition, AR continues to be of increasing relevance due to technological advancement, digitalization and more recently due to the COVID-19 induced needs in different industries to introduce and/or improve communications and safety in a paperless environment.Examples of those use cases include training / service support / UK Ventilator Challenge (which produced 20-years' worth of ventilators in only 12-weeks [48].

III. INDUSTRIAL APPLICATIONS OF AR
AR has found several applications across industrial sectors, some of which are discussed in the following.
Design, Manufacturing and (Dis)Assembly -There is a lot of interest in applying AR in design, manufacturing and assembly, e.g., AR-collaborative design [49], AR-assisted factory layout planning, tele robotics (AR assisted robot collision-free path planning), augmented numerical control machining simulation, and in automotive and aerospace sectors [4], [50], [51].Hou et al. [52] studied a complex assembly task while comparing an AR static screen solution with paper-based instructions.Gattullo et al. [42] proposed to use AR manuals with reduced text while Syberfeldt et al. [53] proposed visual assembly guidance using AR.Neb and Strieg [54] proposed to use ARenhanced assembly instructions.
A recent research by Masood and Egger [3] at the University of Cambridge reported on results of industrial AR experiments using HoloLens (see Fig. 4).From technology point of view, there was a consensus that the user interface provided sufficient visibility of content, however the participants reported that weight was too high for 8-hr working shift.Tracking technology also needed improvement for field service for some participants but was sufficient for others.Health and safety aspect also received mixed views: some reported distraction issues, others raised regulation issues while some others raised no concerns.AR-assisted product disassembly sequence planning system, aiming for near real time interaction and multiple objectives, is also reported in the literature [55].Infrastructure and Construction -AR is beginning to change the way how infrastructure and buildings are designed, built and operated through digital engineering [56].Concept of Building Information Model has been there for some time in this sector, which is now progressing to next levels, and also being linked to developing digital twins that deal with creation of digital models of the physical infrastructure [48].AR has a role to play in such applications of advanced digital technologies.AR displays in such use cases include tablet and phone devices while location tracking may be done by barcodes, GPS, QR codes, image recognition, feature point recognition, edge based tracking, manometer tracking and RGB sensors [8].Crossrail, one of the biggest infrastructure projects in Europe has conducted small scale trials using AR, e.g. at Crossrail Custom House that demonstrated the potential value of AR to construction but also highlighted several factors and considerations which limit its suitability for large scale deployment, e.g.rapid AR technology advancements [8].
Maintenance -AR is increasingly finding applications in maintenance area [57].For example, tele maintenance is a popular area when combined with AR application.Aschenbrenner et al. [7] used 'ARTab' concept concentrating on off-site expert side of tele maintenance and ways to improve their situational awareness of the whole production process and peripheral machinery.Another example is of electromechanical equipment that has wide applications.Zhukovskiy and Koteleva [58] proposed an AR system for servicing such equipment using an AR service bus and a plant service bus.
Similarly, AR is finding its place in remote maintenance services [59] across sectors, e.g., aerospace, automotive, and plant machinery.Smith et al. [60] developed mobile MR tool for fault diagnosis of a three-phase power supply.
Transportation -Driverless cars -Li et al. [6] proposed an augmented autonomous driving simulation system using datadriven algorithms based on data from LiDAR and street cameras.This system augmented real-world images with a simulated traffic flow to create photorealistic simulation images and renderings.
Education and training -AR systems have found several applications in education and training with audience ranging from children to executives.Yamaguchi and Yoshikawa [9] proposed a tangible education system, with AR as a live, direct or indirect, real-world environment whose elements are augmented by the computer-generated sensory input like sound, video or graphics data constructing optical holographs.AR has also a crucial role in medical training.Barson et al. [61] reviewed seven of such applications which included laparoscopic surgical training, and MR training of neurosurgical procedures for example.VR and AR training also has application for emergency management in the built environments to enhance various processes before, during and after an emergency [62].Sebillo et al. [63] presented use of AR interfaces for on-site crisis preparedness.Post-training performance in AR/VR is typically comparable to pre-training performance [64].
Medical and care -Some medical surgeries are challenging as they depend on surgeon's knowledge of the optical visibility of surgical microscope.Kim et al. [5] reported on development of AR based optical coherence tomography (OCT) system integrated with surgical microscope.In this OCT system, depth information of the lesion site was captured and augmented to the ocular eyepiece of the intra-surgical system, providing a valuable aid to surgeons and conventional surgical procedures to enhance the accuracy and verification of the procedure.
Retail and others -With regards to shopping, the customer often feels lost due to the difficulty in finding a product, particularly in several large retail stores, such as shopping malls, sport or food stores.This makes a case for improving customers' shopping experiences using AR applications [65].El-Seoud and Taj-Eddin [66] presented an android mobile AR application for retail fashion shopping while Välkkynen et al. [67] also talked about mobile AR for retail environments.Esch et al. [68] presented results of a survey about anthropomorphism and AR in retail environment.
Other AR applications include smart tourism, where information about destinations, attractions and heritage sites can be provided [69], [70].

IV. HOW IS PHOTONICS RELEVANT IN AR?
This part of the article is concerned about photonics and its relation to AR -the main light-based components/devices, systems, and ideas that are required, or are available.It is observed that photonics is the science and technology based on light, and the photonics field includes optics and optoelectronics, i.e., components, devices and systems using light (i.e., photons).Researchers around the world are conducting research and development to reach the best photonics devices and systems to enable their use in a wide set of different applications across multiple sectors.
With the aim to analyze the academic literature on the intersection of photonics and AR, the following keyword search was conducted on 27 th March 2021 using the Web of Science knowledgebase: • Web of Science Search: TS= (("Augmented reality" OR "Mixed reality") AND ("photon*" OR "optic*")) • Web of Science Results=1,586 The top ten most cited articles were published in Presence-Virtual and Augmented Reality [12], [15] (#1 and #4), IEEE Transactions on Pattern Analysis and Machine Intelligence [16](#2), ACM Transactions on Graphics [17] (#3), Optics Express [18](#5) [5], IEEE Transactions on Medical Imaging (#6) [19], IEEE Conference on Computer Vision and Pattern Recognition (#7) [20], Medical Image Analysis (#8) [21], Journal of Display Technology (#9) [22], and Neurosurgery (#10) [23].It is evident from the titles of the publishing journals and conference that visualization, display and optics are key topics among others.Some key statistical analyses of the academic literature are presented in Figs.5-6.In the following, the sub-systems that are being impacted the highest by photonics-related technologies and the available technologies, the requirements placed upon AR systems and their relation towards photonics-based components are described.Fig. 2 shows in red the sub-systems with a high proportion of photonics components.

A. Visualization Technology
The technology of displays for HHDs, like mobile phones, or projectors are continuously evolving and improving.However, a recent systematic literature review, showed that HMDs are the most common visualization technology used in AR applications [2].In addition, this is supported by an industry survey conducted recently [3].In that survey, 58% of all participants highlight that effective information access is a key advantage of the adopting AR in industry.HMDs offer two distinct advantages: (a) providing a higher immersion by the information layer, and (b) providing hands-free application for the user, which is especially useful in industrial applications.Thus, Table 1 describes a selection of state-of-the-art AR HMDs with varying optical solutions that are currently available.
Only recently, miniaturization facilitation increased computing power for 3D rendering engines while advances in optical systems made it possible to start developing consumer and industrial grade systems for 3D systems with interactable objects.These are currently the most advanced headsets in terms of hardware, especially optical hardware [31].This way of visually presenting digital information has the potential to be the next computing platform after desktop or laptop pcs and mobile computing devices like tablets.
The optical parts of HMDs with the highest effect on the necessary photonics-related technology are usually built from the following sub-systems (see Fig. 2) [2], [12]- [13], [32]: a) Display Engine The display engine generates the light for the AR content.Those micro-displays are based on a variety of technologies.State-of-the art stereoscopic HMDs currently utilize mainly liquid-crystal-on-silicon (LCoS) or laser beam scanner (LBS), as shown in Table 1.In literature, digital light processer (DLP), micro-organic light-emitting diode (mOLED), and micro lightemitting diode (mLED) are named as the most common display engines.The latter two are the newest technological evolutions in the field of display engines [32].
Besides the current HMD solutions, transparent displays have central role in next generation optoelectronics in the age of AR, wearable electronics and IoT.The transparency for LEDs significantly enhances their application by displaying visual information without affecting their original appearance and functionality.As outstanding luminance and transparency are essential for AR "see-through" displays to show vivid images over clear background view, Choi et al. [71] reported transparent quantum dot light-emitting diodes (Tr-QLEDs) with high brightness, transmittance and an ultra-thin form factor [72].
b) Optical combiner Contrary to VR glasses, AR systems need to combine the light of the display engine with the light of the real environment.This task is especially challenging for HMDs.For HHDs, the light from the environment is captured by the camera and an overlayed image is shown through the screen of the device.
For HMDs, the optical combiner is highly complex and often the most expensive part of the AR system architecture.The combiner also significantly influences system performance.[31].In general, the following types of combiners are used: half-mirrors, beam splitters (birdbath), prisms, and waveguides.Here, the state of the art is the usage of waveguides in commercially available devices (see Table 1).Waveguides are essentially prisms with a high number of internal reflections to transfer the light from the optical engine to the eye.Input couplers are used to reflect the light into the waveguide material at the right angle.Output couplers are then used to couple the light out of the waveguide and into the optical path of the light from the real environment.
In the future, dense optical waveguide arrays might be used for complex optical routing systems [73].The goal is to have this array integrated into the glass in front of the eye of the user without the need for more complex optical systems.This lensfree retinal projection concept could reduce the form-factor, weight and energy consumption.However, it is currently in an early research stage, only providing a working solution for a wavelength of 532nm and not a continuous light spectrum [74].Notaros et al. [75] suggested an optical phased array in passive near-eye projectors to generate holographic images through liquid-crystal based modulators.This technology promises a compact form-factor while maintaining low-cost and robustness, as no mechanical parts are involved [75].
Two highly relevant metrics of optical combiners are their efficiency and the field of view they allow.Both of these metrics are highly relevant for users, as Xiong et al. [76] and Lee et al. [77] have shown in their conclusions and recommended active areas of research for further improvement.

B. User Interface
All the systems listed in Table 1 utilize gaze tracking to interact with the users [78].However, in general, this is done by measuring the head's position and movement.The HoloLens 2 enhances gaze tracking through eye gaze tracking [79].As the optics for eye tracking is usually a finite conjugated system, contrary to the optics of the display engine where two optical systems are necessary.

C. Sensor System
For advanced stereoscopic AR HMDs, it is crucial to enable interaction with digital objects, and understand the systems' environment.Depth scanner cameras, head-tracking cameras to provide 6DOF, and gesture sensors are possible photonics related sensor systems utilized in AR.
Currently mainly ultrasonic or infrared stereoscopic systems are used for depth perception (see Fig. 2).For future applications, compact 3D imaging sensors operating as LiDAR systems might replace current solutions [80].This proposed system also relies on the monolithic integration of photonic and electronic circuits, similar to the solution discussed earlier concerning optical combiners [73].

V. SO WHAT? (DISCUSSION)
The foregoing sections have established the following: (a) Industry 4.0 is of high relevance in a broad variety of industries.It facilitates horizontal and vertical digital information flow along value chains up to the end-customer.(b) AR is a tool for humans to obtain (contextualized) information and interact with the physical and digital worlds through a superimposed layer of digital information.(c) As AR relies on optical systems, a variety of its sub-systems is highly intertwined with optics, photonics and the progress made in those disciplines.This is especially true for the visualization technology of HMDs.(d) AR has a broad variety of applications possibly facilitating significant technological and / or commercial benefits.Thus, the question arises why AR is not widely adopted by industry or consumers.
Adoption of AR poses a wide variety of challenges and success factors necessary for its successful implementation.These topics have been confirmed by literature [2], an industrial survey [4] and industrial experiments with operators and nonoperators [3].In general, a significant proportion of mentioned success factors is related to technological issues [4], when categorizing the issues based on the Technology-Organization-Environment (TOE) framework [81].Hence, technology hardware readiness is significantly relevant with the implementation success of AR systems [4].
Fig. 7 shows the AR success factors identified through an industrial survey, where the participants covered a wide range of industries (based on [4]).Hence, the success factors provide a broad perspective.As the performance of AR systems is heavily impacted and influenced by photonic components and sub-systems (see Fig. 2), the photonics industry needs to improve upon these components to facilitate successful industrial AR applications.Those success factors are not exclusively, but mainly related to the challenges of AR systems like HMDs and are translated to metrics highly influenced by the optics of the AR system.These relationships are shown in Table 2 and are discussed in the following.
Ergonomics -Comfort while wearing the HMD is important for professional applications, as the system would need to be worn 8 hours a day, every day.The weight of the HMD was identified as a significant hindrance towards industrial adoption [4].An experiment conducted with shopfloor-operators utilizing a HoloLens 1 clearly showed that the weight of 579g was too much for wearing the device for 8 hours [3].Concerning the field of view, literature highlights this challenge more frequent than industry [4].This was also supported by the experiments conducted with shopfloor operators [3].Thus, reducing the weight to improve wearability should have a higher priority compared to improving the field of view for industrial HMDs.
Usability of user interface (UI) -The usability of the UI depends on accurate gaze tracking and accurate 3D depth tracking of the position of the user's hands.Especially the sensor systems used to facilitate eye gaze tracking and depth perception are relevant from a photonics perspective.Further developments relate to compact LiDAR systems [75], [80].Eye gaze tracking, like in the HoloLens 2, will further improve on the accuracy of the user input detection [78], [79].
Visibility of information -While the brightness of the screen seems to be sufficient for indoor applications [3], for outdoor applications, the brightness needs to be approximately 20 times higher to ensure a sufficient visibility [32].Hence, the light efficiency of the display engine, its contrast and the efficiency of the optical combiner are crucial in delivering improved results.The depth resolution [79] is related to the visibility concerning the positioning of virtual objects in the physical space.Accurate placing and tracking needs depth measurement.While the HoloLens 1 projects all objects at a focal plane 2m in front of the user [79], further development will likely improve on that feature to enable an advanced immersive experience with decreased eye strain.
Hardware robustness -Especially in industrial and professional application, sufficient robustness and the adherence to safety regulations are imperative [3].HSE considerations and the environment (dust, vibrations, handling) need to be taken into account when designing AR HMDs.From a photonics perspective, this concerns especially the form factor, as the packing possibilities of the optical system are highly dependent on the size of the components.
Battery life -For the casual use in entertainment, a battery life of 2-3 hours is likely to be sufficient.However, when utilizing AR in industrial or professional applications, the battery life needs to cover the duration of at least 8 hours.As listed in Table I, only the Realware HMT-1 brings these capabilities.Yet, this device is not providing a stereoscopic experience with interactive digital objects.In order to optimize the battery life, two routes should be taken from a photonics perspective: (a) Increasing the efficiency of the optical components to decrease the power necessary for a high visibility.(b) Decreasing the form factor of the optical components to allow additional room and a higher proportion of the "weight budget" to be taken by the battery.
In order to design HMD systems with enhanced acceptance by users in industry, improvements need to be made especially in the areas of the display engine and the optical combiner, which are the core components of HMDs.Fig. 8 shows a novel AR* framework for photonics-based system design.This framework maps the success factors onto the photonic components to receive rankings for them, which shows the improvement areas with the highest potential impact in facilitating the success factors.The rankings are based on the industrial survey (see Fig. 7) and the relation shown in Table 2.These are derived as follows.
(a) The success factors impacted by photonic components receive a weight based on their relevance (see Fig. 7).
(b) The weight is multiplied with the number of metrics being influenced by the respective area.
(c) The products are summed up.The photonic component with the highest result has the highest importance and thus the lowest rank (rank 1).The AR* framework may be helpful in making system design decisions like trade-offs or prioritization e.g., better coupler or better light engine.However, this framework needs to be further tested in future research to provide industrial use cases.
Improving upon AR HMDs will facilitate its wide usage.As shown in section III, the applications are broad, and use-cases can be identified in every industry.The Industry 4.0 paradigm pushes the integration of data along value chains.AR is the prime technology to seamlessly include humans into this concept and facilitate the interaction with data while engaging with the real world.

VI. CONCLUSIONS
In conclusion, we have demonstrated that photonics is critical in delivering AR within Industry 4.0.Based on previous research, we also contributed by providing new analyses in the form of: (i) the review of academic literature combining AR and photonics, (ii) the review of HMDs with details of optical solutions and their properties; (iii) the challenges for the design of future AR systems and their links to metrics of photonics components, and (iv) AR* framework for photonics-based system design.mapping AR success factors towards photonic components.Future research needs to focus on embedding and further testing the elements of the AR* framework in order to build resilient, changeable and elastic factories of the future.The weight of the success factor was determined by the relevance (see Fig. 7).Table 2 provides the multiplicator between success factor (weight) and photonic components (rank).The sum provides the rank.

Fig 1 .
Fig 1. Benefits (a) and challenges (b) of various Industry 4.0 technologies for UK SMEs on a scale of 1 (least beneficial or least challenging) to 5 (most beneficial or most challenging) [5].

Fig 3 .
Fig 3. Scene creator in Unity with the relevant C# scripts to enable interaction of the scene with the real world and the user.

Fig. 5 :
Fig. 5: Web of Science categories of the results.

Fig. 7 :
Fig. 7: AR success factors identified through an industrial survey.Average normalised importance rank and count for each of the success factor items, based on a multiple-choice question following a ranking question.The success factors highlighted by red rectangles are those related to optical systems of the AR device.(Based on[4]).

Fig. 8 :
Fig.8: AR* framework for photonics-based system design.The framework maps AR success factors towards photonic components.The weight of the success factor was determined by the relevance (see Fig.7).Table2provides the multiplicator between success factor (weight) and photonic components (rank).The sum provides the rank.

TABLE I HMDS
WITH DETAILS OF OPTICAL SOLUTIONS AND THEIR PROPERTIES

TABLE 2 IDENTIFIED
CHALLENGES FOR AR SYSTEMS RELATED TO METRICS OF PHOTONIC COMPONENTS.
Mapping of success factors towards photonic components based on the relevance & the influence