22-gar-age_a feedback-enabled blended ecosystem for vehicle health monitoring

1、Proceedings of the 10th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI 18), September 2325, 2018, Toronto, Canada.Sitara Shah, Snigdha Petluru, Rishabh Singh, Saurabh SrivastavaConduent Labs India Bangalore, Indiasitara.shah; snigdha.p

2、etluru; rishabh.singh; ABSTRACTStandard vehicle maintenance activities can be challenging, time-consuming, error-prone, and expensive. While there is a lot of innovative work that has incorporated latest technologies to provide newer forms of interaction between users a

3、nd vehicles, there has been less inclination towards utilizing these technologies to enhance activities like vehicle maintenance. The ability to draw parallels simultaneously from physical interaction with vehicles and analysis of recorded data is vital to support prompt and effective decision-makin

4、g. To blur the disparity between these real and virtual worlds, we present gAR-age- an ecosystem that enables maintenance personnel to interact with both worlds in a common setting. By learning from historical changes in vehicular components, user behavior, and feedback, this blended ecosystem allow

5、s multi-channel communication among users, featuring personalized contextual insights, thereby enabling users to make data-driven decision on the fly.Author KeywordsBlended Reality; Human-Machine Interaction; Contextual design; Information visualization; User Experience; Personalization; Multi-modal

6、 fusion; Gestural input.CCS ConceptsHuman-centered computing Interactive systems and tools; Human-centered computing Mixed / augmented reality; Human-centered computing Gestural input.INTRODUCTIONStandard vehicle maintenance activities can be challenging, time-consuming, error-prone, and replete wit

7、h sky-rocketing expenses, both anticipated and unexpected. A recent report by APTA 22 shows an annual investment of over 3,000 Million USD on vehicle maintenance activities, including administration, inspection, maintenance, and servicing of personal and revenue vehicles. Transit agencies employseve

8、ral maintenance personnel to perform these activities; however, defects and incident records are not always documented digitally. There is often a chance of human error in the transfer of information from one person to another, leading to overlooked defects and unsafe decision making. Furthermore, d

9、ue to the lack of a common ecosystem to support communication across departments and agencies, there is a delay in forewarning supervisors, allocating tasks, and following up on work orders.A multitude of solutions, including innovative work practices, computer programs, and advanced robotics, has b

10、een created to focus on the optimization of vehicle maintenance activities. But these solutions often leave the user responsible for connecting the dots and at the right conclusions by gathering information from multiple sources. This leads to a higher expectation of the users skills and potentials,

11、 thereby placing a higher cognitive load on them. Furthermore, the flow of information through the system is slower due to the inability of the users to interact with others within the environment which is essential for instantaneous decision-making. There is a need for a common ecosystem that knits

12、 together insights from multiple systems that can communicate with each other.Any technological solution that attempts to provide a common ecosystem for personnel, and provisions for historical analysis of vehicle behavior, must also account for user preferences, roles, and priorities to provide cus

13、tomized insights. This paper proposes a blended ecosystem to enhance traditional practices of monitoring, verification, and correction in maintenance activities seamlessly, by connecting the user to both the system and the real-time environment, where the user is not just a passive consumer of infor

14、mation, but an active contributor as well.Using advanced analytics, computer vision, and machine learning capabilities, the proposed ecosystem renders immersive experiences to users by assimilating insights from multiple data sources to provide a holistic view of the health of the vehicle on a blend

15、ed interface. This ecosystem is supported by an immersive analytics suite that enables users to provide real-time feedback and renders personalized insights to them on-the-fly by learning from the disparities in the ecosystem and captured data. It also ensures multi- channel communication among the

16、users and supports contextual interactions between the users and the environment.Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that cop

17、ies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and

18、/or a fee. Request permissions from P.AutomotiveUI 18, September 23 25, 2018, Toronto, ON, Canada 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-5946-/10.1145/3239060.3239075268Session 7: Special ApproachesAutomotiveUI 18, Toronto, CanadaIn the following

19、 sections, we examine the related works in this space and introduce the proposed blended ecosystem. We also discuss the significance of computer vision in enabling this ecosystem and then delve deeper into the immersive analytics suite, where we define Blended User Feedback, Longitudinal Learning, a

20、nd Personalization engines that support this ecosystem. Finally, we discuss the implications of our design and outline future directions for expanding this solution.RELATED WORKSIn this section, we describe traditional maintenance activities and the challenges involved in the performance of these ac

21、tivities. Then, we discuss existing solutions that have attempted to optimize this process. Finally, we explain the concept of personalization and explore its prevalent benefits identified across various fields.Vehicle Health MonitoringUnexpected on-road vehicular incidents lead to unnecessary expen

22、ses, traffic jams, loss of time, and customer dissatisfaction. But corrective maintenance systems only help when a fault is successfully detected and are accompanied by high costs because of the vehicle being out of operation 16, 17. To avoid this, agencies perform preventive maintenance activities

23、from time-to-time in order to assess the health of the vehicle. Preventive maintenance systems focus on fixed schedules to run proactive checks and distribute costs evenly across time 16. Traditionally,attention to this tangent and create an improved form of interaction.Vision Based Monitoring Solut

24、ionsThere is an abundance of methods and systems that enable technology-powered solutions to vehicle health monitoring, although most work extensively touches upon aviation improvement. One such survey 14 lists a comprehensive set of works that focus on improving engine maintenance in commercial air

25、crafts. There is an abundance of patent literature that focuses on systems and software to support vehicle health monitoring 8, 33, but all of these patents focus on desktop or laptop-based solutions for personnel. The drawback with this form of monitoring is that while personnel need to be physical

26、ly present in front of vehicles and visually inspect them, information about the vehicles health can only be obtained from behind a desk, causing additional overhead for personnel to travel within the facility. There also exist hand-held solutions that attempt to overcome this drawback 24, 29; howev

27、er, these solutions suffer from a myriad of challenges as well, the most significant being their inability to merge real objects to corresponding virtual information on the fly.To overcome this challenge, the mixed reality space can serve as a strong inspiration, backed by literature that has explor

28、ed its incorporation into various industries. Mixed reality is a broad spectrum that defines the space between and including reality and virtuality. This space comprises of various hybrid technologies like Augmented Reality, Virtual Reality, and Blended Reality. Augmented Reality (AR) refers to tech

29、nology focused primarily on generating and overlaying virtual imagery in a real environment through a mediated device 30. In contrast, Virtual Reality (VR) emulates entirely virtual experiences, where the goal is to allow a user to experience a virtual world. The industry has been shifting towards w

30、ider adoption of AR over VR 10, although historically, the first system developed in the 1960s was meant to cater to both AR and VR 23. Because of the provision for both marker-based and marker-less technologies, AR is a more widely used technology by industries in their workflows.Augmented Reality

31、solutions have already created a mark in the vehicle monitoring space. A user study has demonstrated the benefit of AR in helping maintenance personnel locate tasks faster and diminish physical strain on their health 33. In addition, it can help enterprises manage communication about changes needed

32、in a vehicle 3, with industry leaders like BMW adopting AR techniques in wearable devices to support maintenance activities 21.Novel interactions in AR include the ability to show time- sensitive information 21 and context-aware speech that can be utilized during maintenance 35. In addition, AR has

33、been found as beneficial in training and learning-based activities 31. There have also been studies measuring the potential advantages that mixed reality solutions bring to vehicle maintenance than other conventional solutions. Forseasonalmaintenanceactivities,beittechnical,administrative, or manage

34、rial, are planned using guidelinesprovided by the manufacturer or part vendors 4, with a scarce understanding of the current or historical health of the vehicle. There are also additional factors, like whether the landscape is urban or rural, which can affect how preventive maintenance is practiced

35、9.Two traditional methods of vehicle maintenance arepredominant in the industryin-house and third-partycontracting. In-house maintenance occurs on-site anddemands more resources in terms of both costs and manpower, thereby requiring the infrastructure to be situated in a more urban setting 25. In co

36、ntrast, third-party maintenance contracts involve lower costs because facilities can be established in rural settings; in hindsight, this can lead to inconsistencies in terms of the quality owing to the scarcity of skilled labor 9. In addition, the number of vehicles in an agency can also determine

37、costs, as research shows that nearly 15 to 20% of an agencys budget needs to be allocated for maintenance when there are less than 100 vehicles, whereas, this increases to the range of 20 to 30% as the number of vehicles exceeds 100 17.The current focus of research in vehicle maintenance is spread a

38、cross increasing reliability, analyzing faults, improving the connection between maintenance and production 16, and remote diagnostics 32. However, research has been failing to address one key directionmulti-channelcommunicationamongstmaintenancepersonnel within an environment. Our work aims to draw

39、269Session 7: Special ApproachesAutomotiveUI 18, Toronto, Canadainstance, one study showed that AR was more effective than workplace signs and manuals in helping staff assemble vehicles 19 and fix motherboards 6. Furthermore, mixed reality solutions have aided maintenance personnel in identifying in

40、dividual tasks faster by minimizing the number of head movements, validated by user studies 34.However, all of the above works deal with specific use cases without focusing on developing a generalizable construct for the industry. Furthermore, while quantitative measurements are considered as valid

41、indicators, most solutions are validated only by qualitative measures that could be biased when considered in isolation. There is no effort made to capture and analyzing user feedback and behavior while they use this system. Given the drawbacks, we explore thebetween the user and the system, or (c)

42、behavioral context of the captured user interaction.There have been numerous indications about the benefits of personalization. In addition to gaining attention and retaining customer loyalty 2, personalization can help enhance marketing strategies 18, influence online choices 36 and nudge users to

43、perform certain actions, like responding to an email 2. Hence it is easy to see the benefits of incorporating this phenomenon to influence the behavior of maintenance personnel and provide them with a more relevant experience. By including the ability to personalize insights, we provide a unique off

44、ering of an adaptive blended ecosystem that can redefine how maintenance activities are traditionally performed.PROPOSED SOLUTIONAs reflected by our review of related literature, there are shortcomings in current technological solutions that need to be tackled in order to optimize workflows. We prio

45、ritize these shortcomings into 3 crucial factors that have influenced the design of our solution:Data-Driven Decision Making Most systems require the user to switch between multiple contexts and gather information from various sources. Hence, it is important to enable users to make decisions by havi

46、ng an assimilated single source from which they obtain not just data but also insights from this data.possibility of incorporating a deviant Reality, into a possible solution.concept, Blendedwhere the real and vi whereusers can interact with both real and virtual objects in a single realm in what ca

47、n be perceived as a. The merits of Blended Reality in promotinglearning have also been validated by a study, where participants experienced improved communication andlearning in both remote and face-to-face settings using a blended reality environment 26.However, Blended Reality applications lack th

48、e ability to draw user attention through subtle cues. Additionally, such applications only cater to users who are merely consumers of information, as opposed to allowing them to also contribute to the knowledge within the blended reality environment. Our work addresses this gap in the literature and

49、 provides a channel for collecting, analyzing and interpreting feedback captured from users seamlessly through blended reality.The Potential of PersonalizationThe notion of personalization relies on the presumption of hidden preferences of users that can be learnt to provide individualized solutions

50、 or objects of interest 5. While personalization is accompanied by additional costs, the benefits outweigh the additional incurred expenses 15. Introduced in the 1880s 27, personalization is an established and endearing phenomenon in the marketing industry 5, and has seen rapid adoption by several d

51、isciplines including Banking 12, E-commerce 11, and Search Engines 28.Personalization can occur in many ways, either implicitly (non-invasively) or explicitly (invasively) by gathering insights from the system and the user 1. It can often be reflected in the form of change in content, interface, int

52、eraction, and features, and can be targeted towards either an individual or a group of individuals who are determined to be similar in some sense 13.Three distinct types of personalization are outlined 20, where it is claimed that personalization can be driven by (a) interests and preferences of the

53、 user, (b) transactionsFaster Contextualization and ComprehensionInaddition to providing insights, it is also important to ensure that users have sufficient context to understand theseinsights. Most systems take a one-solution-fits-all approach towards this problem and provide a static interaction f

54、or all users. Our solution overcomes this drawback by providing context-driven information that is displayed through visualizations on the blended interface. This helps in faster information comprehension while learning from historical user interaction with the realenvironmentallowsthepersonalizatio

55、nandcontextualization of this information.Multi-Channel Interaction - The third most important quality that is lacking in blended reality is the lack of medium that allows interactions amongst users within an environment. Most systems isolate users, leading to increased delays in communication and p

56、rioritization of tasks. Our system overcomes this drawback by capturing user feedback instantaneously and allowing users to share a common context about the environment through contextual conversations.To summarize, we propose gAR-age, a solution that advocates for the notion of a blended ecosystem

57、an extension to the concept of Blended Reality, which attempts to connect maintenance personnel with the system as well as the environment, where they can interact with a unified270Session 7: Special ApproachesAutomotiveUI 18, Toronto, CanadaFigure 1. Architecture of the gAR-age Ecosystemecosystem t

58、hat blends real-world objects with contextual virtual information.Figure 1 depicts the architecture of gAR-age, where all interactions between the user, system, and environment can occur in-site by means of a hand-held device. The general flow of this ecosystem can be summarized as:1. The user scans an object in the real-world through the hand-held device.2. This scanned object is loaded into an extract database that serves as an intermediate repository of interaction.3. The Computer Vision Engine identifies the object and extracts key insights about it by passing this identity in addi