evaluating-usability-aspects-mixed-reality-solution-for-immersive

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

In medicine or industry, the analysis of high-dimensional data sets is increasingly required. However, available technical solutions are often complex to ...

Ulm University

Tinnitus, the phantom perception of sound without a corresponding external sound, is a frequent disorder which causes significant morbidity. So far there is no treatment available that reliably reduces the tinnitus perception. The research is hampered by the large heterogeneity of tinnitus and the fact that the tinnitus perception fluctuates over time. It is therefore necessary to develop tools for measuring fluctuations of tinnitus perception over time and for analyzing data on single subject basis. However, this type of longitudinal measurement is difficult to perform using the traditional research methods such as paper-and-pencil questionnaires or clinical interviews. Ecological momentary assessment (EMA) represents a research concept that allows the assessment of subjective measurements under real-life conditions using portable electronic devices and thereby enables the researcher to collect longitudinal data under real-life conditions and high cost efficiency. Here we present a new method for recording the longitudinal development of tinnitus perception using a modern smartphone application available for iOS and Android devices with no costs for the users. The TrackYourTinnitus (TYT) app is available and maintained since April 2014. A number of 857 volunteers with an average age of 44.1 years participated in the data collection between April 2014 and February 2016. The mean tinnitus distress at the initial measurement was rated on average 13.9 points on the Mini-Tinnitus Questionnaire (Mini-TQ; max. 24 points). Importantly, we could demonstrate that the regular use of the TYT app has no significant negative influence on the perception of the tinnitus loudness nor on the tinnitus distress. The TYT app can therefore be proposed as a safe instrument for the longitudinal assessment of tinnitus perception in the everyday life of the patient.

Measuring the Moment-to-Moment Variability of Tinnitus: The TrackYourTinnitus Smart Phone App.

Tinnitus is a common medical condition which interfaces many different disciplines, yet it is not a priority for any individual discipline. A change in its scientific understanding and clinical management requires a shift toward multidisciplinary cooperation, not only in research but also in training. The European School for Interdisciplinary Tinnitus research (ESIT) brings together a unique multidisciplinary consortium of clinical practitioners, academic researchers, commercial partners, patient organizations, and public health experts to conduct innovative research and train the next generation of tinnitus researchers. ESIT supports fundamental science and clinical research projects in order to: (1) advancing new treatment solutions for tinnitus, (2) improving existing treatment paradigms, (3) developing innovative research methods, (4) performing genetic studies on, (5) collecting epidemiological data to create new knowledge about prevalence and risk factors, (6) establishing a pan-European data resource. All research projects involve inter-sectoral partnerships through practical training, quite unlike anything that can be offered by any single university alone. Likewise, the postgraduate training curriculum fosters a deep knowledge about tinnitus whilst nurturing transferable competencies in personal qualities and approaches needed to be an effective researcher, knowledge of the standards, requirements and professionalism to do research, and skills to work with others and to ensure the wider impact of research. ESIT is the seed for future generations of creative, entrepreneurial, and innovative researchers, trained to master the upcoming challenges in the tinnitus field, to implement sustained changes in prevention and clinical management of tinnitus, and to shape doctoral education in tinnitus for the future.

Innovations in Doctoral Training and Research on Tinnitus: The European School on Interdisciplinary Tinnitus Research (ESIT) Perspective.

Many research domains still heavily rely on paper-based data collection procedures, despite numerous associated drawbacks. The QuestionSys framework is intended to empower researchers as well as clinicians without programming skills to develop their own smart mobile apps in order to collect data for their specific scenarios.

Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study.

In the recent years, there has been an increasing interest in the potential of internet- and smartphone-based technologies for the support of tinnitus patients. A broad spectrum of relevant approaches, some in the form of studies, others in the form of market products, have been mentioned in literature. They include auditory treatments, internet-based Cognitive Behavioral Therapy (iCBT), serious games, and questionnaires for tinnitus monitoring. The goal of this study is to highlight the role of existing internet-based and smart technologies for the advancement of tinnitus clinical practice: we consider contributions that refer to treatments and diagnostics, and we include contributions refering to self-help measures. We elaborate on the potential and challenges of such solutions and identify constraints associated to their deployment, such as the demand for familiarity with internet-based services and the need to re-design interactive services so that they fit on the small surface of a smartwatch.

Review of Smart Services for Tinnitus Self-Help, Diagnostics and Treatments.

Visual analytics are becoming increasingly important in the light of big data and related scenarios. Along this trend, the field of immersive analytics has been variously furthered as it is able to provide sophisticated visual data analytics on one hand, while preserving user-friendliness on the other. Furthermore, recent hardware developments such as smart glasses, as well as achievements in virtual-reality applications, have fanned immersive analytic solutions. Notably, such solutions can be very effective when they are applied to high-dimensional datasets. Taking this advantage into account, the work at hand applies immersive analytics to a high-dimensional production dataset to improve the digital support of daily work tasks. More specifically, a mixed-reality implementation is presented that will support manufacturers as well as data scientists to comprehensively analyze machine data. As a particular goal, the prototype will simplify the analysis of manufacturing data through the usage of dimensionality reduction effects. Therefore, five aspects are mainly reported in this paper. First, it is shown how dimensionality reduction effects can be represented by clusters. Second, it is presented how the resulting information loss of the reduction is addressed. Third, the graphical interface of the developed prototype is illustrated as it provides (1) a correlation coefficient graph, (2) a plot for the information loss, and (3) a 3D particle system. In addition, an implemented voice recognition feature of the prototype is shown, which was considered to be being promising to select or deselect data variables users are interested in when analyzing the data. Fourth, based on a machine learning library, it is shown how the prototype reduces computational resources using smart glasses. The main idea is based on a recommendation approach as well as the use of subspace clustering. Fifth, results from a practical setting are presented, in which the prototype was shown to domain experts. The latter reported that such a tool is actually helpful to analyze machine data daily. Moreover, it was reported that such a system can be used to educate machine operators more properly. As a general outcome of this work, the presented approach may constitute a helpful solution for the industry as well as other domains such as medicine.

Dimensionality Reduction and Subspace Clustering in Mixed Reality for Condition Monitoring of High-Dimensional Production Data.

The mobile phone app, TrackYourStress (TYS), is a new crowdsensing mobile health platform for ecological momentary assessments of perceived stress levels.

Exploring the Time Trend of Stress Levels While Using the Crowdsensing Mobile Health Platform, TrackYourStress, and the Influence of Perceived Stress Reactivity: Ecological Momentary Assessment Pilot Study.

The management and comprehension of business process models are of utmost importance for almost any enterprise. To foster the comprehension of such models, this paper has incorporated the idea of a serious game called Tales of Knightly Process.

Learning to Read by Learning to Write: Evaluation of a Serious Game to Foster Business Process Model Comprehension.

To deal with drawbacks of paper-based data collection procedures, the QuestionSys approach empowers researchers with none or little programming knowledge to flexibly configure mobile data collection applications on demand. The mobile application approach of QuestionSys mainly pursues the goal to mitigate existing drawbacks of paper-based collection procedures in mHealth scenarios. Importantly, researchers shall be enabled to gather data in an efficient way. To evaluate the applicability of QuestionSys, several studies have been carried out to measure the efforts when using the framework in practice. In this work, the results of a study that investigated psychological insights on the required mental effort to configure the mobile applications are presented. Specifically, the mental effort for creating data collection instruments is validated in a study with N = 80 participants across two sessions. Thereby, participants were categorized into novices and experts based on prior knowledge on process modeling, which is a fundamental pillar of the developed approach. Each participant modeled 10 instruments during the course of the study, while concurrently several performance measures are assessed (e.g., time needed or errors). The results of these measures are then compared to the self-reported mental effort with respect to the tasks that had to be modeled. On one hand, the obtained results reveal a strong correlation between mental effort and performance measures. On the other, the self-reported mental effort decreased significantly over the course of the study, and therefore had a positive impact on measured performance metrics. Altogether, this study indicates that novices with no prior knowledge gain enough experience over the short amount of time to successfully model data collection instruments on their own. Therefore, QuestionSys is a helpful instrument to properly deal with large-scale data collection scenarios like clinical trials.

Measuring Mental Effort for Creating Mobile Data Collection Applications.

The increasing prevalence of smart mobile devices (e.g., smartphones) enables the combined use of mobile crowdsensing (MCS) and ecological momentary assessments (EMA) in the healthcare domain. By correlating qualitative longitudinal and ecologically valid EMA assessment data sets with sensor measurements in mobile apps, new valuable insights about patients (e.g., humans who suffer from chronic diseases) can be gained. However, there are numerous conceptual, architectural and technical, as well as legal challenges when implementing a respective software solution. Therefore, the work at hand (1) identifies these challenges, (2) derives respective recommendations, and (3) proposes a reference architecture for a MCS-EMA-platform addressing the defined recommendations. The required insights to propose the reference architecture were gained in several large-scale mHealth crowdsensing studies running for many years and different healthcare questions. To mention only two examples, we are running crowdsensing studies on questions for the tinnitus chronic disorder or psychological stress. We consider the proposed reference architecture and the identified challenges and recommendations as a contribution in two respects. First, they enable other researchers to align our practical studies with a baseline setting that can satisfy the variously revealed insights. Second, they are a proper basis to better compare data that was gathered using MCS and EMA. In addition, the combined use of MCS and EMA increasingly requires suitable architectures and associated digital solutions for the healthcare domain.

Combining Mobile Crowdsensing and Ecological Momentary Assessments in the Healthcare Domain.

Smartphones containing sophisticated high-end hardware and offering high computational capabilities at extremely manageable costs have become mainstream and an integral part of users' lives. Widespread adoption of smartphone devices has encouraged the development of many smartphone applications, resulting in a well-established ecosystem, which is easily discoverable and accessible via respective marketplaces of differing mobile platforms. These smartphone applications are no longer exclusively limited to entertainment purposes but are increasingly established in the scientific and medical field. In the context of tinnitus, the ringing in the ear, these smartphone apps range from relief, management, self-help, all the way to interfacing external sensors to better understand the phenomenon. In this paper, we aim to bring forth the smartphone applications in and around tinnitus. Based on the PRISMA guidelines, we systematically analyze and investigate the current state of smartphone apps, that are directly applied in the context of tinnitus. In particular, we explore Google Scholar, CiteSeerX, Microsoft Academics, Semantic Scholar for the identification of scientific contributions. Additionally, we search and explore Google's Play and Apple's App Stores to identify relevant smartphone apps and their respective properties. This review work gives (1) an up-to-date overview of existing apps, and (2) lists and discusses scientific literature pertaining to the smartphone apps used within the context of tinnitus.

Smartphone Apps in the Context of Tinnitus: Systematic Review.

Smart sensors and smartphones are becoming increasingly prevalent. Both can be used to gather environmental data (e.g., noise). Importantly, these devices can be connected to each other as well as to the Internet to collect large amounts of sensor data, which leads to many new opportunities. In particular, mobile crowdsensing techniques can be used to capture phenomena of common interest. Especially valuable insights can be gained if the collected data are additionally related to the time and place of the measurements. However, many technical solutions still use monolithic backends that are not capable of processing crowdsensing data in a flexible, efficient, and scalable manner. In this work, an architectural design was conceived with the goal to manage geospatial data in challenging crowdsensing healthcare scenarios. It will be shown how the proposed approach can be used to provide users with an interactive map of environmental noise, allowing tinnitus patients and other health-conscious people to avoid locations with harmful sound levels. Technically, the shown approach combines cloud-native applications with Big Data and stream processing concepts. In general, the presented architectural design shall serve as a foundation to implement practical and scalable crowdsensing platforms for various healthcare scenarios beyond the addressed use case.

Efficient Processing of Geospatial mHealth Data Using a Scalable Crowdsensing Platform.

Tinnitus is often described as the phantom perception of a sound and is experienced by 5.1% to 42.7% of the population worldwide, at least once during their lifetime. The symptoms often reduce the patient's quality of life. The TrackYourTinnitus (TYT) mobile health (mHealth) crowdsensing platform was developed for two operating systems (OS)-Android and iOS-to help patients demystify the daily moment-to-moment variations of their tinnitus symptoms. In all platforms developed for more than one OS, it is important to investigate whether the crowdsensed data predicts the OS that was used in order to understand the degree to which the OS is a confounder that is necessary to consider.

Applying Machine Learning to Daily-Life Data From the TrackYourTinnitus Mobile Health Crowdsensing Platform to Predict the Mobile Operating System Used With High Accuracy: Longitudinal Observational Study.

Modern smartphones contain sophisticated high-end hardware features, offering high computational capabilities at extremely manageable costs and have undoubtedly become an integral part in users' daily life. Additionally, smartphones offer a well-established ecosystem that is easily discoverable and accessible via the marketplaces of differing mobile platforms, thus encouraging the development of many smartphone apps. Such apps are not exclusively used for entertainment purposes but are also commonplace in health care and medical use. A variety of those health and medical apps exist within the context of tinnitus, a phantom sound perception in the absence of any physical external source.

Smartphone and Mobile Health Apps for Tinnitus: Systematic Identification, Analysis, and Assessment.

Process model comprehension is essential in order to understand the five Ws (i.e., who, what, where, when, and why) pertaining to the processes of organizations. However, research in this context showed that a proper comprehension of process models often poses a challenge in practice. For this reason, a vast body of research exists studying the factors having an influence on process model comprehension. In order to point research towards a neuro-centric perspective in this context, the paper at hand evaluates the appropriateness of measuring the electrodermal activity (EDA) during the comprehension of process models. Therefore, a preliminary test run and a feasibility study were conducted relying on an EDA and physical activity sensor to record the EDA during process model comprehension. The insights obtained from the feasibility study demonstrated that process model comprehension leads to an increased activity in the EDA. Furthermore, EDA-related results indicated significantly that participants were confronted with a higher cognitive load during the comprehension of complex process models. In addition, the experiences and limitations we learned in measuring the EDA during the comprehension of process models are discussed in this paper. In conclusion, the feasibility study demonstrated that the measurement of the EDA could be an appropriate method to obtain new insights into process model comprehension.

Article	Year
Measuring the Moment-to-Moment Variability of Tinnitus: The TrackYourTinnitus Smart Phone App. Frontiers in aging neuroscience\| PubMed ID: 28018210	2016
Innovations in Doctoral Training and Research on Tinnitus: The European School on Interdisciplinary Tinnitus Research (ESIT) Perspective. Frontiers in aging neuroscience\| PubMed ID: 29375369	2017
Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study. JMIR mHealth and uHealth\| PubMed ID: 29959107	2018
Review of Smart Services for Tinnitus Self-Help, Diagnostics and Treatments. Frontiers in neuroscience\| PubMed ID: 30177869	2018
Dimensionality Reduction and Subspace Clustering in Mixed Reality for Condition Monitoring of High-Dimensional Production Data. Sensors (Basel, Switzerland)\| PubMed ID: 31510064	2019
Exploring the Time Trend of Stress Levels While Using the Crowdsensing Mobile Health Platform, TrackYourStress, and the Influence of Perceived Stress Reactivity: Ecological Momentary Assessm... JMIR mHealth and uHealth\| PubMed ID: 31670692	2019
Learning to Read by Learning to Write: Evaluation of a Serious Game to Foster Business Process Model Comprehension. JMIR serious games\| PubMed ID: 31917374	2020
Measuring Mental Effort for Creating Mobile Data Collection Applications. International journal of environmental research and public health\| PubMed ID: 32138381	2020
Combining Mobile Crowdsensing and Ecological Momentary Assessments in the Healthcare Domain. Frontiers in neuroscience\| PubMed ID: 32184708	2020
Smartphone Apps in the Context of Tinnitus: Systematic Review. Sensors (Basel, Switzerland)\| PubMed ID: 32204540	2020
Efficient Processing of Geospatial mHealth Data Using a Scalable Crowdsensing Platform. Sensors (Basel, Switzerland)\| PubMed ID: 32570953	2020
Applying Machine Learning to Daily-Life Data From the TrackYourTinnitus Mobile Health Crowdsensing Platform to Predict the Mobile Operating System Used With High Accuracy: Longitudinal Obser... Journal of medical Internet research\| PubMed ID: 32602842	2020
Smartphone and Mobile Health Apps for Tinnitus: Systematic Identification, Analysis, and Assessment. JMIR mHealth and uHealth\| PubMed ID: 32808939	2020
Towards the Applicability of Measuring the Electrodermal Activity in the Context of Process Model Comprehension: Feasibility Study. Sensors (Basel, Switzerland)\| PubMed ID: 32823891	2020

Manfred Reichert

Biography

JoVE Journal Publications

Other Publications