Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Performing data analytics with smart glasses has enormous potential in the context of Industry 4.0, as the data can be analyzed directly at a machine site. Using smart glasses, the operator is able to analyze data, while keeping the hands free for maintenance work. Perform the experiment in a room equipped with a desktop computer and Mixed Reality Smart Glasses.

Ask the participant to put on a skin conductance measurement device and have the participant rest for five minutes to acquire a baseline stress level measurement. Then, inform the participant that the experiment will take 40 to 50 minutes and ask the participant to fill out a State-Trait Anxiety Inventory questionnaire for measuring perceived stress during the experiment. To perform a mental rotation test, have the participant sit in front of the desktop computer and inform the participant that they will have to identify similar objects with different positions in a simulated 3D space.

Inform the participant that only two of the five shown objects will be similar, and that they will have two minutes to complete seven tasks. To perform a spatial sound ability evaluation, explain to the participant that six sounds will also be played for 13 seconds per sound, and that the participant will have to detect the direction from which the sounds are made. To perform an outlier detection skill evaluation, have the participant put on the Mixed Reality Smart Glasses and explain that eight outliers must be found within the world created for the Mixed Reality Smart Glasses.

Explain to the participant that an outlier is a red marked point within a group of white points that will turn green when they detect it by directing their gaze. Further explain that not only will visual help be provided, but that environmental sound cues will also be included to help locate the outliers. Then, inform the participant that the required time for each task, the length of walking, and their final moving position in relation to their starting position will be recorded.

To perform a cluster recognition skill evaluation, explain to the participant that they will need to locate six clusters within the world created with the used Mixed Reality Smart Glasses, emphasizing that the participants will have to distinguish between overlapping clusters by navigating around the clusters using voice commands. When all six of the clusters have been located, have the participant sit at the computer without the smart glasses and explain that the same type of clusters must now be detected within the computer software. At the end of the desktop evaluation, ask the participant to fill out a new State-Trait Anxiety Inventory questionnaire, a self-developed questionnaire for gathering subjective feedback, and a demographic questionnaire.

Remove participants'skin conductance measurement device and thank the participant for participation. In this representative analysis, low and high performers demonstrated no differences in their baseline variables, although the low performers had a higher percentage of female participants than high performers, and high performers were younger than low performers. For the cluster recognition task, low and high performers did not differ significantly between the 2D application and 3D approach.

For the outlier detection task, high performers were significantly faster than low performers and required a shorter walking distance to solve the tasks. Indeed, the BoundingBox, Pathlength, PathVariance, PathMean, AngleVariance, and AngleMean values increased significantly from task to task. Although the recorded time showed little change from task to task using the Mixed Reality Smart Glasses.

Based on the spatial sounds, the participants were able to solve the tasks in the outlier detection case quicker than without using spatial sounds. Notably, the State-Trait Anxiety Inventory scores did not demonstrate significant differences between the pre-and post-task assessments. In addition, small but significantly faster cluster recognition times were observed when participants used the Mixed Reality Smart Glasses compared to those observed when the participants used the desktop computer.

As filtering for experts in this new domain is essential, we have included many pretests. Although we tested the methods in the neutral room, it would be interesting to see real machine instructors in the factory setting analyzing their data.