Preliminary Validation of a Facial-Expression and Movement Analysis Algorithm During Emotion Elicitation Tasks in Healthy Adults

Summary

This study preliminarily validates the FEA algorithm, a proprietary machine-learning algorithm, for detecting responses during emotional stimulus tasks. It examines concordance between the FEA algorithm scores and standardized facial emotion stimuli using the Navarasa framework in healthy adults, assessing whether the system reliably reflects emotion-related responses under controlled experimental conditions.

Abstract

This protocol was designed to preliminarily validate Emoscape, a proprietary machine-learning algorithm based on the Navarasa framework, by testing whether its emotion scores change in concordance with standardized emotion-elicitation tasks in healthy adults. Seventy-two healthy adults were recruited after providing written informed consent. Participants completed the Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7, then viewed facial emotion stimuli from the TRENDS set and Navarasa images, while the FEA algorithm recorded upper-body micro-movements throughout the task. Participants also identified the emotions displayed. Changes in the FEA algorithm scores from baseline were analyzed during exposure to neutral, happy, fear, and anger stimuli from TRENDS, as well as during exposure to Navarasa images. Receiver operating characteristic analysis was used to estimate sensitivity and specificity for discrimination across emotional categories. The FEA algorithm scores showed significant deviations from baseline for TRENDS images mapped to corresponding emotions (Neutral–Shanta, Happy–Haasya, Fear–Bhayanaka, and Anger–Raudra). Concordant changes were also observed in Navarasa images, with all mean differences statistically significant (p < 0.001). Area under the curve (AUC) values varied across categories. The FEA algorithm showed modest sensitivity for pleasant emotions (70–80%) and unpleasant emotions (60–70%). These findings provide preliminary support for the protocol and for the use of the FEA algorithm to assess emotion-related responses during controlled emotion-elicitation tasks in healthy adults.

Introduction

Emotion recognition is an important area of investigation within social cognitive neuroscience, with potential clinical and research applications. Existing methods of emotional assessment quantify alterations in physiological activity, such as electroencephalography (EEG) or galvanic skin resistance (GSR), to infer emotional states. However, these methods are complex and invasive. Direct measurements of facial and bodily movements offer a more immediate and observable index of an individual’s affective state^1,2,3. Machine learning (ML) algorithms, particularly those trained on large, heterogeneous datasets of facial expressions, are used to decode such subtle nonverbal signals and classify emotional states. While conventional ML algorithms such as support vector machines (SVMs) remain widely used, deep learning architectures, most notably neural networks (NNs)^4,5 have demonstrated superior performance, particularly when trained on large-scale datasets, owing to their capacity for automatic feature extraction⁶. An example of a conventional model in the Indian context is an interactive tool that adopts a valence-based dimensional model to determine arousal. Emotion recognition frameworks must be sensitive to cultural context. In the Indian cultural milieu, the Navarasa framework provides a nuanced taxonomy of nine primary emotions, yet it remains largely unexamined within ML-driven emotion recognition research. There is a relative paucity of rigorously validated ML approaches that directly analyze observable behavioral cues—such as facial microexpressions and body movements^7,8. Such expression-driven ML algorithms can be used in real time, are easy to administer, and are non-invasive.

The Facial Expression and movement Analysis algorithm [hereafter referred to as the FEA algorithm] is a proprietary ML-based algorithm that analyzes facial microexpressions and upper-body movements and generates dynamic representations of emotional trajectories and aggregated scores for each of the nine rasas, based on the Navarasa framework. This study aimed to conduct a preliminary validation of the FEA algorithm by examining the concordance between FEA-derived scores and responses to standardized facial emotion stimuli in healthy adult volunteers.

To this end, we draw upon the constructs of the Theory of Mind (ToM) and the mirror neuron system. ToM is a core social-cognitive capacity that enables individuals to infer and interpret the mental and emotional states of others in social contexts⁹. Empirical evidence suggests that exposure to facial emotional stimuli activates mirror neuron networks, which in turn elicit a concordant, vicarious emotional state in the observer¹⁰. The resultant modulation of an individual’s internal, subjective emotional state in response to viewing facial expressions of emotion, if captured by the FEA algorithm, can provide a preliminary validation of the algorithm.

We hypothesize that exposure to facial emotional stimuli corresponding to a specific Navarasa category will elicit a statistically significant deviation from baseline in the FEA-algorithm score for that Navarasa emotion.

Protocol

The study protocol was approved by the Sushruta Hospitals Ethics Committee, Hubballi, Karnataka, India (EC registration number: ECR/372/INST/KA/2013/RR-19), and written informed consent was obtained from all participants prior to enrollment.

The FEA algorithm is a proprietary ML-based method for real-time analysis of facial microexpressions and upper-body movements to infer emotional states using the Navarasa framework. The Navarasa framework, derived from classical Indian aesthetic theory, delineates nine primary emotions: shringara (love), haasya (joy), karuna (sorrow/pensiveness), adbhuta (surprise), shanta (peace), raudra (anger), veera (courage), bhayanaka (fear), and vibhitsa (disgust). The FEA algorithm uses a machine-learning model trained on facial microexpressions and upper-body movements extracted from images of actors portraying the nine Navarasa emotions. The algorithm infers an individual’s emotional state by analyzing subtle facial dynamics and micromovements from visual input captured by a single RGB camera and generates quantitative scores for each of the nine primary emotions. This non-invasive configuration enables unobtrusive, naturalistic emotional assessment, making it suitable for a wide range of research and applied settings.

Model development

The FEA algorithm is based on a neural network (NN) architecture. The training dataset was collected in-house and consists of several hours of recorded video data of Indian actors portraying the nine Navarasa emotions. The dataset was derived from controlled video recordings of several actors/participants, with multiple recordings/tests conducted for each actor/participant across different elicited emotional states. Following pre-processing, including facial landmark detection, temporal consistency checks, quality filtering, and removal of unusable or low-confidence frames, approximately 25,000 structured training samples were retained. During dataset curation, recordings were stratified by more than ten demographic and recording-related parameters, including age, sex, and other relevant attributes, and intentionally balanced to achieve equitable representation across these parameters and minimize demographic bias. Sex distribution was maintained at approximately 52% male and 48% female. Participants broadly covered adolescent and adult age groups, with representation across younger, middle, and older adult cohorts. Actors/participants were based in India and represented multiple Indian regional and cultural backgrounds, as well as multiple international ethnicities present in India. Owing to proprietary restrictions, the exact number of actors/participants, actor/participant-level identity, exact video/recording counts per actor, precise frame counts, and the internal annotation methodology cannot be disclosed, as these form part of Nihilent Ltd.’s proprietary dataset and model-development intellectual property; this limitation is explicitly acknowledged in the Discussion section.

Facial images extracted from these recordings were processed using image processing techniques to derive quantitative features, which were subsequently used as inputs to a neural network for prediction. This model was then evaluated on a target population, and the insights gained were used to iteratively refine subsequent versions. This cycle of development, testing, and improvement was repeated until model performance stabilized. The final predictive model is implemented as a multilayer perceptron (MLP) comprising an input layer with approximately 300 selected features, hidden layers, and an output layer producing a nine-dimensional score vector corresponding to the nine Navarasa emotion categories. More than 700 features were initially derived from the captured data and subsequently reduced through feature selection and model optimization procedures. The MLP output score is then passed to proprietary post-processing algorithms to derive the final aggregated FEA-algorithm scores. Further architectural details—including the precise number of trainable parameters and specific layer dimensions—are proprietary and cannot be disclosed; this is acknowledged as a limitation of the present report. A detailed description of the model development is provided in the supplementary material (Supplementary File 1 and Supplementary Figure 1).

During iterative development, model performance was assessed using cross-validation on the in-house dataset, achieving 96% cross-validated classification accuracy across the nine Navarasa categories. Subsequent independent validation was conducted on more than 15,000 min of video data. A normalized 9 × 9 confusion matrix summarising classification performance across all nine Navarasa categories is provided in the supplementary material (Supplementary File 1).

Following stabilization, the model underwent controlled internal validation under expert supervision. This approach emphasizes empirical validation through direct human subject testing, enabling assessment of real-world applicability and robustness across diverse populations. The model development is summarized in Figure 1.

The patient Health Questionnaire-9 (PHQ-9) and the Generalized Anxiety Disorder-7 (GAD-7) scale: Participants completed self-report questionnaires under the supervision of a trained psychologist to assess their current emotional states. The instruments administered included the Patient Health Questionnaire-9 (PHQ-9), and the Generalized Anxiety Disorder-7 (GAD-7) scale. A score above 9 on either scale was considered clinically significant.

TRENDS

The Tool for Recognition of Emotions in Neuropsychiatric Disorders (TRENDS) is a culturally validated, ecologically sensitive instrument designed to assess facial emotion recognition abilities¹¹. It has been validated for use in Indian populations and consists of 40 photographs of actors depicting the universal basic emotions of neutrality, happiness, fear, anger, and sadness. Images from TRENDS were used as reference stimuli to validate the FEA algorithm’s emotion classification outputs.

Experimental design

The experiment was designed to elicit an algorithmic response to emotional stimuli. Two emotion-elicitation paradigms were developed and used for this study.

TRENDS images paradigm

In the first paradigm, a video sequence initially presented a blank screen for 60 s, followed by the display of emotional stimuli. The emotional stimuli were selected from the TRENDS tool. A total of 20 images were presented, representing neutrality, happiness, sadness, anger, and fear × two genders × two age groups. Each stimulus was displayed for 5 s. The total stimulus presentation time was therefore 20 × 5 s, with an additional 10 s blank screen interval between consecutive images, resulting in a total duration of 300 s. During each 10 s blank interval, participants were required to identify the emotion depicted in the preceding image by selecting one of the five emotion labels displayed on the screen.

Navarasa paradigm

A second, analogous video paradigm was constructed based on the Navarasa framework of nine rasas. A total of 18 images were presented (nine emotions × two genders). Each stimulus was displayed for 5 s, with a 10 s blank screen between images, yielding a total duration of 270 s. The nine emotions were categorized as neutral (shanta), pleasant (haasya, veera, shringara, adbhuta), or unpleasant (Krodha, vibhitsa, bhayanaka, karuna). During each 10 s blank interval, participants were required to classify the emotion depicted in the preceding image by selecting one of three options presented on the screen: neutral, pleasant, or unpleasant. The nine Navaras were grouped into three categories to facilitate the recording of participants’ responses to emotion identification.

Participants

Healthy adult volunteers were recruited from workplace settings and educational institutions. Inclusion criteria comprised individuals aged 18–50 years, of any sex/gender. Exclusion criteria included a history of diagnosed psychiatric disorders, neurological disorders, significant visual impairment, or physical disabilities that restricted head and neck movements.

Procedure

Participants were seated such that the camera was at eye level and positioned 60–70 cm away. The FEA algorithm was used in the study. The application was opened on the laptop, and the researcher entered the subject ID. Throughout the experiment, the FEA algorithm continuously recorded video data to enable subsequent analysis of facial expressions and micro-movements. Initially, a 60 s baseline recording was obtained, during which participants were instructed to relax and refrain from overt expressive behaviors. Following baseline acquisition, participants were exposed to the two video paradigms (TRENDS images followed by Navarasa). All participants viewed the paradigms in the same fixed order to minimize potential confounding effects related to presentation sequence; however, this fixed order could have introduced order effects. During the recording sessions, the researcher remained outside the room to reduce distraction. The researcher monitored the experiment remotely via a mobile application while outside the room. Participants were offered a brief rest interval between the two paradigms (Figure 2).

Analysis

The collected data were mapped to the subject ID entered in the application to ensure data fidelity. The FEA algorithm utilizes a structured, end-to-end computational pipeline for automatic emotion inference, comprising five principal stages: video acquisition, feature detection and extraction, data preprocessing, emotion classification, and output interpretation. Extracted features underwent a series of pre-processing operations, including noise reduction, normalization across subjects and illumination conditions, and interpolation of missing frames. Temporal smoothing was applied to enhance signal stability. The resulting feature vectors were temporally aligned for downstream ingestion by the classification model. These time-aligned, denoised feature matrices served as inputs to the emotion classification component.

Feature extraction

The extraction framework employs a multi-feature extraction-stage algorithmic pipeline that translates standard 2D monocular video image feeds into a reconstructed 3D coordinate space. Rather than relying on raw 2D pixel tracking, the system maps localized spatial coordinates across the face and upper body. The features derived fall into three broad categories: (i) facial landmark regions, encompassing geometric descriptors of facial structure and dynamics (including eye-region cues and facial micro-expression correlates drawn from the Navarasa framework, which is rooted in the Natyashastra—the earliest formalized framework of human emotional expression, recognized by UNESCO); (ii) head-pose information, capturing orientation and movement of the head; and (iii) upper-body movement features, capturing resultant movement of the upper body. Specific feature definitions, including precise landmark indices and keypoint identifiers, are proprietary and cannot be fully disclosed; this is acknowledged as a limitation. The system does not use Facial Action Coding System (FACS) action unit labels directly; instead, it derives geometric and temporal features from facial and body movement. These 3D spatial transformations are aggregated over time to generate a comprehensive behavioral summary vector, which serves as input to the MLP classifier. In total, more than 700 candidate features were initially derived from the captured data and subsequently reduced to approximately 300 selected features through feature selection and model optimization. The structural pipeline proceeds from video acquisition through 2D-to-3D spatial reconstruction, temporal aggregation, and feature normalization prior to classification.

Emotion classification model

The pre-processed feature vectors were provided to a supervised multilayer perceptron (MLP) neural network architecture trained on the in-house annotated dataset corresponding to the nine Navarasa emotional categories: Śṛṅgāra (love), Hāsya (joy), Karuṇā (sorrow), Raudra (anger), Vīra (courage), Bhayānaka (fear), Bībhatsa (disgust), Adbhuta (wonder), and Śānta (peace). The model was explicitly designed to capture spatiotemporal dependencies in expressive behavioral data. At each time step, the architecture produced a vector of predicted emotion intensities for the nine categories.

Post-processing of model outputs

The raw model outputs were subsequently post-processed to yield stable and interpretable emotion scores. Three sequential operations were applied: (i) temporal smoothing to reduce short-term fluctuations, (ii) normalization to ensure comparability across time and across emotion dimensions, and (iii) aggregation to derive summary measures over specified temporal windows or entire sequences. The post-processing procedures ensure cross-subject comparability and control for inter-individual baseline variation.

Output interpretation and visualization

The FEA algorithm generates quantitative and visual representations of emotional dynamics inferred from video data. The two primary forms of output are temporal emotion trajectories. Emotion intensity values are visualized as time-series plots showing the evolution of each emotion throughout the video. This representation provides a continuous view of affective variation, facilitating the examination of transient changes, peak activations, and recovery or de-escalation phases in emotional arousal. Comparative emotional scores: Aggregated emotion values for each of the nine Navarasa categories are displayed as comparative score plots, illustrating the relative magnitude of each emotion across the analyzed interval. These comparative visualizations enable direct comparisons across emotions and provide a concise summary of the overall affective profile. The FEA-algorithm scores generated at baseline were inspected for any flatline or absence of meaningful change, which would indicate inadequate recording.

Statistical analysis

Scores for the nine Navarasa emotion domains, generated by the FEA algorithm during the 10 s interval following each image presentation in the paradigm, were treated as outcome variables. FEA-algorithm scores were generated for each s in the 10 s window (T0, T1, T2… T9). The FEA-algorithm score at T0 was considered the baseline score. The highest FEA algorithm score among the recordings from T1–T9 was defined as Tmax. The change in FEA-algorithm score from baseline (Tmax − T0) was computed and used as the outcome variable in the analyses. The change in FEA-algorithm scores over the time series for each emotion is depicted in Figure 3A–D.

For the TRENDS images, the corresponding target emotion categories were mapped as follows: Shanta > Neutral; Haasya > Happy; Krodha > Anger; and Bhayanaka > Fear. For the Navarasa images, FEA-algorithm scores were grouped and averaged into three composite categories: neutral (Shanta), pleasant (Haasya, Veera, Shringara, Adbhuta), and unpleasant (Krodha, Vibhitsa, Bhayanaka, Karuna).

Both group-level and subject-level analyses were conducted.

Group-level analysis: For each emotion category described above, the change in FEA-algorithm score from baseline to Tmax was compared using paired t-tests. A significant change from baseline in the FEA-algorithm score would indicate good concordance with emotion recognition. However, a significant difference would not necessarily indicate strong classification accuracy.

Subject-level analysis: Any increase in an FEA-algorithm emotion-domain score above the baseline value was classified as a positive emotional response. Accuracy indices were then derived—sensitivity and specificity. Sensitivity and specificity quantify, respectively, the probability that the test is positive when the target condition is present (+) and negative when it is absent (−). When both estimates approach 1, the test performs well in terms of diagnostic accuracy. Estimated sensitivity and specificity were defined as the proportions of participants with and without the reference condition (Navarasa expressions), respectively, who were correctly classified.

A subject was considered to have the target condition present (+) if they correctly identified at least 75% of the depicted emotions. This classification was based on validation studies of TRENDS, in which an accuracy of 60–80% was considered adequate discriminant validity. Receiver operating characteristic (ROC) analysis for a continuous predictor was performed to estimate the area under the curve (AUC) and the corresponding sensitivity and specificity values. A sensitivity above 80% was considered good discriminative ability, and below 80% was considered modest.

Results

The study examined variations in FEA-algorithm scores in response to emotion-eliciting stimuli and assessed the diagnostic performance of the FEA algorithm using receiver operating characteristic (ROC) analysis. Seventy-two healthy volunteers were recruited for the study. The mean (SD) age was 31.9 (11.9) years, and the sample included 26 males and 46 females. All participants had at least 10 years of education, and all were of Indian ethnicity. Their mean (SD) PHQ-9 and GAD-7 scores were 4.4 (4.8) and 4.7 (4.5), respectively.

Changes in FEA-algorithm scores in response to emotional stimuli: FEA-algorithm-derived measures demonstrated statistically significant changes from baseline (T0) to Tmax, as determined by paired t-tests for both stimulus conditions—TRENDS images and Navarasa (p < 0.001; Table 1, Figure 4, Table 2, and Figure 5).

Diagnostic accuracy of the FEA algorithm: The diagnostic performance of the FEA algorithm was evaluated using receiver operating characteristic (ROC) curve analysis, with area under the curve (AUC) values computed for distinct emotional states (Table 3). The highest AUC was obtained for the FEA-algorithm score corresponding to Shanta state in response to neutral facial-emotion stimuli (AUC = 0.665), followed by the positive emotion of happiness (AUC = 0.637) (Figure 6). Sensitivity for neutral and positive emotions (happy, pleasant) was 70–80%, whereas sensitivity for negative emotions (anger, fear, unpleasant) was 60–70%. These sensitivity values are below 80% and represent a modest discriminative performance.

Correlation between FEA-algorithm scores and PHQ-9 and GAD-7 scores: No significant correlations were observed between any of the FEA-algorithm scores and PHQ-9 or GAD-7 scores.

DATA AVAILABILITY:

All raw data are provided in Supplementary File 2.

Figure 1: Schematic diagram of the FEA algorithm model-development pipeline. The schematic depicts the in-house video dataset of Indian actors of both sexes undergoing frame extraction and feature extraction across three categories: facial landmark regions (including Natyashastra/Navarasa-derived eye and facial micro-expression cues), head-pose information, and upper-body movement, which are aggregated into a single feature vector and passed to a multilayer perceptron (MLP) classifier. The MLP output undergoes proprietary post-processing (temporal smoothing, normalization, and aggregation; dashed orange box) before generating the final nine-category Navarasa prediction output. Solid blue boxes denote processing steps; shaded blue boxes denote feature categories; the dashed orange box denotes a proprietary processing step whose internal parameters cannot be disclosed. As this is a process schematic rather than a data plot, error bars and scale bars are not applicable. Please click here to view a larger version of this figure.

Figure 2: FEA-algorithm experiment procedure. Flowchart depicting the sequential stages of the experimental protocol: semi-structured clinical assessment (PHQ-9, GAD-7); participant seating with the camera positioned at eye level, 60–70 cm from the participant; a 60 s baseline recording; the TRENDS emotion-image paradigm (20 images across five emotion categories; 5 s stimulus presentation and 10 s response interval per image; 300 s total); a brief rest interval; and the Navarasa emotion-image paradigm (18 images across nine Navarasa categories; 5 s stimulus presentation and 10 s response interval per image; 270 s total). Arrows indicate the temporal sequence of procedures from enrollment through completion of data acquisition. As this is a process flowchart rather than a data plot, error bars and scale bars are not applicable. Please click here to view a larger version of this figure.

Figure 3: Time-series change in FEA-algorithm scores in response to corresponding TRENDS emotion images (T0 to T9 s). Four panels (A–D) show the group-level, second-by-second trajectory of the FEA-algorithm score (y-axis; arbitrary score units generated by the proprietary scoring algorithm) across the 10 s post-stimulus window (x-axis; T0 = score at stimulus offset/baseline for that trial, T1–T9 = each subsequent s), for (A) the Shanta domain following neutral images, (B) the Haasya domain following happy images, (C) the Bhayanaka domain following fear images, and (D) the Krodha domain following anger images (n = 72 participants per panel). Data points/lines represent the group means at each time point; error bars/shaded bands represent the standard deviation (SD) across participants. Scale bars are not applicable to this time series line plot format. Please click here to view a larger version of this figure.

Figure 4: Bar graph showing mean change in FEA-algorithm scores in response to viewing TRENDS emotion images. Bars represent the group-mean change in FEA-algorithm score (Tmax −T0; y-axis, arbitrary score units) for each of the four TRENDS-mapped emotion categories (Shanta/Neutral, Haasya/Happy, Krodha/Anger, Bhayanaka/Fear; x-axis) (n = 72). Error bars represent the standard deviation (SD) of the change score within each category. All four categories showed a statistically significant change from baseline by paired t-test (p < 0.001; see Table 1). Scale bars are not applicable to this bar graph format. Please click here to view a larger version of this figure.

Figure 5: Bar graph showing mean change in FEA-algorithm scores in response to viewing Navarasa emotion images. Bars represent the group-mean change in FEA-algorithm score (Tmax − T0; y-axis, arbitrary score units) for the three composite Navarasa categories — neutral (Shanta), pleasant (mean of Haasya, Veera, Shringara, Adbhuta), and unpleasant (mean of Krodha, Vibhitsa, Bhayanaka, Karuna) (x-axis) (n = 72). Error bars represent the standard deviation (SD) of the change score within each category. All three categories showed a statistically significant change from baseline by paired t-test (p < 0.001; see Table 2). Scale bars are not applicable to this bar graph format. Please click here to view a larger version of this figure.

Figure 6: Receiver operating characteristic (ROC) curve showing sensitivity at various cut-offs of the FEA-algorithm Shanta score in response to neutral emotion images. The y-axis represents sensitivity (true-positive rate, %) and the x-axis represents 1 − specificity (false-positive rate, %), plotted across the range of possible Shanta-score cut-off values; the diagonal reference line represents chance-level discrimination (AUC = 0.5). Area under the curve (AUC) = 0.665 (see Table 3 for AUC, sensitivity, and specificity values for all emotion categories). Error bars and scale bars are not applicable to this curve format. Please click here to view a larger version of this figure.

Emotion (TRENDS stimuli)	Mean T0	Mean Tmax	Mean Difference	p-value
Neutral – Shanta	0.2261	0.2664	0.0403	<0.001
Happy – Haasya	0.0834	0.1127	0.0294	<0.001
Fear – Bhayanaka	0.0648	0.0848	0.02	<0.001
Anger – Raudra	0.0901	0.1108	0.0207	<0.001

Table 1: Mean T0 and Tmax values of FEA-algorithm scores in response to viewing TRENDS emotion stimuli. For each TRENDS-mapped emotion category (Shanta/Neutral, Haasya/Happy, Krodha/Anger, Bhayanaka/Fear), the table reports the group mean FEA algorithm scores (± SD) at baseline (T0) and at the within-trial maximum (Tmax), the mean change (Tmax − T0), and the result of the paired t-test (p-value) comparing T0 and Tmax (n = 72). All values are reported in arbitrary FEA-algorithm score units; p < 0.05 was considered statistically significant.

Emotion Category (Navarasa stimuli)	Mean T0	Mean Tmax	Mean Difference	p-value
Neutral – Shanta	0.24	0.29	0.047	<0.001
Pleasant (Haasya, Veera, Shringara, Adbhuta)	0.034	0.045	0.011	<0.001
Unpleasant (Krodha, Vibhitsa, Bhayanaka, Karuna)	0.054	0.068	0.014	<0.001

Table 2: Mean T0 and Tmax values of FEA-algorithm scores in response to viewing Navarasa emotion stimuli. For each composite Navarasa category (neutral, pleasant, unpleasant), the table reports the group mean FEA algorithm scores (± SD) at baseline (T0) and at the within-trial maximum (Tmax), the mean change (Tmax − T0), and the result of the paired t-test (p-value) comparing T0 and Tmax (n = 72). All values are reported in arbitrary FEA-algorithm score units; p < 0.05 was considered statistically significant.

Panel A: TRENDS Stimuli
TRENDS Stimuli	AUC	95% CI	Cutoff Δ Emoscape Score	Sensitivity (%)	Specificity (%)
Neutral – Shanta	0.665	0.456–0.873	0.065	82.5	50
Happy – Haasya	0.637	0.493–0.780	0.0345	70.7	46.2
Fear – Bhayanaka	0.583	0.446–0.721	0.018	61.2	54.5
Anger – Raudra	0.532	0.374–0.689	0.023	67.3	45.5
Panel B: Navarasa Stimuli
Navarasa Stimuli	AUC	95% CI	Cutoff Δ Emoscape Score	Sensitivity (%)	Specificity (%)
Shanta (Neutral)	0.596	0.412–0.779	0.03	85.7	44.6
Pleasant (Haasya, Veera, Shringara, Adbhuta)	0.515	0.369–0.660	0.012	70.2	40
Unpleasant (Krodha, Vibhitsa, Bhayanaka, Karuna)	0.565	0.384–0.745	0.007	67.8	38

Table 3: Area under the curve (AUC), sensitivity, and specificity values for FEA-algorithm scores by emotion category. For each emotion category assessed using receiver operating characteristic (ROC) analysis, the table reports the AUC, sensitivity (%), and specificity (%). A sensitivity ≥ 80% was pre-specified as indicating good discriminant ability; values below this threshold were classified as modest (see Statistical Analysis).

Supplementary Figure 1: Normalized confusion matrix for the nine-class Navarasa emotion classification model. Rows represent the true (actual) emotion class and columns represent the model-predicted emotion class; cell values indicate the proportion of true-class instances assigned to each predicted class, normalized to a 0–1 scale (color bar, right). Class labels (1–9) correspond to the nine Navarasa emotion categories; due to proprietary restrictions, the specific number-to-emotion mapping is not disclosed. Overall cross-validated classification accuracy across all nine categories was 96%. Error bars are not applicable to this matrix format.Please click here to download this file.

Supplementary File 1: Description of FEA-algorithm model development (includes internal validation metrics, cross-validated accuracy, and 9 × 9 Navarasa confusion matrix).Please click here to download this file.

Supplementary File 2: File containing raw data.Please click here to download this file.

Discussion

This study sought to provide a preliminary validation of the FEA algorithm by examining the concordance between FEA-algorithm-derived scores and responses to standardized facial emotion stimuli in healthy adult volunteers. These early findings indicate that the FEA algorithm exhibits statistically detectable changes in response to emotion-inducing visual stimuli across various discrete emotional categories, including neutral (shanta), happiness (haasya), fear (bhayanaka), and anger (raudra), as well as broader classifications of pleasant and unpleasant states. This suggests that FEA-algorithm scores are concordant with responses to emotional-stimulus tasks.

The FEA algorithm’s sensitivity was observed to range from 70% to 80% for neutral or positive emotions and from 60% to 70% for negative emotions. These sensitivity values are modest for any test intended for use as a screening measure¹². In comparison, broader machine learning-based emotion inference models that utilize neurophysiological markers, such as the electroencephalography (EEG), report classification accuracies of up to 90%¹³. An ensemble classifier system for emotion classification from EEG and GSR for autism detection has shown detection accuracy of 99.97%¹⁴. However, these models tend to be more computationally intensive and necessitate offline processing. In contrast, a notable strength of the FEA algorithm lies in its focus on observable facial and bodily cues, enabling operation in real-time settings.

The FEA-algorithm procedure depends on several procedural factors critical to the quality and reliability of FEA-algorithm recordings. These include camera position and ambient lighting. Recordings conducted under inconsistent or low ambient lighting may significantly degrade feature extraction quality. Head and body positioning must remain stable throughout each recording session; participants with restricted head or neck mobility were therefore excluded. Minimizing behavioral interference during recording is essential. Participants were seated alone, with the researcher absent from the room, to reduce social desirability effects and self-monitoring that might suppress naturalistic facial expressions.

The incorporation of the Navarasa framework provides a culturally sensitive classification of emotions. The study applied a theory-of-mind (ToM) approach to assess changes in FEA-algorithm scores in response to emotional stimuli. ToM represents a higher-order social cognitive function, and its neurobiological substrates have been documented in the literature¹⁵, thereby supporting the biological plausibility of the observed changes in FEA-algorithm scores.

No correlation was found between FEA-algorithm scores and depression or anxiety scores, suggesting that the internal emotional states of participants were unlikely to have influenced the FEA-algorithm recordings. However, it is difficult to fully control participants’ emotional states during the experiment. Additionally, top-down cognitive processing may inhibit natural emotional responses, particularly in reaction to negative emotional stimuli. Future studies should assess participants’ subjective experiences while viewing emotional stimuli, as this could further validate whether passive viewing indeed induces a subjective emotional state. The modest sensitivity with the FEA algorithm alone could potentially be enhanced with additional neurophysiological data, such as EEG, heart rate variability (HRV), or galvanic skin response (GSR). This should be assessed in future validation studies. The validation was performed on a healthy adult population. The utility of the FEA algorithm in assessing subjective emotional states in clinical conditions such as depression and anxiety warrants further investigation through case-control studies.

Several limitations of the present study warrant acknowledgement. The proprietary nature of the FEA algorithm limits the transparency of its processing pipeline. Specifically, the exact number of actors/participants, actor/participant-level identity, exact video/recording counts per actor, exact feature definitions (landmark indices and keypoint identifiers), and detailed MLP architectural parameters (precise layer dimensions and parameter counts) cannot be disclosed. The internal validation metrics (cross-validated accuracy of 96%; independent validation on >15,000 min of video) are reported in the supplementary material. The class labels representing emotion names are anonymized due to proprietary constraints. We acknowledge that these constraints limit independent replication.

The study observed stimulus concordance between FEA-algorithm scores and emotion-recognition accuracy in healthy adults, with modest sensitivity for discriminating between emotional states. This provides preliminary validation for the FEA algorithm, a proprietary ML-based algorithm that analyzes upper-body micro-movements and generates dynamic representations of emotional trajectories and aggregated scores for each of the nine rasas, based on the Navarasa framework. Future iterations of the algorithm should aim to improve its discriminatory power across all emotional categories by incorporating additional physiological or contextual data. Validation in broader clinical populations requires further investigation.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Emoscape	Nihilent Ltd, Pune	1	Developer of Emoscape
Manoshanti Centers for Emotional Well Being	GNR Health Care Pvt Ltd.	2	Clinical and research partner
Tool for recognition of emotions in neuropsychiatric disorders	copyright with corresponding author of the study	3	Tool used for validation in the study