VoiceNet: A Responsible Artificial Intelligence Multilingual Speech Framework for Gender and Age Classification

Summary

VoiceNet-RAI integrates MFCC features, multilingual wav2vec 2.0 embeddings, and EfficientNet-Lite for gender and age classification from voice data. The framework achieves up to 97.87% accuracy on the English-language subset and supports responsible applications in speech recognition, authentication, and digital forensics.

Abstract

Accurate gender and age classification from voice data is important for personalized, secure, and ethical human-computer interaction (HCI). With the growing use of large pretrained speech models such as wav2vec 2.0, there is a need to leverage these representations responsibly for multilingual and privacy-aware demographic inference. This study proposes VoiceNet-RAI, a deep learning (DL) framework that integrates Mel-Frequency Cepstral Coefficients (MFCCs) with transformer-based multilingual wav2vec 2.0 embeddings within an EfficientNet-Lite architecture to support accurate, fair, and transparent classification across diverse linguistic contexts. The framework combines contextual and spectral information to improve robustness under variable recording conditions. Experiments on multilingual speech data covering 28 languages demonstrate strong classification performance, with the English subset achieving 97.87% accuracy, 98.61% precision, 98.70% recall, and a 98.65% F1-score. External validation using the Mozilla Common Voice dataset yielded 98.27% accuracy for gender classification, demonstrating generalizability to an independent dataset. Ablation analysis further showed that combining MFCC and wav2vec 2.0 features improved performance compared with raw and MFCC-only representations. These findings demonstrate the potential of hybrid spectral-contextual speech representations for robust and responsible gender and age classification.

Introduction

Voice recognition is a core component of human communication through which individuals express their feelings, thoughts, and goals. Human voices are produced through a biological process in which air is exhaled from the lungs and shaped into sound by articulatory organs such as the lips, tongue, and teeth¹. The ear plays an essential role in voice identification and can distinguish between male and female voices based on characteristics such as volume and pitch. One characteristic that contributes to substantial differences in vocal pitch between males and females is sexual dimorphism². Age and gender classification from....

Protocol

This study used publicly available BVC Challenging Voice Set and Mozilla Common Voice datasets without direct participant involvement. Therefore, institutional ethics approval and additional informed consent were not required. Only data necessary for gender and age classification were processed, and no attempt was made to identify individual speakers. The complete workflow methodology of the proposed framework is shared in Figure 1.

Data acquisition

The dataset used in the proposed approach consisted of .wav audio files obtained from the BVC Challenging Voice Set, a ....

Results

Experimental results for age and gender classification were evaluated using MFCC-based and hybrid feature representations with ML and DL models. The dataset was partitioned into training and testing sets at an 80:20 ratio, with 80% used for training and 20% for testing. Speaker-level partitioning was applied to prevent recordings from the same speaker from appearing in both sets.

The hyperparameters and implementation details of the proposed framework are summarized in Table 3.......

Discussion

This study developed and evaluated an automated framework for gender and age classification from voice data by integrating MFCC features and wav2vec 2.0 embeddings with EfficientNet-Lite. In the initial experiments, EfficientNet-Lite achieved an accuracy of 83.48%, precision of 82.87%, recall of 84.28%, and F1-score of 83.54% using the original features. Following the integration of MFCC features, performance improved substantially, with an accuracy of 92.64%, precision of 93.79%, recall of 94.92%, and F1-score of 94.84%.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
BVC Challenging Voice Set	Biometrics Vision and Computing Group / OgeNI	BVC Challenging Voice Set	https://github.com/OgeNI/BVC_Challenging_Voice_Set
EfficientNet-Lite	Google	EfficientNet-Lite, exact variant used	https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite
Graphics Processing Unit	NVIDIA	Tesla K80	https://www.nvidia.com/
Mozilla Common Voice Dataset	Mozilla Foundation	Common Voice, version used in experiments	https://www.kaggle.com/datasets/mozillaorg/common-voice/data
NumPy	NumPy Developers	Exact version used in experiments	https://numpy.org/
Processor	Intel	Intel Core i7-8265U	https://www.intel.com/
Python	Python Software Foundation	Exact version used in experiments	https://www.python.org/
PyTorch	PyTorch Foundation	Exact version used in experiments	https://pytorch.org/
Scikit-learn	Scikit-learn Developers	Exact version used in experiments	https://scikit-learn.org/
SciPy	SciPy Developers	Exact version used in experiments	https://scipy.org/
wav2vec 2.0 Base	Meta AI	facebook/wav2vec2-base	https://huggingface.co/facebook/wav2vec2-base
Windows 11	Microsoft	Windows 11, exact edition/build if available	https://www.microsoft.com/windows/windows-11