Knowledge-guided cross-modality attention for smartphone-based stress recognition in applied health informatics

Purpose – Stress is a major risk factor for both mental and physical disorders, yet many recognition systems still rely on a single modality or static fusion and seldom include domain knowledge or input-quality indicators. These limitations reduce robustness, interpretability and generalization. This study develops a smartphone-based framework for real-time three-class stress recognition (neutral, low stress and high stress) that integrates acoustic, linguistic and behavioral data. Design/methodology/approach – We propose CMAF-Net, a cross-modality attention fusion network that embeds psychological priors directly into attention scoring and incorporates a reliability-aware gating mechanism driven by audio signal-to-noise ratio, automatic speech recognition word-error rate and behavioral sparsity. Data from 87 participants speech prompts, transcripts and passive behavioral sensing were labeled with the Perceived Stress Scale (PSS-4). Evaluation used subject-independent five-fold validation, comparisons with strong early- and late-fusion baselines, ablation studies isolating the effects of prior vectors and cross-modal bias, missing-modality and noise-stress tests, calibration analysis and on-device deployment with post-training quantization. Findings – CMAF-Net achieved 88.5 % accuracy, 0.89 macro-F1 and 0.92 AUC, significantly outperforming recent multimodal baselines. Ablation experiments confirmed the benefits of both knowledge-guided attention and reliability-aware gating under noisy or incomplete inputs. Post-training dynamic quantization reduced model size by 58 % and delivered ∼180 ms mean inference latency on a mid-range smartphone without measurable accuracy loss while maintaining low calibration error. Originality/value – This work formalizes prior-informed cross-modality attention for smartphone-based stress recognition and couples it with reliability-aware modality gating, comprehensive calibration and robustness evaluation and reproducible on-device deployment, yielding a scalable, privacy-preserving approach suitable for real-world health informatics monitoring.