Multimodal emotion recognition for emphatic virtual agents in mental health interventions

<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd">
  <record>
    <leader>00000cab a2200000   4500</leader>
    <controlfield tag="001">MAP20260002712</controlfield>
    <controlfield tag="003">MAP</controlfield>
    <controlfield tag="005">20260211190523.0</controlfield>
    <controlfield tag="008">260205e20251208esp|||p      |0|||b|eng d</controlfield>
    <datafield tag="040" ind1=" " ind2=" ">
      <subfield code="a">MAP</subfield>
      <subfield code="b">spa</subfield>
      <subfield code="d">MAP</subfield>
    </datafield>
    <datafield tag="084" ind1=" " ind2=" ">
      <subfield code="a">922.134</subfield>
    </datafield>
    <datafield tag="100" ind1=" " ind2=" ">
      <subfield code="0">MAPA20260002125</subfield>
      <subfield code="a">Huerta Espinoza, Marcelo Alejandro</subfield>
    </datafield>
    <datafield tag="245" ind1="1" ind2="0">
      <subfield code="a">Multimodal emotion recognition for emphatic virtual agents in mental health interventions</subfield>
      <subfield code="c">Marcelo Alejandro Huerta Espinoza, Ansel Y. Rodríguez González and Juan Martinez Miranda</subfield>
    </datafield>
    <datafield tag="520" ind1=" " ind2=" ">
      <subfield code="a">Depression and anxiety disorders affect millions of individuals globally and are commonly addressed through psychological interventions. A growing technological approach to support such treatments involves the use of embodied conversational agents that employ motivational interviewing, a method that promotes behavioral change through empathic engagement. Despite its critical role in therapeutic efficacy, empathy remains a significant challenge for virtual agents to emulate. Emotion Recognition (ER) technologies offer a potential solution by enabling agents to perceive and respond appropriately to users' emotional states. Given the inherently multimodal nature of human emotion, unimodal ER approaches often fall short in accurately interpreting affective cues. In this work, we propose a multimodal emotion recognition model that integrates verbal and non-verbal signals (text and video) using a Cross-Modal Attention fusion strategy. Trained and evaluated on the IEMOCAP dataset, our approach leverages Ekman's taxonomy of basic emotions and demonstrates superior performance over unimodal baselines across key metrics such as accuracy and F1-score. By prioritizing text as the main modality and dynamically incorporating complementary visual cues, the model proves effective in complex emotion classification tasks. The proposed model is designed for integration into an existing conversational agent aimed at supporting individuals experiencing emotional and psychological distress. Future work will involve embedding the model in the conversational agent platform for emotionally distressed users, aiming to assess its real-world impact on engagement, user experience, and perceived empathy</subfield>
    </datafield>
    <datafield tag="650" ind1=" " ind2="4">
      <subfield code="0">MAPA20080611200</subfield>
      <subfield code="a">Inteligencia artificial</subfield>
    </datafield>
    <datafield tag="650" ind1=" " ind2="4">
      <subfield code="0">MAPA20110010515</subfield>
      <subfield code="a">Salud mental</subfield>
    </datafield>
    <datafield tag="650" ind1=" " ind2="4">
      <subfield code="0">MAPA20210005503</subfield>
      <subfield code="a">Medicina bioelectrónica</subfield>
    </datafield>
    <datafield tag="650" ind1=" " ind2="4">
      <subfield code="0">MAPA20080550400</subfield>
      <subfield code="a">Depresión</subfield>
    </datafield>
    <datafield tag="650" ind1=" " ind2="4">
      <subfield code="0">MAPA20260002156</subfield>
      <subfield code="a">Ansiedad</subfield>
    </datafield>
    <datafield tag="700" ind1="1" ind2=" ">
      <subfield code="0">MAPA20260002132</subfield>
      <subfield code="a">Rodriguez Gonzalez, Ansel Y.</subfield>
    </datafield>
    <datafield tag="700" ind1="1" ind2=" ">
      <subfield code="0">MAPA20260002149</subfield>
      <subfield code="a">Martinez Miranda, Juan</subfield>
    </datafield>
    <datafield tag="710" ind1="2" ind2=" ">
      <subfield code="0">MAPA20260002095</subfield>
      <subfield code="a">IBERAMIA, Sociedad Iberoamericana de Inteligencia Artificial
 </subfield>
    </datafield>
    <datafield tag="773" ind1="0" ind2=" ">
      <subfield code="w">MAP20200034445</subfield>
      <subfield code="g">08/12/2025 Volume 28 Number  76 - December 2025 , p. 28 - 39</subfield>
      <subfield code="x">1988-3064</subfield>
      <subfield code="t">Revista Iberoamericana de Inteligencia Artificial</subfield>
      <subfield code="d"> : IBERAMIA, Sociedad Iberoamericana de Inteligencia Artificial , 2018-</subfield>
    </datafield>
    <datafield tag="856" ind1=" " ind2=" ">
      <subfield code="u">https://journal.iberamia.org/index.php/intartif/article/view/2508</subfield>
    </datafield>
  </record>
</collection>