Invited lecture

When, What, and How: Designing Empathic Behaviors for Social Robots

Christian Arzate Cruz · Honda Research Institute Japan

AFAR Lab, University of Cambridge · Hosted by Hatice Gunes · November 17, 2026

Overview

A robot that responds to how people feel has to answer three questions: when to be empathic, what to say, and how to show it. In this lecture I present how we answer each one on Haru, Honda's tabletop social robot.

Part 1, Responding Empathically, covers detecting when a user is seeking empathy, guiding large language models toward empathic replies with empathy cues, and expressing empathy through backchannels, facial and motor expressions, and an expressive voice. Part 2, Expressing Emotion, shows how emotion-aware co-speech gestures keep emotion visible while the robot speaks.

Slides

Videos

Full lecture

  • When, What, and How: Designing Empathic Behaviors for Social Robots

    Invited lecture at the AFAR Lab, University of Cambridge, November 17, 2026. Recording coming soon.

Demos shown in the lecture

Papers

When to be empathic

What to say

Co-speech gestures

Datasets & Code

Ours

  • whEE: When and How to Express Empathy

    Code, prompts, and results for detecting when a user seeks empathy and generating empathic replies. Includes EDR (Empathetic Exchanges with a Robot), conversations recorded in the presence of Haru. GitHubRO-MAN 2025

  • EmpatheticExchanges

    Conversations from EmpatheticDialogues split into speaker–listener exchanges, each labeled for behavioral empathy and annotated with empathy cues: sentiment, valence–arousal–dominance, EPITOME mechanisms, empathetic intents, and mimicry. Also comes in an extended version with more labeled conversations. GitHubIEEE Access 2024

  • mEX: Multimodal EmpatheticExchanges

    Text exchanges with behavioral-empathy labels plus video-based cues (arousal and valence, mimicry, speaker status), with the analysis notebooks and multimodal baselines. GitHubIEEE Access 2026

Public datasets we use

  • iEmpathize

    Sentences labeled as seeking empathy, providing empathy, or neither. Hosseini & Caragea, “It Takes Two to Empathize: One to Seek and One to Provide,” AAAI 2021. GitHubPaperCSV in whEE

  • TwittEmp

    Tweets annotated for empathy. Hosseini & Caragea, “Distilling Knowledge for Empathy Detection,” Findings of EMNLP 2021. GitHubPaperCSV in whEE

  • EPITOME

    Framework and dataset for the three ways empathy is expressed in text: emotional reactions, interpretations, and explorations. We use these mechanisms as empathy cues and in our text-generation prompt. Sharma et al., EMNLP 2020. GitHubPaper

  • BEAT2

    60 hours of motion-captured co-speech gestures from 25 speakers with a wide range of emotions; the training data for our gesture models. Liu et al., CVPR 2024. Project pagePaper