---
title: Voice assistants forget how you sounded, because the transcript throws the sound away
description: Preprints published October 30 show that standard speech pipelines discard emotion, prosody and voice events, that audio-native models recover some of that signal, and that capable speech recognition now runs on a microcontroller.
canonical: https://hangingcontext.com/stories/2026-10-31/voice-assistants-forget-how-you-sounded/
last-updated: 2026-10-31
url: https://hangingcontext.com/stories/2026-10-31/voice-assistants-forget-how-you-sounded/
published: 2026-10-31T00:37:47Z
edition: 2026-10-31T00
newsroom: artificial-intelligence
beat: models-labs
trust: confirmed
source_count: 3
cited_manifest_ids: ['1793346404577730094', '1793360486738224056', '1793359933822673187']
---

# Voice assistants forget how you sounded, because the transcript throws the sound away

*Preprints published October 30 show that standard speech pipelines discard emotion, prosody and voice events, that audio-native models recover some of that signal, and that capable speech recognition now runs on a microcontroller.*

Confirmed · 3 sources · Artificial Intelligence · Published 2026-10-31 00:37 UTC

## Key facts

- In a preprint, standard speech-to-text pipelines systematically discard paralinguistic signals such as emotion, prosody and voice events. (self-reported) [cites: 1793346404577730094]
- In a preprint, giving text-based models paralinguistic metadata lifts accuracy by 0.09 to 0.38 on a long-horizon voice memory benchmark. (self-reported) [cites: 1793346404577730094]
- In a preprint, audio-native models score 0.354 to 0.412 on paralinguistic cues against 0.325 for blind models. (self-reported) [cites: 1793346404577730094]
- In a preprint, a block-recurrent speech model with two shared blocks recovers 97.4% of the accuracy of large reference models while cutting model size by an order of magnitude. (self-reported) [cites: 1793360486738224056]
- In a preprint, a 65.7-million-parameter speech transducer quantised to 2 and 4 bits fits in 26.65 MB and runs in real time on an ESP32-P4 microcontroller without a neural accelerator. (self-reported) [cites: 1793359933822673187]

## The transcript is not the conversation

In a preprint, researchers introduce VoiceLongMemEval, a benchmark for testing whether AI assistants remember paralinguistic metadata, emotion, prosody and voice events, across long-horizon, multi-session conversations. Their central finding is that standard speech-to-text pipelines systematically discard those signals: the words survive into text, and how they were said does not. That creates what the authors call an affect gap. An assistant that remembers what a user said in a previous session, but not that they sounded distressed, hesitant or sarcastic when they said it, is missing information a human listener would have kept without thinking. [cites: 1793346404577730094]

The preprint quantifies how much that missing information matters. Providing paralinguistic metadata to text-based models yields an accuracy boost of 0.09 to 0.38 on the benchmark, a wide range that suggests the value depends on the task and on which cues are supplied. Audio-native models, which process sound directly rather than through a transcript, outperform blind models in extracting paralinguistic cues, scoring 0.354 to 0.412 against 0.325. Those absolute scores are low, and the authors present them as evidence of a gap rather than a solved problem: even the better approach is far from reliable at remembering how someone sounded across sessions. [cites: 1793346404577730094]

## Recognition itself is moving to the edge

Two further preprints show the other half of the voice story: recognition is getting small enough to run where the user is, rather than in a data centre. One introduces a block-recurrent architecture for speech recognition on resource-constrained edge devices such as wearables, replacing deep parameterised layers with weight-shared recurrent blocks to reduce model size by an order of magnitude. A model with two shared blocks is reported to recover 97.4% of the accuracy of large reference models on Open ASR Leaderboard benchmarks, while weight reuse reduces memory-access overhead and inference energy, and a mixture-of-experts low-rank adaptation method enables user-specific personalisation at lower inference cost than general-purpose systems. [cites: 1793360486738224056]

> **From Synorb** — Claims with a source link. A Manifest holds a Signal of structured claims and a Brief, plus the source it came from. Your agent can check a sentence against it. [Read the guide](https://synorb.com/connect/start)


The second preprint goes further down the hardware ladder. A 65.7-million-parameter transducer, quantised to 2 and 4 bits with Hadamard-rotated groups and custom SIMD kernels, fits into 26.65 MB of memory and runs in real time on an ESP32-P4 microcontroller, at 0.90 seconds per 1.28-second chunk, without a dedicated neural accelerator. The author reports that GPTQ quantisation is the primary driver of accuracy retention in the 2 and 4-bit process. The point for product teams is not the microcontroller itself. It is that the cost and privacy calculus for voice changes when recognition can run on the device: audio need not leave the room to be transcribed, which matters precisely because, as the first preprint shows, audio carries more than words. [cites: 1793359933822673187]

## Remembering tone is a design choice with privacy attached

Read together, these preprints frame a choice that voice product teams cannot avoid much longer. If an assistant is meant to remember a user across sessions, it must decide what to remember: words only, or words plus how they were said. Keeping paralinguistic signals improves performance on the benchmark, but it also means storing more personal information, for longer, about emotion and voice characteristics. Running recognition on-device reduces one privacy risk, by keeping audio local, while remembering tone across sessions creates another, by keeping a richer profile. The honest product question is not whether voice assistants should remember how you sounded. It is what they remember, for how long, and who can see it, because the transcript alone was never the whole conversation. [cites: 1793346404577730094, 1793360486738224056, 1793359933822673187]

## What is still unresolved

- Whether paralinguistic memory helps in real conversations, or only on the benchmark's structured multi-session tasks.
- How long paralinguistic profiles should be retained, and whether users can inspect or delete them, which the preprints do not address.
- Whether on-device recognition accuracy holds in noisy real rooms, rather than in the tested benchmark and microcontroller conditions.

## Decision card

- **Useful for:** Developers building voice assistants or wearables; Product teams deciding what voice data to retain; Edge AI teams deploying speech on constrained hardware
- **Not for:** Teams building text-only chat products; Anyone needing a privacy compliance ruling
- **Compare against:** Text-only pipelines that discard audio cues; Cloud speech recognition for every utterance; General-purpose speech models without personalisation

## Provenance ledger

| Manifest id | Title | Source | Claims used | Published |
|---|---|---|---|---|
| `1793346404577730094` | VoiceLongMemEval: Do Assistants Remember How You Sounded? | [arxiv-llm-evals-benchmarks](https://arxiv.org/pdf/2609.00570v3) | 1793346826514818865, 1793346826521936894, 1793346826527492110 | 10-30 00:00 |
| `1793360486738224056` | Depth through Recurrence: Towards Ultra-Efficient On-Device ASR | [arxiv-model-efficiency-engineering](https://arxiv.org/pdf/2610.37833v1) | 1793360718298592790, 1793360718300308661, 1793360718302596665, 1793360718304512107 | 10-30 00:00 |
| `1793359933822673187` | Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4 | [arxiv-ai-infra-inference-ops](https://arxiv.org/pdf/2610.38739v1) | 1793360197539562656, 1793360197543300011, 1793360197551378659 | 10-30 00:00 |

## Run these Manifests yourself

Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.

1. **Get a Synorb Key.** Click Generate Free Agent Key at https://synorb.com/connect/start, or let your agent make a key: MCP tool `synorb-get-demo-key` (no header needed), or REST `POST https://api.synorb.com/api/connect/demo-key` with a JSON body such as `{"client":"your-agent-name"}`. The key is the `api_key` field of the answer. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; all of them are starter Beacons.
2. **Read each cited Manifest by id.** REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief.
3. **Check the claim.** Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them.

Cited Manifest ids:

- `1793346404577730094`
- `1793360486738224056`
- `1793359933822673187`

```sh
for id in 1793346404577730094 1793360486738224056 1793359933822673187; do
  curl -s -H "Authorization: Bearer $SYNORB_KEY" "https://api.synorb.com/manifests/by-id/$id"
done
```

> **From Synorb** — Ask what Synorb covers. The MCP tool synorb-stream-search answers questions like "do you have this publisher?" without using any Manifest allowance. [Connect and ask](https://synorb.com/connect/start)

## Related stories

- [Training cost is now search cost: configs, rollouts, data mixes and optimisers all get cheaper searches](https://hangingcontext.com/stories/2026-10-31/training-budgets-are-search-budgets/)
- [The computer-use agent clicked too early: stale screens, moving goals and false finishes](https://hangingcontext.com/stories/2026-10-31/computer-agents-act-on-stale-screens/)
- [Synthetic users are fluent, confident, and wrong about the parts of life the internet never wrote down](https://hangingcontext.com/stories/2026-10-31/synthetic-data-blind-spots/)
- [Serving is the new model race: Baseten and NVIDIA squeeze seconds out of weight syncs and storage](https://hangingcontext.com/stories/2026-10-31/inference-serving-squeezes-seconds/)

## Links

- HTML: https://hangingcontext.com/stories/2026-10-31/voice-assistants-forget-how-you-sounded/
- JSON: https://hangingcontext.com/stories/2026-10-31/voice-assistants-forget-how-you-sounded/index.json
- Markdown: https://hangingcontext.com/stories/2026-10-31/voice-assistants-forget-how-you-sounded/index.md

Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.

---
Reuse (terms v1): Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product. Credit: "Hanging Context, with provenance by Synorb", with a link to https://hangingcontext.com/stories/2026-10-31/voice-assistants-forget-how-you-sounded/. Terms: https://hangingcontext.com/terms/reuse/