Commit Graph

108 Commits

Author SHA1 Message Date
Christian Gick
08f4e115b9 fix: filter own key events, fix RoomOptions None, wait for participant
- Skip bot own encryption_keys events in on_unknown handler
- Always pass valid RoomOptions to AgentSession.start()
- Wait up to 10s for remote participant to connect before starting pipeline

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:33:12 +02:00
Christian Gick
6e1e9839cc fix: use timeline events for E2EE key exchange (not state events)
Element Call distributes encryption keys as timeline events, not room
state events. Changed bot to publish keys via room_send and fetch from
/messages endpoint instead of /state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:28:56 +02:00
Christian Gick
4e1e372ca2 fix: use caller's E2EE key (not own), fetch via HTTP API
All participants must use the SAME shared key. Bot was generating
its own key which couldn't decrypt user's audio. Now:
1. Fetch caller's key from room state via HTTP API
2. Fall back to waiting for key via sync handler
3. Publish the SAME key back (not a new one)
4. Only connect with E2EE if key available

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:17:33 +02:00
Christian Gick
753d6543d4 fix: generate and publish E2EE key, always connect with encryption
Element Call encrypts media by default. Bot must:
1. Generate its own 32-byte E2EE key
2. Publish it to room state (io.element.call.encryption_keys)
3. Connect to LiveKit with HKDF E2EE enabled
4. Use caller's key when received, own key as fallback

This fixes: Nicht verschlüsselt warning, silent audio (encrypted
frames couldn't be decoded by VAD/STT)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:06:34 +02:00
Christian Gick
74758a3f13 debug: enable livekit.agents debug logging for STT/VAD diagnosis
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 17:02:24 +02:00
Christian Gick
75970fc06b fix: link AgentSession to remote participant + debug speech events
- Pass participant_identity via RoomOptions so AgentSession knows
  which audio track to consume (was silently ignoring user audio)
- Add USER_SPEECH and AGENT_SPEECH event handlers for debugging
- Simplify greeting to exact text to prevent hallucination
- Use httpx for room state scan (nio API was unreliable)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 15:03:24 +02:00
Christian Gick
85df4b295f fix: E2EE key timing + verbose logging + shorter greeting
- Reorder: send call member event BEFORE creating VoiceSession
- Store VoiceSession BEFORE start so sync handler can forward keys
- Increase E2EE key wait from 3s to 10s
- Add INFO-level logging for key lookup + room state scan via HTTP API
- Tighten voice system prompt to prevent long rambling greetings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 14:55:52 +02:00
Christian Gick
e5e8b56482 fix(e2ee): Add E2EE HKDF to voice.py, bot uses patched Dockerfile
voice.py runs in bot container, not agent container.
- Wait 3s for encryption key before connecting
- Build E2EE options with HKDF when key received
- Bot container now uses patched Dockerfile (needs FFI)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 17:13:53 +02:00