NoneType causes TypeError in patched room.py proto assignment.
Empty bytes is falsy so shared_key is not set in proto,
initializing key provider in per-participant mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
shared_key locks provider in shared-key mode, making set_key()
ineffective for per-participant decryption. Remove shared_key so
SDK initializes in per-participant mode. Also: failure_tolerance=-1
to prevent premature track closure on decrypt failures,
ratchet_window_size=16 to match Element Call.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
E2EE key setup may briefly appear as participant disconnect.
Keep session alive to allow audio to flow once keys are settled.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call uses per-participant keys, not shared key mode.
Bot now generates its own key, publishes it, and sets both
keys via key_provider.set_key() after connecting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call now rejects unencrypted audio. Use caller's key
as shared_key so both sides encrypt/decrypt with the same key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call uses per-participant keys, LiveKit Python SDK shared key mode
cannot properly decrypt. Reverting to working state (no LiveKit E2EE).
Bot still publishes keys so Element Call shows encryption indicator.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both bot and caller must use the same key in shared key mode.
Bot now reuses caller's key and publishes it back, instead of
generating a separate bot key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-participant set_key alone with empty shared_key caused silent incoming audio.
Now connects with caller key as shared_key, then overlays per-participant keys.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Bot tells which model it uses when asked
- Injects current UTC datetime into prompt
- Responds in users language instead of always German
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call uses per-participant keys via MatrixKeyProvider.onSetEncryptionKey(),
not shared key mode. This was causing silence with E2EE enabled.
- Set bot's own key and caller's key separately via e2ee_manager.key_provider.set_key()
- Live-update caller key when received after connect
- Fallback to set_shared_key if per-participant API unavailable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Audio pipeline confirmed working without E2EE. E2EE key derivation
mismatch with Element Call needs separate investigation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bot now publishes the same key as the caller so both sides can decrypt.
Falls back to no-encryption if no caller key received.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Skip bot own encryption_keys events in on_unknown handler
- Always pass valid RoomOptions to AgentSession.start()
- Wait up to 10s for remote participant to connect before starting pipeline
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call distributes encryption keys as timeline events, not room
state events. Changed bot to publish keys via room_send and fetch from
/messages endpoint instead of /state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All participants must use the SAME shared key. Bot was generating
its own key which couldn't decrypt user's audio. Now:
1. Fetch caller's key from room state via HTTP API
2. Fall back to waiting for key via sync handler
3. Publish the SAME key back (not a new one)
4. Only connect with E2EE if key available
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call encrypts media by default. Bot must:
1. Generate its own 32-byte E2EE key
2. Publish it to room state (io.element.call.encryption_keys)
3. Connect to LiveKit with HKDF E2EE enabled
4. Use caller's key when received, own key as fallback
This fixes: Nicht verschlüsselt warning, silent audio (encrypted
frames couldn't be decoded by VAD/STT)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pass participant_identity via RoomOptions so AgentSession knows
which audio track to consume (was silently ignoring user audio)
- Add USER_SPEECH and AGENT_SPEECH event handlers for debugging
- Simplify greeting to exact text to prevent hallucination
- Use httpx for room state scan (nio API was unreliable)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Reorder: send call member event BEFORE creating VoiceSession
- Store VoiceSession BEFORE start so sync handler can forward keys
- Increase E2EE key wait from 3s to 10s
- Add INFO-level logging for key lookup + room state scan via HTTP API
- Tighten voice system prompt to prevent long rambling greetings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix state_key format: try @user:domain:DEVICE_ID (Element Call format),
then @user:domain, then scan all room state as fallback
- Publish bot E2EE key to room so Element shows encrypted status
- Extract caller device_id from call member event content
- Also fix pipecat-poc pipeline with context aggregators (CF-1579)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rust:latest produces FFI needing CXXABI_1.3.15 (GCC 14 libstdc++).
GCC 14 libstdc++ needs GLIBC 2.38. Bookworm only has 2.36.
Trixie has GLIBC 2.38+ — fixes the CXXABI_1.3.15 runtime error.
Also reverts to rust:latest since bookworm GCC 12 cant compile webrtc C++20.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rust:latest links against GLIBC_2.38 libstdc++ which is incompatible with bookworm.
rust:bookworm (1.93.1) produces FFI binary compatible with bookworm libstdc++.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Stop VoiceSession when call leave event received
- Copy libstdc++ from rust build stage to fix CXXABI_1.3.15 mismatch
- Read caller encryption key from room state before starting VoiceSession
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
voice.py runs in bot container, not agent container.
- Wait 3s for encryption key before connecting
- Build E2EE options with HKDF when key received
- Bot container now uses patched Dockerfile (needs FFI)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call uses HKDF-SHA256 + AES-128-GCM for frame encryption,
while the LiveKit Rust SDK defaults to PBKDF2 + AES-256-GCM.
- Multi-stage Dockerfile builds patched Rust FFI from EC-compat fork
- Generates Python protobuf bindings with new fields
- patch_sdk.py modifies installed livekit-rtc for new proto fields
- agent.py passes E2EE options with HKDF to ctx.connect()
- bot.py exchanges encryption keys via Matrix state events
- Separate Dockerfile.bot for bot service (no Rust build needed)
Ref: livekit/rust-sdks#904, livekit/python-sdks#570
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call uses SHA256(room_id + "|m.call#ROOM") encoded as unpadded
base64 for LiveKit room names (via lk-jwt-service). The bot was using
the raw Matrix room ID, causing agent and user to join different rooms.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add memory-service (FastAPI + pgvector) for semantic memory storage.
Bot now queries relevant memories per conversation instead of dumping all 50.
Includes migration script for existing JSON files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
