christian/matrix-ai-agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(MAT-140): Set E2EE decryption keys for video tracks, not just audio

Browse Source 
Video tracks (camera + screen share) were never getting E2EE keys set
via set_key() because the condition on track_subscribed only matched
audio tracks (kind==1). This caused DEC_FAILED for all video frames,
making look_at_screen return encrypted garbage or fail entirely.

Also added track source logging to distinguish camera vs screen share.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Gick
2026-03-09 16:06:52 +02:00
parent 3c06ededdf
commit f5e08257eb
1 changed files with 4 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
voice.py
View File

@@ -672,10 +672,12 @@ class VoiceSession:
# skips HKDF derivation → raw key stored → DEC_FAILED.
# Solution: set caller key HERE, after frame cryptor is initialized.
# Store video track for on-demand vision (look_at_screen tool)
# Screen share = source "screen_share" or "screenshare"; camera = "camera" or default
if int(t.kind) == 2: # video track (LiveKit: 1=audio, 2=video)
track_source = getattr(pub, 'source', None) or "unknown"
self._video_track = t
logger.info("Video track stored from %s for on-demand vision", p.identity)
if int(t.kind) == 1 and e2ee_opts is not None: # audio track only
logger.info("Video track stored from %s source=%s for on-demand vision", p.identity, track_source)
if int(t.kind) in (1, 2) and e2ee_opts is not None: # audio + video tracks
caller_id = p.identity
logger.info("E2EE_DIAG: track_subscribed for %s, have %d caller keys",
caller_id, len(self._caller_all_keys))