christian/matrix-ai-agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: retry video frame capture after 2s on E2EE decryption failure

Browse Source 
When text bot captures a frame during active call and gets 8x8 garbage
(E2EE not yet decrypted), retry once after 2s to allow key propagation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Gick
2026-03-10 14:20:21 +02:00
parent 1a0a2ec305
commit cdd876fe24
1 changed files with 26 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
bot.py
View File

@@ -1881,6 +1881,32 @@ class Bot:
logger.info("Captured %dx%d frame from active call for text query", rgba.width, rgba.height)
else:
logger.warning("Frame too small (%dx%d) — E2EE video decryption likely failed", rgba.width, rgba.height)
# Retry once after 2s — E2EE key may still be propagating
await asyncio.sleep(2)
try:
stream2 = rtc.VideoStream(vs._video_track)
frame2 = None
async for f2 in stream2:
frame2 = f2
break
try:
await stream2.aclose()
except Exception:
pass
if frame2:
vf2 = getattr(frame2, 'frame', frame2)
rgba2 = vf2.convert(rtc.VideoBufferType.RGBA)
if rgba2.width >= 64 and rgba2.height >= 64:
img2 = Image.frombytes("RGBA", (rgba2.width, rgba2.height), bytes(rgba2.data))
buf2 = io.BytesIO()
img2.convert("RGB").save(buf2, format="JPEG", quality=85)
img_b64 = base64.b64encode(buf2.getvalue()).decode()
image_data = (img_b64, "image/jpeg")
logger.info("Retry captured %dx%d frame from call", rgba2.width, rgba2.height)
else:
logger.warning("Retry still too small (%dx%d)", rgba2.width, rgba2.height)
except Exception as exc2:
logger.warning("Frame retry failed: %s", exc2)
except Exception as exc:
logger.warning("Failed to capture frame from call: %s", exc)