Add Skyvern container (public.ecr.aws) with dedicated PostgreSQL,
connected to LiteLLM proxy for LLM calls. Replace browser_executor
stub with full Skyvern API integration (create task, poll, extract).
Implement skyvern pipeline step for workflow chaining.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a video track is subscribed (screen share starts), Element Call
rotates the E2EE key. Instead of waiting for DEC_FAILED, proactively
poll the timeline for the new key (6x @ 500ms = 3s window).
Also reduce DEC_FAILED threshold from 3→1 and cooldown from 5s→2s
for faster recovery when the proactive poll misses the rotation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element appends U+FE0F to emoji reactions (👍️ vs 👍). Strip before
matching against approval map.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
matrix-nio parses m.reaction as ReactionEvent with .reacts_to and .key
fields. UnknownEvent handler never fired for reactions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Matrix needs formatted_body as HTML, not raw markdown. Added _md_to_html
for bold/italic/code conversion.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Separate 15s poll for manual triggers (lastStatus=pending) from the
5-minute full job sync. Run Now button now fires within seconds.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Require ALL criteria to clearly match. Exclude general articles,
category pages, unconfirmed locations. When in doubt, exclude.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LLM sometimes returns extra text around the JSON array. Use regex to
extract the array pattern instead of parsing the full response.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Brave Search results are passed through LiteLLM (claude-haiku) when
job config includes a `criteria` field. LLM returns indices of matching
results, filtering out noise before posting to Matrix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cron package that syncs jobs from matrixhost portal API, schedules execution
with timezone-aware timing, and posts results to Matrix rooms. Includes
Brave Search, reminder, and browser scrape (placeholder) executors with
formatter. 31 pytest tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace brittle exact-string matching with keyword/substring classifier
that handles edge cases (punctuation, partial matches, German variants).
Detect article language and present all prompts in the users language.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
'Audiozusammenfassung' and 'Textzusammenfassung' now correctly trigger
the audio/text summary flows instead of falling through to regular LLM
which says it can't create audio files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When text bot captures a frame during active call and gets 8x8 garbage
(E2EE not yet decrypted), retry once after 2s to allow key propagation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The re-fetch check was placed after the 5s cooldown return, so it never
executed. Now it triggers after 3+ DEC_FAILED regardless of cooldown.
Also relaxed stale key age filter from 60s to 300s to handle key
rotation during ongoing calls.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Element Call rotates E2EE keys by re-sending index 0 with a new value
when screen share starts. The LiveKit frame cryptor caches derived AES
keys per index, so overwriting index 0 does not force re-derivation.
Fix: detect when index 0 value changes and map to incrementing internal
index so the frame cryptor gets a fresh key slot. Sets all accumulated
keys on late arrival so cryptor can try both during transition.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Timeline key fetch now filters by sent_ts (max 60s age) to avoid
using keys from a previous call session
- After 3+ consecutive DEC_FAILED events, automatically re-fetches
key from timeline in case rotation happened
- Tracks DEC_FAILED count per participant, resets on OK
This should fix the issue where the bot picks up stale encryption keys
from previous calls and can't decrypt the current caller's audio.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Plays immediate spoken feedback so the user knows the bot is processing
their screen share / camera before the vision API responds.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The on_e2ee_state callback crashed with NameError on time.monotonic()
when video tracks (screen share) arrived, preventing E2EE key re-derivation
and causing the bot to miss screen-share related questions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Render scanned/image-based PDF pages to PNG at 200 DPI and send to AI
model as image content when text extraction returns empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: aggressive video re-keying (set_key at 0.3/0.8/2/5s intervals)
briefly cleared encryption_key between SetKey and HKDF callback, causing
DEC_FAILED oscillation. Single set_key per track subscription is sufficient.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR #904 callback-based HKDF hack only fired for the first frame cryptor
(audio), leaving video frame cryptors with PBKDF2 - DEC_FAILED oscillation.
PR #921 integrates HKDF natively at the WebRTC C++ level, applying uniformly
to all frame cryptors (audio + video).
Also removes aggressive video re-keying workaround and adds 5s cooldown
to DEC_FAILED re-keying handler to prevent tight loops.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Video frame cryptors may not be fully initialized when set_key() is
first called during on_track_subscribed. Audio works immediately but
video oscillates OK↔DEC_FAILED with the same key.
Add staggered re-keying at 0.3s, 0.8s, 2s, 5s after video track
subscription to ensure the key is applied after the frame cryptor
is fully ready.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
