Commit Graph

9 Commits

Author SHA1 Message Date
Christian Gick
efb976a27c feat: activity video track (pulsing orb) for voice sessions
- ActivityVideoPublisher renders animated orb on 160x120 canvas
- Integrated into both agent.py and voice.py
- Updates confluence-collab submodule
2026-03-06 15:58:51 +00:00
Christian Gick
1000891a97 fix: Improve voice noise tolerance and focus on latest message
- Raise VAD thresholds (activation 0.65→0.75, min speech 0.4→0.6s,
  min silence 0.55→0.65s) to reduce false triggers from background noise
- Add "focus on latest message" instruction to all prompts (voice + text)
- Add "greet and wait" behavior for new conversations instead of auto-continuing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:30:14 +02:00
Christian Gick
7791a5ba8e feat: add Confluence recent pages + Sentry error tracking (MAT-58, MAT-59)
MAT-58: Add recent_confluence_pages tool to both voice and text chat.
Shows last 5 recently modified pages so users can pick directly
instead of having to search every time.

MAT-59: Integrate sentry-sdk in all three entry points (agent.py,
bot.py, voice.py). SENTRY_DSN env var, traces at 10% sample rate.
Requires creating project in Sentry UI and setting DSN.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 08:44:57 +02:00
Christian Gick
fc3d915939 feat(e2ee): Add HKDF E2EE support for Element Call compatibility
Element Call uses HKDF-SHA256 + AES-128-GCM for frame encryption,
while the LiveKit Rust SDK defaults to PBKDF2 + AES-256-GCM.

- Multi-stage Dockerfile builds patched Rust FFI from EC-compat fork
- Generates Python protobuf bindings with new fields
- patch_sdk.py modifies installed livekit-rtc for new proto fields
- agent.py passes E2EE options with HKDF to ctx.connect()
- bot.py exchanges encryption keys via Matrix state events
- Separate Dockerfile.bot for bot service (no Rust build needed)

Ref: livekit/rust-sdks#904, livekit/python-sdks#570

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:29:06 +02:00
Christian Gick
2917f6515a fix: set agent_name via rtc_session decorator, revert CLI flag
AgentServer in livekit-agents 1.4.x does not support --agent-name CLI
flag. The agent_name must be set on @server.rtc_session() decorator.
Also reverts docker-compose.yml command back to plain python agent.py start.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:25:34 +02:00
Christian Gick
7032fef5c4 fix: use standard ctx.connect() for audio pipeline (CF-1170)
Custom rtc.Room skipped ctx.connect(), leaving framework audio
input pipeline uninitialized. STT/VAD never received audio frames.
Switch to standard approach: ctx.connect() + ctx.room.
Added debug event logging for speech pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 13:55:15 +02:00
Christian Gick
ee4efd01ef fix: agent cleanup on disconnect + targeted audio input
- Agent disconnects custom room when all real participants leave
  (prevents zombie participants blocking auto-dispatch)
- Bot sends m.call.member state event on call detection
  (Element Call shows bot as joined)
- Use RoomInputOptions(participant_identity=...) to target real user
  audio input (framework agent-AJ_xxx participant was confusing RoomIO)
- Removed incorrect bot dispatch (Matrix room ID != LiveKit room name)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 13:41:53 +02:00
Christian Gick
7bc7318c5b refactor: Use ElevenLabs directly for both STT and TTS
- STT: elevenlabs.STT() (Scribe) instead of Whisper via LiteLLM
- TTS: elevenlabs.TTS() (already direct)
- LLM: still routed through LiteLLM/OpenRouter
- No extra API accounts needed — only ElevenLabs + OpenRouter

CF-1147

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 07:37:44 +02:00
Christian Gick
fa65fbeb3d feat: Matrix AI voice agent (LiveKit + LiteLLM)
Bot @ai:agiliton.eu accepts room invites, dispatches LiveKit agent.
Agent joins call with STT (Groq Whisper) → LLM (Sonnet) → TTS (ElevenLabs)
pipeline, all routed through LiteLLM.

CF-1147

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 07:31:52 +02:00