- POST /notify: send encrypted message to any room
- GET /messages: read decrypted messages from any room
- GET /health: health check
- Authenticated via BOT_API_KEY header
- Port 9100 exposed in docker-compose
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element X only reads encryption keys from encrypted to-device
messages, not room events or call.member state. Bot now sends
its key via Olm-encrypted to-device to all call participants,
matching Element Call's encryptAndSendToDevice behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element X sends keys as a single dict {index, key} not a list
[{index, key}]. The handler iterated over dict keys ('index','key')
instead of the actual key data. Also extracts device_id from
member.claimed_device_id (Element X format).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
nio's _handle_olm_event silently drops unknown Olm event types
with 'Received unsupported Olm event'. Element X sends E2EE call
keys as encrypted to-device io.element.call.encryption_keys events.
After Olm decryption they were dropped before reaching any callback.
Patch intercepts the drop and forwards to on_to_device_unknown handler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element X sends E2EE keys via encrypted to-device messages targeting
the device_id from the call.member state event. Bot was advertising
device_id='AIBOT' but its actual Matrix session is on device
'PEYRKFEXFP'. Keys were sent to a non-existent device.
Now uses the real device_id from nio credentials so Element X's
encryptAndSendToDevice reaches the correct device.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element X (26.03.3+) sends io.element.call.encryption_keys as
to-device messages, not room timeline events. Added
UnknownToDeviceEvent callback to catch these and deliver keys
to active voice sessions.
Also added m.room.encrypted decryption attempt in timeline scan
as fallback for older Element versions that send encrypted timeline
events.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element X embeds E2EE keys inside memberships[].encryption_keys,
not at the top level of the call.member state event content.
Bot was only checking content.encryption_keys, so it never found
the caller's key — causing 'Warten auf Medien' (waiting for media)
because encrypted audio couldn't be decrypted.
- Added _extract_enc_keys_from_content() helper handling both formats
- Updated on_unknown handler, VoiceSession creation, and key fetch
- Bot now publishes keys in both formats for compatibility
- Updated voice.py state fetch to check memberships[] fallback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Element Call v0.17+ embeds encryption_keys in call.member state events
instead of separate timeline events. In E2EE rooms, timeline events are
encrypted and the bot HTTP fetch cannot decrypt them, causing DEC_FAILED.
- Extract caller keys from call.member state event on join
- Embed bot key in call.member state event
- Check call.member state in key fetch (before timeline fallback)
- Handle key updates in call.member during active calls
- Update voice.py key poller to check call.member state first
- Add debug logging for UnknownEvent types in call rooms
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Same pattern as files: download and cache in _recent_images without
responding. When user next @mentions the bot, the cached image is
available as context. Applied to both plain and encrypted image handlers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Files uploaded to group rooms are now downloaded, parsed, and stored
in _room_document_context even without @mention. When the user later
mentions the bot, the document context is automatically included.
Previously files were silently dropped if the caption didn't contain
a mention, so the bot would say it can't access uploaded PDFs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bot user is @ai:agiliton.eu but display name is 'Claude'.
Element renders mentions using display name, so the old check
for 'ai' in message body never matched '@Claude: ...' messages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add pitrader_script executor for running PITrader scripts (pi-scan,
playbook, execute_trades) as pipeline steps with vault credential
injection and JSON output capture.
Extend claude_prompt step with vision support (image_b64 in trigger
context). Add image pipeline trigger to on_image_message handler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
matrix-nio parses m.reaction as ReactionEvent with .reacts_to and .key
fields. UnknownEvent handler never fired for reactions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cron package that syncs jobs from matrixhost portal API, schedules execution
with timezone-aware timing, and posts results to Matrix rooms. Includes
Brave Search, reminder, and browser scrape (placeholder) executors with
formatter. 31 pytest tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When text bot captures a frame during active call and gets 8x8 garbage
(E2EE not yet decrypted), retry once after 2s to allow key propagation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Render scanned/image-based PDF pages to PNG at 200 DPI and send to AI
model as image content when text extraction returns empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8x8 frames are encrypted garbage from E2EE video decryption failure.
Skip frames < 64x64 to avoid sending black/noise images to the LLM.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Text bot can now capture video frames from active call when user
types vision-related queries ("siehst du meinen bildschirm", etc.)
2. Voice transcript injected into text bot context during active calls
3. Text messages injected into voice transcript with [typed in chat] prefix
4. Bot text replies injected back into voice transcript
This enables seamless context sharing between voice calls and text chat.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add HTTPS instruction to system prompt so LLM never generates http:// links.
Fix bare matrixhost.eu/settings references to use full https:// URLs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add scheduled messages/reminders system:
- New scheduled_messages table in memory-service with CRUD endpoints
- schedule_message, list_reminders, cancel_reminder tools for the bot
- Background scheduler loop (30s) sends due reminders automatically
- Supports one-time, daily, weekly, weekdays, monthly repeat patterns
Make article URL handling non-blocking:
- Show 3 options (discuss, text summary, audio) instead of forcing audio wizard
- Default to passing article context to AI if user just keeps chatting
- New AWAITING_LANGUAGE state for cleaner audio flow FSM
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integrates _ensure_cross_signing() into Bot.start() flow. On first run, generates
and uploads cross-signing keys, then signs the bot device. On subsequent restarts,
detects existing cross-signatures and skips. Seeds persisted for device recovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Allows the bot to paginate back up to 500 messages in a room
to find specific content, beyond the default 10-message context window.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Route ~90% of simple chat to claude-haiku (4x cheaper), escalate to
claude-sonnet for code blocks, long messages, technical keywords,
multimodal, and explicit requests. Sentry tags track model_used,
escalation_reason, and token usage breadcrumbs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Make user_id required on all request models with field validators
- Always include user_id in WHERE clause for chunk queries (prevents cross-user data leak)
- Add bearer token auth on all endpoints except /health
- Add composite index on (user_id, room_id) for conversation_chunks
- Bot: guard query_chunks with sender check, pass room_id, send auth token
- Docker: pass MEMORY_SERVICE_TOKEN to both bot and memory-service
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Scanned passport PDFs have completely garbled OCR text that makes
the LLM think they're not passports, even though the AI-generated
title and summary correctly identify them. Added explicit instruction
to trust title/summary fields.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When doc_context is available, limit history to just 4 messages (2 exchanges)
to prevent stale answer patterns from overriding fresh document search results.
Without RAG results, keep 10 messages for normal conversation context.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The AI reply often contains full document content (passport details, etc.)
which the memory extraction LLM incorrectly stores as user facts. Limiting
to 200 chars avoids including document content while keeping the gist.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
30 messages of "only one passport" history overwhelmed fresh RAG results.
Reducing to 10 messages (5 exchanges) provides enough conversation context
without letting stale patterns dominate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two changes:
1. Reorder messages: doc_context now placed RIGHT BEFORE the user message
(after chat history), so fresh search results override historical patterns
where the bot repeatedly said "only one passport"
2. Strengthen doc_context instructions: explicitly tell LLM that fresh search
results override chat history, and to list ALL matching documents
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The memory extraction prompt was extracting facts from RAG search results
(e.g., passport holder names) and storing them as if they were facts about
the user. Added explicit instruction to only extract facts the user directly
states about themselves.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With only 3 results, passport queries often miss family members since
all passport files have similar low relevance scores. Increasing to 10
ensures all related documents are included in LLM context.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Raise VAD thresholds (activation 0.65→0.75, min speech 0.4→0.6s,
min silence 0.55→0.65s) to reduce false triggers from background noise
- Add "focus on latest message" instruction to all prompts (voice + text)
- Add "greet and wait" behavior for new conversations instead of auto-continuing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add rag_key_manager.py: stores encryption key in private E2EE room
- Bot loads key from Matrix on startup, injects into RAG via portal proxy
- No plaintext key on disk (removed RAG_ENCRYPTION_KEY from .env)
- Pass owner_id (matrix_user_id) to RAG search for user isolation
- Stronger format_context instructions for source link rendering
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DocumentRAG class now prefers local RAG endpoint (RAG_ENDPOINT env var)
over central portal API. When RAG_ENDPOINT is set, searches go to the
customer VM encrypted RAG service on localhost:8765. Falls back to
portal API for unmigrated customers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When v2 API returns 401 (scope mismatch with classic OAuth tokens),
fall back to v1 REST API which accepts classic scopes. Also provides
clear error message asking user to re-authorize if both fail.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch from /wiki/rest/api/content to /wiki/api/v2/pages.
V2 requires space ID instead of key, so resolve via /api/v2/spaces first.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove all !ai command handling (help, models, set-model, search, etc)
- Remove legacy user_keys system (WildFiles API key storage)
- Remove docs connect/disconnect commands
- Bot now responds to all DM messages and @mentions naturally
- Settings managed exclusively via matrixhost.eu portal
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove WILDFILES_BASE_URL and WILDFILES_ORG env vars
- Rename _wildfiles_org_cache to _documents_cache
- Update _has_documents() to use provider=documents
- Remove "wildfiles connect" command alias (keep "docs connect")
- Remove WILDFILES env vars from docker-compose.yml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DocumentRAG now calls MatrixHost /api/bot/documents/search instead of
the WildFiles API. Removes device auth flow and legacy org provisioning.
Bot authenticates via existing BOT_API_KEY pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Connect the Matrix AI bot to customer WildFiles orgs via the MatrixHost
portal API instead of requiring manual !ai wildfiles connect. The bot
now auto-resolves the user document org on every message, enabling
seamless RAG document search for all MatrixHost customers.
- Add _get_wildfiles_org() with portal API lookup and session cache
- Update DocumentRAG.search() to accept org_slug (no API key needed)
- Add DocumentRAG.get_org_stats() for org-based stats
- Update context building to use portal org lookup with legacy fallback
- Add !ai docs connect/disconnect aliases
- Rebrand all user-facing messages from WildFiles to Documents
- !ai wildfiles connect now checks portal first, shows auto-connect msg
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When browse_url fails with DNS resolution error (common with STT-misrecognized
domain names like "klicksports" instead of "clicksports"), automatically try a
web search to find the correct domain and retry. Applied to both text and voice bot.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add semantic search over past conversations alongside existing memory facts.
New conversation_chunks table stores user-assistant exchanges with LLM-generated
summaries embedded for retrieval. Bot queries chunks on each message and injects
relevant past conversations into the system prompt. New exchanges are indexed
automatically after each bot response.
Memory-service: /chunks/store, /chunks/query, /chunks/bulk-store endpoints
Bot: chunk query + formatting, live indexing via asyncio.gather with memory extraction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both bots can now fetch and read web pages via browse_url tool.
Uses httpx + BeautifulSoup to extract clean text from HTML.
Complements existing web_search (Brave) with full page reading.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
