Docs

Under the hood.

Skim the architecture and design choices to learn how Rosey is built.

How it works

Every channel hands off to the same agent loop. The agent reads a roster, writes to a shared memory file, and schedules anything time-shaped onto a persistent job queue. Replies fan back out to whichever channel makes sense.

Telegram webhook · /telegram WhatsApp · Cloud API webhook · /whatsapp WhatsApp · Groups Baileys sidecar Alexa skill · /alexa Agent loop Claude · roster-aware small set of family tools stays quiet in groups Shared memory household.md · knowledge/ single source of truth Persistent scheduler APScheduler + SQLite fire · escalate · miss scheduled fires fan back through the channels
Python · Quart
async webhook server
Anthropic Claude
agent reasoning
APScheduler
persistent reminders
Node · Baileys
WhatsApp groups sidecar
Memory

A filesystem the agent actually owns.

Rosey's long-term memory is a directory of plain markdown files, managed by the agent through Anthropic's memory tool. No vector database, no embeddings — just files the agent reads and writes turn by turn, organized however helps it most. The approach follows Anthropic's context engineering best practices: inline a small catalog so the agent knows what's available, then let it pull just the files relevant to the current turn.

01 Files, not databases

Everything Rosey remembers lives under a single /memories directory: household.md, reminders.md, events.md, a knowledge/ folder of one-topic-per-file notes, per-sender thread snippets, plus whatever else the agent decides to organize. You can cat, grep, edit, or back it up like any other file system. No proprietary store, no schema to migrate.

02 Just-in-time context, not everything at once

On every turn the system prompt inlines a small snapshot of the memory directory — a catalog of filenames plus a tiny per-topic index. The agent picks what it needs and reads only those files. Long-lived families with thousands of memory entries stay cheap to run because the agent never loads everything into context.

03 Plain markdown is the format

Memory files are human-readable on purpose. You can audit what Rosey remembers about your family at any time, edit anything by hand, and walk away with the full corpus if you stop using her — it's just text. Reminders, events, and lists all follow simple line formats so you can write them yourself too.

04 Isolation by directory

One household, one directory on disk. Self-hosted, it sits on your machine. On our hosting, it lives in a partition keyed to your household ID — never co-mingled with another family's data. Hosted text messages also pass through a local redaction layer that swaps common PII for placeholders before model calls, with the private mapping kept outside the agent's memory folder.

/memories layout
# the household's durable state /memories/ ├── household.md # who's in the family ├── reminders.md # scheduled nudges ├── events.md # shared calendar ├── groceries/ │ ├── list.md │ └── history.md ├── pantry.md ├── knowledge/ │ ├── INDEX.md # topic catalog │ ├── pediatrician.md │ ├── wifi.md │ └── ... └── threads/ └── tg:123456.md # per-sender history
tools available to the agent
# Anthropic's memory tool: 6 operations memory.view(path) # list a directory memory.read(path) # read a file memory.write(path, text) # create / replace memory.str_replace(path, old, new) memory.create_dir(path) memory.delete(path)
Design choices

Prioritize reliability and privacy.

01 Channels are independent

Each channel is a thin webhook adapter that converts to a canonical identifier (tg:, wa:, alexa:) and hands off. WhatsApp going down doesn't take Telegram with it — including the reminder fan-out.

02 Roster maps people, not devices

One markdown file lists each person's channel identifiers. @Alex in a reminder body resolves to every channel they're on, in parallel. New phone? Edit one line.

03 Reminders survive restarts

APScheduler with a SQLite jobstore on a Fly volume. Each reminder is three jobs: fire, escalate, miss. The container can crash mid-schedule and reconcile picks up where it left off.

04 Group chat without the noise

A fuzzy gate runs before the agent in groups: only addressed messages get a reply. Rosey doesn't interrupt; she answers when summoned.

household.md
# Family - **Alex** — tg:8600355980, wa:+15048755536 - **Sam** — wa:+15049991122 - **Maya** — (too young for a phone) # Pets - Mishka, dog, beagle mix, vet: Pawsitive Care (415) 555-0142 # Sitters - Priya — wa:+14155552288, $22/hr, prefers texts
scheduler.py · the reminder lifecycle
def schedule_reminder(text, when, recipients): # 3 jobs, one row each, persisted to SQLite fire = trigger(when) escalate = trigger(when + "+10m") miss = trigger(when + "+1h") for r in recipients: store.add("fire", fire, r, text) store.add("escalate", escalate, r, text) store.add("miss", miss, r, text) # on boot: def reconcile(): # re-attach jobs that should still fire, # skip ones already acknowledged. ...
Setup

Running Rosey, end to end.

About an hour if you've shipped a webhook before. Telegram is the easiest starting channel; WhatsApp and Alexa are bolt-ons you can add later.

01 Clone the repo

git clone https://github.com/ankittandon/rosey && cd rosey

02 Get a Telegram bot token from @BotFather

Free, ~30 seconds. Open Telegram, message @BotFather with /newbot, and save the token he hands you (looks like 1234567890:AAEh…).

03 Get an Anthropic API key

Head to platform.claude.com → API Keys, create one, and add a few dollars of credit. See Costs below for what to expect.

04 Deploy to Fly.io

Install flyctl, then fly launch, paste your secrets (ANTHROPIC_API_KEY, TELEGRAM_BOT_TOKEN, TELEGRAM_WEBHOOK_URL), and fly deploy. The webhook URL secret switches the bot from polling to webhook mode automatically.

05 Edit memories/household.md with your family

Each person sends /start to the bot once — Rosey replies with their Telegram chat ID. Paste them into household.md alongside their name. If the file's empty, Rosey trusts everyone (fine for initial testing).

06 Text your bot

Open Telegram, find your bot by username, send a message. She'll respond. From there, add WhatsApp / Alexa / group chats as needed — they're optional bolt-ons documented in the repo's README.