How close the AI's suggestions are to what you actually send — and where to teach it. The more you train it, the closer to 100%.
Reply training stats are temporarily unavailable; showing the rest of Insights with empty reply defaults. Scheduling, automation harness, and available Convex signals still render below.
One operating grade across reply quality, scheduling accuracy, and live source coverage. This is the daily answer to: can we automate more without losing control?
Customer reply events are missing. Keep the Customer Replied workflow published before adding more automation.
No measurable daily activity yet.
The system needs dashboard sends, graded drafts, or schedule recommendations today before a useful report can be produced.
Runtime ledger for automation decisions: what was ready, what ran, what was blocked, and which rules should be promoted next. This is the harness layer that lets Aveyo increase autonomy without guessing.
Operator's modal corrections become scheduler eval candidates. Good clicks are tracked separately from wrong slot, stale, not confirmed, declined, ambiguous, and missed-confirmed cases.
Safe replies can proceed through Convex when live gates are enabled. Uncertain replies are prompted here with the exact reason, and each decision feeds the reply training ledger.
No current reply is ready for action; keep warming drafts and collecting reviewed outcomes.
The eight workstreams that make this measurable: better retrieval, graded replies, automation readiness, scheduling accuracy, source history, rule extraction, Insight visibility, and cost control.
Cheap vectors handle recall. Codex handles final judgment before the prompt sees examples.
Reranker live; waiting on matches · Keep accepted replies, Operator edits, outbound sends, and rules flowing into Convex; use Codex only to reorder the short candidate list.
0 compared, 0% useful, 0 bypassed.
0% F grade · Send more replies from the dashboard so suggested-vs-sent pairs accumulate.
Converts accepted replies, useful edits, schedule matches, and route-aware savings into time and dollar value. Conservative assumptions are visible so this can support a monthly retainer conversation without hiding the math.
No drafts rated in this window yet. Approve or edit a few proposed replies and the AI's accuracy shows up here.
Scores suggested replies against what Operator actually sends. The automation score is stricter: exact matches, useful edits, coverage, and enough samples all matter.
Repeated misses become rule candidates. Response types graduate only when the evidence says they are ready.
Numerator/denominator audit. Pending, duplicates, internal/test rows, invalid addresses, and manual exclusions stay visible here without skewing the score.
Local SQLite store · 0 drafts · 0 outbound events · 0 messages stored
Readiness requires grade, confidence, sample volume, and failure review.
0/0 lanes ready · Collect more paired examples until response types have enough evidence to graduate.
Appointment telemetry is required before scheduling automation can be trusted.
0 appointment events · Compare suggested booking windows against actual appointment dates, then separate route-aware wins from misses.
Historical source-system data is the fastest way to raise measurement confidence without waiting weeks.
0 GHL · 0 Jobber events · Wire Jobber webhooks or direct API sync next; GHL alone cannot prove operational savings.
Rules are extracted from repeated misses, not vibes.
0 patterns · 0 critical · Keep mining edited/rejected drafts for reusable rules, tags, and triggers.
The dashboard now ties learning quality to operational value and automation readiness.
8 workstreams visible · Use this board as the Monday retainer proof: grade, value, readiness, missed savings, and next action.
Performance is measured as value per dollar and time saved, not just model accuracy.
0m saved · $0 · Track rerank cost separately from draft cost, then compare it against reply minutes saved and route minutes saved.
Learning needs more paired reply samples before the trend is trustworthy. Compares the latest 1 day block against the previous block, using only observed suggested-vs-sent outcomes.
Send the next replies from the dashboard and approve/edit the drafts so every suggestion gets a measured outcome.
Value model: approved/verbatim replies save 2 minutes, useful edits save 1 minute, exact or same-day scheduling matches save 6 minutes, and route savings count when the scheduler records avoided drive time.
