create AI_template

docs/project-overview.md

@@ -0,0 +1,82 @@
+# Project Overview: Starter Template (Provider‑agnostic)
+
+---
+**Last Updated:** 2025-01-17  
+**Phase:** Phase 0 (Planning)  
+**Status:** Approved  
+**Owner:** Product Team  
+**References:**  
+- `/docs/phases-plan.md` (development roadmap)  
+- `/docs/content-structure.md` (app/screen structure)
+---
+
+## 1. Project Goal
+
+Provide a universal starting point for new products that leverage AI assistance. The template outlines how to structure documentation and technical decisions for projects that may include data ingestion, processing/classification (rules + embeddings + LLM), human approvals, reporting, and optional billing.
+
+Emphasis areas: trust (auditability, optional reasoning traces), speed (near‑real‑time processing), and clarity (human approvals and transparent reporting), adaptable to your domain.
+
+## 2. Target Audience
+
+- Define your primary users and stakeholders for the target domain.
+- Typical teams value:
+  - automation with explainability (reasoning traces and event logs);
+  - reliability (idempotent ingestion, safe retries);
+  - compliance (audit trail, role‑based access).
+
+## 3. Unique Value Proposition
+
+1. **Automated processing with explainability**  
+   Rules + embeddings (`pgvector`) first, LLM fallback via a single helper, with optional `reasoning_trace` per record.
+2. **Human‑in‑the‑loop approvals**  
+   Review/override UI with optional rule creation; every action is logged (e.g., `source_agent`).
+3. **Reliable ingestion**  
+   OAuth2/webhooks for external providers; idempotent event handling; queue workers for retries.
+4. **Reporting & optional billing**  
+   Dashboards/exports per tenant; subscription management with tenant‑scoped access control.
+5. **Multi‑tenant and secure by design**  
+   Tenant isolation, RBAC, webhook verification, audit log (`EventLog`).
+
+## 4. Key Features (example template)
+
+### 4.1. Onboarding & Integrations
+- Tenant signup/login (Clerk/Auth.js or equivalent).
+- Connect external data sources (OAuth2, webhooks, file uploads, etc.).
+- Configure subscription/billing if applicable.
+- Example domain model (multi‑tenant): `Tenant`, `User`, `Record`, `Rule`, `Attachment`, `Report`, `EventLog`, `Embedding`; `EventLog.source_agent` (e.g., default value), `Record.reasoning_trace` JSONB.
+
+### 4.2. Ingestion
+- Webhooks/workers normalize provider payloads into a unified schema.
+- Idempotent writes; deduplication; `INGESTED` events with `source_agent`.
+
+### 4.3. Classification/Processing
+- Rule engine first; embeddings similarity next; LLM fallback via `callLLM()`.
+- Optionally persist `reasoning_trace` JSONB on records; log `PROCESSED` (or similar).
+
+### 4.4. Approval & Rules
+- UI for reviewing/overriding outcomes; optional rule creation from overrides.
+- Log `APPROVED`, `RULE_CREATED`; track who/when/why.
+
+### 4.5. Reporting & Events
+- Dashboards and exports per tenant.
+- Read‑only `/api/events` for downstream agents and observability.
+
+### 4.6. Billing & Access
+- Subscriptions via a payment provider; tenant‑scoped roles.
+- Audit log (`EventLog`) with `source_agent`.
+
+## 5. Non‑Functional Requirements (High‑Level)
+
+- **Reliability:** idempotent ingestion/webhooks, durable queues, retries, monitoring.
+- **Security:** RBAC per tenant, webhook signature verification, secret management, audit/event logging; no raw card data stored.
+- **Performance:** near real‑time processing and acceptable latency for human approvals; fast UI for lists/filters.
+- **Explainability:** every automated decision includes `reasoning_trace`; events logged with `source_agent`.
+- **Localization:** primary UI in English; other locales optional.
+- **Operational tooling:** queue-based pipelines (BullMQ + Redis) with observability and dead-letter handling for ingestion/categorization/reporting jobs.
+
+## 6. Constraints and Assumptions
+
+- Web-first product (desktop + mobile web). Native apps not in scope for V1.
+- Payments via a provider; exact choice is project‑specific.
+- LLM calls should go through a single abstraction to swap models easily.
+- Multi‑agent readiness: consider `EventLog.source_agent` and optional `reasoning_trace`.