1. Front Matter
Title: AI Idea Validator
Author: Sean Patrick Caintic (scorevi)
Updated by: Christian Denzon (AI Idea Validator overhaul, July – September 2026)
Reviewers: Joshua Tribiana Uriel, Krizha Onise Cortez, Patrick Babala, Clyde Ador
Created: February – March 2026
Last Updated: September 30, 2026
Status: Approved and Merged
References:
Issue: #24 — [1.2] AI Validator
Milestone: [1] Diagnose (Project Setup)
Sub-issues & related issues: #185 Gemini AI Backend Infrastructure, #186 Idea Validation & Curriculum Generation API, #187 Validator Frontend & UX, #188 Import Decision & Curriculum-to-Canvas Converter, #189 Visual Canvas Import Integration, #190 Test Coverage & QA, #191 Content Safety Layer, #192 Validation History, #245 Enhancements — Smart Merge, Structured Error Feedback & Validation Diff View, #285 Quest Mode Support, #286 Import Canvas — Quest Mode Support
Update PRs: #674 AI-generated content cards, choice edges & role gate, #679 Quest creation error handling & content card ID type, #681 Rendering, code block editor, Game Over flow & import merge fixes, #707 Reviewer queue bugs (#23/#24/#25), #721 ImportMergeDialog layout, #724 Creator route role enforcement, #778 Dashboard entry point & creator UI fixes, #840 Auth round-trips & suspension enforcement
2. Introduction & Goals
Problem Summary: Creators need a way to validate their quest topics before investing time in content creation. The AI Idea Validator is a Gemini-powered system that assesses topic viability and generates structured curriculum outlines including conflict points, crisis scenarios, and recommendations — all importable into the Visual Canvas editor. A July – September 2026 overhaul (PRs #674, #679, #681, #707, #721, #724, #778, #840) hardened the end-to-end flow: generated cards render with supported node types, import can merge into existing quests, the reviewer queue distinguishes AI-linked quests from manual ones, and the whole auth chain moved to Supabase Auth.
Goals:
Provide AI-powered topic validation with viability scoring (0-100)
Generate structured curriculum outlines with modules, in either linear (narrative) or exploration (branching) quest mode
Identify conflict points and crisis scenarios for engagement
Enable direct import of AI output to the Visual Canvas, including Smart Merge into an existing quest
Cache responses to reduce redundant API calls
Implement content safety filtering (local deterministic + Gemini AI layers)
Enforce platform-wide AI token quotas (sitewide, agency, member)
Non-Goals:
Real-time content generation during quest editing
AI-generated media assets (images, videos)
Automatic quest publishing without creator review
Glossary:
Viability Score: 0-100 rating of topic's educational potential
Curriculum Module: A unit of learning content with duration estimate
Conflict Point: Branching decision point for learner engagement (exploration mode)
Crisis Scenario: Assessment activity placed after specific modules (exploration mode)
Quest Mode:
linear(sequential narrative) orexploration(branching with game-over detours)Smart Merge: Import strategy that updates matching nodes in an existing canvas, skips duplicates, and appends genuinely new nodes
Canvas Metadata: React Flow nodes/edges JSON structure
3. High-Level Architecture
System Diagram:
Data Flow: Input → auth + role gate → token quota gate → Zod validation + input-quality gate → AI content safety (local hard-block → Gemini classifier) → cache check → (miss) Gemini call → parse structured JSON → mode-aware validation → persist to
idea_validations→ return result.Import path:
ValidatorOutput→ deterministiclib/canvas-importconversion → deterministic layout pass (POST /api/ai/compute-canvas-layout— a TypeScript-only engine, no AI call; the client keeps the converter's positions if it fails) →sessionStorage(ai-curriculum,ai-import-decision) → quest editor → Smart Merge / Replace All → React Flow nodes → content cards.
Technologies Used:
Google Gemini — topic validation, curriculum generation, and content safety classification (canvas layout is deterministic — no AI). The active model and API key are admin-configured in
gemini_config(DB defaultgemini-3.1-pro-preview;gemini-3.5-flashruntime fallback when no active row exists).Zod — input/output schema validation, including the input-quality gate inside
superRefineSupabase — Postgres persistence, caching, usage tracking, Storage, Row Level Security
Supabase Auth — authentication and session management (Clerk was fully decommissioned in September 2026; PRs #823, #826)
Next.js App Router + React Flow — canvas node conversion and rendering
Monaco editor (
@monaco-editor/react) and TipTap — code-block editor and reflection editor in the quest playerDOMPurify +
markdownToHTML— sanitized rendering of AI markdown output
4. Detailed Design & Implementation
Data Model / Schema:
idea_validations Table:
Column | Type | Description |
|---|---|---|
| UUID PK | Auto-generated |
| TEXT | Supabase Auth user UUID stored as text ( |
| TEXT | Original input topic |
| TEXT | Optional context |
| TEXT | Optional objectives |
| INTEGER | 0-100 score (CHECK constraint) |
| TEXT | AI reasoning |
| JSONB | Array of modules (mode-aware shape) |
| TEXT[] | Exploration mode: conflict items saved by the validator; legacy rows may contain plain strings |
| TEXT[] | Exploration mode: crisis items; legacy rows may contain plain strings |
| TEXT[] | Array of recommendation strings |
| TEXT | Estimated total duration (mirrors the numeric output field) |
| TEXT | Gemini model used (default |
| TEXT |
|
| TEXT | Linear mode only; NULL for exploration (migration |
| TIMESTAMPTZ | Auto-timestamp |
| TIMESTAMPTZ | Maintained by |
Final effective structure (after the February – March 2026 migrations):
idea_validations ( id uuid primary key default gen_random_uuid(), creator_id text not null, -- Supabase Auth user UUID (auth.users.id) topic text not null, context text, learning_objectives text, viability_score integer not null check (viability_score between 0 and 100), viability_rationale text not null, curriculum_outline jsonb not null, conflict_points text[], -- exploration mode crisis_scenarios text[], -- exploration mode recommendations text[], total_estimated_duration text, model_name text not null default 'unknown', quest_mode text not null default 'exploration' check (quest_mode in ('linear', 'exploration')), narrative_progression text, -- linear mode only created_at timestamptz not null default now(), updated_at timestamptz not null default now()); -- RLS (20260911_supabase_rls_auth_uid.sql sweep):-- SELECT / INSERT / UPDATE / DELETE all use: creator_id = auth.uid()::text
Implementation Notes:
Indexes:
idx_idea_validations_creator_created(creator_id,created_at DESC) andidx_idea_validations_creator_topic_created(creator_id,topic,created_at DESC) — the history sidebar and variant dedup read patterns.RLS: enabled; SELECT/INSERT/UPDATE/DELETE policies all use
creator_id = auth.uid()::textafter the September 2026 RLS sweep (20260911_supabase_rls_auth_uid.sql; previouslyauth.jwt() ->> 'user_id'). The API routes use the service-role client, so RLS is defense-in-depth and the routes also filter by the authenticated user.Creator identity:
creator_idstores the Supabase Auth UUID. Older migration comments say "Clerk" — that is stale naming only; never treat this value as a Clerk ID.The validator only persists fresh (non-cached) results; cache hits skip the insert to avoid duplicate history rows.
Feature migrations (apply in order where not already deployed):
Migration | Purpose |
|---|---|
| Initial table + indexes + RLS + updated_at trigger |
| Dedup |
| Linear-mode narrative field |
|
|
|
|
|
|
Supporting Tables:
gemini_cache— response cache with 30-day TTL for validations; SHA-256cache_keyderived from the normalized input (including the active model);hit_count/last_accessed_attelemetry;cleanup_expired_cache()RPC;gemini_cache_statsview.ai_usage_tracking— single source of truth for every AI call (featurevalidator/ purposetopic-validation, and content-safety checks): tokens, cost, success/error, model, agency attribution. Monthly views:ai_usage_monthly_by_feature,ai_usage_monthly_by_agency,ai_usage_realtime_stats.gemini_pricing— per-model pricing configuration (5-minute in-memory cache inpricing.service.ts); one active row enforced by a partial unique index.gemini_config— admin-managed active model + API key (+ optionalgoogle_cloud_project_id);api_keymay be NULL, in which caseGOOGLE_GEMINI_API_KEYis used as fallback.gemini_responses,gemini_api_logs— legacy audit tables, retained for history but no longer written by the validator flow (superseded byai_usage_tracking).
API Specification:
POST /api/ai/validate-idea— validate a topic and generate a curriculumAuth:
getAgencyAuthContext()+verifyCreatorAccess()(ADMIN / CREATOR / AGENCY; LEARNER and REVIEWER blocked); Supabase Auth session required.Pre-flight:
enforceTokenQuota(userId, 1500)— 3-tier token gate (sitewide → agency → member).Request:
{ topic: string (2-200 chars, required), context: string (max 1000), learningObjectives: string (max 2000), model?: "gemini-3-flash-preview" | "gemini-3.1-pro-preview" | "gemini-2.5-pro" | "gemini-2.5-flash", bypassCache?: boolean, questMode?: "linear" | "exploration" (default "exploration") }topic,context, andlearningObjectivesalso pass the input-quality gate (gibberish, excessive special chars, repetition).The
modelfield is accepted for compatibility; the service always executes the active model fromgemini_config, and the cache key includes that model.
Response (200):
{ success: true, data: ValidatorOutput, error: null, message }whereValidatorOutputcontainsviabilityScore,viabilityRationale,curriculumOutline,recommendations,totalEstimatedDuration,cached: boolean,timing, plus mode-specific fields:narrativeProgression(linear) orconflictPoints+crisisScenarios(exploration); discriminating fieldquestMode.Errors: see the error code table below.
GET /api/ai/idea-validations— creator's validation historyAuth: Supabase session (
createServerSupabaseClient()), suspension check (assertNotSuspended).Query params:
limit(default 50, max 50),offset(default 0),sortBy(created_at|title),sortOrder(asc|desc, defaultdesc). Note:sortBy=titleis currently ineffective — the route maps it totopic, which the params schema rejects andcatches back tocreated_at(see Known Limitations).Response (200):
{ success: true, data: IdeaValidationRow[], error: null, meta: { count, limit, offset } }, filtered bycreator_id.Note: history is read-only via API. The earlier save component (
AIIdeaValidatorForm.tsx) is no longer mounted.
POST /api/creator/validate-idea— content-safety pre-check (lightweight)Auth:
getCreatorAuthContext()(PR #724) + token quota gate.Request:
{ topic: string (1-500), context?: string (max 1000), learningObjectives?: string (max 1000) }.Runs the same AI content-safety pipeline without calling Gemini for curriculum generation.
Response (200):
{ success: true, data: ContentSafetyFullResult, message: "Content validation passed" }; blocked input → 422CONTENT_POLICY_VIOLATIONwithcategory,reasoning,flaggedFields,evasionDetected(this route's error payload does not includeisLocalFallback; the main validate-idea route does).
POST /api/ai/compute-canvas-layout— deterministic node layout for importsAuth: Supabase session + suspension check.
Request:
{ canvas: CanvasMetadata, questMode: "linear" | "exploration" }.Runs the TypeScript-only layout engine (
lib/ai/canvas-layout.service.ts) — no AI call — and returns the same canvas with computed x/y coordinates.Response (200):
{ success: true, data: { canvas: CanvasMetadata } }; if the request fails, the client keeps the converter's placeholder positions (non-fatal).
GET /api/ai/token-quota— current month quota status for the floating token-budget widgetResponse:
{ allowed, tier: "none" | "member" | "agency" | "sitewide", reason, details: { sitewideUsage, sitewideLimit, sitewideRemaining, agencyUsage, agencyLimit, agencyRemaining, agencyName, userUsage, userLimit, userRemaining } }.
Removed endpoint: the former GET /api/ai/active-model no longer exists. The active model is resolved server-side by getActiveGeminiModel() (reading gemini_config) and never exposed as a standalone endpoint.
Error Codes:
Status | Code | When |
|---|---|---|
400 |
| Malformed JSON or Zod validation failure ( |
401 |
| No Supabase Auth session (auth chain throws before the route's inline check) |
403 |
| Role gate failure (LEARNER/REVIEWER) or suspended account |
422 |
| Local hard-block or Gemini safety classifier rejection (includes |
422 |
| Gibberish, excessive special characters, or repetition in topic/context/objectives |
429 |
| 3-tier token quota exhausted (includes |
429 |
| Gemini quota/rate limit surfaced from the provider |
500 |
| AI parse failure, incomplete output structure, missing API key, unexpected server error |
503 |
| Active Gemini model misconfigured or unavailable |
504 |
| Gemini timeout (message surfaces the timeout) |
Logic & Workflows:
Validation Flow:
Creator submits topic, context, and objectives (with quest mode) on
/creator/ai-validator.Auth + role gate: Supabase session resolved,
verifyCreatorAccess()enforces ADMIN / CREATOR / AGENCY (LEARNER and REVIEWER blocked).Token quota gate:
enforceTokenQuota(userId, 1500)— sitewide → agency → member caps; blocked → 429 with tier details. This runs before the body is parsed.Zod validates the input schema and runs the input-quality gate (
content-safety.tsheuristics) insidesuperRefine.AI content safety: normalized evasion scan → deterministic local hard-block (authoritative) → Gemini classifier (circuit-breaker protected).
Cache check — SHA-256 key of the normalized input including the active model; 30-day TTL; mode mismatch forces regeneration;
bypassCache: trueforces a fresh call.On miss: Gemini call with the mode-specific prompt (temperature/structured JSON output), then
parseModelAwareJSONparses the response.Mode-aware output validation (shared fields + linear/exploration fields); incomplete output → 500 without persisting.
Persist fresh results to
idea_validations(cached hits skip persistence); token usage recorded toai_usage_trackingwith cost attribution.Return the validated result with
cachedandtimingmetadata.
Quest Mode (linear vs exploration):
linear: sequential narrative. Output addsnarrativeProgression(2-3 sentence arc) and per-modulecoreConcept,practiceActivity, optionalprerequisiteModules(1 activity per module on import).exploration: branching. Output addsconflictPoints(title, description, related modules, resolution) andcrisisScenarios(title, description, required competencies, placement, outcomes) — 3-5 activities per module on import.Cached entries without a mode discriminator are treated as
exploration(matching the original prompt shape); requesting the other mode regenerates.Linear route responses may carry a null/absent
narrativeProgressionwhen the AI omits it — the route still succeeds, so consumers must guard for it.
Content Safety Layers:
Layer 1 (always on, zero latency):
lib/validation/content-safety.ts— gibberish, excessive special characters, and repetition checks exported and invoked fromvalidatorInputSchema.superRefineand the client pre-flight (validateAllFields). Result codes:CLEAN,GIBBERISH_DETECTED,EXCESSIVE_SPECIAL_CHARS,REPETITIVE_CONTENT(PROHIBITED_CONTENTis declared in the contract but is produced by the policy layers, not by these heuristics).Layer 2 (deterministic hard-block):
lib/validation/local-content-safety.ts— text normalization (evasion-aware) plus deterministic classifiers (EXPLICIT_SEXUAL,ILLEGAL_ACTIVITY,VIOLENCE_GLORIFICATION,HATE_SPEECH, prompt-injection →COMPANY_POLICY_VIOLATION) with educational-context allowlists. A local block is authoritative: the model can never downgrade it.Layer 3 (Gemini contextual classifier):
lib/ai/content-safety.service.ts— temperature 0; creator input fenced between<<<UNTRUSTED_CREATOR_INPUT>>>/<<<END_UNTRUSTED_CREATOR_INPUT>>>delimiters with fence-breakout escaping; response Zod-validated (CLEAN,EXPLICIT_SEXUAL,ILLEGAL_ACTIVITY,HATE_SPEECH,VIOLENCE_GLORIFICATION,NON_EDUCATIONAL,COMPANY_POLICY_VIOLATION).Circuit breaker: 3 failures within a 2-minute window → all safety checks use the local classifier for 5 minutes; after cooldown the next call is retried against Gemini (half-open) and a success resets the breaker.
If the AI safety step throws an unexpected error, the route logs it and fails open (validation proceeds) — monitor
ai_usage_trackingand server logs for safety-call anomalies.
Token Quota & Usage Tracking:
enforceTokenQuota(userId, estimatedTokens)checks three tiers in order: sitewide monthly budget (platform_config.sitewide_monthly_token_budget, with the hardcodedSITEWIDE_MONTHLY_TOKEN_BUDGETfallback inlib/constants.ts), agency aggregate, per-member cap; it fails closed on unknown state.The validator pre-flight reserves ~1,500 tokens (estimate only); actual usage is measured after the call via
ai.models.countTokens(input/output roles; length/4 fallback) and recorded bytrackAIUsageSafe().Feature attribution:
validator/topic-validationfor validations;content-safety/content-safety-checkfor safety calls.
Curriculum-to-Canvas Conversion (current pipeline: lib/canvas-import/):
Entry point:
convertCurriculumToCanvas(validatorOutput, { questMode })→CanvasMetadata { nodes, edges, quest_mode, folders: [] };convertWithStats()returns conversion counters.Node mapping:
Title →
Textnode (id: "title",data.isStart: true)Curriculum module →
Scenarionode (id: "module-{order}",data.isEnd: truefor the last module)Deciding question (per non-last module) →
Questionnode with choices; each wrong choice routes to a game-over branchGame-over branch →
Scenarionode labelled"Game Over: {title}"with aTextchild labelled"What Went Wrong"(exploration places these alternating above/below the flow to avoid overlaps)Module children →
Text,Quiz,Reflection,Code, orLinknodes depending onsuggestedActivities(keyword map) with a rotating fallback (Quiz → Reflection → Code → Link)
Supported child types are the canvas registry types only. Since PR #681 the pipeline emits
Reflection(notEssay) andCode(notCodeblock), fixing blank cards in the visual canvas, quest player, and form editor.MAX_MODULES = 10; exploration emits 3-5 activities per module, linear emits 1.lib/utils/curriculum-to-canvas.tsis the superseded legacy converter — no code imports it;lib/canvas-import/is authoritative. The stale@seereference in the schema file still points at the old path.
Import Flow & Smart Merge:
Creator clicks Import Curriculum →
ImportDecisionDialog(create new quest vs import into an existing quest; view preference canvas or outline; quest mode preserved).AIValidator.handleImportConfirmvalidates importability, converts vialib/canvas-import, calls/api/ai/compute-canvas-layout(non-fatal), and storesai-curriculum+ai-import-decisioninsessionStorage.create-new:
POST /api/creator/create-quest(title retried with a"(n)"suffix up to 10 times on duplicates) →PUT /api/creator/update-quest-canvas→ navigate to/quest-editor/{id}/content/{visual-canvas|form-editor}.import-existing: navigate directly to the existing quest editor. On the Visual Canvas path,
useCanvasFunctionsreads the sessionStorage payload and opensImportMergeDialog; on the form-editor (Outline) path, the nodes are converted to content cards instead (no merge dialog).ImportMergeDialog(PR #721 layout fix) offers Smart Merge, Replace All, or per-node decisions (cycleupdate → skip → appendper incoming node); the dialog keeps a fixed header/footer with an internally scrolling per-node list.Smart Merge (
lib/utils/smart-merge.ts): exact normalized label match → update; fuzzy token overlap ≥ 0.5 → update; otherwise append with fresh UUIDs; edges are remapped and deduplicated. (A third "type singleton" tier exists in the code but itsSINGLETON_TYPESset is currently empty — the tier is disabled and always resolves as append.)Imported top-level nodes (title
Text, module and game-overScenarionodes) receivedata.isAiGenerated: trueand render the purple "AI" badge (PR #674); the form editor shows the badge on cards created from flagged canvas nodes (Visual Canvas import path). Child activity nodes (Question/Quiz/Reflection/Code/Link) do not currently render the badge.The form-editor path converts canvas nodes to content cards via
POST /api/creator/convert-canvas-to-cards; a node'sisAiGeneratedflag is propagated to parent and child card content when set (Visual Canvas import path).
Rendering & Formatting Fixes (PR #681):
AI-generated markdown (
## headings,**bold**, lists) is converted bylib/utils/markdown-to-html.ts(markdownToHTML) in the learner-facing renderers (QuestContentRenderer,ReflectionRenderer), sanitized with DOMPurify at render time. The canvas-import builders currently store raw markdown strings for canvas text nodes (no conversion inside the builders), so markdown formatting is guaranteed in the quest player/learner renderer, not necessarily on the canvas itself.The quest player's code block was replaced with a Monaco-based editor (
CodeblockRenderer) with language detection and a sandboxed preview; reflections use TipTap with word-count limits.
Reliability & Platform Notes (July – September 2026):
PR #674: AI-generated content cards carry
isAiGeneratedthrough canvas → schema → cards → UI badges; scenario edges connect each choice handle (up to 3) to the next scenario;quest_modeis propagated during AI batch import;verifyCreatorAccessperforms auser_roleslookup for standalone creators;quest-generator.servicelogs detailed status/code/stack.PR #679:
quest_content_cards.id(andactivity_submissions.content_card_id) migrated uuid → text so synthetic AI IDs (ai-scenario-…) flow through sync without 500s; fixed delete/reorder errors and mixed AI/manual card corruption; enhanced quest-creation error handling.PR #707: the reviewer queue resolves a quest's AI report in two steps — direct
quests.validation_idlink first, then an exact title↔topic fallback for legacy quests; when neither matches, no report is shown ("Manual"). The PR removed the previous unsafe fallback that attached a creator's most recent validation to any quest (which made manual quests display unrelated AI reports). Reviewers additionally got fullscreen read-only enforcement, comment/analytics access, and node centering.PR #724: every
/api/creator/*route — includingvalidate-ideaandvalidate-quest-content, but excluding the intentionally public share endpoints (share/scorm,share/fetch,share/analytics) — enforces the CREATOR role viagetCreatorAuthContext();ApiResponseHelper.handleError()mapsAuthorizationError → 403,AuthenticationError → 401,ServiceUnavailableError → 503instead of 500.PR #778: creator dashboard "Generate Ideas" card now links to
/creator/ai-validator; dead Settings button removed; Radix modal scroll-lock fix for the sticky creator header/sidebar.PR #840: middleware verifies session JWTs locally; suspension enforcement moved into the route auth chain (
assertNotSuspended) —GET /api/ai/idea-validationsand the other AI routes reject suspended accounts with 403ACCOUNT_SUSPENDED; central error mapping viaApiResponseHelper.handleError.Clerk → Supabase Auth: the platform completed the migration in September 2026 (#823 client UI, #826 decommission). All AIV routes authenticate with Supabase sessions;
creator_idstores the Supabase Auth UUID.
Key Files:
Path | Role |
|---|---|
| Main validation endpoint (quota, safety, cache, Gemini, persistence) |
| Validation history (GET, pagination) |
| Content-safety-only pre-check endpoint |
| Deterministic canvas layout for imports (no AI call) |
| Quota status for the token-budget widget |
| Single source of truth for input/output Zod schemas ( |
| Prompt builders, Gemini call, parsing, mode-aware validation, token counting |
| SHA-256 cache keys, 30-day TTL, hit stats |
| Gemini content classifier + circuit breaker |
| Input-quality heuristics (gibberish/special/repetition) |
| Deterministic hard-block + prompt-injection detection |
| 3-tier quota enforcement |
| Usage/cost attribution to |
| Model pricing (5-min cache) |
|
|
| Node/edge builders (title, modules, deciding questions, game-over branches, children) |
| Keyword map, fallback types, module/activity limits |
| Import matching ( |
| Markdown → HTML for AI text |
| Structured error categories for |
| Validator page ( |
| Canonical validator UI (form, progress, results, import) |
| Import destination & view decision |
| Smart Merge / Replace All / per-node decisions |
| History sheet |
| Validation detail modal (+ variant selector) |
| Switch between AI variants of the same inputs |
| Structured error feedback UI |
| Monaco code editor for code blocks |
| TipTap reflection editor |
| Canvas import ( |
Legacy / no longer mounted: components/creator/AIIdeaValidatorForm.tsx, ValidatorForm.tsx, ValidatorResults.tsx, ValidationDiffCard.tsx, and lib/utils/curriculum-to-canvas.ts remain in the tree but are not used by the current page — AIValidator.tsx and lib/canvas-import/ are authoritative.
5. Infrastructure & Operations
Dependencies:
Google Gemini — AI inference (active model + key from
gemini_config;GOOGLE_GEMINI_API_KEYfallback)Supabase — Postgres persistence, caching, usage tracking, Storage, Auth
Monitoring & Alerting:
ai_usage_trackingrecords every AI call with tokens, cost, success/error, and agency attribution; monthly views (ai_usage_monthly_by_feature,ai_usage_monthly_by_agency) support quota reviews.gemini_cachestats view tracks cache hit counts and token savings; cache expiry is handled bycleanup_expired_cache().Content-safety circuit breaker logs trips (
3 failures / 2 min → 5-min local fallback).Persistent failures to insert history rows are logged non-fatally — alert on repeated
Failed to save validationentries.Token quota gate failures surface as 429s with tier details; monitor per-tier usage to avoid surprise lockouts.
Configuration:
gemini_config(admin): activemodel_name(DB column defaultgemini-3.1-pro-preview; runtime fallbackgemini-3.5-flashwhen no active row exists),api_key(optional),google_cloud_project_id(optional).Sitewide token budget:
platform_config.sitewide_monthly_token_budget(hardcoded fallbackSITEWIDE_MONTHLY_TOKEN_BUDGET = 5,000,000inlib/constants.ts).Env:
NEXT_PUBLIC_SUPABASE_URL,NEXT_PUBLIC_SUPABASE_ANON_KEY,SUPABASE_SERVICE_ROLE_KEY,GOOGLE_GEMINI_API_KEY(fallback when thegemini_configrow has no key).
Deployment Plan:
Run the feature migrations (see table above), including
20260716_change_content_card_id_to_text.sqlfor AI-generated content card IDs, and the RLS sweep20260911_supabase_rls_auth_uid.sql(creator_id = auth.uid()::text).Ensure an active
gemini_configrow exists (model + key) or setGOOGLE_GEMINI_API_KEY.Deploy API routes and UI components; verify
/creator/ai-validatorand/quest-editor/{id}/content/visual-canvas.Verify cache behavior (miss → hit),
ai_usage_trackingrows, quota widget (/api/ai/token-quota), and the safety circuit breaker state.Smoke-test import: create-new, import-existing (Smart Merge + Replace All), and form-editor card conversion.
6. Testing & Quality Assurance
Test Strategy:
Suite | Coverage |
|---|---|
|
|
| Live Gemini integration (excluded from default Jest run; requires credentials) |
| Canvas conversion: sequence chain, game-over branches per wrong choice, node counts — currently outdated/red: it still asserts the pre-#681 |
| Mandatory learning objectives across quest/adventure/validator schemas |
| Circuit breaker trip/cooldown/reset + safety result schema invariants |
| Prompt-injection resistance, fencing, educational false-positive allowlists |
| Input-quality heuristics (gibberish, special chars, repetition) |
| Deterministic local classifier + breaker behavior |
|
|
| Token estimation and cost tiers |
Run:
npm test(Jest + coverage) ornpx jest <path>. The integration suite is excluded byjest.config.js(\.integration\.test\.ts$) and must be run explicitly with live Gemini credentials.Manual QA checklist: validate in both quest modes; verify verdict score + outline render; import create-new and import-existing (Smart Merge, Replace All, per-node cycle); confirm AI badges; check history sidebar and variant switching; confirm reviewer report only appears for AI-linked quests; confirm quota 429 surfaces cleanly.
Known Limitations:
Gemini response quality varies by topic complexity; linear mode may omit
narrativeProgression(tolerated).Cache invalidation is TTL-based only (30 days) plus the per-request
bypassCacheflag — there is no admin purge UI.Content safety may produce false positives on edge cases; an unexpected safety-service exception fails open (logged).
Legacy
conflict_points/crisis_scenariosrows may contain plain strings; the UI renders both string and structured shapes.GET /api/ai/idea-validations?sortBy=titleis a no-op: the route mapstitle→topic, which the params schema rejects and falls back tocreated_at.quests.validation_idis not populated during quest creation yet, so AI-created quests surface their reviewer report through the exact title↔topic fallback rather than the FK; duplicate titles could match the wrong validation.Orphaned UI/legacy files listed in the Key Files section can confuse navigation; treat
AIValidator.tsx+lib/canvas-import/as canonical.
7. Maintenance & Support
Troubleshooting:
422 Content Policy Error → topic/context/objectives triggered a local hard-block or the Gemini classifier; rephrase input; inspect
category/flaggedFields/isLocalFallbackin the response.422 Input Quality Error → gibberish, excessive special characters, or repetition; ask for a clearer topic.
429 Token Limit Exceeded → check
GET /api/ai/token-quotaand the reported tier (sitewide/agency/member); free budget or raise the cap.429 Rate Limit Exceeded → Gemini provider quota; retry later or check the Google project quota.
503 Model Not Found → verify the active
gemini_configrow and model name.504 Timeout → Gemini slow/overloaded; retry or bypass cache.
Cache not updating → TTL-based (30 days) by design; use
bypassCache: true. Switching the active model produces a different cache key.Empty curriculum → topic too narrow; add context or learning objectives.
Blank/invisible cards after import → confirm node types are the supported registry types (
Reflection/Code, notEssay/Codeblock) and thatlib/canvas-import(not the legacy converter) is in use.Raw markdown shown as literal text → markdown conversion lives in the learner renderers (
markdownToHTML+ DOMPurify inQuestContentRenderer/ReflectionRenderer); the canvas-import builders store raw markdown for canvas text nodes, so verify rendering in the quest player and do not expect the canvas to format it.Smart Merge dialog overflow → layout was fixed in #721; verify the fixed-header / scrollable-body / fixed-footer structure is intact.
Wrong or missing AI report in the reviewer queue → the queue resolves the direct
quests.validation_idlink first, then an exact title↔topic fallback for legacy quests; "Manual" means neither matched. The FK is not populated at quest creation yet, so verify title uniqueness to avoid fallback mismatches.Validation history sorting by title appears to do nothing → known no-op (
sortBy=titledegrades tocreated_at); use the defaultcreated_atsort.403 on validation → role gate (LEARNER/REVIEWER blocked) or suspended account; check
user_roles, agency membership, andapp_users.status.
Changelog:
1.0 (Feb 2026): Initial implementation
1.1 (Mar 2026): Added Quest Mode support (#285)
1.2 (Mar 2026): Smart Merge & Error Feedback (#245)
2.0 (Jul – Sep 2026): Reliability & rendering overhaul — supported node types instead of
Essay/Codeblock(#681), markdown rendering + Monaco code blocks (#681), AI card flags & choice-based edges (#674),quest_content_cards.iduuid → text (#679), reviewer queue report matching (#707), ImportMergeDialog layout (#721), creator role enforcement on all creator routes + correct 401/403/503 mapping (#724), dashboard entry point (#778), suspension enforcement + auth round-trip cuts (#840); platform auth migrated Clerk → Supabase Auth (#823, #826).
Document Version
1.0 - Initial version, 2026-04-27 07:58 UTC
1.0.1 - Minor revisions, 2026-04-27 07:59 UTC
1.0.2 - Minor revisions, 2026-04-28 06:18 UTC
1.0.3 - Minor revisions, 2026-06-29 07:45 UTC
1.0.4 - Minor revisions, 2026-06-29 07:47 UTC
1.0.5 - Minor revisions, 2026-06-29 09:09 UTC
2.0 - Major revision by Christian Denzon: post-overhaul architecture (Supabase Auth, smart merge, canvas import pipeline, content safety + circuit breaker, 3-tier token quota, updated schema/API/tests/troubleshooting), 2026-09-30. PRs: #674, #679, #681, #707, #721, #724, #778, #840.