Feature Owner: scorevi (Sean Patrick Caintic)
Updated by: Christian Denzon (AI Idea Validator overhaul, July – September 2026)
Module: Diagnose
Priority: P0
Sprint #12: Fully Implemented (v2.0 reliability & rendering overhaul merged July – September 2026)
Date: 2026-06-29
Last Updated: September 30, 2026
EXECUTIVE SUMMARY
What is this feature? The AI Idea Validator provides two independent endpoints for evaluating creator content ideas: a lightweight content safety check and a comprehensive curriculum generation engine. It assesses viability, generates curriculum outlines, identifies conflict points, and produces structured recommendations. Creators can then import the generated curriculum into the Visual Canvas — either as a new quest or merged into an existing one.
Why does it matter? Creators need confidence that their educational ideas are viable before investing time in content creation. Bad ideas caught early save hours of wasted development. The validator ensures content quality and curriculum coherence from the ideation stage, and the July – September 2026 overhaul made the generated content render correctly end-to-end (supported card types, code blocks, game-over branches, and markdown formatting in the learner renderer) and importable into existing quests without breaking them.
What's the MVP scope?
Content safety: zero-latency input-quality pre-flight, deterministic local hard-block, and a Gemini classifier with prompt-injection/evasion detection and circuit-breaker protection.
Full curriculum validation with viability scoring (0-100), curriculum outlines, conflict detection, crisis scenarios, recommendations, and narrative progression.
Quest modes: Narrative (Linear) and Exploration (Branching) — the mode changes the generated structure.
Import to the Visual Canvas: create a new quest or merge into an existing quest (Smart Merge, Replace All, or per-node decisions); Outline View available for linear quests.
Validation history with variant switching and a 30-day response cache with an explicit "Regenerate Idea" bypass.
Platform constraints: CREATOR-role access via Supabase Auth (Clerk was decommissioned, September 2026) and the 3-tier token quota (sitewide → agency → member).
1. USER PAIN POINT & SOLUTION
Current State (Without Feature)
Creators submit ideas without any structured validation. Low-quality or incoherent educational content reaches development, wasting creator and platform resources.
Pain Point
Type | Impact |
|---|---|
Emotional | Anxiety about whether an idea is "good enough" to build |
Functional | No objective framework to evaluate educational viability |
Business | High content rejection rates post-development; wasted AI generation costs |
Future State
Creators submit an idea and receive a viability score (0-100), a structured curriculum outline, identified conflict points and crisis scenarios, actionable recommendations, and narrative progression guidance — all before writing a single line of content. The curriculum can then be imported straight into the Visual Canvas as a new quest or merged into an existing one, with AI-generated nodes clearly badged.
Marketing Hook
"Validate your educational vision in seconds. Know what works before you build."
2. 4D FRAMEWORK MAPPING
Phase | Mapping |
|---|---|
Diagnose | Core feature — AI evaluates idea quality, viability, and curriculum structure |
Design | Curriculum outline feeds into quest/adventure structure design |
Develop | Recommendations and conflict resolutions guide content creation |
Deliver | Validated content flows into the quest editor (Visual Canvas or form editor) as a new quest or a Smart Merge into an existing one |
3. USER FLOWS
Entry Point
Creator Dashboard → AI Suggestion card → "Generate Ideas" (links to
/creator/ai-validator; wired in PR #778).Creator sidebar → Builder Tools → "AI Idea Validator".
Direct route:
/creator/ai-validator.
Success Criteria
Viability score returned (0-100) with rationale.
Curriculum outline generated as structured JSON.
Conflict points and crisis scenarios identified (exploration), or narrative progression mapped (linear).
Curriculum importable to the Visual Canvas — as a new quest or merged into an existing quest — with AI badges on the top-level nodes and correct rendering (code blocks, reflections; markdown formatting in the learner renderer).
Validation history persists past results; cached results are flagged and can be regenerated.
Main Flow
Creator signs in (Supabase Auth) and opens
/creator/ai-validator.Creator enters a quest topic (required, 2-200 characters), optional context (max 1000 characters) and learning objectives (required in the UI, max 2000 characters), and picks a quest mode: Narrative (Linear) or Exploration (Branching).
The client runs the zero-latency input-quality pre-flight (gibberish, repetition, excessive special characters).
"Validate Topic" →
POST /api/ai/validate-idea: role gate → 3-tier token quota → schema + input-quality validation → AI content safety → cache check → Gemini generation → mode-aware output validation.Progress phases display ("Connecting to Gemini" → "AI content safety analysis" → "Gemini thinking & generating" → "Structuring curriculum" → "Saving to database" → complete), then results render: Viability badge + rationale, Curriculum Outline (N modules), Narrative Progression (linear only), Conflict Points (N) and Crisis Scenarios (N) (exploration only), and Recommendations.
If the result came from the 30-day cache, a "Regenerate Idea" action appears to bypass the cache and generate a fresh alternative.
The history sidebar lists the creator's past validations; "Load to Form" restores an entry's inputs (the current results panel is cleared), while "View Details" opens the stored result in a modal; variants of the same topic can be switched.
"Import Curriculum" opens the import decision: Create New Quest or Import to Existing Quest. Visual Canvas is always offered; Outline View is available for linear quests.
Import converts the curriculum into canvas nodes (title, scenario modules, deciding questions, game-over branches, Text/Quiz/Reflection/Code/Link children) and runs the deterministic layout engine (falling back to the converter's positions on failure). On the Visual Canvas path, importing into an existing quest opens the Smart Merge dialog (Smart Merge / Replace All / per-node update-skip-append).
Imported top-level nodes (title, module, and game-over scenario nodes) are marked with a purple "AI" badge; the form editor shows the badge on cards created from flagged canvas nodes. Fresh (non-cached) validations are saved to
idea_validationsfor history.
Edge Cases
Toxic/harmful content: blocked by the deterministic local hard-block (authoritative — the model can never downgrade it) or the Gemini safety classifier; the response reports the category and flagged fields.
Prompt-injection attempts in context/learning objectives are neutralized (untrusted-input fencing) and cannot flip the verdict to allow.
Gibberish/repetition: rejected with a 422 input-quality error; ask the creator to rephrase.
Empty learning objectives: the UI requires them (validated at ≥2 characters); the API schema itself treats the field as optional.
Over-length input: topic > 200 / context > 1000 / objectives > 2000 are rejected with a 400 validation error (not truncated).
AI failure/timeout during safety: circuit breaker (3 failures within 2 minutes → 5-minute local fallback) keeps validation moving.
AI failure/timeout during generation: 504 timeout (retry) or 503 when the active model is misconfigured.
Repeated identical inputs: the cached result is returned for 30 days unless "Regenerate Idea" is used.
Import title collision: the system retries with a "(n)" suffix (up to 10 attempts).
Importing into a quest with many nodes: the merge dialog stays usable (fixed header/footer, per-node scrolling); Smart Merge updates matches and appends new nodes; Replace All overwrites. (Smart Merge / Replace All apply to the Visual Canvas import path; the Outline View path converts directly to content cards.)
Layout request failure during import: the converter's placeholder positions are kept (import still succeeds — the layout engine is deterministic and never calls AI).
Decision Points
Review recommendations (informational — there is no per-item accept/reject).
Choose the quest mode before validating — it changes the generated output shape.
Import as a new quest or merge into an existing quest; choose Smart Merge / Replace All / per-node decisions; choose Visual Canvas or Outline View (linear only).
Regenerate (bypass cache) with modified input.
4. INFORMATION ARCHITECTURE
Primary
Input data: topic, context, learning objectives, quest mode.
Viability score (0-100) and viability rationale.
Curriculum outline.
Secondary
Conflict points (exploration only), crisis scenarios (exploration only).
Narrative progression (linear only), recommendations, total estimated duration (shown in the import decision summary).
Model name (audit trail), cached flag, quest mode, validation timestamps.
Actions
Validate Topic, Regenerate Idea, Import Curriculum (Create New Quest / Import to Existing Quest → Smart Merge / Replace All / per-node decisions), View History / load variant.
5. WIREFRAMES
Excluded — existing UI (AIValidator.tsx).
6. WIREFLOWS
Excluded — existing UI implemented.
7. PROTOTYPE
Excluded — feature is fully implemented.
8. BACKEND SCHEMA
idea_validations table (verified columns)
Column | Type | Notes |
|---|---|---|
id | UUID | PK |
creator_id | TEXT NOT NULL | Supabase Auth user UUID stored as text (legacy "Clerk" naming — never a Clerk ID) |
topic | TEXT NOT NULL | Creator input |
context | TEXT | Optional |
learning_objectives | TEXT | Optional in schema; required by the UI |
viability_score | INTEGER NOT NULL | CHECK (0-100) |
viability_rationale | TEXT NOT NULL | AI reasoning |
curriculum_outline | JSONB NOT NULL | Structured curriculum data |
conflict_points | TEXT[] | Exploration mode |
crisis_scenarios | TEXT[] | Exploration mode |
recommendations | TEXT[] | Array of recommendations |
total_estimated_duration | TEXT | Estimated total duration |
model_name | TEXT NOT NULL | DEFAULT 'unknown' |
quest_mode | TEXT NOT NULL | DEFAULT 'exploration'; CHECK (linear / exploration) |
narrative_progression | TEXT DEFAULT NULL | Linear only; TEXT, NOT JSONB |
created_at | TIMESTAMPTZ | Auto-timestamp |
updated_at | TIMESTAMPTZ | Maintained by trigger |
Important notes:
NO
content_warningscolumn exists.NO
model_variantcolumn — usemodel_nameinstead.narrative_progressionis TEXT, not JSONB.learningObjectivesin the Zod schema:z.string().max(2000).trim().optional()— NOT an array (the UI enforces it as required).Row Level Security: creators can only read/write their own rows (
creator_id = auth.uid()::text).quests.validation_idlinks a quest to its source validation for the reviewer AI report (column exists, but is not yet populated at quest creation — reviewers currently resolve reports via the exact title↔topic fallback).Full DDL and migration list: see the Internal Technical Guide.
9. API ENDPOINTS
Method | Path | Auth | Purpose | File |
|---|---|---|---|---|
POST | /api/creator/validate-idea | Supabase Auth + Creator role | Content safety ONLY (lightweight standalone check) |
|
POST | /api/ai/validate-idea | Supabase Auth + Creator role | Full validation (quota, safety, cache, Gemini, persistence) |
|
GET | /api/ai/idea-validations | Supabase Auth | Creator's validation history (pagination) |
|
POST | /api/ai/compute-canvas-layout | Supabase Auth | Deterministic node layout for imports (no AI call) |
|
GET | /api/ai/token-quota | Supabase Auth | Monthly token quota status (widget) |
|
Import target endpoints: POST /api/creator/create-quest, PUT /api/creator/update-quest-canvas (canvas save), and POST /api/creator/convert-canvas-to-cards (form-editor path).
Key distinction: TWO separate validation endpoints. The main UI action uses /api/ai/validate-idea, which runs the content-safety layers internally; /api/creator/validate-idea is a lightweight, API-only safety check (the creator UI does not call it — only the debug enforcement-test page probes it). The former GET /api/ai/active-model endpoint was removed — the active Gemini model is admin-configured server-side.
Content Safety (/api/creator/validate-idea)
Lightweight safety check; no curriculum generation.
Uses
lib/ai/content-safety.service.tsplus the deterministic layers inlib/validation/.Prompt-injection/evasion detection with normalization and untrusted-input fencing.
Circuit breaker: 3 failures in 2 minutes → 5-minute cooldown → local fallback.
Full Validation (/api/ai/validate-idea)
Comprehensive curriculum generation (viability scoring, outline, conflicts, recommendations).
Auth: Supabase Auth + Creator role gate; suspended accounts are rejected.
3-tier token quota gate before generation; usage recorded to
ai_usage_tracking.Uses
lib/ai/validator.service.ts; fresh results are persisted toidea_validations(cached hits are not re-saved).
10. DATA REQUIREMENTS
Frontend Needs
Topic, context, and learning objectives inputs (with character counters and limits).
Quest mode selection (Narrative / Exploration).
Viability score display (0-100) with rationale.
Curriculum outline for review.
Conflict points and crisis scenarios for creator awareness (exploration); narrative progression (linear).
Recommendations (display only).
Cached-result indicator and the Regenerate action.
History sidebar for past validations with variant switching.
Token quota display (floating budget widget).
API Calls
POST
/api/ai/validate-idea(validation; main action).GET
/api/ai/idea-validations(history).POST
/api/ai/compute-canvas-layout(import layout).POST
/api/creator/create-quest+ PUT/api/creator/update-quest-canvas(create-new import).POST
/api/creator/convert-canvas-to-cards(form-editor import).(Available) POST
/api/creator/validate-idea, GET/api/ai/token-quota.
Caching
Validation results are cached for 30 days, keyed by the normalized input including the active model; repeats return
cached: true."Regenerate Idea" bypasses the cache and generates a fresh alternative for the same inputs.
Cached hits do not create duplicate history rows; switching the active model produces a different cache variant.
11. PERFORMANCE CONSIDERATIONS
DB Optimization
JSONB column for
curriculum_outlineallows flexible querying.TEXT[] arrays for conflict points, crisis scenarios, recommendations.
Indexes:
(creator_id, created_at DESC)and(creator_id, topic, created_at DESC).Token quota is checked before any AI call (pre-flight ≈1,500 tokens for validation).
History queries are scoped by an explicit creator filter (the service-role client bypasses RLS; RLS remains table-level defense-in-depth).
Response Time
Input quality: near-instantaneous (local logic).
Content safety: local hard-block is zero-latency; the Gemini classifier adds AI latency.
Full validation: dependent on the active Gemini model's API latency; cached results return immediately.
Circuit breaker prevents cascading safety failures during AI outages.
Import layout is deterministic and instant; if the layout request fails, the converter's positions are kept.
Access Control
Supabase Auth session required on all endpoints (Clerk was decommissioned in September 2026).
Allowed: Platform Admin, Creator, the legacy platform AGENCY (owner) role, and agency members with the Creator role. Blocked: Learner, Reviewer (403).
Suspended accounts are rejected (403 ACCOUNT_SUSPENDED).
Creator can only view their own validation history (explicit creator filter; the service-role client bypasses RLS, which remains table-level defense-in-depth).
Auth Logic
creator_idis TEXT and stores the Supabase Auth user UUID (legacy Clerk string naming; no UUID FK constraint by design).
Validation
Learning objectives: Zod
z.string().max(2000).trim().optional()(UI requires the field).Content safety: input-quality heuristics, deterministic hard-block, Gemini classifier, prompt-injection/evasion detection before AI processing.
3-tier token quota (sitewide → agency → member) protects against cost overruns; circuit breaker mitigates AI outages and abuse.
13. ERROR HANDLING
Error | Response |
|---|---|
Content flagged as unsafe | 422 CONTENT_POLICY_VIOLATION (category + flagged fields); generation never runs |
Input quality (gibberish/repetition) | 422 INPUT_QUALITY_ERROR; ask the creator to rephrase |
Missing learning objectives | Allowed by the API schema; the UI prevents submission |
AI API timeout/failure | 504 timeout (retry) or 503 MODEL_NOT_FOUND when misconfigured |
Provider rate limit | 429 RATE_LIMIT_EXCEEDED |
Token quota exhausted | 429 TOKEN_LIMIT_EXCEEDED with the blocking tier |
Malformed/incomplete AI output | 500 INTERNAL_ERROR; nothing persisted |
Safety service exception | Logged; validation proceeds (fail-open) |
3 safety failures in 2 minutes | Circuit breaker triggers; 5-minute cooldown; local fallback |
Unauthorized / forbidden | 401 authentication error / 403 authorization or suspension error |
Duplicate quest title on import | Automatically retried with a "(n)" suffix |
14. TESTING CHECKLIST
Happy Path
Sign in as a Creator; open
/creator/ai-validatorfrom the sidebar (Builder Tools) or the dashboard "Generate Ideas" card.Enter a topic, context, and learning objectives; choose Exploration (Branching); validate → score, outline, conflict points, crisis scenarios, and recommendations render.
Validate the same inputs again → cached result is returned (and the "Regenerate Idea" action appears).
Click "Regenerate Idea" → a fresh alternative is generated (bypasses cache).
Switch to Narrative (Linear); validate → narrative progression and linear module details render.
Open the history sidebar → load a previous validation; switch between variants of the same topic.
Import Curriculum → Create New Quest → Visual Canvas: title, module, deciding-question, and game-over nodes appear; title/module/game-over nodes show the purple AI badge; code blocks open in the code editor; reflections render; verify markdown formatting in the quest player (learner renderer).
On the Visual Canvas path, import into an existing quest via Import to Existing Quest → Smart Merge preview → merge and verify updated/appended nodes; repeat with Replace All and with per-node update/skip/append decisions.
Import with the Outline View (linear mode) → content cards are created in the form editor (no merge dialog on this path; AI badges appear only for cards exported from flagged canvas nodes).
Publish an AI-created quest → the reviewer queue shows its AI report (resolved via the exact title↔topic fallback until
validation_idis populated); a manually created quest with no matching validation shows "Manual".
Edge Cases
Unauthorized account (Learner) or suspended account is denied.
Idea with no learning objectives (API) / attempt to submit without them (UI blocked).
Idea at max character limits (topic 200, context 1000, objectives 2000); over-limit input is rejected.
Content safety blocks harmful input; prompt-injection text in context/objectives does not flip the verdict.
Circuit breaker triggers after 3 rapid safety failures and recovers after cooldown.
Token quota exhaustion surfaces a clear 429 and the quota widget reflects usage.
Gemini model misconfiguration returns 503.
Duplicate quest title during create-new import auto-suffixes "(n)".
Smart Merge dialog with many nodes stays scrollable; skipped/updated/appended decisions are respected.
Cached results do not duplicate history rows; a model switch produces a new cache variant.
15. OPEN QUESTIONS
None — feature is fully implemented. (v2.0: still none outstanding for the overhaul. Engineering follow-ups — populating
quests.validation_idat creation, history sort by title, and refreshing the outdated canvas-import test suite — are tracked in the Internal Technical Guide.)
16. OUT OF SCOPE
Multi-language curriculum validation. — Still out of scope.
Peer review integration for validation scores. — Partially addressed (v2.0): reviewers now see the AI validity report for AI-linked quests in the reviewer queue (PR #707); score-based peer review remains out of scope.
17. SUCCESS METRICS
AI validation adoption rate (% of creators validating before creating).
Average viability score distribution.
Import-to-canvas conversion rate (create-new and merge) with no rendering regressions after the v2.0 overhaul.
Circuit breaker trigger frequency (should be near zero).
Token quota block rate stays low and expected.
18. DEPENDENCIES
Google Gemini (validation, content safety) — active model from
gemini_config;GOOGLE_GEMINI_API_KEYfallback. (Canvas layout is deterministic and does not use AI.)Supabase Auth (Creator role) — migrated from Clerk, completed September 2026.
Supabase Database —
idea_validations,gemini_cache,ai_usage_tracking,gemini_pricing,gemini_config.lib/ai/validator.service.ts(validation engine).lib/ai/content-safety.service.ts+lib/validation/(safety layers).lib/canvas-import/+lib/utils/smart-merge.ts(import and merge workflow).Quest Editor / Visual Canvas + form editor.
Token quota (
platform_configbudget) and the token budget widget.Reviewer Queue (AI report resolution for AI-linked quests).
19. TIMELINE
Completed — Feature fully implemented in Sprint #12 (v1.0, June 2026). The v2.0 reliability & rendering overhaul merged July – September 2026 (PRs #674, #679, #681, #707, #721, #724, #778, #840).
Backend / Frontend (v1.0): Sean Patrick Caintic (scorevi).
v2.0 overhaul: Christian Denzon.
QA: Internal development testing.
Note: API Endpoints and Schema — please refer to its Internal Technical Guide (AI Idea Validator): https://kb.wyzquests.com/internal-technical-guides/1-2-ai-idea-validator
Document Version
1.0 - Initial version - 2026-06-29 08:11 UTC
1.1 - Added Document Version section and update author to have full name - 2026-06-29 08:43 UTC
2.0 - Hand-off updated for the reliability & rendering overhaul: correct node types + AI badges, markdown/code-block rendering, Smart Merge import flow, reviewer AI report matching, validation caching + regenerate, Supabase Auth + role/quota enforcement, and refreshed schema/API/error/QA sections. Updated by Christian Denzon, 2026-09-30. PRs: #674, #679, #681, #707, #721, #724, #778, #840.