HIL (Human-in-Loop) Specification

Formal specification for Human-in-Loop (HIL) workflow in midimaze and task-registry.

Definition

HIL (Human-in-Loop) is a workflow mechanism for tracking content and tasks that require explicit human review, decision-making, or completion before they can be considered finished.

Purpose

HIL serves three primary functions:

1. Quality Gate: Ensures AI-generated or automated content receives human review
2. Decision Tracker: Flags items where multiple approaches exist and human judgment is needed
3. Completion Signal: Marks partially-complete work needing human finishing touches

Scope

HIL applies to:

• Articles: Vault content requiring review/completion
• Tasks: Action items needing human decision
• Emails: Flagged correspondence requiring response

States

State Transitions:

Agent creates → needs_to_be_decided
                    ↓
Human starts →  pending_decision
                    ↓
Human completes → resolved OR ""

Implementation

Vault Articles (Frontmatter)

yaml
---
created: 2025-12-15T10:00:00
updated: 2025-12-15T10:00:00
slug: example-article-abc123
hil: needs_to_be_decided
---

Rules:

• Empty hil: "" means article is complete
• Non-empty, non-resolved means HIL active
• Once resolved, field can be cleared or set to resolved

Registry (Convex Database)

Articles Table:

typescript
{
  slug: "example-article-abc123",
  vaultPath: "Computer Tech/example-article.md",
  hil_status: "needs_to_be_decided",
  hil_note: "Need to verify technical details in section 3",
  hil_assigned_to: "nakul",
  created_at: 1702650000000,
  hil_updated_at: 1702650000000
}

Tasks Table:

typescript
{
  content: "Review and complete Yazi article",
  articleId: "yazi-navigation-abc123", // Links to article
  hil_status: "needs_to_be_decided",
  status: "in_progress"
}

Workflows

1. Agent Creates Article with HIL

User: "ma yazi navigation"

midimaze-mcp.create_article():
  1. Generate content (may be incomplete)
  2. Set frontmatter: hil: needs_to_be_decided
  3. Sync to registry with hil_status
  4. Create linked task in registry
  
Registry state:
  - Article: hil_status = "needs_to_be_decided"
  - Task: content = "Complete Yazi Navigation article"
          articleId = "yazi-navigation-abc123"

2. Morning Review Shows HIL Items

User: "Show morning review"

midimaze-mcp.morning_review():
  1. Query registry.get_hil_items()
     → Returns all articles with hil_status != resolved
  2. Display:
     - 3 articles need review
     - Yazi Navigation (needs_to_be_decided)
     - Email workflow (pending_decision)

3. Human Resolves HIL

User: "Mark yazi article as resolved"

registry-mcp.resolve_hil({
  slug: "yazi-navigation-abc123",
  decision_note: "Added keyboard shortcuts section, ready to publish"
}):
  1. Update article: hil_status = "resolved"
  2. Update linked task: status = "completed"
  3. Optionally update vault frontmatter

Use Cases

Use Case 1: Incomplete AI-Generated Content

Scenario: OpenCode generates article but can't find source for a claim.

HIL Setup:

yaml
hil: needs_to_be_decided
hil_note: "Verify claim about eza performance in paragraph 3"

Resolution: Human verifies claim, adds citation, marks resolved

---

Use Case 2: Decision Between Approaches

Scenario: Multiple valid ways to implement a feature.

HIL Setup:

yaml
hil: needs_to_be_decided
hil_note: "Choose between SQLite vs Convex for local caching"

Resolution: Human decides approach, documents decision, marks resolved

---

Use Case 3: Email Requiring Response

Scenario: Flagged email needs human decision on response.

HIL Setup:

typescript
registry.create_task({
  content: "Respond to student question about mixing console",
  hil_status: "needs_to_be_decided",
  emailId: 12345
})

Resolution: Human drafts response, sends email, marks resolved

---

Morning Review Integration

HIL items surface in morning_review workflow:

typescript
async function morning_review() {
  const hil_items = await registry.get_hil_items(registryId);
  const flagged_emails = await outlook.get_flagged_emails();
  
  return {
    hil_decisions: hil_items.length,
    hil_articles: hil_items.filter(i => i.type === 'article'),
    hil_tasks: hil_items.filter(i => i.type === 'task'),
    flagged_emails: flagged_emails.length,
    recommendations: [
      `Review ${hil_items.length} items needing decisions`,
      `Process ${flagged_emails.length} flagged emails`
    ]
  };
}

Registry Schema Requirements

Articles:

typescript
articles: defineTable({
  slug: v.string(),
  vaultPath: v.string(),
  hil_status: v.union(
    v.literal(""),
    v.literal("needs_to_be_decided"),
    v.literal("pending_decision"),
    v.literal("resolved")
  ),
  hil_note: v.optional(v.string()),        // Why HIL was set
  hil_assigned_to: v.optional(v.string()), // Who should resolve
  hil_updated_at: v.optional(v.number()),  // Last HIL change
})

Tasks:

typescript
tasks: defineTable({
  content: v.string(),
  status: v.union("pending", "in_progress", "completed", "cancelled"),
  hil_status: v.optional(v.union(
    v.literal(""),
    v.literal("needs_to_be_decided"),
    v.literal("pending_decision"),
    v.literal("resolved")
  )),
  articleId: v.optional(v.string()), // Link to article if relevant
})

Future Enhancements

Multi-User HIL (midimaze 2.0)

typescript
{
  hil_status: "needs_to_be_decided",
  hil_assigned_to: "[email protected]",
  hil_note: "April should review mixing technique accuracy"
}

HIL Workflows

typescript
// Auto-resolve when linked task completes
task.status = "completed" 
  → article.hil_status = "resolved"

// Auto-create HIL when article has TODOs
scan article for "- [ ]"
  → if found, set hil: needs_to_be_decided

HIL Analytics

typescript
// Track resolution time
{
  hil_created_at: 1702650000000,
  hil_resolved_at: 1702750000000,
  hil_resolution_time_hours: 27.8
}

// Dashboard metrics
- Average HIL resolution time: 2.3 days
- HIL backlog: 12 items
- HIL by category: articles (8), tasks (3), emails (1)

Open Questions

1. Should HIL be required or optional?

   • Should all agent-created content have HIL by default?
   • Or only when agent detects uncertainty?

2. Should vault frontmatter be source of truth or registry?

   • Currently: Frontmatter is primary, registry syncs
   • Future: Registry is primary, frontmatter is secondary?

3. What about HIL for coding tasks?

   • Currently focused on articles/emails
   • Should coding tasks also use HIL workflow?

4. Auto-resolution criteria?

   • When should HIL auto-resolve?
   • Linked task completion? Manual only?

See Also

• Task Registry Architecture
• Morning Review Workflow
• midimaze-mcp Design
• Article Creation Workflow