Research Roadmap
_Last updated: 2026-07-31 (fourth trigger: short-posts cluster formalization; Jean/Tailscale field note added as adoption-barrier evidence and as substrate for new config-as-state-machine investigation)_
_Next review: On significant repository change_
---
Active Investigations
1. Agent State Machine Model Validation
What: The Agents and State Machines document defines 12 states and 25 invariants for Pi-sdk-based agents. This is a formal model — it needs to be validated against real agent traces.
Open questions:
Do all 25 invariants hold in practice, or are some unreachable/redundant?
Are there states missing from the model that real agents exhibit?
How does the model map to non-Pi-sdk agents (LangChain, CrewAI, custom)?
Evidence: State machines to visualize the work of agents, How to Background Agents, State Machines — Knowledge Map (learning paths now sequence the agentic-coding arc)
Next step: Collect state traces from Product Scout, Gardener, and any other running agent. Compare observed transitions against the formal model. Report violations.
---
2. Trust Graph Topology in Open Source
What: The Graduated Trust essay proposes a Web-of-Trust-based hop-distance scoring system for OSS contributions. But the real-world topology of OSS trust graphs is unknown.
NEW substrate evidence (2026-07-28): The Community Models in Seed document provides the actual Seed infrastructure for two trust models — formal communities (mutual membership, role-based permissions for contribution scoring) and user web of trust (one-way follows for discovery/filtering). This gives us a concrete architecture to map the research against:
Formal community membership → hop-0 (maintainers) and hop-1 (vouched contributors with role-based permissions)
Web of trust → hops 2+ (discovery paths through followed accounts)
The research question becomes: what's the empirical hop distribution of the web of trust for a typical Seed space?
Open questions:
What does the trust graph of a typical OSS ecosystem look like? (npm, PyPI, crates.io)
How many hops between a random contributor and a core maintainer?
Does the small-world property hold? Are there bridging nodes?
What path quality weighting makes sense empirically?
NEW: How do Seed's formal community boundaries map to hop-distance zones? Is a community member always hop-0, or are there sub-roles?
How would graduated trust scoring apply to the Devin case study? Would Devin have a trust path to a project maintainer? What friction level would it encounter at each hop distance?
Next step: Map the Devin case study to Seed's community models — what would the trust path look like through formal communities vs. web of trust? Which model would Devin's contributions route through?
---
3. Multi-Agent Coordination Patterns on Content-Addressed Stores
What: Multiple agents writing to the same content-addressed document store is a novel pattern.
Open questions:
Do document-level leases work in practice, or do we need block-level leases?
How does content addressing simplify or complicate conflict detection?
What's the minimum viable coordination protocol for 2-5 agents?
Evidence: Product Backlog — Opportunity 7 — Multi-Agent Coordination Protocol.
Next step: Run a conflict experiment: two agents edit the same document simultaneously. Measure result. Design lease protocol.
---
4. Self-Healing Knowledge System Design Patterns [NEW]
What: The LLM Wiki spec defines three self-healing mechanisms: version tracking (detect source changes, patch incrementally), deletion handling (remove dependent content when source is deleted), and periodic regeneration (full rebuild after N cycles to prevent drift). These form a reusable pattern language for LLM-maintained knowledge bases.
Open questions:
What is the optimal N for periodic regeneration? Too frequent = expensive, too infrequent = drift.
How do you detect and measure "knowledge drift" — the gradual divergence between wiki content and source documents?
Can the contradiction log serve as a drift metric? (More contradictions = more drift = regeneration needed sooner.)
Do the self-healing patterns generalize to other domains (code documentation, scientific literature review, legal research)?
The human-curated flag creates an append-only mode. Does this work in practice, or do humans need more granular override?
Evidence: LLM Wiki project document — self-healing section, human-override section, contradiction log section
Next step: Prototype a minimal wiki pipeline with one source document. Measure drift over 10 incremental LLM edits. Determine optimal regeneration frequency.
---
Proposed Investigations
5. State Machine Adoption Barriers
What: State machines are great, but hard to establish documents adoption friction.
Hypothesis: The lack of good debugging/visualization tools is the primary barrier.
NEW evidence (2026-07-31): The Jean Web Access behind Tailscale field note documents a real configuration failure — two deployment modes "easy to mix up" — a concrete instance of state confusion in the wild, supporting the tooling/debuggability hypothesis beyond UI code.
Next step: Survey XState community on adoption pain points.
---
6. AI Contribution Quality Metrics
What: How do we measure whether an AI-generated contribution is genuinely useful vs. plausible but wrong?
Open questions:
Can we detect "slop PRs" by analyzing code patterns (test coverage, dead code, unnecessary abstraction)?
What signals distinguish a thoughtful AI contribution from a noisy one?
The Devin case study is a rare documented example of high-quality AI contributions (6 merged PRs). What patterns in the Devin workflow produce quality? Can those patterns be formalized into metrics?
Evidence: Graduated Trust proposal, Devin Booking System
Next step: Collect 10 real slop PRs from open source. Analyze patterns against the Devin case study patterns. Propose detection heuristics.
7. Community Models ↔ Trust Graph Mapping [NEW]
What: The Community Models in Seed document defines two trust models, but doesn't specify how they interact. Can a user be in both a formal community and a web of trust simultaneously? Which model takes priority for hop-distance calculation when both apply?
Open questions:
When a user is both a formal community member (hop-0) and a web-of-trust follower (hop-2 from a maintainer), which distance wins? Minimum? Average?
Can web-of-trust edges override formal community role distance? (Example: a maintainer follows someone outside the community — does that give them hop-1 access?)
What happens when formal community permissions conflict with web-of-trust signals? Who decides precedence?
Evidence: Community Models in Seed — the two models defined; Graduated Trust — where they'd be applied
Next step: Model the interaction between the two trust models. Propose a precedence rule: formal community membership wins for permission gating, web of trust for discovery/relevance. Test against a hypothetical Seed space with 3 community members and 10 followers.
8. Deployment Configuration as State Machine [NEW 2026-07-31]
What: The Jean Web Access behind Tailscale field note describes two valid modes (Tailscale Serve vs direct tailnet HTTP) whose misconfigurations produce confusing failure modes. This looks like a small state machine: host binding (127.0.0.1 vs tailnet IP) × proxy mode (Serve vs direct) → reachability matrix.
Open questions:
Can deployment topologies be modeled as state machines where misconfiguration = invalid transitions?
Would a config-state visualization have prevented the "works in browser but not in native app" class of bugs?
Does this generalize to other self-hosted services (reverse proxies, auth origins, WebSocket endpoints)?
What are the other recurring failure modes of self-hosted apps (TLS/proxy ordering, websocket proxying, token leakage in URLs)?
Next step: Formalize the Jean/Tailscale config as a 2-3 state machine; enumerate invalid transitions; check whether the documented failure modes all correspond to invalid states.
9. Diagnostic Template Generation from Config State Machines [NEW 2026-07-31 — final revision]
What: The final Jean Web Access behind Tailscale revision makes its diagnostic structure explicit: Recommended setup → Why the direct IP may stop working → Quick checks (/readyz, auth probe with expected bodies) → "Is this a Jean issue?" diagnosis. Question: can this troubleshooting-doc structure be generated mechanically from a config state machine (enumerate valid states → emit checks + expected outputs for each)?
Open questions:
Can the readyz body ({"http":true,"ok":true,"websocket_broadcaster":true}) be treated as an invariant of the valid state, checked automatically?
Which parts of a troubleshooting guide are derivable (checks, expected outputs, decision tree) vs. craft (narrative, "why" sections)?
Would a generated template reduce the "easy to mix up" class of errors across self-hosted services?
Evidence: Jean Web Access behind Tailscale (Recommended setup + Quick checks + "Is this a Jean issue?"); the Seed team's dev-docs pattern note — this structure is the admired pattern; Opportunity Backlog #12 (Jean-Style Troubleshooting Template)
Next step: Formalize #8's state machine, then derive the quick-check matrix from valid states and compare against the post's manual checks. This becomes the core of the Troubleshooting Template opportunity.
_Final-revision pass 2026-07-31: appended cross-account evidence to #5/#8 evidence sets implicitly via the post's Related links; added Proposed Investigation #9 (Diagnostic Template Generation). Prior content preserved._
What IA patterns exist across P2P and local-first tools worth borrowing?
Evidence Update — 2026-07-31 (Vibe code pipeline trigger — the 4-step Linear→Agent pipeline)
Investigation #1 (Agent State Machine Model Validation) — new substrate. The pipeline vision (4-step spec) plus the author's own link to the state machines post gives the formal model a second real-world workload to validate against: task lifecycle (assigned → in_progress → ready_for_review → merged). The community MCP template is an existing implementation whose behavior can be traced.
Investigation #6 (AI Contribution Quality Metrics) — new angle. The pipeline's terminal step ("create PRs when the issue is fixed") begs the quality question: what evidence distinguishes a fixed issue from a plausible-but-wrong fix when the PR was produced autonomously? The source post's new Graduated Trust cross-link makes this the explicit link between pipeline execution and trust. See new Proposed Investigation #10 below.
10. Verification Protocol for the Linear→Agent Pipeline [NEW 2026-07-31]
What: The Vibe code like a PRO pipeline spec (fetch → expand → spawn → PR) ends at "create PR". The open question is what makes that PR trustworthy: what evidence should a background agent return when it closes a Linear issue, and can that evidence be machine-checked before a human reviews?
Open questions:
What is the minimal evidence bundle for an autonomous PR (diff, test output, state trace, task-to-code mapping)?
Can task acceptance criteria be encoded as states (assigned → in_progress → ready_for_review → merged) with guard conditions, extending the 12-state agent model (Investigation #1)?
How do you timebox/loop-guard an agent that fails repeatedly on the same issue (prompt thrash → backoff → escalate to human)?
Does the state-machine visualization approach (state machines post) make pipeline failures diagnosable at a glance?
Evidence: Vibe code like a PRO — 4-step spec; How to Background Agents; community MCP template; Agents and State Machines
Next step: Model the pipeline as a small state machine (assigned → in_progress → ready_for_review → merged, with failed/escalated traps); enumerate the evidence bundle per transition; compare against how the community MCP template currently reports results.
_Updated: 2026-07-31 (Vibe code trigger, second pass) — refreshed source fragments to the live revision of the source post; noted its new Graduated Trust cross-link in #6 evidence. All prior content preserved._
Evidence Update — 2026-07-31 (Short Posts hub restructure trigger — author-labeled clusters)
#1 (Agent State Machine Model Validation) — new substrate. The author's reorganization of Short Posts placed How to Background Agents under the State Machines lane — the author classifies the practical background-agent workflow as state-machine territory, giving the formal model another observable workload (agent spawn → task lifecycle → PR). The lane also cross-links the Knowledge Map.
#8 (Deployment Config as State Machine) — permanent exhibit. The full Jean troubleshooting doc is now embedded in the All Posts lane of the hub — including Recommended setup, Why the direct Tailscale IP may stop working, Quick checks, and Is this a Jean issue?. The manual quick-check matrix stays visible in the hub for comparison against any mechanically generated template.
#9 (Diagnostic Template Generation) — pattern visible in situ. The embed means the troubleshooting structure is now consumed inside an index hub, not only as a standalone post — an additional requirement for generated templates: which parts must remain embeddable (checks, expected outputs) and which are narrative craft?
_Updated: 2026-07-31 (Short Posts hub restructure trigger) — appended lane/embed evidence to #1/#8/#9. No new investigations. Re-applied on top of a concurrent execution (res-vibe-footer2) that had overwritten this document; concurrent content preserved in full._
Evidence Update — 2026-08-03 (homepage trigger: document machine as formal-model substrate)
#1 (Agent State Machine Model Validation) — second formal model available. The document machine spec defines an explicit transition system (loaded → editing → publishing; version-history viewing; discard-draft; rebase with re-entry guards) implemented in XState v5 (465 lines per the Project Plan). This gives the state-machine validation program a second, in-production workload, and Product Backlog Opportunity #2 (State Machine Inspector) is the natural trace-collection tool. The external Case Study: Document Editor on the UI with State Machines hub independently documents version-history/branching behaviors that can be checked against the model.
#9 (Diagnostic Template Generation) — new domain. The document machine's quick-check analog: which states/guards can be checked from outside ("is a draft present?", "is a rebase in flight?") for a troubleshooting template.
11. Document Machine State Model Formalization [NEW 2026-08-03]
What: The document machine spec enumerates events/lifecycles that update document content. This is a small, well-specified state machine: loaded → editing → publishing, plus version-history viewing (render old version WITHOUT applying current draft data), discard-draft (only when a draft exists), and a rebase flow that must not run while saving or during another rebase.
Open questions:
Can the "no draft data when rendering an old version" and "no editing from an old version" rules be enforced structurally (state + render guard) rather than by convention?
Does the 12-state agent model (Investigation #1) generalize to document editing — same invariants, different events?
Can rebase conflicts be modeled as trap states with explicit recovery (currently "ignore and transition to rebase idle")?
Which transitions are observable from outside (sync/polling invalidation), i.e., what does the State Machine Inspector (Product Backlog #2) need to surface?
Evidence: document machine spec; Project Plan — New Publish mental model; Publish Guard; Draft Publishing Chain; Multi-draft; Document Web Editing; external Case Study: Document Editor
Next step: Formalize the machine (states × events × guards) from the spec; verify each documented rule (e.g., "discard changes impossible without draft") is a structural invariant; publish the state diagram as the anchor of the "Document Machine" essay (Content Roadmap #22).
_Updated: 2026-08-03 (homepage trigger) — added Investigation #11; appended evidence to #1/#9. All prior content preserved._
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime