project-development

Project Development Methodology

This skill covers the principles for identifying tasks suited to LLM processing, designing effective project architectures, and iterating rapidly using agent-assisted development. The methodology applies whether building a batch processing pipeline, a multi-agent research system, or an interactive agent application.

When to Activate

Activate this skill when:

• Starting a new project that might benefit from LLM processing
• Evaluating whether a task is well-suited for agents versus traditional code
• Designing the architecture for an LLM-powered application
• Planning a batch processing pipeline with structured outputs
• Choosing between single-agent and multi-agent approaches
• Estimating costs and timelines for LLM-heavy projects

Core Concepts

Task-Model Fit Recognition

Evaluate task-model fit before writing any code, because building automation on a fundamentally mismatched task wastes days of effort. Run every proposed task through these two tables to decide proceed-or-stop.

Proceed when the task has these characteristics:

Characteristic	Rationale
Synthesis across sources	LLMs combine information from multiple inputs better than rule-based alternatives
Subjective judgment with rubrics	Grading, evaluation, and classification with criteria map naturally to language reasoning
Natural language output	When the goal is human-readable text, LLMs deliver it natively
Error tolerance	Individual failures do not break the overall system, so LLM non-determinism is acceptable
Batch processing	No conversational state required between items, which keeps context clean
Domain knowledge in training	The model already has relevant context, reducing prompt engineering overhead

Stop when the task has these characteristics:

Characteristic	Rationale
Precise computation	Math, counting, and exact algorithms are unreliable in language models
Real-time requirements	LLM latency is too high for sub-second responses
Perfect accuracy requirements	Hallucination risk makes 100% accuracy impossible
Proprietary data dependence	The model lacks necessary context and cannot acquire it from prompts alone
Sequential dependencies	Each step depends heavily on the previous result, compounding errors
Deterministic output requirements	Same input must produce identical output, which LLMs cannot guarantee

The Manual Prototype Step

Always validate task-model fit with a manual test before investing in automation. Copy one representative input into the model interface, evaluate the output quality, and use the result to answer these questions:

• Does the model have the knowledge required for this task?
• Can the model produce output in the format needed?
• What level of quality should be expected at scale?
• Are there obvious failure modes to address?

Do this because a failed manual prototype predicts a failed automated system, while a successful one provides both a quality baseline and a prompt-design template. The test takes minutes and prevents hours of wasted development.

Pipeline Architecture

Structure LLM projects as staged pipelines because separation of deterministic and non-deterministic stages enables fast iteration and cost control. Design each stage to be:

• Discrete: Clear boundaries between stages so each can be debugged independently
• Idempotent: Re-running produces the same result, preventing duplicate work
• Cacheable: Intermediate results persist to disk, avoiding expensive re-computation
• Independent: Each stage can run separately, enabling selective re-execution

Use this canonical pipeline structure:

acquire -> prepare -> process -> parse -> render

1. Acquire: Fetch raw data from sources (APIs, files, databases)
2. Prepare: Transform data into prompt format
3. Process: Execute LLM calls (the expensive, non-deterministic step)
4. Parse: Extract structured data from LLM outputs
5. Render: Generate final outputs (reports, files, visualizations)

Stages 1, 2, 4, and 5 are deterministic. Stage 3 is non-deterministic and expensive. Maintain this separation because it allows re-running the expensive LLM stage only when necessary, while iterating quickly on parsing and rendering.

File System as State Machine

Use the file system to track pipeline state rather than databases or in-memory structures, because file existence provides natural idempotency and human-readable debugging.

data/{id}/
  raw.json         # acquire stage complete
  prompt.md        # prepare stage complete
  response.md      # process stage complete
  parsed.json      # parse stage complete

Check if an item needs processing by checking whether the output file exists. Re-run a stage by deleting its output file and downstream files. Debug by reading the intermediate files directly. This pattern works because each directory is independent, enabling simple parallelization and trivial caching.

Structured Output Design

Design prompts for structured, parseable outputs because prompt design directly determines parsing reliability. Include these elements in every structured prompt:

1. Section markers: Explicit headers or prefixes that parsers can match on
2. Format examples: Show exactly what output should look like
3. Rationale disclosure: State "I will be parsing this programmatically" so the model prioritizes format compliance
4. Constrained values: Enumerated options, score ranges, and fixed formats

Build parsers that handle LLM output variations gracefully, because LLMs do not follow instructions perfectly. Use regex patterns flexible enough for minor formatting variations, provide sensible defaults when sections are missing, and log parsing failures for review rather than crashing.

Agent-Assisted Development

Use agent-capable models to accelerate development through rapid iteration: describe the project goal and constraints, let the agent generate initial implementation, test and iterate on specific failures, then refine prompts and architecture based on results.

Adopt these practices because they keep agent output focused and high-quality:

• Provide clear, specific requirements upfront to reduce revision cycles
• Break large projects into discrete components so each can be validated independently
• Test each component before moving to the next to catch failures early
• Keep the agent focused on one task at a time to prevent context degradation

Cost and Scale Estimation

Estimate LLM processing costs before starting, because token costs compound quickly at scale and late discovery of budget overruns forces costly rework. Use this formula:

Total cost = (items x tokens_per_item x price_per_token) + API overhead

For batch processing, estimate input tokens per item (prompt + context), estimate output tokens per item (typical response length), multiply by item count, and add 20-30% buffer for retries and failures.

Track actual costs during development. If costs exceed estimates significantly, reduce context length through truncation, use smaller models for simpler items, cache and reuse partial results, or add parallel processing to reduce wall-clock time.

Detailed Topics

Choosing Single vs Multi-Agent Architecture

Default to single-agent pipelines for batch processing with independent items, because they are simpler to manage, cheaper to run, and easier to debug. Escalate to multi-agent architectures only when one of these conditions holds:

• Parallel exploration of different aspects is required
• The task exceeds single context window capacity
• Specialized sub-agents demonstrably improve quality on benchmarks

Choose multi-agent for context isolation, not role anthropomorphization. Sub-agents get fresh context windows for focused subtasks, which prevents context degradation on long-running tasks.

See multi-agent-patterns skill for detailed architecture guidance.

Architectural Reduction

Start with minimal architecture and add complexity only when production evidence proves it necessary, because over-engineered scaffolding often constrains rather than enables model performance.

Vercel's d0 agent achieved 100% success rate (up from 80%) by reducing from 17 specialized tools to 2 primitives: bash command execution and SQL. The file system agent pattern uses standard Unix utilities (grep, cat, find, ls) instead of custom exploration tools.

Reduce when:

• The data layer is well-documented and consistently structured
• The model has sufficient reasoning capability
• Specialized tools are constraining rather than enabling
• More time is spent maintaining scaffolding than improving outcomes

Add complexity when:

• The underlying data is messy, inconsistent, or poorly documented
• The domain requires specialized knowledge the model lacks
• Safety constraints require limiting agent capabilities
• Operations are truly complex and benefit from structured workflows

See tool-design skill for detailed tool architecture guidance.

Iteration and Refactoring

Plan for multiple architectural iterations from the start, because production agent systems at scale always require refactoring. Manus refactored their agent framework five times since launch. The Bitter Lesson suggests that structures added for current model limitations become constraints as models improve.

Build for change by following these practices:

• Keep architecture simple and unopinionated so refactoring is cheap
• Test across model generations to verify the harness is not limiting performance
• Design systems that benefit from model improvements rather than locking in limitations

Practical Guidance

Project Planning Template

Follow this template in order, because each step validates assumptions before the next step invests effort.

1. Task Analysis

---

Content truncated.