#4 — Prompting
November 1, 2023•3 min read
Why it matters: Effective prompt engineering is becoming table stakes for AI-powered startups. The right approach can dramatically improve your product's performance while reducing costs.
The Essentials
Start with prompts for rapid prototyping before investing in fine-tuning or custom models. Most founders overlook basic techniques that could deliver 80% of the value at 20% of the effort.
The bottom line: Master these five core strategies to outperform competitors who are wasting resources on unnecessary complexity.
Core Techniques That Actually Work
1. Fundamental Techniques Every Founder Should Know
N-shot prompting delivers outsized ROI:
- Include 5+ representative examples to guide your model
- Ensure examples match your production data distribution
- Demonstrate tool usage patterns when applicable
Chain-of-Thought (CoT) reduces hallucinations:
- Force your model to show its work before concluding
- Specify exact reasoning steps to improve accuracy
- Example implementation: "First extract key decisions from the transcript, then verify against meeting details, finally synthesize into key points"
ReAct framework combines reasoning with action:
- Explicitly instruct your model to reason, plan, then execute
- Design clean interfaces between your LLM and external tools
- Integration quality here often separates winning products
2. Structure Drives Performance
The insight: Structure isn't just about aesthetics—it's about performance. Well-structured inputs dramatically improve output quality.
Model preferences matter:
- Claude responds best to XML
- GPT models prefer Markdown and JSON
- Adapt your approach to your chosen model family
Pro tip: Prefilling response templates guarantees your outputs start exactly as needed.
3. Focus Beats Complexity
The trap: Most founders create bloated, catch-all prompts that underperform.
The solution: Break complex tasks into focused micro-prompts:
- Instead of one massive meeting summarizer
- Create separate prompts for extraction, verification, and synthesis
Why it works: Single-purpose prompts are easier to optimize, test, and maintain.
4. Context Optimization Is Your Edge
Challenge assumptions about what context you actually need.
Ruthlessly eliminate:
- Redundant instructions
- Self-contradictory language
- Poor formatting that wastes tokens
Structure context to highlight relationships between components.
5. Evaluation Is Non-Negotiable
Before engineering: Establish reliable evaluation metrics.
The minimum viable process:
- Manually label ~100 evaluation examples
- Create initial prompt and benchmark performance
- Iterate based on concrete metrics, not intuition
- Test on held-out examples before deployment
What Actually Doesn't Matter
Skip the pleasantries. "Please" and "thank you" don't improve output quality.
Forget threats and tips. Modern models don't need to be cajoled or warned.
The takeaway: Focus your engineering time on techniques that move metrics, not on prompt superstitions that waste your runway.
Keep reading
#5 — Information Retrieval / RAG
RAG systems outperform finetuning for knowledge integration, offering startups faster updates and lower costs.
#6 — Tuning and Optimizing Workflows
Move beyond basic prompting to deliver more reliable AI products at lower costs.
#7 — Working with models
AI startups face critical decisions on LLM integration, migration, versioning, and sizing that can determine success or failure.