Prompt LibraryEngineering
LLM Model A/B Comparison Protocol Prompt
Compare two models on a task with a fair protocol.
The prompt
Design a fair A/B comparison between model {{model_a}} and {{model_b}} on the task below.
Specify:
- 20 prompts (same for both models, same temperature, same system prompt).
- The evaluation: blind LLM-judge or human, and the rubric.
- How to handle ties and variance.
- The metric that decides the winner.
Then state one confound that could make the comparison unfair.
Task: {{task}}Replace the {{fields}} with your own context and tighten the rules to match your domain.
More Engineering prompts
LLM Reranking Prompt for RAG Retrieval
Use a model as a cross-encoder-style reranker for retrieved chunks.
LLM Function-Call Error Recovery Prompt
Give a model a decision tree for handling failed tool calls.
LLM Model Routing Policy Design Prompt
Decide which requests go to a cheap model vs a frontier model.
