Measuring agent skill quality

Positive feedback tells you the skill worked at least once. Negative and neutral feedback are the better product signals because they show where the skill lost context, stalled, or needed a human to recover.

• Positive: the skill completed the intended job.
• Neutral: the skill partly worked, but the agent could not fully prove the result.
• Negative: the skill failed, blocked the run, or gave the agent the wrong next step.

The rating is the sorting layer. The written feedback is where you find the actual repair. Look for repeated nouns, missing prerequisites, unclear commands, and vague completion standards.

Do not promote a skill because the first run was impressive. Promote it when repeated runs produce similar positive evidence and the remaining negative feedback is explainable.