← Back to library

▶ quest log

Self-Improving Agent Skills From an Open Source Optimizer

Claude Code / SkillsAdvanced@marc.kazsaved JUL 21

◇ Could this help me?

Yes on the idea, with the prerequisite treated as the real work. Optimizing instructions against a scored benchmark instead of by intuition is a legitimate and well-studied direction, and rejecting any edit that fails to beat the benchmark is the right safeguard to build in. The catch the clip skips is that the whole approach is only as good as the evaluation set, so a weak benchmark produces instructions that are confidently over-fitted to it, and self-rewritten instructions should still be read by a person before they run anywhere that matters. This is a general practice.

✦ Walkthrough+10 xp / step
0 / 5 steps · adopt for +40
$ suggested_prompt+40xp

I want to improve a skill or prompt through measurement rather than guesswork. Help me build an evaluation set first: propose a set of representative task inputs, define what a correct output looks like for each, and design a scoring method that would actually catch a subtly wrong answer. Then set up a loop that proposes an instruction edit, scores it against that set, and keeps the edit only if it beats the current score. Show me every rewritten instruction for review before it is adopted.

More from Claude Code / Skills

Enter world →