Agent Skill vs Code: What Should Live in Instructions?

An agent skill is best for procedural guidance: how to approach a job, what to inspect, when to stop, and what evidence to return. Code is best for deterministic execution: parsing, validation, API calls, transformations, tests, and repeatable automation.

The strongest skills often include code, but they do not replace code. OpenAI's Codex skill format explicitly allows a skill directory to include SKILL.md plus optional scripts/, references/, and assets/ (OpenAI Codex skills). Anthropic's Agent Skills documentation uses the same basic shape: a skill packages instructions, scripts, and reference material so an agent can load the right capability when it is relevant (Anthropic Agent Skills).

The design question is not "skill or code?" It is "which parts need judgment, and which parts need determinism?"

If you are still deciding what an agent skill is, start with What Is an Agent Skill?. This article assumes the task is already repeatable enough to deserve a skill, and focuses on the boundary between instructions, scripts, tools, tests, and application code.

Put guidance in SKILL.md when the agent needs to make a contextual decision. A good skill explains the job, the trusted source order, the stop rules, and the completion evidence. It should not ask the model to pretend it is a parser, a linter, a browser, or a CI system.

Put trigger conditions, required inputs, trusted source order, task sequence, judgment criteria, escalation rules, output format, and completion evidence in the skill.

Example:

If validating blog frontmatter, inspect only Markdown files under app/content/blog.
Check required fields, canonical path, date consistency, and body starts with H2.
Report invalid files with the exact field that failed.

That is procedural. It explains the job. It also creates a clear contract for the author: if you later add a script, the skill already knows when to run it and what evidence to return.

This is the same pattern described in How to Write an Agent Skill That Actually Works: name one repeatable job, tell the agent where to look first, and require visible evidence. The skill is the operating procedure. It is not the only enforcement layer.

Put logic in code when the answer should be the same every time for the same input. Code is where you want exactness, repeatability, type discipline, and failure modes that can be tested without asking a model to be careful.

Put YAML frontmatter parsing, schema validation, link checking, screenshot capture, SQL execution, data transforms, file conversion, and deterministic scoring in scripts, tools, validators, or application code.

Example:

node scripts/validate-blog-frontmatter.js app/content/blog

The script can fail fast and produce exact errors. The skill can tell the agent when to run it and how to interpret the results.

There is one operational catch: a bundled script is still code that must run in a real environment. In a GitHub Copilot discussion about Agent Skills in VS Code, one user pointed out that skills are easier to author than MCP because they can combine prompt, context, and optional scripts, but script dependencies and runtime expectations still have to be solved somewhere (Reddit discussion). That is the right concern. A skill can say "run the validator"; the repository still needs the dependency, command, permissions, and test path to make that instruction reliable.

Skill instruction:

Run the frontmatter validator before reviewing prose.
If the validator fails, fix metadata first and rerun.
Then inspect headings, source links, and cannibalization risk manually.

Script responsibility:

Parse Markdown files.
Verify required frontmatter fields.
Check canonicalPath matches slug.
Check body starts with H2.
Exit nonzero on failure.

This split is cleaner than asking the model to mentally validate every file. Use the model for review and judgment; use code for exact checks.

A concrete content QA setup might look like this:

Layer	Responsibility
SKILL.md	"Review only the assigned article, use the local source brief first, and report final word count."
scripts/validate-blog-frontmatter.js	Parse frontmatter and fail if canonicalPath does not match the slug.
src/content/blog.test.ts	Ensure the blog loader accepts the article and the body starts with H2.
Human review or eval	Decide whether the piece actually answers the reader's question and avoids generic prose.

The skill should know that metadata failures come before prose edits. The script should know nothing about editorial taste. That boundary keeps both parts maintainable.

Browser work is another place where the boundary becomes obvious. A skill can say:

After changing a checkout page, open the local app, test the happy path, capture one desktop screenshot and one mobile screenshot, and report the exact route and command used.

That is useful guidance, but it is not a substitute for browser automation. Playwright's own docs recommend locators based on user-facing attributes such as page.getByRole() and advise running tests frequently in CI, ideally on each commit or pull request (Playwright locators, Playwright best practices).

So the split should be:

• Put the QA sequence, route list, screenshot expectations, and escalation rules in the skill.
• Put selectors, assertions, retries, screenshots, and browser setup in Playwright tests or scripts.
• Put the required CI workflow in .github/workflows, not in prose. GitHub's Node.js Actions guide shows the normal pattern: install dependencies, build, and run tests in a workflow (GitHub Actions for Node.js).

The skill can require the agent to run npm run test:e2e -- checkout.spec.ts before saying the page works. The Playwright test is what proves the button exists, the redirect happens, and the confirmation state appears.

MCP tools expose callable functions to models and can connect AI applications to external data sources and systems (MCP introduction). The MCP tools specification describes tools as model-controlled actions exposed by servers and recommends human-in-the-loop visibility and approval for safety-sensitive operations (MCP tools).

That means code can live outside the skill as an MCP tool. The skill still matters because it explains:

• which tool to use
• when to use it
• what inputs are safe
• how to interpret output
• when to ask for approval

For example, a create_github_issue MCP tool should own the schema, authentication, API request, and error handling. A skill should say when creating an issue is appropriate, what evidence should appear in the issue body, whether the user must approve the draft first, and what to do if the repo is ambiguous.

Do not hide policy inside the tool description if the policy is part of the work. Do not hide API mechanics inside the skill if the API call needs repeatability.

Structured outputs and function calling are useful when a model must hand deterministic software a predictable shape. OpenAI's Structured Outputs guide describes JSON Schema-constrained responses, and the function calling guide describes tool calls whose inputs and outputs are structured enough for an application to process (OpenAI Structured Outputs, OpenAI function calling).

That changes where you put the boundary:

• The skill can say what the final report should mean.
• A JSON Schema can enforce which fields the report must contain.
• Application code can validate the schema, reject invalid payloads, and decide whether to retry.
• Tests can lock the contract so a future model or prompt change does not silently break downstream code.

If a sales-call skill must return objections, buying_signals, next_actions, and follow_up_email, the field names belong in a schema or typed interface once another system consumes them. The skill can still explain how to judge a weak objection or when the transcript is too poor to trust. The schema handles shape. The skill handles judgment.

Skills need evaluation, but "be accurate" is not an eval. If a behavior matters repeatedly, write a test, an eval, or a review fixture that can fail.

OpenAI's evals documentation frames evaluations as tests for whether model outputs meet style and content criteria, especially when upgrading models or changing an LLM application (OpenAI evals). That is directly relevant to skills. A skill can describe the expected behavior, but the eval should contain cases that catch regressions.

For example:

Requirement	Skill instruction	Test or eval
"Never edit unrelated files"	State the scope and stop rule	Fixture repo with unrelated dirty files; pass only if output leaves them untouched
"Return source-backed research"	Require primary sources first	Eval checks citations and flags unsupported claims
"Validate article metadata"	Run the validator before prose review	Unit test fails on missing canonicalPath
"Confirm browser workflow"	Capture evidence before final answer	Playwright test verifies the page state

The practical point is simple: if you would be upset when the instruction is ignored, consider whether a deterministic check can enforce it.

Use instructions when:

• human judgment is required
• the right sequence matters
• the agent must adapt to context
• the output needs explanation

Use scripts when:

• the operation is deterministic
• exact parsing or validation matters
• repeated manual checking would be error-prone
• speed or scale matters

Use MCP tools or function calls when:

• the agent needs to reach an external system
• inputs should be schema-validated before an action
• authentication, permissions, or audit logs matter
• the action may need explicit approval

Use application code when:

• the behavior is part of the product, not just the agent workflow
• users depend on the behavior outside the agent
• the output feeds other product systems
• you need versioned APIs, database writes, or durable state

Use both when:

• the task has a repeatable procedure plus mechanical substeps
• the agent needs scripts but also needs boundaries
• the output must be both correct and explainable

If a mistake can be prevented by a deterministic check, write code. If a mistake comes from unclear judgment or missing procedure, write a better skill.

An agent skill can say "verify every blog post has a title, canonical path, author, and valid date." That instruction is useful, but if the same check runs every day, it should probably become code. A parser test can fail deterministically when frontmatter is missing. A skill should not be the only thing standing between a broken content file and production.

The better design is to split responsibility. Put mechanical checks in tests, scripts, schemas, or linters. Put judgment in the skill: which posts need source review, what quality bar applies, when a draft is too thin, and how to report uncertainty. Code catches the repeatable rule. The skill guides the human-like judgment around the rule.

The first mistake is putting too much faith in prose. "Always validate the JSON" is weaker than a schema validator that rejects invalid JSON. "Check the page works" is weaker than a Playwright test that fails when the user cannot complete the flow.

The second mistake is over-coding judgment. A linter can tell you a field is missing. It cannot decide whether a customer interview summary captured the buyer's real objection. That belongs in the skill, supported by examples and review criteria.

The third mistake is hiding operational policy in code where authors cannot see it. If the agent must ask before sending email, deleting data, opening a paid API, or publishing a public artifact, that rule should be visible in the skill as well as enforced by the tool or application.

The fourth mistake is treating scripts as throwaway helpers. If a skill depends on a script, the script needs the same discipline as other code: clear inputs, useful errors, dependency notes, and tests when the behavior matters.

Is an agent skill just code with instructions around it?

No. A skill is the workflow package. Code may be part of the package, but the skill's job is to tell the agent how to complete the work: where to start, what to trust, when to stop, and what evidence to return.

Should every skill include a script?

No. Many skills are pure procedural guidance. Add a script when there is a repeated mechanical step, such as parsing files, validating schemas, querying an API, converting assets, or running a browser check.

When should a script become an MCP tool?

Move from a local script to an MCP tool when the action needs a reusable interface across agents or clients, especially when it touches external systems, shared credentials, approvals, or audit trails.