Find the Step Where Readers Abandon Your Skill

If readers buy or claim your skill but few report a useful result, map the path from access to the first completed task. Count the same readers at each observable step, give them a consistent amount of time, and investigate the largest unexplained gap.

The gap tells you where to look. It does not tell you why someone stopped. A missing record may mean confusion, a postponed task, unrecorded activity, or an actual product problem. Treat “abandoned” as a hypothesis until you have evidence about the experience.

This guide provides an original funnel worksheet with invented numbers. It is a manual analysis design, not a claim that Skillfully or an AI provider supplies all the events, identities, or conversation access needed to build it automatically.

For a book-based skill, the first meaningful task should be something a reader recognizes as progress. Examples include choosing an opening scene, checking an instruction sheet, or preparing a concrete practice plan. “Used AI” is too broad to tell you whether the companion helped.

Consider a fictional author, Dorian, whose book teaches people to prepare a short audio documentary. His companion's first task is to help a reader produce a recording brief: the subject, the scene to capture, three sounds to seek, and one practical constraint.

The first-use journey might be:

1. Obtain access to the companion.
2. Open the setup guidance.
3. Reach the intended AI environment with the skill available.
4. Begin the recording-brief exercise.
5. Produce a brief that meets the stated rule.

This is a proposed journey. Before counting it, Dorian must establish which steps he can actually observe. If setup happens in another application and there is no reliable record, he should leave that step unknown or use a clearly labeled voluntary report.

A funnel follows a chosen sequence within a chosen time window. Amplitude's documentation describes those parameters explicitly: the events, their order, and the time allowed for conversion. Changing them changes what the chart means. Amplitude's funnel interpretation guide

For Dorian's fictional worksheet, use these conventions:

• Starting group: 40 distinct readers who received access during the same invitation period.
• Observation window: seven days from each reader's access confirmation, chosen for this example rather than as a benchmark.
• Maturity: all 40 have had the full seven days when the worksheet is reviewed.
• Counting unit: one reader, counted at most once per step.
• Order: count a later step in this funnel only when the same reader has the preceding steps in order.
• Exclusions: author practice accounts and known test runs are excluded and documented.

Do not sum activity totals from unrelated systems and call that a funnel. Forty purchases, thirty page visits, and twenty instruction loads may describe overlapping groups, repeated visits, or different people. They become a reader funnel only when the identity and sequence can be reconciled appropriately.

Analytics software can support different counting methods. Mixpanel, for example, distinguishes counting unique users from counting total actions. Use the setting that matches the question, then record it with the result. Mixpanel's funnel guide

Every number in this table is invented. Assume that the fictional records can be matched to the same readers and that the observable setup steps have been checked for accuracy.

The largest numerical gap is between opening guidance and confirming availability: 12 of 32 readers have no recorded next step. The missing share at that transition is 37.5%, calculated as 12 divided by 32. It is not 37.5% of all readers; relative to the starting 40, the same gap is 30%.

Keep both the count and denominator visible. A dramatic percentage based on two people may deserve investigation, but it does not carry the same evidence as a larger, consistently measured group.

The table also shows why “only 45% started” is incomplete. The first question is whether the recording is reliable. If it is, the twelve-person setup gap is a more specific starting point than rewriting the audio-documentary method from scratch.

Now suppose Dorian invites all 40 readers to a follow-up. Sixteen respond, including twelve of the eighteen recorded starters. Of those twelve starters, ten voluntarily share briefs that meet the completion rule and two say they stopped before finishing. The other four respondents had not reached the recorded start step.

These are still synthetic numbers. They illustrate a separate evidence layer:

Do not label the eight starters without an inspected brief as eight failures. Two reported stopping; six did not supply an outcome. Nor should ten completions among twelve responding starters become the completion rate for all eighteen starters.

If you later obtain reliable completion events for everyone, you can extend the funnel. Until then, keep confirmed artifacts, self-reports, and unknown outcomes distinguishable. A recorded instruction retrieval is also not proof that the AI followed the method or produced a satisfactory brief.

An onboarding problem may involve understanding when to use the skill, not merely locating the install button. In a public discussion, u/ShroomHog wrote:

“There are so many skills out there that I am confused how to best apply them to my work.”

The poster described uncertainty about installing and invoking a skill for an existing coding project. This is one person's question, not a measured rate of confusion among authors' readers. It is useful because it separates interest in the tool from clarity about the next action. Original question

For Dorian's twelve-person setup gap, a short follow-up could ask: “What was the last step you reached, and what did you expect to happen next?” Provide an optional place for a redacted screenshot if that would help. Do not request private conversations just to classify a setup problem.

Use categories after reading the answers: did not try yet, unclear starting instruction, wrong environment, access problem, technical error, or another reason. Preserve uncertainty when a reply does not fit. A convenient category is not a diagnosis.

Suppose, in the fictional exercise, four people from the setup gap reply. Two cannot tell which application the instructions refer to, one postponed the task, and one reports an error. Those four accounts do not establish the reason for all twelve missing transitions.

They do support a narrow next action: review the instructions for application ambiguity and reproduce the reported error through an authorized test. The postponed reader may need a more appropriate reminder or no intervention at all.

The change record might look like this:

Do not promise an improved conversion rate from these changes. The next group may differ in experience, motivation, device, or invitation source. Record those differences and look for repeated evidence before attributing a change in the numbers to your edit.

Walk through the observable steps with an authorized practice account and confirm that the expected records appear. Identify your own activity so it stays out of the reader group. If that check has not happened, label the funnel provisional.

Watch for duplicate accounts, delayed events, missing timestamps, and readers who reach the task through a different route. A reader who skips your guidance because they already know the setup may succeed outside the strict sequence. Count that separately rather than treating the reader as a failure or quietly changing the definition mid-report.

Amplitude's FAQ notes that a funnel and a simple event chart can differ because the funnel applies sequence and conversion-window rules. That is a useful reminder to compare definitions before assuming two dashboards disagree about the underlying activity. Amplitude's funnel FAQ

If the gap is inside the method rather than setup, inspect the work. Our guide to measuring agent skill quality can help you judge whether the recording brief or another promised output actually meets your criteria.

A useful funnel review produces a specific question, the evidence needed to answer it, and a person responsible for the next step. It does not require a new chart for every possible reader behavior.

For Dorian, that question is whether the setup guidance clearly identifies the supported application. Completion quality remains a separate investigation, with missing outcomes visible. That separation prevents a small dataset from turning into a confident but unsupported story about why readers leave.

To build a measurable first-use journey around your established method, visit Skillfully and choose Book onboarding. Bring the reader task and the points in the journey you need to observe.