Find the behavior that predicts conversion. Then prove it.

Six free agent skills for lifecycle teams: find your leading indicator, target it, and test it with a real holdout.

Get the skills View on GitHub
  • MIT licensed
  • Python standard library
  • 136 passing tests
A real run on the bundled example data
$ npx skills add sendlyr/skills
✓ 6 skills installed

> What should we track for our 7-day trial?

Primary signal   goal_set ≥1 in 3 days
Converts if hit  36.4%   if missed 3.2%
Lift             +33.2pp [+30.8, +35.5]
Holdout check    replicated

Predictive, not causal. Test it with a holdout.

Six skills, one honest loop.

Each skill is a plain-English playbook for your agent, plus one small Python script. Run them in order.

  1. 01finding-leading-indicators

    Which early behavior predicts the outcome?

    Scores every behavior by lift × coverage, with leakage, maturity and out-of-time checks. score_signals.py

  2. 02designing-lifecycle-states

    Who needs which single ask?

    Builds a small state model, then flags unreachable states, duplicate asks and distinctions that don’t matter. state_audit.py

  3. 03designing-state-journeys

    When to send, and when to stop.

    Caps, spacing and state checks enforced at send time, plus an audit of where your sends went. audit_sends.py

  4. 04grounding-lifecycle-emails

    AI-written emails that stay true.

    The model writes structured copy; rules catch false claims and guessed names before a person approves. claim_check.py

  5. 05designing-holdout-experiments

    Seal the question before the answer.

    Sample size from a minimum worthwhile effect, deterministic assignment, and a sealed plan. holdout.py

  6. 06writing-decision-readouts

    Read it out honestly.

    Delivery is checked first, then one evidence class. The readout is allowed to say no. readout.py

Then the loop closes. The readout’s decision becomes the next sealed hypothesis, or a stop.

Our experiments showed nothing. Here is why.

We ran this with eReflect’s Typesy in 2026. These are the real numbers, published with permission.

“Neither experiment failed to show an effect. Neither was ever in a condition to measure one.”

Targeting

73% of emails went to people already past the key step.

623 of 848 Typesy Homeschool sends reached households past the key step. The stuck ones converted at 4%.

Delivery

Nobody measured whether the treatment arm was treated.

Only 80% of eligible treatment households got an email. The readouts counted sends, never reach.

Causation

We moved the behavior. Conversion did not move.

Typesy VIP: the indicator rose 11.6 points; conversion moved 0.1, against a 77% baseline.

Read all 13 lessons ↗ Read the Typesy case study →

Every script has to find the planted answer.

The repo ships Practica, a synthetic trial app with traps built in. The tests fail if a script misses one.

  • The real cliff

    Users who never set a goal almost never pay.

    Ranked first: 36.4% converted after setting a goal, 3.2% without, and it replicated. score_signals.py

  • A confounded decoy

    Motivated users open more email.

    Email opens rank low. They track motivation, not a step that drives paying. score_signals.py

  • A leak

    Billing pages get viewed after paying.

    Counted only before the outcome, the “perfect” signal turns rare and weak. score_signals.py

  • The wrong actor

    The system, not the learner, sends streak reminders.

    Flagged not actionable. An email can only ask its reader to act. score_signals.py

  • A flawed state model

    A legacy model with five planted problems.

    All five found, including a state no user can ever reach. state_audit.py

  • A broken experiment

    An outage, cap drift and control contamination.

    Refused to learn: treatment reach was 51.5% against a 90% gate. readout.py

One line in the agent you already use.

CSV in, report out. Python 3.11 or newer, no other installs, and raw rows never leave your machine.

  1. 1

    Install

    Claude Code

    /plugin marketplace add sendlyr/skills
/plugin install sendlyr@sendlyr

    Codex, Cursor, Gemini CLI and others

    npx skills add sendlyr/skills

    Or copy any folder under skills/ into your agent’s skills directory. Each one is self-contained.

  2. 2

    Run it on the bundled example

    Terminal, from the repo root

    python3 skills/finding-leading-indicators/scripts/score_signals.py \
  --accounts examples/practica/observational/accounts.csv \
  --events examples/practica/observational/events.csv \
  --outcome-window-days 14 --decision-window-days 7 \
  --email-actor learner --out /tmp/signals
  3. 3

    Or just ask your agent

    Use finding-leading-indicators on examples/practica and tell me what to track.

Want us to run it with your data?

The Signal Validation Pilot takes one candidate behavior through analysis, a holdout and a readout, with your team.

Request a Signal Validation Pilot