pytyche.experiment.sequential

The SequentialExperiment state machine — the L1 round-by-round loop.

Each advance() (equivalently next(exp)) runs one round: size the round from the schedule, hand the current NextRoundPlan to the generator, fit the joint hurdle BCF on the CUMULATIVE data, analyze, score the round’s cells, ask the recommendation engine for the next plan, and (sim mode) evaluate against the cumulative ground truth. All state commits atomically at the end of the round — a failed step leaves history untouched.

The cumulative row order is variant-major: all rounds of the first-seen variant, then all rounds of the next. Per-round artifacts aligned to a round’s own concatenation (truth arrays, the cell column) are re-indexed into that order via the position map _build_cumulative returns.

Module Attributes

Generator

The single data-source contract.

Functions

sequential_experiment(*, generator, ...[, ...])

Construct a SequentialExperiment — the L1 entry point.

Classes

SequentialExperiment(*, generator, schedule, ...)

Stateful round-by-round adaptive experiment loop.

Exceptions

UncalibratedWarning

Emitted once per SequentialExperiment running uncalibrated.
pytyche.experiment.sequential.Generator[source]

The single data-source contract. The loop invokes generator(round_idx, plan) at the start of each round and receives (observed, truth) — truth is non-None in sim mode and None in real-data mode (the mode may not flip between rounds). Every returned visitors frame must carry the reserved cell column recording each visitor’s allocated cell id, and generators assigning adaptively record per-visitor propensities in the reserved propensity columns (propensity at K = 2, propensity_1..propensity_{K-1} at K >= 3).

alias of Callable[[int, NextRoundPlan], tuple[ObservedExperimentData, CalibrationTruth | None]]

exception pytyche.experiment.sequential.UncalibratedWarning[source]

Bases: UserWarning

Emitted once per SequentialExperiment running uncalibrated.

Fires on the first advance() when the instance was constructed with calibration=None. Supply an SBC-fitted Calibration artifact via the calibration= constructor parameter to correct interval coverage and silence the warning.

class pytyche.experiment.sequential.SequentialExperiment(*, generator, schedule, treatments, cells=None, min_control_weight=0.05, min_explore_weight=0.05, max_segment_depth=3, min_segment_share=0.1, calibration=None, graduation_rule=None, seed=0, progress=False)[source]

Bases: object

Stateful round-by-round adaptive experiment loop.

The instance is its own iterator: next(exp) (or exp.advance()) runs one round and returns the resulting Experiment snapshot; iteration raises StopIteration when the schedule is exhausted. Configuration is locked at construction; per-round cell overrides go through advance(cells=...).

Scope: this surface assumes designed experiments — the operator emits the assignment rules (the plan’s cells and policies), the platform assigns per those rules, and per-visitor propensities are known and recorded. Heavily-confounded observational inference is out of scope for this API; for that setting, consider econml or DoubleML.

Parameters:

  • generator (Callable[[int, NextRoundPlan], tuple[ObservedExperimentData, CalibrationTruth | None]]) – generator(round_idx, plan) returning (observed, truth | None) — the single data-source contract. Sim-mode generators return ground truth; real-data generators return None (the mode may not flip between rounds). Every visitors frame must carry the reserved cell column recording each visitor’s allocated cell id.

  • schedule (Schedule) – Per-round visitor counts (Schedule protocol).

  • treatments (Sequence[str]) – The full treatments universe, control first. Drops are recomputed from allocation history each round; dropped names never leave this universe.

  • cells (list[Cell] | None) – Optional round-0 cell-structure override replacing the cold-start Control/Explore 50/50 split.

  • min_control_weight (float) – Guaranteed Control-cell share on engine-proposed plans.

  • min_explore_weight (float) – Guaranteed Explore-cell share on engine-proposed plans.

  • max_segment_depth (int) – Policy-tree depth bound for the per-round analysis and recommendation.

  • min_segment_share (float) – Minimum per-leaf population share.

  • calibration (Calibration | None) – SBC-fitted artifact applied on-path each round, or None (uncalibrated; warns once per instance).

  • graduation_rule (GraduationRule | None) – None uses ExpectedLossRule at its defaults.

  • seed (int) – Master seed — per-round fit seeds derive from it, and it seeds the policy tree’s stability bootstrap.

  • progress (bool) – When True, each round’s fit renders tqdm progress bars on stderr (passed through as fit_hurdle_bcf(progress=True)). Default False — silent.

Raises:

ValueError – When min_control_weight + min_explore_weight is not below 1.0 (no room for the Optimized cell).

advance(cells=None)[source]

Run one round and return its Experiment snapshot.

cells overrides this round’s cell structure only; subsequent rounds revert to the recommendation engine’s proposals. Overrides must satisfy the weight-sum invariant but are NOT subject to the engine’s min-weight floors.

All state commits only after every step succeeds — a failure at any step (missing cell column, calibration regime mismatch, …) leaves history and the cumulative data unchanged.

Raises:

  • StopIteration – When the schedule is exhausted.

  • ValueError – When the override cells’ weights do not sum to 1.0, when round data lacks the reserved cell column or labels visitors with ids outside the round’s plan, when a variant name falls outside the treatments universe, or when the round’s identity (experiment_id, metric, sim/real mode) conflicts with prior rounds.

Parameters:

cells (list[Cell] | None)

Return type:

Experiment

run_to_completion()[source]

Advance until the schedule is exhausted or a candidate emerges.

Returns self for chaining.

Raises:

ValueError – When the schedule is open-ended (n_rounds is None) — the operator must provide a stop condition.

Return type:

SequentialExperiment

property history: list[Experiment]

Completed rounds, oldest first (a defensive copy).

property latest: Experiment | None

The most recent round, or None before the first.

property current_round_idx: int

Number of completed rounds (the next round’s index).

property next_recommendation: NextRoundPlan | None

The engine’s plan for the next round; None before round 0.

graduation_candidates(sustained_rounds=2)[source]

Latest-round candidates sustained for at least sustained_rounds.

Parameters:

sustained_rounds (int)

Return type:

list[GraduationCandidate]

has_graduation_candidate()[source]

Whether any candidate meets the default sustained-rounds bar.

Return type:

bool

has_credible_segments(stability_threshold=0.8)[source]

Whether any latest-round segment clears stability_threshold.

Parameters:

stability_threshold (float)

Return type:

bool

dropped_treatments()[source]

Universe treatments absent from the latest plan’s active list.

Return type:

list[str]

property summary: str

Deterministic multi-paragraph prose over the series so far.

property confidence: Literal['high', 'medium', 'low']

One-word label from the credibility + graduation state.

pytyche.experiment.sequential.sequential_experiment(*, generator, schedule, treatments, cells=None, min_control_weight=0.05, min_explore_weight=0.05, max_segment_depth=3, min_segment_share=0.1, calibration=None, graduation_rule=None, seed=0, progress=False)[source]

Construct a SequentialExperiment — the L1 entry point.

The verb form of the SequentialExperiment constructor with the identical signature; see the class docstring for the parameter contract. Iterate the returned instance (next(exp)) to advance one round at a time, or call run_to_completion() on a bounded schedule.

Scope: this surface assumes designed experiments — the operator emits the assignment rules (the plan’s cells and policies), the platform assigns per those rules, and per-visitor propensities are known and recorded. Heavily-confounded observational inference is out of scope for this API; for that setting, consider econml or DoubleML.

Parameters:
Return type:

SequentialExperiment