pytyche.experiment.experiment

Per-round experiment snapshot: Experiment, CellObservation, and the per-cell observed-performance computation.

An Experiment is one completed round of a sequential experiment — the fitted posterior, its analysis, the cells that shipped, what each cell observed, and the recommendation engine’s plan for the next round. compute_cell_observations() produces the per-cell scoreboard from the joint posterior conditioned on the actual cell-assignment indicators.

Functions

compute_cell_observations(posterior, cells, ...)

Per-cell observed performance from the joint posterior.

Classes

CellObservation(id, n_visitors, ...)

Observed per-cell performance for one round.

Experiment(round_idx, posterior, analysis, ...)

One round of a sequential experiment.
class pytyche.experiment.experiment.CellObservation(id, n_visitors, observed_rpv, observed_rpv_ci, lift_vs_control, lift_vs_control_ci, lift_vs_other_cells, policy_summary)[source]

Bases: object

Observed per-cell performance for one round.

The scoreboard for the cells-as-blend story: when an operator ships a hypothesis cell alongside the recommendation engine’s cell, this is where per-cell RPV and cross-cell lift appear. All intervals follow the library-wide 80% credible convention.

id

Matches the cell’s id.

n_visitors

Visitors actually assigned to this cell this round.

observed_rpv

Empirical revenue per visitor over the cell’s members.

observed_rpv_ci

80% credible interval on the cell’s mean RPV from the joint posterior conditioned on the member rows.

lift_vs_control

Point estimate of observed_rpv minus the Control cell’s; exactly 0.0 on the Control cell itself.

lift_vs_control_ci

80% credible interval on the lift; (0.0, 0.0) on the Control cell.

lift_vs_other_cells

Point estimates of lift vs each other non-Control cell, keyed by cell id (no self key, no control key). Empty on the Control cell — Control IS the reference, so per-cell lifts vs Control live on those cells’ lift_vs_control, not redundantly reversed here.

policy_summary

cell.policy.describe().

Parameters:

  • id (str)

  • n_visitors (int)

  • observed_rpv (float)

  • observed_rpv_ci (tuple[float, float])

  • lift_vs_control (float)

  • lift_vs_control_ci (tuple[float, float])

  • lift_vs_other_cells (dict[str, float])

  • policy_summary (str)

class pytyche.experiment.experiment.Experiment(round_idx, posterior, analysis, cells_shipped, cell_observations, next_recommendation, truth_comparison)[source]

Bases: object

One round of a sequential experiment.

Composes the existing public single-shot types (the fitted posterior and its AnalysisResult) with the sequential-specific context: cells shipped, per-cell observations, the next-round recommendation, and the sim-only truth comparison.

round_idx

Zero-based round index.

posterior

The round’s fitted posterior (fit on cumulative data); identical to analysis.posterior.

analysis

The round’s analysis summary.

cells_shipped

The cell structure that actually served traffic.

cell_observations

Per-cell observed performance, matching cells_shipped order.

next_recommendation

The engine’s plan for the next round. None only on the engine’s provisional in-flight view; yielded experiments always carry a plan.

truth_comparison

Sim-mode truth metrics; None in real-data mode.

Parameters:
property observed: ObservedExperimentData | None

The observed data the round’s posterior was fit on.

Read-only alias of posterior.observed.

summary_one_line()[source]

Deterministic one-line summary of the round (template-based).

Return type:

str

pytyche.experiment.experiment.compute_cell_observations(posterior, cells, cell_assignment)[source]

Per-cell observed performance from the joint posterior.

cell_assignment holds one cell id per concatenated visitor row (the extraction-adapter row order: observed.variants frames concatenated in variant-list order). Point estimates are empirical means of the members’ revenue column; intervals condition the joint posterior on the membership indicators — each visitor contributes the level draws of its REALIZED arm (K = 2: the p0/p1 × sev0/sev1 channel selected by the row’s treatment indicator; K >= 3: the p_samples/sev_samples (n, S, K) slice at the realized arm), each cell’s per-draw member mean is reduced through the calibration-aware 80% interval path (pytyche.analysis._intervals.credible_interval).

Parameters:

  • posterior (HurdleBCFResult) – The round’s hurdle posterior, carrying observed data and retained per-visitor level draws.

  • cells (list[Cell]) – The round’s cells; the output matches this order. Must include the 'control' cell (the lift reference).

  • cell_assignment (ndarray) – Array of cell ids, one per concatenated visitor row.

Return type:

list[CellObservation]

Returns:

One CellObservation per cell, in cells order.

Raises:

ValueError – When the posterior carries no observed data or no per-visitor level draws (neither the K = 2 channel arrays nor the K >= 3 p_samples / sev_samples), when cell_assignment length mismatches the visitor rows, when a cell has no assigned visitors, or when no cell has id 'control'.