Significance at 5%

GoTo's weekly experiment review needs a verdict per test, not a column of z-scores. Postgres has no normal CDF, so the standard move is the same one you'd make with a printed statistics table: keep the critical values in a table and join against them.

experiment_results holds one row per experiment per variant. z_critical holds two-sided critical values by alpha.

Return experiment, z_score (rounded to 4 decimal places), and is_significant — true when the absolute z-score reaches or exceeds the critical value at alpha 0.05. Use the pooled two-proportion z-test:

p_pool = (x_c + x_t) / (n_c + n_t)

SE = sqrt( p_pool * (1 - p_pool) * (1/n_c + 1/n_t) )

z = (p_t - p_c) / SE

Order by experiment.