# Methods summary

This is a selected public summary for *Does a higher click-through rate mean a cheaper conversation?*, ALIQ Studio research note version 1.0, dated 9 November 2026. It describes the retained retrospective rules, selected implementation details, the exploratory diagnostics added during the study, and the test plan fixed for later data. It is not the unedited historical protocol, an external preregistration, or a claim that the underlying records are publicly reproducible. Private source locations and real identity/ownership mappings are omitted.

## Design and scope

The study is a descriptive secondary-data audit of selected Malaysian-targeted WhatsApp advertising. The original window is 5 January–30 August 2026; the later window is 31 August–25 October 2026. Both use complete Monday–Sunday weeks. The later analysis followed knowledge of the original result. Further magnitude, arithmetic-fragility, temporal, equal-pair and final-stage diagnostics are exploratory: they were specified after earlier findings were known.

The source material comprises retained daily advertising metrics and saved ad, ad-set and campaign objects. It is an access-dependent archive, not a probability sample or verified national SME frame. No campaign intervention, new advertising spend or customer-message analysis was performed for this research. Reporting results are not unique customers, qualified enquiries, sales or profit.

## Recorded request settings

The retained request metadata for both periods records Meta Marketing API `v26.0`, `level=ad`, and daily `time_increment=1`. The pinned requests do not record explicit values for `action_attribution_windows`, `action_report_time`, `use_account_attribution_setting` or `use_unified_attribution_setting`. Their omission is not interpreted as a runtime default or proof that a setting was disabled. The separately returned `attribution_setting` structure is retained for exact matching; its values are not disclosed.

## Platform definitions

Read from Meta's Business Help Centre on 8 November 2026, by rendering the pages in a headless browser (a plain fetch returns only the page title):

- **Messaging conversations started:** "The number of times that a messaging conversation was started with your business after at least seven days of inactivity, attributed to your ads." For WhatsApp Business, "this metric only includes conversations started from clicks on your ad." The `7d` in the API action name `onsite_conversion.messaging_conversation_started_7d` therefore most plausibly refers to this seven-day inactivity rule, not to an attribution window.
- **Link clicks:** "The number of clicks on links within the ad that led to advertiser-specified destinations, on or off Meta technologies", where destinations "are based on your campaign, ad set and ad settings". The page does not say whether the tap that opens WhatsApp counts; no assumption is made.

These are current help-page wordings. They are not the Marketing API field documentation, and they may differ from the definitions in force on historical dates.

## Eligibility and identity handling

The daily-record grain is account × ad × saved date. Duplicate keys, incomplete captures, invalid dates and negative measurements are rejected. Each ad-day must join to the required archived ad, ad-set and campaign objects; overlapping breakdown extracts are not summed into daily records.

The primary universe requires MYR, `REPLIES` optimisation, a saved WhatsApp-only destination, and saved targeting whose stated included countries are all Malaysia. Identified recruitment is excluded using the employment category and explicit recruitment terms in retained names or creative text. These classifications use saved metadata; snapshots do not establish that targeting, destination, offers or text were unchanged historically.

Private account-to-entity associations are resolved at their documented identity date. Documented common ownership is handled internally using the approved mapping. These associations are not proof of historical management dates. Real names, associations, the number of client businesses and individual-client results are not included in this supplement.

## Matching and primary ranking

A comparison is an unordered pair of different ads within the same account, ad set, calendar week, objective, returned attribution setting and destination. Only calendar dates with positive impressions for both ads enter either side. Each ad must have at least three shared dates, RM30 spend and 1,000 impressions on those dates. These are exposure rules, not precision guarantees. No minimum conversation count is imposed on the primary analysis.

The inspected implementation requires matching saved `date_start` and `date_stop` labels. It retains `date_start` unchanged for endpoint filtering and shared-date matching. Weekly grouping parses the date label and subtracts its weekday offset to identify Monday. No analytical timezone conversion is applied; this does not establish the provider's underlying timezone.

The same pair may recur, and different pairs may share an ad, ad set or owner. Pair/week counts are not independent trials or counts of businesses. The earliest-pair diagnostic keeps the first rankable week per distinct pair across both periods; it does not remove shared-ad or owner dependence.

## Measures and missing records

- **Link CTR (%)** = 100 × summed `inline_link_clicks` / summed impressions. Daily percentages are not averaged.
- **CPM (RM)** = 1,000 × summed spend / summed impressions.
- **Reported conversations** = summed returned `value` for `onsite_conversion.messaging_conversation_started_7d` only.
- **Cost per reported conversation (RM)** = summed spend / reported conversations, with a positive denominator.

For a structurally returned `actions` list, an absent named action contributes zero recorded events under the analysis convention. A missing or non-list field is unknown and excluded from the primary calculation. This convention is not evidence of no real conversations or an independently verified platform zero-handling rule. Duplicated occurrences of the named action are rejected.

Both-zero comparisons remain in the selection flow but are not directionally rankable. A side with zero recorded conversations has no finite cost; it is not assigned RM0. Exact CTR and cost ties are distinguished. Rankings use unrounded cross-products. Let H be the higher-CTR ad and L its alternative. The sign of `spend_H × conversations_L − spend_L × conversations_H` determines the recorded cost order whenever it is rankable.

## Exploratory magnitude and one-unit arithmetic checks

For positive conversation totals on both sides, the finite H/L cost ratio is `(spend_H / conversations_H) / (spend_L / conversations_L)`. Median and quartiles use linear interpolation at `(n−1)p`. Zero-event cases are not inserted as zero or infinity into that distribution.

For factors 1.10, 1.25, 1.50 and 2.00, the bands count H costing at least the factor times L, L costing at least the factor times H, and finite ratios strictly between reciprocal cutoffs. Equality belongs to an outer band. Exact rational cutoffs preserve symmetry. These are analyst-selected descriptive thresholds, not established practical or statistical significance rules.

The one-unit check holds spend, impressions and CTR orientation fixed. It adds or removes one recorded conversation unit on one side at a time, permitting removal only where that side has at least one unit. It records reversal, tie, both-zero and loss of the original strict order. Each comparison enters the union at most once. This is a deterministic arithmetic stress check, not a confidence interval, event-error rate or probability of future reversal.

## Equal-pair diagnostic (added 6 November 2026)

For each distinct pair p, let a_p be its agreeing rankable weeks and n_p its total rankable weeks. The diagnostic is the arithmetic mean of a_p/n_p across the 30 represented pairs, using exact `Fraction` arithmetic and one rounding to one decimal: **59.5%**. It differs from the primary ratio sum(a_p)/sum(n_p), which gives more influence to repeatedly observed pairs, and from retaining only each pair's earliest rankable week. The actual pair counts and exact fraction are not supplied publicly.

## Analyses added at the final stage (8 November 2026)

Written into a specification on 8 November 2026 after preliminary versions had been run; the code was then frozen before the final run. All are post hoc and exploratory. Results are in [v08-results.json](v08-results.json); the code is [v08_analysis.py](v08_analysis.py), with synthetic checks in [test_v08.py](test_v08.py).

**Clustered intervals.** Percentile bootstrap, 20,000 replicates, `random.Random(20261108)`. Whole clusters are resampled with replacement and pooled agreement is recomputed. Clusters are distinct ad pairs (account plus unordered ad identifiers) or owner groups. For the earliest-per-pair view, the owner-clustered bootstrap is reported with an exact Clopper–Pearson interval and two-sided binomial test that treat the 30 pairs as independent. The original protocol forbade a binomial interval on the dependent pair/week count; these intervals are a labelled deviation, not population error margins. With few owner groups, percentile bootstrap intervals are likely to be too narrow.

**CTR-gap split.** CTR ratio = (clicks_H ÷ impressions_H) ÷ (clicks_L ÷ impressions_L), exact rationals, always above 1. Lead cut-point 1.5 (comparisons with a ratio of exactly 1.5 join the upper group). The reviewer's first run used three bands with boundaries at 1.2 and 1.5, round numbers set before any CTR-gap result was seen; making 1.5 the single lead cut-point, the median split, quartile bands and cut-points 1.3, 1.4, 1.75 and 2.0 all came afterwards. The difference in agreement (upper minus lower group) gets pair- and owner-clustered intervals with the same bootstrap settings. A leave-one-owner-out check reports only whether the direction held.

**One-unit robustness partition.** Agreement among comparisons that no admissible one-unit scenario reverses, and among those that one does. This conditions on the outcome and is descriptive only.

**Arithmetic benchmark (added after a separate review of the final-stage analyses).** By the identity cost ratio H/L = Q ÷ CTR ratio, with Q = (CPM_H/CPM_L) × (R_L/R_H) and R = reported conversations per link click, H is cheaper exactly when Q is below the CTR ratio. The script asserts this identity exactly for all 58 finite comparisons. The benchmark keeps each comparison's CTR ratio and shuffles Q across comparisons (within each period; and separately across both periods), 20,000 times, `random.Random(20261109)` and `20261110`. It reports the mean agreement in each gap group, the mean gap, and the share of shuffles whose gap is at least the observed one. Results are in [v08-arithmetic-null.json](v08-arithmetic-null.json); code in [v08_arithmetic_null.py](v08_arithmetic_null.py). Shuffling treats dependent comparisons as interchangeable, so this is a benchmark, not a formal test.

**Second calculation path.** A separate script, not importing the analysis code, re-enumerates every candidate pair from the retained eligible daily records under the stated grouping, shared-date and exposure rules, recovers exactly the 63 qualifying comparisons and their dates, rebuilds each side's totals, and reproduces the final-stage counts in the manuscript's Sections 4.4–4.6, including the distinct-pair counts. It is part of the restricted study record because it reads the daily records. The other rows of Tables 2 and 4 come from earlier retained results with their own calculation checks.

**Protocol deviations, labelled.** The original protocol ruled out a binomial interval on the pair/week count (the clustered intervals are a deviation) and asked for a leave-one-owner-out agreement range. At the author's direction that range is not published; this is also a deviation.

## Other aggregate cells

Sensitivity, decomposition, follow-up and longer-window cells reported in the note are copied from the retained result files into [restored-cells.json](restored-cells.json). The number of client businesses, their identities, per-owner results and leave-one-owner-out ranges remain restricted.

## Sensitivity and temporal methods

The retained design includes lower/higher exposure floors, outcome-volume restrictions, an explicit-event-every-shared-date check, identical nonempty saved text, an update-predating-window check and earliest-pair restrictions. They change the selected population or assess snapshot/reporting limitations; none replaces the primary result.

Following-week checks require the immediately next calendar week and the same grouping dimensions and shared-date exposure floors. They report sequential eligibility attrition rather than treating continued delivery as random. The combined-interval extension permits a source-period boundary transition only when the complete matching dimensions agree. A deterministic nonoverlap check reserves each selected pair's training/test weeks before screening follow-up availability and does not replace failed candidates. These checks do not recover outcomes for stopped ads or establish a superior selection policy.

Longer-window checks use complete two- and four-week blocks anchored to Monday 5 January 2026. Floors scale to six shared dates/RM60/2,000 impressions per side for two weeks and twelve dates/RM120/4,000 for four weeks. Incomplete final blocks are excluded by calendar. Different included populations prevent interpreting changing agreement as the causal effect of waiting.

## Test plan for later data

[ctr-gap-test-protocol.md](ctr-gap-test-protocol.md) fixes, before any data dated after 25 October 2026 was read, how the CTR-gap pattern will be tested on 26 October–20 December 2026: data rules, a sample-size rule decided on exposure counts alone, the estimand, a pair-clustered bootstrap with a fixed seed, and a verdict rule. Its verdict rule was replaced the same day, before any test data was read, after a review found the first version's verdicts could overlap, treated a negative estimate as a contradiction regardless of uncertainty, and let undefined bootstrap draws vanish (see the plan's "Change 1"). Its SHA-256 hashes, before and after the change, are held in the study record; it has not been registered externally.

## Reporting reconciliation and limits

For retained ad/calendar-week cohorts, the reconciliation compared archived daily totals, freshly returned daily totals and freshly requested whole-week totals. It uses full calendar weeks, separately from shared-date ranking. Equality among those examined sources establishes internal reporting consistency, not independent inbox activity, true zeros or full measurement completeness.

The public files contain selected pooled outputs, restored aggregate cells and calculation code. Restricted raw records, source manifests, request jobs, identity keys and ownership diagnostics are not supplied. Full raw-data reproducibility, national inference and causal claims are not established by this supplement.
