Keep an experiment journal for better store decisions

After reviewing a store change, the team needs to decide what comes next. Keeping the original expectation, conditions, and observations together can make that decision easier to examine. We suggest a blank journal for one catalog or interface experiment. It contains no completed test, numerical outcome, or customer case study.

1. Separate the problem from the proposed change

Describe the question to investigate and the evidence behind it. If there is no supporting observation yet, label the problem as an assumption. Record the proposed change separately: which part of the page or process would be altered? This lets the team reconsider the implementation while retaining the original question.

Define expected behavior and how it could be observed. For example, a study might explore whether guidance about product variants helps someone complete a selection task. That is a possible research direction, with no claim that the guidance has been implemented or proved useful. Keep expectations out of the results field.

2. Agree on conditions before starting

Record the change version, review period, and available data. Decide what the result will be compared with and which differences could limit the comparison. For a review with people, prepare the task and observation method. A quantitative comparison needs a separate plan for adequate evidence.

Agree on stopping conditions, technical failures that require a pause, and the decision owner. There is no single duration or data volume suitable for every question. If the team cannot justify its criterion yet, keep that as an open planning issue before launch.

3. Use a blank journal

• Problem and supporting material: [complete before starting].
• Proposed change and version: [complete before starting].
• Expected behavior: [complete before starting].
• Review and comparison method: [complete before starting].
• Period, conditions, and evidence requirements: [complete before starting].
• Stopping conditions and owner: [complete before starting].
• Actual observations: [complete after the review].
• Limitations and concurrent changes: [complete after the review].
• Decision, rationale, and next action: [complete after analysis].

Keep a history of revisions. If a criterion changes after the start, record when and why. Otherwise, a later reader may be unable to connect the conclusion to the question the team originally intended to answer.

4. Review limitations before deciding

First check whether measurement worked and the conditions remained as planned. Missing events, a different page version, or a simultaneous offer change may restrict the comparison. Record exactly what remains unknown and what needs fixing before another attempt.

Evidence may be insufficient to support or reject a hypothesis. Preserve that as a distinct outcome. A before-and-after comparison without control of conditions does not establish why a measure changed. The decision should reflect the evidence available and retain the limitations explicitly.

WGP develops and supports online stores. Frontbox includes a responsive storefront that also works on phones, plus development and support, and connects to a 1C-Bitrix store. A mobile app is a separate paid add-on. This journal can support discussions about future changes; it does not imply a built-in experimentation feature.

5. A checklist for the next action

• Check the completeness of observations and comparison conditions.
• Separate measurement failure from the hypothesis outcome.
• Choose an action and record its rationale.
• If continuing, define the conditions for the next review.
• Name the person responsible for that review.

Start with one hypothesis and complete only the preparation fields. Leave observations and conclusions blank until real evidence is available.