Recommendation
SHIP
Hypothesis
Hiding the calorie filter buttons will increase the added_to_cart metric for users evaluating meals without calorie-based filtering.
Treatment (variant)
Menu without calorie filter
https://www.blueapron.com (same URL, variant applied via JS) Result
Hiding the calorie filter lifts add-to-cart rate by +18.8% (48.5% → 57.6%, p<0.01) with no measurable impact on checkout reach (~12% in both variants). Menu visitation was universal.
Wins
What treatment did better
- Per-serving price scan as the real decision moment+12%
- Solo and couple shoppers rejecting the Family Style bundle+11%
Losses
What treatment did worse
- Time-pressed users gravitating to 15-minute and Assemble & Bake meals-5%
Open questions
Worth investigating next
- Visitors bypassing the homepage hero for the Menu link
- Weeknight cooks rejecting the 65-minute bundle prep time
Metrics
Relative lift over Control · 95% CI · two-proportion z-test
Group byFilter
Metric
Control
Treatment
-50%0+42%
Δ (95% CI)
p
visited_menu
100.00%
99.80%
-0.2%
[-0.6%, +0.2%]
0.317
added_to_cart
48.48%
57.58%
+18.8%
[+6.0%, +31.5%]
0.004
visited_checkout
12.32%
11.52%
-6.6%
[-39.3%, +26.2%]
0.695
Wins
What worked — supports shipping.
Losses
What broke — fix or rethink.
Open questions
Worth investigating — fuel for the next test.
Session replays
989 sessions
069691c4control✓ visited_menu✓ added_to_cart✗ visited_checkout
AI
Simulation Q&A
Ask about the overall simulation or this specific session. Watching session 069691c4 (control).
Try: “What's the strongest evidence to ship?” or “Why did added_to_cart move +18.8%?”
Loading replay...
Transcript · 30 entries
0:00I’m just trying to get the quick read here, not turn this into homework. If I can see what kinds of meals they’re actually showing and whether the pricing feels reasonable without getting funneled into some giant sign-up thing, that’s enough for me. Blue Apron has been around forever, so part of me is curious if it still feels a little dated or if they’ve made it easier to browse like a normal person.
0:00That privacy box is parked right in the middle of the food, and I’m not in the mood to read corporate housekeeping before I even see dinner. I just want it out of the way so I can tell if they’re showing me real meals or just another polished homepage with a giant “trust us” energy.
0:00click — /
0:00“Meet the new Blue Apron” tells me nothing, and “Shop now” on a meal kit site usually means “start giving us your email.” The “Menu” link up top is the one that feels like it might actually show me the food first, which is really all I need right now because if the meals don’t look appealing there’s no point caring what the weekly total is.
0:00click — /
0:00This is already better because at least I’m looking at an actual menu instead of a giant promise, but they’re asking for my zip code right away and I’m not doing that just to browse. I can see “Meal Kits,” “Dish by Blue Apron,” and “Assemble & Bake,” so I want to get down to the actual dishes and see whether there’s enough variety to make this feel fun instead of repetitive.
0:00scroll — /menu
0:00There we go, at least I’m seeing numbers now. Forty-five ninety-nine for four and sixty-eight ninety-nine for six is useful, but those family bundles aren’t really my life most weeks unless the kids happen to be in town, and I’m more interested in the regular meal kits anyway because that’s where the interesting flavors usually live.
0:00scroll — /menu
0:00This is the first useful screen because I can actually compare without squinting. Eleven ninety-nine, twelve ninety-nine, thirteen ninety-nine per serving and the times are short, which is appealing on a weeknight, but this “Baked by Melissa” section feels a little more polished-prepared than what I’m in the mood for. I want the actual meal kits tab because that’s probably where the more creative recipes and maybe a vegetarian option are hiding.
0:00click — /menu
0:00This is more my speed. I’m seeing things like shrimp stir-fry at eleven ninety-nine a serving and that spring pesto pasta at twelve ninety-nine, so the price range seems kind of in the expected meal-kit zone, not cheap exactly but not shocking either. What I still need is whether they make it easy to narrow this down, because if I’m craving lighter or meatless meals some weeks and the filter is annoying, I know that’ll get old fast.
0:00click — /menu
0:00This is exactly the kind of thing I wanted to see, because if the filter menu is this straightforward then at least they understand how people actually browse dinner. I can see “Vegetarian” sitting right there under protein, so I’m just going to tap that and see whether the site responds cleanly instead of making me feel like I’m sorting through a spreadsheet.
0:00click — /menu
0:00The button lit up, good. That’s such a small thing but I notice it because half these sites make you wonder whether anything happened. If they can show me a handful of vegetarian meals with prices in one pass, that’s probably enough for me to decide whether this would actually fit into my week or if it’s more of a once-in-a-while splurge.
0:00click — /menu
0:00Interesting, the vegetarian prices drop a bit, which I like immediately. Eight ninety-nine for the veggie enchilada skillet and nine ninety-nine for the miso glazed tofu bowls is more comfortable than the chicken and shrimp dishes I just saw, even if these two are marked sold out. I want to nudge up just a little and see whether there are a couple more options, because two sold-out meals by themselves don’t quite tell me how solid the menu really is.
0:00scroll — /menu
0:00“Creamy Roasted Red Pepper Pasta” is exactly the kind of weeknight thing I’d actually talk myself into ordering, because it feels comforting without being boring, and nine ninety-nine a serving is a lot easier to swallow than sixteen forty-nine for that burger. I still want one closer look at a real dish page in case it tells me whether this is serving-two territory with decent leftovers or something annoyingly vague.
0:00click — /menu
0:00This is actually helpful because now I can see the structure. Two servings or four servings makes sense for me since I’d either have dinner plus leftovers or something to share if someone’s over, and nine ninety-nine a serving reads very differently once I translate it into an actual meal total in my head. I’d rather see the cart do the math than sit here multiplying while my tea gets cold.
0:00click — /menu
0:00It’s making me explicitly choose “2 servings” first, which is slightly fussy but fine. I’d rather have that than get surprised by a four-serving default and some weird total, because for me the whole question is whether a simple pasta dinner lands closer to twenty dollars or forty.
0:00click — /menu
0:00Now that I can see “Add — $19.98,” the math feels real instead of abstract. For one pasta meal with leftovers, twenty bucks is not outrageous to me at all, especially compared with takeout, but I know meal kits love to look reasonable until some extra fee appears one screen later. I’m curious enough to let the cart show its hand once, and if it immediately tries to shove me into sign-up, that tells me something too.
0:00click — /menu
0:00The cart bubble in the corner still just says nineteen ninety-eight, which is promising because at least nothing inflated instantly. I’m willing to click that once because if there’s a shipping charge or a weird minimum, this is where it should show up, and if it just dumps me into account creation then I’ve basically learned the limit of what they’ll reveal without the whole commitment dance.
0:00click — /menu
0:00Mmm, no, that $11.49 shipping on top of a $19.98 two-serving pasta is exactly the kind of total that makes me think I can just swing by the store and cook something fun myself tonight.
OutcomesVariantvisited_menuadded_to_cartvisited_checkout
AttributesAgeSexIncomeHousehold
SessionVariantPersonaOutcomesDuration
069691c4controlElena Masterson✓ visited_menu✓ added_to_cart✗ visited_checkout3:55
902ca335treatmentEvelyn Parker✓ visited_menu✓ added_to_cart✗ visited_checkout3:01
5eac55f1treatmentEmily Carter✓ visited_menu✗ added_to_cart✗ visited_checkout3:40
35326e0dcontrolAlex Chen✓ visited_menu✓ added_to_cart✓ visited_checkout3:52
0329ec85controlDerek Lawson✓ visited_menu✗ added_to_cart✗ visited_checkout2:36
4418674btreatmentNathan Chen✓ visited_menu✓ added_to_cart✗ visited_checkout4:39
8ab27574controlLisa Morgan✓ visited_menu✗ added_to_cart✗ visited_checkout3:19
6c088003treatmentMichelle Carter✓ visited_menu✓ added_to_cart✓ visited_checkout3:12
7b0fbcaatreatmentMartin Caldwell✓ visited_menu✗ added_to_cart✗ visited_checkout5:33
b31c3206controlLaura Bennett✓ visited_menu✗ added_to_cart✗ visited_checkout2:04
1 / 99