Statsig

An unexpected perk of a working in person? Cookie contests! 🍪 Today Morgan organized an A/B/n taste test with 9 variants - check out her description (and Siddharth's reaction) below ⬇ We have some good statisticians working on the analysis... so stay tuned for the results! 📊

If you're in SF, keep an eye out for Statsig's latest billboard campaign... Cat Lee, Jessie Ong, Elizabeth George and Brock Lumbard put together another great campaign. Here's my personal favorite (which is extremely visible on the 101) Lots more promotion coming next week! 🎉

Feature Fridays Week 29: Experimenters rejoice! Today, we’re introducing new Meta-analysis views and the Experiment Knowledge Base 🤓 Throughout the summer we’ve been rolling out a series of new views that enable meta-analysis across your corpus of experiments. 🏏 Today, we’re introducing the “Batting Averages” view, which lets you see how easy or hard it is to move a metric. - This is helpful for setting reasonable goals and sniff-testing claims about the next experiment moving a metric by x%. - You can filter by a set of shipped experiments and see how many moved a metric by 1% versus 10%. As with other meta-analysis views, you can filter by team, tag, or whether the results were statistically significant. 📊 We’ve also updated the “Experiment Timeline” view to include summary stats such as "how many experiments shipped with control" so you no longer have to manually tally stats. 📜 We're also introducing the Experiment Knowledge Base (KB), a searchable repository of experiment learnings across teams. - While meta-analysis views provide a structured way to glean insights across experiments, the KB is perfect for free-text search. - It’s a great way for new teammates to explore and find experiments a team ran or where a specific topic was mentioned.  - This promotes an experimentation culture, provides more context to stakeholders, and drives more ideas to test! Screenshots attached below, but you can dive right into Statsig to start driving more learnings and insights today! #experimentation #datascience

Did you follow us during Product Analytics Week? 🗓 We announced a new feature every day last week to make our product analytics capabilities stronger than ever. Here’s a summary of what we shipped: 1. Integrated workflows that make product analytics work seamlessly across experimentation, feature flags, user session streams, session replays, and the rest of the Statsig platform. 2. Segmentation & cohort analysis that gives you a clear picture of how specific segments of users engage with your product. 3. User journeys to help you understand the different paths users take in your product, including what they do before and after any event of interest. 4. Revamped funnels that provide more flexibility in defining conversion flows and offer deeper insights into every funnel step. 5. And finally, Product Analytics is now available directly on top of YOUR warehouse allowing you to instantly drill down into metrics. 📊 Catch up on everything we shipped here and jump into the Statsig console to see these features in action right away: https://lnkd.in/g8WwPZ5n #analytics

We’re committed to advancing the Thumbtack product through data-driven experimentation. Richard Domurat, Director, Economics, shares how our scientific approach leverages in-product experiments and advanced analytical techniques to continuously improve user experience. Learn more about how our Engineering teams are driving innovation here: https://lnkd.in/gDgyHXVD. #Experimentation #Engineering #FutureOfHomeCare

Over the last few months, the design team has delivered some meaningful updates, and I’m thrilled to talk about our refreshed design system, Pluto. It’s all about simplifying and enhancing the user experience, making it easier for our customers to use Statsig and drive value for their businesses every day. We had a solid design system that the team has faithfully used for the past three years. However, as the company has scaled, we’ve faced challenges in making our product more cohesive and approachable for a broad range of users. We needed to build a scalable foundation that supports both our existing products and those yet to come. This UI refresh marks the beginning of deeper improvements to our Pluto Design System, so stay tuned for more exciting details and upcoming changes. We’ve been gradually rolling out these updates in phases since August. Exciting times ahead! 🚀 Kudos to the talented team of designers Soo Hyun (Andy) Moon, Daniel Kim, Cynthia Xin, and our world-class engs Pierre Estephan, Brian Do, Nick Jiang, Alex Coleman, and many more!

Statsig Life 📸 by Timothy Chan #statsigsnapshots

Product Analytics Week Day 5: We’re wrapping up the week with a big announcement 👀 We’re now bringing Product Analytics directly into your warehouse: Snowflake, Databricks, Google Cloud BigQuery and Amazon Web Services (AWS) Redshift 📊 Product Manager vineeth madhusudanan explains the value propositions of Warehouse Native Product Analytics: - Stay in control over your data without needing to egress it to a third-party analytics tool. - Combine business metrics like support requests and refunds with your clickstream data to gain deeper insights without ever leaving the platform. - Maintain a single source of truth without worrying about data duplication/drift.  Warehouse Native Product Analytics works seamlessly with our experimentation and powers a flywheel: xfn teams are empowered to self-serve, identify drop-offs, and generate more hypotheses to test. Shoutout to Joe Zeng, Daniel West, Craig Sexauer, Liz Obermaier, Nicole Smith, and our customers at Monzo Bank and Vista, who quickly resonated with the offering and continue to help us improve the product! --- This is available in broad beta on Statsig Warehouse Native and is free for the rest of 2024. Jump right in — instantly drill down into metrics, filter and slice by any property including experiment groups, and analyze funnels to drive more conversions 🚀 #analytics #data #warehouse

Product Analytics Week Day 4: Statsig ❤️ Funnels Today, we’re announcing a spate of new features that make Funnels more flexible and powerful, enabling you to precisely identify where users drop off in key product flows. Product Designer Soo Hyun (Andy) Moon shares how this is a very personal feature — something he uses daily — as he showcases the slick new UI and enhancements we’ve shipped. These features provide richer insights, more configuration options, and make it easier for every stakeholder to quickly understand the data at a glance 📊 A couple of highlights: - Table overhaul and new data columns that give you a detailed view of key metrics like conversion rates and average time at each step of your funnel, with more options to view and rename funnel steps. - More flexibility and control when constructing a funnel with features like first-time filters to better understand users' initial experiences, and the ability to combine multiple related events (such as adding an item to a wishlist or cart) into a single funnel step. As with everything we’re announcing this week, funnels are tightly integrated with the rest of the Statsig platform, unlocking even more value from your data: - Group-by capabilities: Break down your funnel by any event/user property, feature gates, and experiments to see how different factors impact conversion. - Conversions in context: Funnels integrate seamlessly with session streams and session replays, offering event-level insights into a user's journey through the funnel. Plus, you can watch the session unfold to fully close the loop! Shoutout to Akin Olugbade, Nicole Smith, and Brian Barak who worked on these capabilities, and to our customers Ahmed Muneeb, Joakim Aglo, Nick Carneiro, and Kevin Anderson who have given us great feedback from the beginning to make our Product Analytics and all-in-one data platform better 🙌 Sign up and construct your first funnel today! #analytics

We all encountered the same frustration: A/B tests show promising results, but once the feature is launched, the anticipated uplift just doesn’t materialize. Here are six reasons why it happens: 1. Human bias in analysis and interpretation 2. False positives 3. Sequential testing and overstated effect sizes 4. Novelty effect and user behavior 5. External validity and real-world factors 6. Limited exposure in testing In my recent post published on Statsig blog, I discussed these key reasons behind this discrepancy and offered insights on how to manage expectations effectively. Read more here or DM me to learn how Bell Statistics can help your testing process. https://lnkd.in/dzattSWK