How do you manage experiments in your company?
Borja DR
11 replies
We’re experimenting more and more with various aspects, such as different user onboardings, UI colors, landing pages, etc. However, we sometimes encounter situations where different teams run experiments that overlap each other leading to inaccurate results. Have you faced similar challenges, and how did you address them?
Replies
Owen Michael Donovan@owenmichaeldonovan
We typically use a hybrid approach - setting up structured A/B tests for key features, combined with more rapid iterative experiments based on user feedback and analytics for UI/UX tweaks. Having a solid experimentation framework is key, but being flexible to pursue interesting opportunities that emerge is also important. Tools like Optimizely help a lot with managing it all.
Share
I think experimenting and A/B testing are super important for optimizing user experience. For UI/UX experiments, I like using tools like Optimizely or Google Optimize to easily set up multivariate tests on key pages. For backend or infrastructure changes, we do limited rollouts to a small % of traffic first before full release. The key is having good metrics to track the impact. But you're right, every experiment is a chance to learn something new that opens doors for the next opportunity to improve!
@jasperelliotsinclair Thanks for sharing! I agree that experiments are very important. How many experiments would you say you run per month?
We usually A/B test key experiences like onboarding flows, landing pages, and core features. It's a mix of dev experiments pushed to staging, UI/UX tweaks mocked up by the design team, and copy variations from marketing. The key is having clear success metrics and giving each test enough traffic to reach stat sig. Also building a culture of experimentation so teams are always thinking about the next improvement to try. Amplitude and LaunchDarkly help a ton with experiment management and analysis.
@zacharywilliamking That's very interesting, thanks for sharing! How many experiments would you say that your company runs per month?
It varies from project to project. UI/UX and landing pages are very important for user experience. In my opinion every experience opens the door for another opportunity.
@amelia_hansley that right! How frequently do you run experiments in your company?
@borja_diazroig Well, for a company website, My main focus is Content Writing, Keyword Research, and UI/UX. Different experiments goes hand in hand with time. All we need to know is to understand the problems and pain points of our audience.
Sometimes it is difficult to align everyone so we try to use excel spreadsheets to keep the dates, names, and impact of experiments (sort of a Gant Chart of things). It is helpful but it feels very clunky at times.