Very cool approach to ML testing. I like how you track against commits and help define goals as you define the pipeline. One question - how do define the "root cause" that you mention when solving failed goals?
Great integrated product for giving visibility into ML models, highly recommend for anyone looking for a way to benchmark, evaluate, and iterate on their models (which everyone should!)
This is awesome! Having a great debugging workspace on par with software engineering debugging has always been a pain point to me when working on finance data and autonomous driving. What are some of the use cases you enable today?
Hey @lawlm thanks! Companies and orgs in the finance space stand to benefit greatly from Openlayer. Many of the people we work with use us to help de-bias and improve the accuracy of models that predict whether to give loans, or whether a transaction is fraudulent, for example. These use cases are especially important because they have a large impact on people's lives, so it's critical to invest in evaluating these models.
Optimizing an ML model at scale requires a bunch of different tools and lots of work by the engineers + data scientists. Love that Openlayer can do all of this for a company (detect errors, suggest new optimizations, etc.), definitely a game-changer for ML teams! 👏🏾
I love OpenLayer because it allows not just engineers, but PMs, analysts, and managers to participate in the ML development process. Finally, a way to catch errors before the product gets into the hands of users!
Congratulations!!, Do you provide dynamic editing of ML pipeline based on observability results? that is: if error rate is high than the expected (or via another rule) dynamically swap the deployed ML model with a different version.
Another question, Is your observability integrates with the open source tooling such as https://opentelemetry.io/, https://prometheus.io/, https://www.jaegertracing.io/
Thanks, Good Luck!
Openlayer is a great tool with a ton of potential for helping ML teams! The team does an amazing job of implementing feedback and feature requests effectively and efficiently, and we can't wait to see the platform grow!
Very excited to see this launch - I've watched Vikas, Gabe, Rish, Erica and the team build this over the last couple years. They've been through many iterations to nail down a thoughtful, beautiful product and workflow. We're big fans of Openlayer at Dream3D!
Congrats to the Openlayer team on the launch! It's inspiring to see this innovative solution bridging the gap in ML testing. Excited to see its impact.
We've been using Openlayer for the past couple of months, and it has been a valuable asset to our team. The platform's timeline feature is excellent for tracking progress, and collaborating with the team is effortless. The Openlayer team is highly responsive to feedback and feature requests, making it a top-notch platform for gaining insights into machine-learning models
