Subscribe
Sign in
Deepchecks Monitoring
p/deepchecks-monitoringOpen Source Monitoring for AI & ML
Start new thread
Shir Chorev

Deepchecks Testing Package - Open source tool for testing & validating your model & data

by•

Find problems in your ML models and data before it’s too late!
With just a few lines of code, Deepchecks creates elaborate reports about your data’s integrity, distributions, and your model’s performance.
Check it out: https://github.com/deepchecks/deepchecks

Add a comment

Replies

Best
Shir Chorev
Maker
Hi Product Hunt! 🙋‍♀️ I am Shir, co-founder and CTO of Deepchecks. Excited to share a bit about the journey of developing and launching an open-source package for testing machine learning models! At Deepchecks we’re seeing LOTS of models in production. While monitoring them is a must, we noticed that the tools for something much more basic was missing: How should these models be tested before they’re even deployed? 🤔 Like every startup - we started with experimentation. We named the first prototype the “Preemptive report”. Despite the REALLY compelling name 😅 the first people who saw it responded with: “How come this doesn’t exist yet??” At the same time, we realized that testing for problems with your data, model, and ML process, is relevant also much earlier in the research phase. The smiles on the faces of Data Scientists when running our package on a new batch of data, told us that this understanding is correct... As we are a company of tech-savvy engineers, it was clear that the way to go for a dev-tool used during research, was to build it open source, and we started working on the package towards its planned release. After iterations with hundreds of Data Scientists from the amazing community (to which we owe many thanks 🙏) and receiving great feedback and suggestions (for usability, additional ideas for checks, requests for integrations with tools, and many more), we’re happy to announce the deepchecks testing package here on Product Hunt 🚀 Show your support by giving us a star ⭐️ on github: https://www.github.com/deepchecks/deepchecks Our commitment is to keep this project open source, and to continue building a comprehensive ecosystem of tools, for “Continuous Validation” of ML pipelines, from research to production – for Tabular, CV, and NLP Data & Models. More links to jump right in: - Get Started - Join Our Slack Community for more updates and info about ML validation and testing - Open an Issue on Github for feature requests and feedback
Mayank Jain
@shirch this is fantastic. We are in AI and ML space, and seem to be a great fit for startups in this space. Congratulations and best of luck!
philip tannor
Maker
👋 Hi all! 👨‍🦰 I’m Philip Tannor, I’m the CEO and one of the co-founders at Deepchecks. Thank you SO MUCH for the support you all have been giving us since we released this package, it really means the world to Shir, myself, and the rest of the team. 👨‍👩‍👧‍👦 We’ve always had a passion for the community-focused approach, but that’s not how Deepchecks started. After building out a (closed) solution for monitoring ML models in production, we kept looking for a way to provide value to a much wider community. It took us over a year of hard work, but we ended up realizing that we can provide the ML community with the (extensive) open-source package that you see here today while focusing our commercial efforts elsewhere (e.g. where other non-ML-practitioner users are involved, etc.). 🚀 The open-source package was released at the beginning of this year, and in the meantime, the feedback has been incredible. It’s really a privilege to get to work on a package that’s getting so much love from the professional community. If you haven’t yet, give it a try - should only take a few minutes. And can’t wait to get your feedback and your support (both here and on GitHub - https://github.com/deepchecks/deepchecks ). 👂 We’re constantly looking to add more checks & suites, as well as to support more types of data and frameworks. Since our roadmap is based primarily on community feedback, please let us know what you do or don’t like, as well as what you think we’re missing! Shir and I will also be here throughout the day to answer any questions that you may have.
philip tannor
Maker
🙏🙏🙏 Thank you all SO MUCH for the open source love. 🤩 If you like what we're doing, the best ways to support us are by starring our repo (https://github.com/deepchecks/de...) or by leaving feedback via our GitHub issues (such as opening a feature request or a bug report here - https://github.com/deepchecks/de...).
Shir Chorev
Maker
@ptannor Some of our best checks came from user's requests & ideas 😍!! (One simple & lovable example is the IsSingleValue check) And starring the repo is a great way to show your support ⭐️🙏⭐️
Fares
Congrats on the launch @ptannor @shirch !
Shir Chorev
Maker
@fares_aktouf Thanks for the shoutout!
philip tannor
Maker
@shirch @fares_aktouf thanks Fares! Much appreciated... Would love to hear your feedback if/when you have any
Gaurav Goyal
Sounds pretty cool @shirch. Congratulations on the launch. Going to share it with ML team to try for upcoming stuff :)
Shir Chorev
Maker
@gauravgoyal_gg Amazing, looking forward for their feedback! Are you working with Tabular or Computer Vision data?
Tom Ahi Dror
Kudos team Deepchecks! It's a pleasure to see a repo with such great docs (: Just out of curiosity, how is this different than Great Expectations?
Shir Chorev
Maker
@tom_ad Thanks! ❤️ Docs are key for open source adoption 😃 (inviting the readers to check out swimm.io!) About Great Expectations - first of all, its a great tool and we love what they're doing! In short: Deepchecks is focused in everything about testing ML-related projects, and thus our checks (model evaluation, distribution and leakages, and also the data integrity ones) are tailor built for these scenarios. One example - finding similar samples with "Conflicting Labels" is a data integrity check that is relevant only within the ML context. Summing up a few usage-related differences: - GE is meant for Data Engineers, when ML is not always around, when our focus is on Data Scientists that are working on an ML Project - GE works great after you configure it thoroughly, Deepchecks is designed to give a solid out-of-the-box experience, even if this can be improved after configuration - Deepchecks is also built for unstructured data (work in progress)
Clément Delangue
congrats for the launch team!
philip tannor
Maker
@clement_delangue thank you so much, and great to see you here! BTW at Deepchecks we're huge fans of Huggingface, this is a good place to shout out that you can get the packages to work together - https://docs.deepchecks.com/stab.... Currently for computer vision models and hopefully, NLP will be later in the year
Shauli Rozen
Thanks for sharing ! The testing package has been super helpful for my team!
Daniel Tannor
Can confirm whenever I'm validating my models Deepchecks makes me feel warm and fuzzy inside.
Shir Chorev
Maker
@daniel_tannor Always cool to see when brothers have a good / interesting relationships 😅
Almog Baku
Really cool product. I'm following you guys for a while :) good luck w/ the launch
Shir Chorev
Maker
@almogbaku Thanks for the ongoing support!
Arseny Kravchenko
Deepchecks solve a very important problem for the applied ML community - I wish them luck and rapid growth!
philip tannor
Maker
@arseny_info thank you so much! And great to see your work on Kaggle as well, can't wait to hear more detailed feedback
Amit Moran
Awesome work guys! Love it. Good luck :)
Shir Chorev
Maker
@amit_moran Thanks, appreciate it!
Ori Kabeli
That's an awesome product. We used to do such tests manually at our team and recently migrated to using it, and the design of the library is delightful to work with! Congrats!
Shir Chorev
Maker
@ori_kabeli1 Great to hear! Looking forward for hearing more feedback 🙏🤩
Shir Chorev
Maker
@karimsaif Happy to hear you liked it!
Ariel Biller
Hi, mod of r/mlops here. Not affiliated with DeepChecks at all (unless you count my desperate attempts to make them change the name). The work that they have been doing in the OSS space is admirable and they deserve every bit of love they got here! Kudos! Can't wait to feature you guys as OSS of the month!
philip tannor
Maker
@lstmeow haha sorry Ariel, the name Deepchecks is here to stay! And r/mlops rocks - for those of you that don't know it, check it out - https://www.reddit.com/r/mlops/
Erez Simon
Sounds like a must-have solution for ML and DS programs! Can't wait to give it a test drive on my project! Thanks for making this important contribution to our community!👏👏👏
philip tannor
Maker
@erezx thanks a million, and can't wait to hear your feedback!
Aishwarya Srinivasan
This is an amazing tool to integrate data and model validation in the MLOps pipeline! Kudos team 👏
philip tannor
Maker
@aishwarya_srinivasan thanks, it's so great to have your support!
Roy Miara 🕶
Really important product in today's ML toolchain, really nice open source work. Good luck team!
Shir Chorev
Maker
@miararoy Excited that's the way you see it as well, thanks for sharing! 🤓
Paula Marañón
Great product! I have previously used Deepchecks on a tabular project and working with it on an image project now. It is highly recommended.
Shir Chorev
Maker
@paula_maranon Many thanks! looking forward to hearing your feedback
philip tannor
Maker
@paula_maranon glad to hear that! And it was great to see your work on Medium