For the past few months, Aman Dhesi and I have been helping companies build AI pipelines. In the process, we built a Superpipe Studio to solve our own problems and thought it might be useful for others. Superpipe Studio solves three problems we were having while developing pipelines with our partners and customers. 1. Dataset management When we were working with companies we found it very difficult to set a baseline to try and beat, either because they didn’t have labeled data or because the labels were incorrect. Superpipe Studio makes it easy for us to make apples to apples comparisons we can trust, starting with data we can trust. 2. Experiment comparisons Last month we launched superpipe-py, a framework to build, evaluate, and optimize LLM pipelines via experimenting with pipeline parameters (prompts, models, # rag results). Superpipe Studio allows you to compare experiments in a long lived environment instead of just a notebook. 3. Logs and observability Experimentation doesn’t stop once we deploy. We need to know how our pipelines are doing in production and debug any issues. Superpipe Studio gives you an easy interface to dive deep on your live data, understand losses, and add labeled data to your golden sets. Free and open source Superpipe Studio was a tool we built for ourselves to make building pipelines easier. We’re most interested in using Superpipe to build useful AI pipelines so we made Superpipe and Superpipe Studio free and open source for anyone to use or deploy on their own infra. Github: https://github.com/villagecomput... Docs: https://docs.superpipe.ai/studio/