How to test your AI product?

As AI has become a significant part of our lives, and becoming an integral part of software products, question comes to mind - how do you test it? Or, maybe, the more important question is - can you test it?? Well, as some of our business partners and clients have come to the point where these questions require answers, I’ve taken the liberty to dive into it. Lets, first of all, set the ground here: AI is not the only component of your product. Its probably the most unique one, but it is surrounded by some user interface (graphic, API or SDK), it probably saves some data to some kind of a DB. All of these components require testing, otherwise your product will lose its value. Now that we’ve established that, how do you test the AI component? Well, conceptually, you can’t really know if the machine is right or wrong - that is its added value, it provides something the human brain cannot. However, you can test the AI component, by using itself as a reference. In software testing, we usually isolate one of the variables and test the code accordingly. So, lets say your AI component is trained by a certain data set. And you inject some kind of input values to retrieve a result. You can create a matrix, combining different input values and different training data sets, isolating each one at a time, and compare the results. The combination of the results, crossing them and comparing them, will provide you great clarity of your AI’s quality. There’s still much to be learned and investigated here. We are at the beginning of the road, but its an exciting one. We will let you know how it goes - follow me for future updates. Please share your thoughts about this concept, I’d really like to get some more ideas from the crowd.