For every founder their product is their baby!
But that as you rightly said can work against you and is a big problem.
I too am guilty as any passionate founder, but from my experience have come to realize that building a product that you love is not the right approach. Buil something that you are passionate about, but build it in the way that users want.
I feel that's the secret sauce!