If I have to say “I’m not saying,” then I’m probably saying what I say I’m not saying. And people know it.
What if we cut down on the negations and the qualifications and tried to speak only in assertions? We can assert negative things positively, like “I don’t like this idea,” and that is probably more useful than “I’m not saying it’s a bad idea, but . . . “
It feels like the positive assertion of a negative thought is kinder than the non-assertion, or even the half assertion, of it.
YoRocko! 
It’s funny to read about you saying this, Rocky. It isn’t a saying (sorry) I catch you using much at all. As for myself, I catch myself saying “That’s what I think” or “That’s just for me” after I assert something. I could stand to be less wishy-washy with facts and times and just assert. That’s what I think. (Oops.)