As a manager, you are honest and you have integrity. Of that, I have no doubt. That, however, is not the issue. The challenge you have to deal with is how to manage the inconsistencies that make you appear to be dishonest.
If you manage people, then, every day, you have to display – consistently – a searing honesty and a deep level of integrity. This is something you have consciously to try to do, and you have to practise doing it to become good at displaying it. 
In my new book, “Charismatic Leadership”, I look at how authenticity is a key trait of truly charismatic people. 
Leaders have to focus on building trust, because trust is possibly the single most important element of building effective teams. However, you cannot build trust if people don’t trust you. To do that, people need to know who you are. They need to know that you know what you’re talking about. And they need to believe that you have their best interests – and an honourable cause – at heart. They trust you for your character, your competence and your integrity – and these must be authentic. 
A chain reaction of mistrust.
They don’t trust you when you remain aloof, hard to read and mysterious. In this case, they will always wonder what your agenda really is and whether you’re being completely truthful. More than anything else, this will undermine their willingness to give maximum effort, and they will be inclined always to keep one eye on you and one eye on their work, thus dissolving their focus and effectiveness. Worse still, that wariness will translate itself into a lack of willingness to trust colleagues. Once you start a chain reaction of mistrust, you also trigger a lack of respect. A lack of respect translates into a lack of respect for teammates, and for customers. And all of that inevitably leads to disastrous results. When you build trust, you build strong relationships, which lead to better business. The business benefits of trust are enormous. 
Without integrity, leaders will soon fail. If, for example, you keep choosing what’s convenient over what’s right, your team will quickly lose faith. As leaders, we always have to make decisions that, in some ways, define who we are. Without a strong set of values to help guide us, our decision-making will soon become inconsistent at best, and potentially confusing and damaging at worst. 
Under close scrutiny, your integrity can be found wanting.
As leaders we are being scrutinised every single moment, and everything we say, everything we do and every decision we make will be picked over by our teams, who will be quick to interpret those actions through their own set of filters. To avoid being thought of as potentially dishonest, or lacking in principles, it becomes necessary to be radically transparent with people – absolutely straight with them about what decisions you’re making and why. It is then especially important to ensure there is no gap between your words and actions. This is an area, from my experience, that most leaders are simply unable to see as a weakness. Followers often pay more heed to what you do than what you say.
If you say bullying is unacceptable but do nothing about the super- salesman who is domineering and a bully, the signal you are sending is very different to your words. Followers will take their cue from how you behave, not your words, and the damage is done.
 This gap between your actions and your words is potentially one of the most toxic to your leadership effectiveness. Your followers will be watching for consistency – both in your language and in your behaviour. If you even slightly change your story, or treat one member of the team differently to the others, this will send danger signals that you are not to be trusted. You have to be acutely aware of being consistent or explain fully why not when you behave in an inconsistent manner. 
I have no doubt that you are honest, sincere and principled. To convince your followers, you need to practise the behaviours that demonstrate your honesty and integrity every day. Without fail.