Hide What is Used to Roll in a Public Roll?

Is there any way to only show the result of a dice roll and make it so they can't see what dice and modifier is being rolled when they hover over it?

Unfortunately not without the API, and even with the API you'll get a slightly different 'hover/rollover' tooltip than you get from a 'real' roll, so players will always know if a roll was legitimately made.  

If your intent is to remove the roll information from the tooltip you can surround your rolls with extra [[ and ]]. For example:- [[ [[1d20 + 5]] ]] So instead of the tooltip saying "Rolling 1d20 + 5 = (14) +5" it would say "Rolling 19 = 19" This will remove all information about the roll including things like critical successes and failures so it may not actually be suitable.