Separate from the issue of your players seeing a list of characters they they shouldn't see in the 'Speaking As' list, if your players have characters in that list they they are supposed to have access to, but aren't true 'characters', they can use Stylus to hide those characters from the list, or to only display the characters they should see.
For example, I use the 'Language Trick' to be able to only whisper between characters who understand the same language, or for some 'Macro Mules', or for spell templates, so I don't want those to show up in the 'Speaking As' list, but I do want my players to have Edit & Control access to them.
Here's the code:
/* Hide Specific Character */
#speakingas option[value=character\|character_id]
{ display: none; }
/* Hide all BUT specific characters */
#speakingas option:not([value=player\|player_id]):not([value=character\|character_id])
{ display: none; }
Example code:
#speakingas option:not([value=character\|-KHIA-W9OoZp9eE6UdWy]):not([value=player\|-M4MqKQpTCRs8I6I65Rh]):not([value=character\|-M7YiSUD9rhnHKmKmBnY]):not([value=character\|-M7Yi0tFL482VJrJBblW]):not([value=character\|-M7Yi3hBPf_U-VYId7q0]):not([value=character\|-M7YiVxk1Hz1ebQYkDhh]):not([value=character\|-M7YiXjwmM2ztpWWzsj7]):not([value=character\|-MfJu8dkIod0GA9EMclI])