Limiting what others can hear?

I often like to talk to single players alone sometimes if there is something that only they can see/hear/discover. Is there a way to stop other players from hearing the DM and a selected [amounts of] player[s]? Or do I just have to say "guys, take off your headset until I wave my hand at the screen", or use whispers?

There's no mechanic for it within Roll20.

I think you have to rely on other players understanding IC and OOC knowledge. Anyone who acts as though their character knows all of the information that they, as players, know invokes a strangely specific tide of bad luck. Out of curiosity in an in-person tabletop game did you normally leave the room with the relevant player(s) or how did you handle it there?

I usually left the room with them. I used it for instructions more than knowledge, e.g. when I had a horror campaign, and I told one of my players that he was going slowly insane, and to RP it out believing that another player was plotting against him. Generally knowledge is easier to control with character and real world knowledge, but when an entire atmosphere depends on it, I like to be able to pull people away from the rest of the group.

Roll20 Voice doesn't offer this kind of channel-or-sub-room splitting yet. You already know about the /w Whisper method for private text chats to any player(s), that is built-in and offered here. As an external workaround, you could either use a different voice program that offers this feature (some do, letting you break off a "room" with any subset of participants), or you could continue using Roll20 for the main voice but telephone the person on-the-side in another program for the side chats only.

I know that players can click the speaker icon over anyone that is connected to the game to mute them, but you'd have to rely on the players actually doing it. No way that I know of for the DM/GM to mute himself to selected players.

You could also use a second campaign, add the same players to both, but only have the one you want alone in the second campaign. Comes with overhead, but let's you see who is there. As for the IC/OOC, it depends on the type of campaign you are running. I've got a good group where it isn't an issue, but when we're doing murder mysteries or similar stories, the players don't want to know. It's like someone telling you the ending to a book you've just started, if the players are good, it doesn't change the outcome, but it does ruin the suspense and speculation.

You can whisper things via the chat for short things. Pre-written handouts also works. For longer split-ups, you can switch between the groups or people just before any information relevant to the other group comes up. When Kikoskia (on YouTube) ran Call of Cthulhu, group A began questioning a police NPC about a cultist NPC that group B was about to visit. Kikoskia had group A wait for the answer until group B had been thoroughly killed by the cultist NPC, before answering group A "That guy is *extremely* dangerous, and should be approached with extreme caution!". That's how you handle things. For even longer things, you can ask people not in the group to log out of the campaign until you message them to log in again.