I could be entirely mistaken, as I've never had to kick anyone that was logged in before, but I don't think the kick option will remove anyone who's actively logged in. It may be overkill... but you can copy your campaign into a new one, send out new invite links to your players, and delete the old one.