I use the GM Layer to not only make notes on the Map, but also to prepare Encounters, have tokens there that will pop up etc. This sometimes makes moving around groups of things harder than it has to since I accidentally select the Area-Numbers or my Area Notes as well. I'd love if there was another layer that acts like the "Map and Background" layer, but is not visible to players to Permanent Notes, markings etc.