This has actually been brought up many times in the past. From what I've gathered, it's far more difficult to implement than you would think. Roll20 uses vectors for the drawings and "erasing" them isn't a simple task due to the way that they function. Even vector drawing programs like Adobe Illustrator suffer from issues with this.
You can use the selection tool to grab individual parts of drawings (depending on whether you drew them as a continuous line or as separate pieces) and delete them with the delete key or hit CTRL+Z to undo the last one.