While you can't buy mounts out of the PHB with your starting gold, you can get a mount if you are granted one from a spell or class feature.