How do I make a macro for Prismatic Spray (D&D5E)?

The Prismatic Spray spell needs to be rolled x number of times depending of the number of enemies. That part is easy. The problem starts when I roll the d8 for the effect. If the result of d8 is 5 or less, it should roll 10d6 damage. If the result is 6 or 7, no further rolls on my part. But if the result is 8, it should roll the d8 again twice and add the two effects (rerolling any 8). How do I insert a conditions like these to avoid making several rolls manually?

There are no conditionals in the base macro system.  You might be able to do something with API scripting, but I have never learned that stuff, others have. What I would do is, make a macro that generates which beams hits, that has chat button macro calls that you can press to roll the effects of that beam. You may also be able to do something with rollable tables, and I believe the Recursive Table script could in turn have the effects of the beams rolled up out of the rollable table beam selection result.  Perhaps one of our API experts, or The Aaron himself, will be along shortly to explain that.

I'm interested in APIs and I would like to know more about how I could do this, but I believe I'll have to go the API wing of the forum to get an answer on this then.