The templates on macro rolls (at least in D&D5e) don't seem to work with the detached chat window. I'm using Safari 8.08 on OS X 10.10. I imagine this is a known issue but I didn't see anything about it. It's also not the end of the world. The two screenshots used the same macro. The first one is the in app chat, the second is the detached chat window that pops up if you double click the chat button.