Using the button in the app's settings clears the chat for the open session but doesn't clear the games overall chat log. Refreshing the page will reference the chat log again on loading and show it in the chat pane. To full clear the log you need to go to the game details page and use the 'clear chat archive' option there, once you confirm the action the chat log will be purged. I always recommend backing it up first if there may be useful info contained within.