Greetings! In the past week, we’ve been working hard to improve the stability and performance for users. These infrastructure changes don’t always make it into the weekly release notes, which focus more on code changes, but we wanted to highlight some of the ways we’re working to improve the site behind the scenes. The big changes this week are to our load balancer, the API server, our roll server, the jukebox, and the video and voice chat. First, the load balancer was really excited to make connections and have a party, and a whole lot of people wanted to come in! The load balancer was shaking everyone’s hands at the door, but there were too many hands, and kept forgetting to let go of one connection before starting another. From inside the party (the database), everything looked pretty good, but the load balancer was overwhelmed and wasn’t letting anyone in. We fixed that by moving the load balancer a little further inside. Now, the party guests get told to line up for their handshakes, and the load balancer has enough room to make all the connections it needs to. The API server is the server that handles all the API sandboxes in games owned by Pro users. These sandboxes are isolated spaces that can’t affect the rest of the site, but which allow scripts to run in the VTT to handle various tasks and enhance your gaming experience. There were several reports in the past couple of weeks of API sandboxes shutting down unexpectedly, which causes the APIs to stop working during your games. The customer support and development teams spent a lot of time researching the causes of these shutdowns and found that there were some outdated network settings that could be optimized. They also found that when a sandbox went idle or had an error, information about it stayed in the system even though the sandbox itself was no longer active. Kind of like when you throw away an old sandwich, but the smell sticks arouned in your fridge? Like that, but the API sandbox was the sandwich you tossed out, and the error info was the lingering smell from it still stinking up the fridge. To fix that, we set up a system that clears out the fridge more frequently, reorganized it so it’s easier to find stuff inside it, and we increased the size of the fridge. So now, we have a nice, clean, walk-in cooler. The second big change was to the Quantum roll server . You might have noticed in the last week or so that dice rolls have lagged more than usual, or sometimes straight-up failed to roll. The roll server was sharing space with another service we use for load balancing. And with a lot more users on the site, both of those services needed all the resources they could get! So we moved ol' Quantum into its own space. It likes the new digs. It’s got its own fridge, TV, and sofa, and so far, it has room for everyone’s dice to roll without interruption. That load balancer is happier, too, now that the roommate has moved out. It said it’s thinking of turning the extra room into an art studio. Sometimes, performance tuning is all about the tiny places where you can trim down one process to save time and overhead. The improvements to the jukebox and video/voice chat servers are examples of those. You might remember that on May 5 , we had done some optimization for the video and voice chat servers. While those were good changes, and helped improve the health of the servers, we did find that, after a long period of time, the server connection would time out. It turns out, those servers keep a lot of information in their logs! We set them to rotate the log files a little more often, getting rid of yesterday’s news to make room for more current data. Finally, the jukebox got a little performance tuning as well. Have you ever had a kid or friend incessantly ask you the same question over and over, like “are we there yet?” The Jukebox progress bar was set to refresh every 50 ms, which is about .05 of a second. That’s super smooth, but it also means that the Jukebox was pulling data every .05 seconds, and that was one place where we could trim a little bit and streamline things. We changed that to refresh every .5 seconds. You might notice a slight difference in how smooth the progress bar moves, but most importantly, we reduced that one data call by a factor of 10. As always, we’re keeping our eyes on issues as they emerge, and we know there have been reports of games not loading . This can be caused by many factors, so if you are experiencing this, please follow the troubleshooting steps here . We fixed one of the causes this morning, where games using the community-created GURPS character sheet would not load if your language was set to something other than English. This type of issue comes up once in a while, and we have a fix for the underlying cause in development. If you’re still having trouble loading into your games after troubleshooting, please report it in this Google form . Your reports help us diagnose issues like these and fix them. There's also a forum thread directing you to these troubleshooting and the form. Anyway, we wanted to fill you in on some of these updates. We know this kind of work often happens invisibly, and you might not always know when something has been worked on or fixed. Thanks for listening! Check it Out The most recent community roundtable was Thursday, May 21. Catch the video here ! Looking for previous release notes? Read the full list on the Roll20 Help Center , or check out the most recent note here.
