Roll20 uses cookies to improve your experience on our site. Cookies enable you to enjoy certain features, social sharing functionality, and tailor message and display ads to your interests on our site and others. They also help us understand how our site is being used. By continuing to use our site, you consent to our use of cookies. Update your cookie preferences .
×
Create a free account

Losing data from "state" variable: Is there a limit on how much the state variable can contain?

1494861669

Edited 1494862246
I'm writing a kinda long, kinda complex API script for a game I'm about to start, and I'm running into a problem with some data mysteriously vanishing from the state variable I've set up. In a nutshell, the script assigns players to one of four quadrants on the board (one of which is shown to the left), then populates those quadrants with graphics that represent their abilities and pools, which they can interact with. Most interactions are done via "bins", which are regions defined by grid coordinates that trigger different functions when counters or other things are dragged onto them. Both the photo and the skull symbol in the image at left have been registered as "bins", and the three rows of trait dots can be dragged & dropped onto them for various effects. Problem:  For some reason, whenever I restart the sandbox, my quadrant assignments will sometimes not persist.  This makes little sense to me, because the quadrants are stored very simply.  For example, if I check the state variable immediately before restarting the sandbox, I'll get this: state.DotTracker.Quadrants = {             TL: "-KjgjouPXDxIwG3XCa34", // (character IDs linked to that quadrant)             TR: "-KjMWf6MutGjCwewZaEh",             BR: "-KjcGCjeuUXgT7cutPga",             BL: "-KjmbIllYx5_RjfmiiyN"         } ... but, when I log the same state info to the console as the very first step after restarting the sandbox, I'll get this: state.DotTracker.Quadrants = {             TL: "-KjgjouPXDxIwG3XCa34", // (character IDs linked to that quadrant)             TR: "-KjMWf6MutGjCwewZaEh",             BR: "-KjcGCjeuUXgT7cutPga"         } It seems to only affect the last assignment I made, which is strange because I can't figure out what makes the last assignment any different from the assignments that came before---they all use the same function. I've considered that it might be a reference/pointer issue (as I know references to objects/functions don't persist), but I've tried everything to exclude this possibility---most recently peppering my code with random "state.DotTracker.Quadrants = _.clone(state.DotTracker.Quadrants)". My last thought is that the state variable might have a limit on how much data it can store?  The method I'm using to keep track of my bins involves a very large 2-dimensional array, and I'm wondering if that might be making the state variable... unstable... or something? I've included the relevant parts of my code below; hopefully it's enough, I pruned out everything I thought irrelevant but wanted to keep the main formatting intact in case there's a scope problem or something. Thanks in advance for any help you can provide! var DotTracker = DotTracker || (function () {     'use strict';     //--- SNIP ---// (a bunch of variable declarations and utility functions)     var cleanupObjectReferences = function () {         /* First "substantial" function run when sandbox starts up: declares variables, deletes unlinked graphics, etc.            big props to Aaron C for a lot of this (though I'm sure I've mangled it in ways that physically hurt him) :) */                  state.DotTracker = state.DotTracker || {};                  state.DotTracker.Chars = _.omit((state.DotTracker.Chars || {}), ["null", null, "undefined", undefined]);         state.DotTracker.Quadrants = _.omit((state.DotTracker.Quadrants || {}), function (value) {             return !_.isString(value);         });         state.DotTracker.Bins = state.DotTracker.Bins || {};         state.DotTracker.Bins.XYMap = state.DotTracker.Bins.XYMap || [];         state.DotTracker.Bins.Index = state.DotTracker.Bins.Index || {};                          /* These "Bins" are graphic objects on the board that you can drag other objects into, to trigger various functions.            When an object is dragged, it checks its new position (in grid coords) against Bins.XYMap (a 2-dimensional array of X/Y grid coordinates).            If a bin occupies that position, XYMap will return its binName (string); otherwise, returns undefined.                 E.g. object's grid position is (12, 75), it looks up Bins.XYMap[12][75] to see if there's a binName there.                      If so, accesses all the bin's details via Bins.Index[binName]. */         //--- SNIP ---// (wacky code to clear unlinked objects and solve problems from crashing; not relevant                       //  to my problem, as the error occurs before this point.     };          var assignQuadrant = function (quad, charID) {         /* This is the main function to set a character up on one of the four quadrants on the board.  It creates a stat block, name, photo, and other graphical elements specific to that character and the system I'm running.  */                  // First, clear out any characters currently in that quadrant, as well as the quadrant iself:         state.DotTracker.Chars = clearCharacter(charID); // returns _.omit(state.DotTracker.Chars, charID)         state.DotTracker.Quadrants = clearQuadrant(quad); // returns _.omit(state.DotTracker.Quadrants, quad)         // Create a template object for the character, referencing the various objects and stats...         state.DotTracker.Chars[charID] = { vitality: {}, willpower: {}, spite: {}, shadeSymbol: "", quadrant: quad }; // Assign the charID to the Quadrants lookup object: state.DotTracker.Quadrants[quad] = charID;                  // ... and then construct the graphical objects and bins for the character in this quadrant:         _.each(["vitality", "willpower", "spite"], function (trait) {             state.DotTracker.Chars[charID][trait] = makeDotLine[trait](charID);         });         state.DotTracker.Chars[charID].shadeSymbol = makeShadeSymbol(charID);         restoreAllValues(charID); // grabs current values of stats from character sheet     };          var handleInput = function (msg) {         /* Event Handler:  Chat Input (pruned to include only the relevant parts) */         var args = msg.content.split(/\s+/);         switch (args.shift()) {             case '!dt':                 switch (args.shift()) {                     case 'assign':                         var quad = args.shift();                         var token = getObj("graphic", msg.selected[0]._id);                         var id = token.get("represents"); //--- SNIP ---// (just some error-catching code)                         assignQuadrant(quad, id);                         break;                     //--- SNIP ---// (a bunch of other commands, not really relevant)                     default:                         showHelp(who);                         break;                 }                 break;         }     };          var checkInstall = function () { // And right here is when I check the status of the Quadrants object, and find entries missing.         log("SUPER EARLY: " + JSON.stringify(state.DotTracker.Quadrants || {});         log("");         log("*********** INITIALIZING ************");                  cleanupObjectReferences();     };     var registerEventHandlers = function () {         on('chat:message', handleInput);     };     return {         RegisterEventHandlers: registerEventHandlers,         CheckInstall: checkInstall,     }; }());
1494863525
The Aaron
Pro
API Scripter
The limit for the state is 2megs.  It's kind of hard to measure what that means, but you can get an estimate using this script: log(`State approximate size (startup): ${(JSON.stringify(state).length/1024).toFixed(3)}kb`); on('ready',function(){   _.delay(()=>{     log(`State approximate size (ready): ${(JSON.stringify(state).length/1024).toFixed(3)}kb`);   },100); }); Since Javascript arrays are not sparse... > let foo=[]; [] > foo[123]=2 2 > JSON.stringify(foo); "[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,2]" ...you'd probably be better off storing your XYMap as an object, and then build a key like: let key = `${xCoord}x${yCoord}`; or similar. > let bar = {}; undefined > bar["12x3"]=2; 2 > JSON.stringify(bar); "{"12x3":2}" There's nothing obvious jumping out at me, but you never know what might be hiding in all that redacted code.. =D 
1494864523
The Aaron
Pro
API Scripter
(Also, I'm betting the "Aaron C" you're talking about is me. =D  Don't worry, the nausea passed quickly! =D~ )
1494868796

Edited 1494868816
The Aaron said: (Also, I'm betting the "Aaron C" you're talking about is me. =D  Don't worry, the nausea passed quickly! =D~ ) Oh, it's definitely you---you were a huge help to getting me started on this, way back when I was looking for the Best 'Hack' to Create a Graphical Button .  Plus your scripts in the archive have been a great resources; thank you! After tinkering around a bit, I do have a few questions that might help me pin down the problem. (My state variable never went above 83kb in size, so it's not a capacity issue.)  I suspect my problem has something to do with scope, or I'm somehow assigning pointers to the state variable instead of actual values.  I can't think of any other way that data from state would disappear in between checking the state variable --> restarting the sandbox --> immediately checking it again. Is there anything I need to know about using the state variable inside functions or the like?  Can I freely assign, say, "state.DotTracker.Quadrants[quad] = charID;" at any scope and expect it to stick (as long as charID is a value)? Is there any way a string (character ID, specifically) might be passed as a reference/pointer instead of a value?   Is there any functional difference between ending all of my functions and variables with commas, vs. ending them with semi-colons?  Thanks again!  Let me know if you really want to see my code; I'm happy to pasteBin it but it's getting lengthy :)
1494869854
The Aaron
Pro
API Scripter
I'm always a fan of "bring on the code!"  =D 1. The state is a global object, so it can be accessed in any scope.  However, if you define a variable in a narrower scope with the name 'state', it will hide hide the global version. 2. Not that I can think of. 3. Just a matter of how you define them.  I tend to have a single var/let/const and then just define multiple things with it. Some people argue that using it again on each line is better, but I'm not really a fan.  Just personal taste. =D My guess is that somewhere you care taking a subtree of your state object and making manipulations to it for display, not realizing that you have it by reference and are thus changing the state with what you're doing.  Look for places where `state` is to the right of `=` and you can probably find it quickly.
1494870124
Lithl
Pro
Sheet Author
API Scripter
Yes, that's fine. No. JavaScript is always pass-by-value (although the "value" in question for objects including arrays is a reference). However, even setting reference values in state will lose that reference, since the state object is ultimately stored as a JSON string. Yes, absolutely. It depends on the context in which they exist. Compare: function example1(param1, param2) { } // comma is illegal here, and semicolon is not expected (but won't break anything) var example2 = function(param1, param2) { }; // semicolon is expected here, and omitting it will cause automatic semicolon insertion var example3 = function(param1, param2) { }, // this is the comma operator, essentially concatenating "var example3 = ..." with "var foo = true", // omitting the second "var" foo = true; { example4: function(param1, param2) { } // semicolon is illegal here, and comma is optional since it's the last property in the object literal } { example5: function(param1, param2) { }, // comma is required here since it's not the last property in the object literal foo: true }
1494870362
Lithl
Pro
Sheet Author
API Scripter
The Aaron said: I tend to have a single var/let/const and then just define multiple things with it. Some people argue that using it again on each line is better, but I'm not really a fan.  Just personal taste. =D My personal preference is to define multiple things with var, and put that var at the top of its scope, but only define a single thing with let/const, and put the let/const as close to where it's used as I can. This is because var declarations are hoisted to the top of the scope, while let/const are not, so I want my code to accurately reflect what's actually occurring at runtime. Of course, the style guide I have to follow at work prohibits defining multiple variables with a single var/let/const... =/
1494870495
The Aaron
Pro
API Scripter
=D  Classic Brian. =D  I was hoping you'd drop in with the technicals. 
My guess is that somewhere you care taking a subtree of your state object and making manipulations to it for display, not realizing that you have it by reference and are thus changing the state with what you're doing.  Look for places where `state` is to the right of `=` and you can probably find it quickly. Woo, thanks a bunch---you were right on the money.  I was passing state as a parameter into a separate function, then manipulating the parameter and returning it.  Took me a while to figure it out, but it all appears to be working! Thanks again!
1494880521
The Aaron
Pro
API Scripter
Great!
Bah, I spoke too soon, my "fix" just evaded the problem by dodging the function I needed entirely. But definitely, the problem is I'm passing state properties by reference, and then manipulating them as values, which of course is causing all kinds of problems. Is there a safe way to extract data from the state variable by value, rather than by reference/pointer?
1494892110

Edited 1494892576
Lithl
Pro
Sheet Author
API Scripter
Ryan said: Is there a safe way to extract data from the state variable by value, rather than by reference/pointer? var statecopy = JSON.parse(JSON.stringify(state)); There are some caveats to using JSON to create a deep copy of an object, but since state gets turned into a JSON string for long-term storage anyway , those caveats are irrelevant in this particular case. You can see a much  more complicated deep copy function  here , running from line 673 to line 789 in that file. It circumvents the potential pitfalls of deep copy with JSON, but it's also 115 lines longer than deep copy with JSON, and requires extension every time you want to deep copy an object type that requires special handling (with an example of extending for arrays at line 775 and deep copying dates at line 786). =)
Okay, well now I'm fairly clueless.  I have no idea what's going on. I've stringified-then-unstringified my entire state namespace, logged it and compared to make sure it matched the original, and it still results in all my state data disappearing. If I comment out lines 434-438 of my code, disabling the registering of bin "regions" into the map grid object, the original problem goes away; the state variable persists as normal and everything stays when I reset the sandbox.  I have no idea what could possibly be happening in these four lines to mess with my state variable: ... for (var i = gridBounds.left; i < gridBounds.right; i++) {     for (var j = gridBounds.top; j < gridBounds.bottom; j++) {         state.DotTracker.BinsMap[`${i}x${j}`] = { name: binName, charID: charID };     } } ... Here is my entire script, just in case you're feeling masochistic; I've pruned it down and tried to keep it nice and organized, but I'm an armchair coder at best!  Pastebin: DotTracker.js
1494934244
The Aaron
Pro
API Scripter
I think the state might be a red herring.  This is probably a logic bug, rather than the state failing to properly store data.  I think what is happening is when you call assignQuadrant(), it calls clearCharacter(), which has the side effect of removing the entry for the character's current quadrant before assigning it to a new quadrant.  Nothing appears to restore that quadrant entry.
Ack, I should have reported back as I found a solution: Rather than mapping out the locations of all the bins, I simply replaced the "BinsMap" with a function that iterates through all the bins whenever something is dragged & dropped, and checks whether the dropped object's position falls inside any of the bins' bounding boxes. With this alternative, I was able to remove those four lines of code I identified above, and everything works perfectly! The Aaron said: I think the state might be a red herring. This is probably a logic bug, rather than the state failing to properly store data. I think what is happening is when you call assignQuadrant(), it calls clearCharacter(), which has the side effect of removing the entry for the character's current quadrant before assigning it to a new quadrant. Nothing appears to restore that quadrant entry. I checked over my code, and I think this part is behaving as it should:  Basically, clearCharacter() is called in case I'm moving a character from one quadrant to another.  So it clears the character's current quadrant (the one he's moving from), and then assigns him to a new quadrant (which populates all of the quadrant entries).  So the former quadrant entry is supposed to be blanked out, as there isn't a character in that quadrant anymore. Thanks again for your diligent help, I can't tell you how much I appreciate it---my game setup is really coming along nicely!
1495470545
The Aaron
Pro
API Scripter
Cool!  Glad you got it working. =D