Large Scripts - How big is too big?

I've got a little bit of programming in my background but never had the chance to work with javascript. I decided that a good way to learn was to try making some simple API scripts for Roll20 and after much swearing at the computer I finally managed to produce a simple little  calendar script . I put it on the Roll20 forums and had some requests for changes. Most of the requests I was able to implement and it was a great opportunity for me to further my knowledge of javascript. Fast forward 6 months and my script now covers 4 worlds, lunar cycles, time and weather tables. I enjoy making requested changes and I'm still just super thrilled that people are actually using it. My concern at this point is bloat. I haven't noticed it slowing down my games or anything yet, but I'm wondering if that's something I should be concerned with? I've got 11 state variables stored and then I do a lot of calculations on the fly. It seems to me that keeping the number of state variables to a minimum is important as there is limited storage space. Is that correct or should I be using more variables? I'm working on adding events to my calendar at the moment and I'm struggling with how I want to implement them. I could store them in the state variable, or I could put them into a handout or character sheet. It makes the most sense to me to put events into an array in the state variable, but I'm concerned that someone might add a lot of events to their game, eating up the memory for the state variable. As someone who only knows enough javascript to be dangerous, I'd really appreciate the comments/thoughts of those of you who understand this more than I do! Thanks in advance.

The only real size bloat issues that I've run into with scripts are the size of the state, which is limited to 2mb in size, and the time it takes for any calculations used, which is 20-30 seconds max depending on where in the infinite loop detection ping cycle your calculation falls. For your events question, I'd recommend having the script make an archived handout which would hold the events in JSON format in the GM notes and then making a chat interface for creating, editing, and deleting events from the handout, you would then just save the handout's ID to the state.

As you might imagine, I've got some thoughts on this... =D The size of the code is not actually that important, but it tends to be an indicator of something else that could influence runtime: design choices.  I tend to find the hardest part of any programming problem is not really the functionality per se, but the way the data is organized.  You can use a very simplistic structure that causes you to need to do a lot more work to use it, or you can use a cleverer structure that makes using it much easier.  Coming up with that clever structure is often where the most work (and most fun!) is. There's something of an adage in computer science that you can "trade memory for speed."  Caching is a very straight forward example.  You could calculate the sine of an angle every time you need it, or you could calculate it once each time an angle is passed to your sine function and just return the cached result on subsequent calls (you can even prime the cache with the sine of the most common angles and not need to calculate them at all!).  Another apropos example is indexing.  You could have an array of objects you care about and just search through it for the right one each time, or you could keep a lookup from some key bit of information (name, id, etc) that gives you the index of the item in the array and save the time looking. As far as state goes, it is limited in size, but (at this point) is still pretty large.  If you're filling 2mb, you're probably doing something wrong.  You could definitely store events in the state and probably not worry too much about it. (This is precisely what I do in the Imperial Calendar Script.)  Something else to consider though is portability.  The State is locked to a particular Game, and not easy to transport.  If you wanted to have the same Calendar in two different games, you'd have to manually copy the data out and enter it back in.  If I were doing a calendar script again, I'd create a Character that represented the Calendar and add the events as Attributes on that Character.  That way the Calendar "Character" could be moved to another game to copy it's events over.  I'd probably store the data Base64 encoded (I have a script for that in the repo) so it couldn't get corrupted by the browser and on script startup, I'd find the Calendar Character and use it's ID to grab all the Attributes for that Character and prime an in memory cache of events, keeping the Attribute ID along with the data I read from it.  If the event gets removed or modified, you can use that ID to remove or modify the corresponding Attribute.  If the Attribute gets removed for some reason outside of your script, you could recreate it with the data in your cache (provided you were currently running). One way to address bloat is by inserting some abstraction (Another adage, largely tongue-in-cheek, is "there's no problem in computer science that can't be solved with another layer of abstraction.").  If you can find a way to abstract the interface to lunar cycles, seasons/days/weeks/months/years, weather, etc. you can create implementations of that interface for each of the different worlds.  Even if it's a simple as a collection of expected functions to get the names of the week, names of the month, length of the year, etc. Hope that helps. =D

To Scott's point about execution time, if you have big tasks that need to be done, it's best to break them in to small discrete steps and execute each in a _.defer() or _.delay() or the like.  A good example of this is when the Search API script indexes all of the handouts, characters, and attributes in a Game.  It builds a few hundred thousand objects to make an index for the search engine, but it only does one object at a time.  Using _.defer() and the like will allow execution to return to the calling scope so that it doesn't miss heartbeats and believe that the script is hung.

Yep, I figured I'd let you explain how to address the problems I raised when you inevitably stopped by since you're the one that taught me how to get around them :)

=D

Scott and Aaron, thank you so much for taking the time to answer this. I think I was worried for nothing, though the best practice tips you've given me will serve me well in this script and every one I write in the future! I'm continuing to work on streamlining my code and thoroughly enjoying the challenge. I love the idea of an archived handout for the events. Doing it that way will allow GMs to put a decent amount of information in the notes for each event AND will allow them to transfer the events between games. I really appreciate the tips on how to handle this!

No problem.  I was suggesting a Character and associated Attributes, but you could use one or more Handouts for events as well.  Doing it in a single character makes it easy to move the whole thing, but having handouts lets you show them to players easily.  You could have a feature to create a handout from an event, and keep it up to date when the event changes if you wanted to support that. =D

I prefer the handout over the character sheet when possible because it's quicker and easier for me to get a single gmnotes field that you know will have valid JSON (Should still have handling for incorrectness here of course) than to get a bunch of attributes/abilities from a single character and then manipulate them into JSON. It also doesn't contribute to attribute bloat.

Kirsty said: I decided that a good way to learn was to try making some simple API scripts for Roll20 and after much swearing at the computer I finally managed to produce a simple little  calendar script .  And the kill counter! Woot!

You could use handouts.  Particularly if you used a single handout with a large base64 encoded block in it, but I think it loses some utility as you'd have to be constantly decoding and encoding the whole thing when you make changes.  You could use multiple handouts, but then if you want to migrate to a different game, you'd have to move all of them (which may more may not be a bad thing).  Also, lack of being able to organize journal entries into folders means you'd constantly be dropping new entries in the root. I don't think that attribute bloat is going to be too much of a concern for having just one more character.  Most attributes from the character sheet shouldn't even be created unless you specifically open the Character Sheet tab.  The get/set semantics on attributes are easier and faster than handouts (handouts have that extra async get step and an issue where the whole content is synced across all players on save, so having a monolithic handout storage will be a large blip on each change.), and they don't suffer from automatic manipulation the same way handout text does (expansion of html entities, etc).  You will want to be sure that your attribute name identifies it in such a way that you can separate it from automatically created attributes.  You could use the max field set to a specific identifier or just use a specific name that will be explicitly for your calendar: let calchar = /* fetch somehow at startup */; // using name to store the Event's name for convienence let eventAttrs = findObjs({ type: 'attribute', characterid: calchar.id, max: 'KirstyCalEventRecord' }); let calchar = /* fetch somehow at startup */; // using an explicit name (same for all event attributes) for the attribute to separate them from regular character sheet attributes let eventAttrs = findObjs({ type: 'attribute', characterid: calchar.id, name: 'KirstyCalEventRecord' });

The Aaron said: [...]  The get/set semantics on attributes are easier and faster [...] Do you have a quick example for how to set attributes? I'm fiddling around with a script for managing the party's money and was kinda struggling with that. I got it working by now, but the whole thing got a bit quirky...

Once you have the attribute, it's just the regular .get() and .set() calls: let attr = findObjs({type:'attribute',name:'tacosauce'})[0]; if(attr){ let cur = attr.get('current'); // .get() log(`Taco Sauce level: ${cur}`); attr.set({current: parseInt(cur,10)+1}); // .set() } Contrast that to: let handout = findObjs({type:'handout',name:'tacosauce'})[0]; if(handout){ handout.get('notes',function(notes){ // .get() log(`Taco Sauce note: ${notes}`); handout.set({notes: notes + '<br>...and again...'}); // .set() }); } Note also that  .set() on handouts has a cooldown to prevent overloading all the connected clients when things are updated fairly rapidly (that blip I alluded to above).

Ah... I see. I used getAttrByName for getting the attribute values and was looking for something like setAttrByName first.  var pp = parseInt(getAttrByName(character.id, "pp")); Thanks!

Yeah, getAttrByName() is only for reading.  If not for the fact that it understands autocalc fields, I'd never have cause to use it.  It's real intent is to duplicate the effects of an @{XXX|Attr} type reference, where it resolves against the character sheet.  Since autocalc fields have no actual object, they would otherwise be invisible to the API.  With the advent of sheetworkers, autocalc fields are out of vogue, so maybe getAttrByName() will be at some point. =D  My biggest complaint with getAttrByName() is that it ALWAYS returns a value.  There's no way to tell with it if an attribute exists with a 0 value or doesn't exist at all.  That's mostly an unimportant distinction, but I prefer knowing (particularly for things like GroupInitiative). =D

All right then. I'll have some re-coding to do... ;)

With the advent of sheetworkers, autocalc fields are out of vogue, so maybe getAttrByName() will be at some point It still has some use, namely returning the default value of an attribute if no attribute object exists... which can also be problematic, as you later point out.