API Dev Server update 10/17: Sheet Worker Support

Update on both fronts: Translations: This is a weird one. It appears that the failing translations is not a function of how getTranslationByKey is called, but some sort of strange state that the campaign can get into. Once it's in this state, API sheetworkers can't successfully call getTranslationByKey under any circumstances. I haven't yet been able to work out exactly what circumstances cause the campaign to get broken. If I start with a super simple test campaign it doesn't happen initially. I think it may first be triggered when I try to set the campaign sheet to the Shaped Sheet (as a custom sheet) - but interestingly after this point it makes no difference if I then erase that sheet and replace it with a simple testcase - from that point on translations don't work under the API in that campaign. I also notice that initially I get a message in the API log saying something like "Loading translations into worker" - but then after it breaks I don't see this any more. Although I don't know exactly how or why the campaign gets into this state, I can usually get it to break pretty quickly. If you want to see a barebones game where this problem manifests, check out my campaign id 39204. You just have to type something into the chat and hit enter and you'll see the error in the API log - and the sheetworker it's executing is extremely simple so hopefully it should be fairly simple to look into. Callbacks: While investigating the translations issue, I noticed that in a brand new campaign with just the shaped sheet and script, the callback executed when I expected it to. The original campaign I was testing with had exactly the same code, and was executing the same commands, but had 16 characters, a few handouts and 4 maps. This suggests to me that the timeout is resulting not from extended calculation within the script itself, but from the time taken to access attributes, which, as I've noted elsewhere , appears to scale linearly with campaign size(!), perhaps due to a missing/poorly configured index, or some strange artefact of your firebase configuration. Perhaps if there's a reasonable fix for the poor performance of attribute access it might be possible to avoid having to increase the timeout - which might well end up needing to be unreasonably large on campaigns with even a moderate number of characters (e.g. 100+)?

Perhaps the list is longer than other sheets, but I can't think of why the Shaped list would present problems.

Riley D. said: Kryx said: Perhaps the list is longer than other sheets, but I can't think of why the Shaped list would present problems. Yeah I don't think it's anything you guys did since it works on the client it's something on our end...I'm just saying, maybe there's something going on there that's causing a specific issue...I'll look into it when I get done with the cached attributes fix. Interestingly, I've had the Shaped Sheet translations file working correctly with a simple test sheet with a minimal sheetworker, so I don't think there's a problem with the translations file itself. It seems that the problem manifests when using the sheet itself in the campaign, but as I said, it then leaves the campaign somehow broken in a way that isn't fixed by changing back to a simple sheet again.

Great timing Riley - I just asked about this on the character sheet forum.  Excited about seeing this in action.

Would it be possible to add an argument to filterObjs() to restrict to one of those sets? var playerChars = filterObjs((o)=>''!==o.get('controlledby'), 'characterid');

I don't know to what extend this holds for the sheet worker setAttrs(), but even when you have the attributes already, setting   them  seems to be a rather slow process. I ran into this problem with ChatSetAttr for Storm King's Thunder-sized campaigns. The exact times vary of course, but I just ran a small test by hacking my scipt: In a somewhat large campaign with 161 characters and 75k attributes (in case campaign size matters), I built up essentially a list holding 1 attribute object per character. Setting all of these to a certain value in a loop takes around 140ms per attribute (with normal set, not setWithWorker). That really seems excessively long, and that may well hold back API sheet worker performance if they are affected by the same slowdown.

Cool beans. =D

Riley D. said: If you give me a Gist of your benchmark and a link to the game you were running it on I'll certainly take a look and see what I can find out. Okay, sent you a message with a link and a reduced test script. Thanks for taking a look! EDIT: This is my benchmark code:  Gist , if anyone else is interested. I've been getting really inconsistent results here in differently-sized games. It seems that the size of the game (dramatically) affects how much time it takes to set a single attribute (to a fixed value) - mind you, this is after I've already found all of the attributes and put them neatly into an object, this has nothing to do with filterObjs() or findObjs().

Riley D. said: In theory it should be faster, but let me know what you find -- if you could re-run that test case from your other post with the timings and let me know if there are any speedups that would be great. In addition if you want to give me the link to a game that has a lot of Characters (and attributes) I will try and use that in the future to fine-tune things. So I've just rerun my import process and I'm not seeing any real difference in the top-level behaviour under setWithWorker - the sheet worker completed callback still happens a lot earlier than it ought to. The campaign that I referenced in my other post belonged to another user and I no longer have access to it unfortunately. I will make a test campaign with a bunch of characters later today and post some more profiling stats both using findObjs directly and using setWithWorker, but it looks like there might still be some tweaking to do here.

Ok, so quick update: I've rerun a test on a campaign for 450 characters and findObjs is much faster - looks like we've got something much closer to O(1) now which is great! The same test that I ran in the other thread gives these results: [ {"name":"charCreated","time":0.007}, {"name":"charRetrieved","time":0.022}, {"name":"attrcreate","time":0.025}, {"name":"attrFound","time":0.034}, {"name":"attrRemoved","time":0.041}, {"name":"charRemoved","time":0.044} ] I think that's already going to make a huge difference to a lot of API scripts! The campaign with all the characters is id 39237 (I'll PM you a campaign link but I assume you've probably got magic powers to access it anyway!) and the script is the same one that is listed in the bug thread linked above. I'll try and play with some more minimal test cases involving sheetworkers to see if I can pin down why that use case isn't working better now.

Lucian said: . I'll try and play with some more minimal test cases involving sheetworkers to see if I can pin down why that use case isn't working better now. I'll be very interested in seeing the results of this... in particular,   if my suspicion holds that setting attributes is as slow in API sheet workers as it seems to be in the API normally.

Eeek sorry! I've just realised that my test wasn't valid as my 450 characters didn't have all their attributes since their import wasn't complete. I will try and redo the test properly now.

Yeah, I've managed to fully import about 60 characters (sorry, it's a slow process at the moment) and findObjs is already up to 160ms from 30ms previously. It's looking like the scaling is still not great - as I mentioned before, I'd really hope  for at worst O(log(n)), and with decent hash-based indexing it should be closer to constant time in a lot of cases.... Have to go out now but I'll try and do some more investigation tomorrow.

So, can I get a bit of clarification on this.  Does this mean that we can now use (as long as they have been updated with the new code), api such as the !setattr !alterattr, !decattr to change character sheets and have that register with sheet worker calculations? If not, what exactly does this mean for the users, that is what exactly will this allow people to do that isnt possible now.  Forgiveness if I missed something in the previous posts.

This allows changes made via the API to cause sheet worker code to be run.  Example:  an API script could increase a strength attribute by 4 points. This can now trigger the recalculation code in the sheet worker which updates the strength modifier attribute. 

Hallelujah!  Praise the Lord! This has been the bane of my, and my players life for a while, since I started switching to sheet workers since most of my games use shapeshifters heavily or things like Augment Summoning. I cannot say thank you enough for making this possible.

Well, as you can see from this thread, it's not quite working without a hitch just yet, and only on the dev server. But I agree that this is a great feature to have.

Nice. It looks like we're making some real progress here! Thanks for your clarification of the cases that you are addressing. I wasn't sure what the relationship was between findObjs and the sheetWorker setAttrs/getAttrs - that's now much clearer. My main concern performance-wise is scaling nicely with the number of characters. The shaped sheet is quite feature-rich and does tend to have quite a lot of attributes, but Mark has put a lot of work into trying to prune it down wherever he can and for NPCs (which tend to be the bulk of characters in a campaign) the attribute count/character tends not to be too big. Looping through a few hundred attributes shouldn't be too expensive - as your final test seems to have demonstrated. Going to do a bit more testing now to confirm, and I'm going to do complete the import of more characters so we have an even heavier dataset to try against. My comment about about O(1) was made relative to 'n' being the number of characters rather than the number of attributes; since the number of attributes / character doesn't vary that dramatically in most cases (except for PCs with loads of spells), an index on character will ensure that lookup is pretty close to constant time for most practical purposes. Assuming I can confirm your findings with findObjs, the next big testing point for me will be when you think you've got the indexing applying to the sheetWorker calls - and hopefully this will in turn fix the issue with the callbacks as well. I'd like to say a huge thank you for working through this with me - it's a rare privilege to be able to work directly with the developers of a platform to address issues (especially about performance!); having been on the other side of this particular fence, I understand that it's often hard to do so in the face of other development priorities and demands on your time and I want you to know that it's genuinely appreciated! I think that working through some of these issues will ultimately result in some really noticeable improvements for a wide range of Roll20 users - even if most of them don't necessarily understand all the technical ins-and-outs of what you're grappling with here!

Ok so I've completed the import of a bunch more characters now - there are ~150 characters with a total of 70k attributes in my test campaign. Performance of findObjs still seems to be much better than previously: < 10ms rather than substantially over 100. It looks like it's not quite constant time, but it's certainly fast enough for me and I can't think of any applications that are appropriate for the Roll20 platform where it would need to be substantially faster. If this can be applied to the code that the API sheetworkers use I think we'll be in business. The only slight fly in the ointment is that this appears to have come at the cost of a significant slowdown in API loading times. I can't profile it exactly because obviously I don't have access to the loading code, but a crude test suggests it takes over 60 seconds to get from reloading the sandbox to firing the first on('ready') event. Don't get me wrong - I'd far prefer to pay the cost once on startup rather than having findObjs be slow every time; but a 60+ second delay on API bootup might catch people out (perhaps a log message indicating what is happening would be useful?) - and maybe there's some quick-win optimisations that could be done at some point to bring this down a little bit?

Oh, just saw your edit. Will check now.

I've rerun the sheetworker test and it's still calling back to early - and still taking rather a long time to complete overall. I need to spend some more time profiling it to try and get to the bottom of where the time is going. It's quite hard to do because of all of the interleaved callbacks - the sheetworkers do a lot of getAttrs with setAttrs in the callback, and these then trigger other sheetworkers that do more, etc.  I'm going to have to put a fair bit more logging in to work out where the big chunks of time are going - whether there's some "smoking gun" slowdown or whether there's just a lot of little bits of work getting done that all add up in the end. In the meantime, the other thing that might be worth double-checking is the code that actually triggers the sheetworkers. I've got a vague hunch that's based on something I saw in the client a while back. I seem to remember seeing that in a large campaign, there were quite substantial gaps between the different "tranches" of sheetworker execution - and it seemed like there was a fair amount of time being spent just calculating which attribute listeners to fire after each batch of value changes were made. But obviously this is just me stabbing wildly in the dark, so feel free to ignore if that makes no sense based on the code...

If you're in there looking at test cases already, here's another one that would be nice to sort out....&nbsp; <a href="https://app.roll20.net/forum/post/2217884/api-regr" rel="nofollow">https://app.roll20.net/forum/post/2217884/api-regr</a>... =D

Just to follow up on my previous post: "1478811836964 initialize.sheetOpened getSetItems" "1478811836997 initialize.sheetOpened getAttrs callback called" "1478811836998 initialize setFinalAttrs" "1478811836999 initialize.sheetOpened setFinalAttrs" "1478811837066 initialize setFinalAttrs callback called" "1478811837066 proficiencyBonus.update getSetItems" "1478811837067 initialize.generateSkills getSetRepeatingItems" "1478811837073 savingThrows.updateSavingThrow getSetItems" "1478811837073 savingThrows.updateSavingThrow getSetItems" "1478811837073 savingThrows.updateSavingThrow getSetItems" "1478811837073 savingThrows.updateSavingThrow getSetItems" "1478811837073 savingThrows.updateSavingThrow getSetItems" "1478811837073 savingThrows.updateSavingThrow getSetItems" "1478811837074 savingThrows.updateSavingThrow getSetItems" "1478811837074 savingThrows.updateSavingThrow getSetItems" "1478811837074 savingThrows.updateSavingThrow getSetItems" "1478811837075 character.updateLevels getSetRepeatingItems" "1478811837075 abilityChecks.updateInitiative getSetItems" "1478811837076 equipment.updateArmor getSetRepeatingItems" "1478811837077 spells.updateShowHide getSetItems" "1478811837077 abilities.updateModifier getSetItems" "1478811837077 abilities.updateModifier getSetItems" "1478811837077 abilities.updateModifier getSetItems" "1478811837077 abilities.updateModifier getSetItems" "1478811837077 abilities.updateModifier getSetItems" "1478811837077 abilities.updateModifier getSetItems" NOTE "1478811837185 settings.updateShapedD20 getSetItems" "1478811837198 initialize.sheetOpened setFinalAttrs callback called" This needs a bit of explanation. getSetItems is a function that encapsulates the process of doing getAttrs followed by setAttrs. setFinalAttrs is the function that this outer function calls to set the values using setAttrs. Each different usage of these gets a name to identify the specific sequence involved. The above is an excerpt from the beginning of the sheet setting itself up.It does a whole bunch of updates - setting up default values etc. The point that says "NOTE" is, as best as I can tell, the first point at which a "dependent update" gets kicked off - i.e. a listener is fired as a result of a change made in a prior sheetworker script. There's a gap of over 100ms there - and this is a pattern repeated throughout the rest of the trace - there's a substantial gap of around 100ms before each initial call to a sheet worker event handler. The call to setFinalAttrs labelled 'initialize.sheedOpened' changes (among other things) an attribute called roll_setting. There's a listener on this attribute that triggers the function 'settings.updateShapedD20'. Shortly after that's triggered, the completion callback for the original initialize.sheetOpened setAttrs call comes through. It's clear that there's a substantial delay between the original setAttrs call that changes roll_setting, and the triggering of the next sheetworker. This delay isn't filled with any other activity inside the sheetworker scripts themselves (as you can see from the jump in the trace above) so I guess it's either time spent actually doing the processing involved in setAttrs, or in calculating how to trigger the next set of sheetworkers. Obviously without any insight into the internal code behind this, I can't tell what's going on there.

Riley D. said: Okay I've just pushed out another update that should in theory speed up both findObjs() a little more and setAttrs() calls. Let me know if that helped. In addition, we probably should start drilling down deeper if possible...I wonder if there's a way to specifically benchmark the setAttrs() vs findObj() calls?&nbsp; I'm really &nbsp;sorry for continuing to bring this up, I promise I will let it lie from now on: have you checked out the benchmark I sent you? It's not setAttrs(), but it specifically benchmarks the time it takes to call a lot of attribute.set(value) operations, which may at least be similar internally, modulo some findObjs()?

Riley D. said: Okay I've just pushed out another update that should in theory speed up both findObjs() a little more and setAttrs() calls. Let me know if that helped. In addition, we probably should start drilling down deeper if possible...I wonder if there's a way to specifically benchmark the setAttrs() vs findObj() calls?&nbsp; Hi Riley, Sorry for the radio silence - I've been moving a truckload of our possessions half way across Europe + a whole load of stuff has gone wrong/blown up/otherwise made life difficult. Realistically I'm not going to be able to do much in the way of benchmarking for a week or so but I'm still really keen to make this stuff work - I hope you can bear with me and I will get back to it to try and give you some more useful benchmarks. Thanks, Lucian

I totally missed this entire thread until yesterday since i usually read the Pro forum, not this one. From the survey we recently put out, a lot of people would like us to do the following 2 things: 1. Integrate the 3 token bars so we'd call the sheetworker update on whatever field they mapped to the token bar. (hit points, for instance) 2. auto decrement/increment the "used" or "ammo" field when a user clicks on a rollbutton (most common use case is field to update will be on same repeating row as the button) For the 1st, i expect someone is already working on this, because it is totally generic and would work on any sheet. If so can someone point me to it to try in Dev? For the 2nd.. &nbsp;i think we need some help on the API side. I only wrote one long ago, and i never even moved it to the new one-click format yet.

chris b. said: 2. auto decrement/increment the "used" or "ammo" field when a user clicks on a rollbutton (most common use case is field to update will be on same repeating row as the button) Lucian may be able to give you some advice after he returns from vacation. Or you can check out his code. I'd recommend putting a field for the amount used - we're adding that now. For 5e it helps for cases like Lay on Hands.

Is it possible to trigger a sheetworker event from the API yet? &nbsp;Or is this planned? &nbsp;Currently you can change sheet attributes using the API but it isn't recognized by sheetworker to fire various functions, correct?&nbsp;

Vince said: Is it possible to trigger a sheetworker event from the API yet? &nbsp;Or is this planned? &nbsp;Currently you can change sheet attributes using the API but it isn't recognized by sheetworker to fire various functions, correct?&nbsp; You could use sheet worker to do those things but there isn't an event that corresponds to it (like button press) - that I know of. &nbsp;API is still the best way (I use SetChatAttr and Aaron's Ammo API for that). &nbsp;This release in this threat just makes it when an API does make a change to a field that a sheetworker is "watching" via an event - it will register as a change. &nbsp;I ran into this as an issue on my character sheet design, when I used an API to make a change - I notice that the sheetworker didn't recalculate. &nbsp;&nbsp;

Kryx said: chris b. said: 2. auto decrement/increment the "used" or "ammo" field when a user clicks on a rollbutton (most common use case is field to update will be on same repeating row as the button) Lucian may be able to give you some advice after he returns from vacation. Or you can check out his code. I'd recommend putting a field for the amount used - we're adding that now. For 5e it helps for cases like Lay on Hands. Kryx - so like a current / max? or a current / max / left over?

Don't suppose there's any update on how close this is to hitting live, is there? &nbsp;I'd love &nbsp;to be able to toggle buffs on the Pathfinder character sheet via API script...

Riley D. said: Scott C. said: Not related to this, but to the previous API update. Any ETA on when noarchive will be fixed? It still isn't working after the last update (although the error is now more sporadic). Bug Report:&nbsp; <a href="https://app.roll20.net/forum/post/4093752/api-bug-noarchive-equals-true-still-funky/?pageforid=4093752#post-4093752" rel="nofollow">https://app.roll20.net/forum/post/4093752/api-bug-noarchive-equals-true-still-funky/?pageforid=4093752#post-4093752</a> It's on my list, I don't have any further info at this time, I'll update that thread when I do. Hey Riley, any current update on this. I am running into this issue as well.&nbsp;

Riley D. said: Yeah at this point since the only thing that seems to be blocking this is the potential speed of it, I'll probably go ahead and just push it out to Main in the near future (next week or two) and then we can always keep working on speed improvements after it's on there. Did this ever get pushed live?

Hey, yeah, sorry for the long radio silence. Real life got me good this time! I think it makes sense to push this live as is and I will try and find the time to help debug the performance issues as soon as I can.

Gary W. said: Riley D. said: Yeah at this point since the only thing that seems to be blocking this is the potential speed of it, I'll probably go ahead and just push it out to Main in the near future (next week or two) and then we can always keep working on speed improvements after it's on there. Did this ever get pushed live? No :(. Seems to have been abandoned for now...

I wonder if they realize how much of a quality of life type of improvement it would be? The changes over the past year, combined with some incredibly savvy programming, have brought us to the a new level of sophistication in our games. Taken the drudgery out of the game and allowed us to focus on the story and the joy of being with friends.&nbsp; The fact that a script can make a change to a character sheet and not have that change noticed by the sheet worker is a big pain in the rear end.&nbsp; Take casting bless in DnD 5e. A very useful spell- but one that can bog a game down in background details. It adds 1d4 to every saving throw and to hit roll for up to three characters. Awesome! But using it means that all those macros each character is using to resolve combat has to be changed. Or you need to remember to roll an extra d4 with every attack and save. The shaped sheet has the capability to resolve that- but it means adding d4cs0cf0 in three spots on each of those three characters, and then removing them when the spell is over. If the players are not tech savy that means you have one or two people making the changes for everybody. It really slows the game down. Jakob's chatsetattr script allows the gm to make the needed change to all three character sheet in seconds. But since the sheet worker isn't triggered when a script makes a change you still need to go into each sheet manually to trigger the recalc.&nbsp; This may not sound like an important update- but to the people using the scripts it would really make life easier.&nbsp; Please roll out the fix. If something is preventing you from doing so please explain the problem. And regardless, please come up with a way to make this happen!

Gary W. said: I wonder if they realize how much of a quality of life type of improvement it would be? The changes over the past year, combined with some incredibly savvy programming, have brought us to the a new level of sophistication in our games. Taken the drudgery out of the game and allowed us to focus on the story and the joy of being with friends.&nbsp; The fact that a script can make a change to a character sheet and not have that change noticed by the sheet worker is a big pain in the rear end.&nbsp; Take casting bless in DnD 5e. A very useful spell- but one that can bog a game down in background details. It adds 1d4 to every saving throw and to hit roll for up to three characters. Awesome! But using it means that all those macros each character is using to resolve combat has to be changed. Or you need to remember to roll an extra d4 with every attack and save. The shaped sheet has the capability to resolve that- but it means adding d4cs0cf0 in three spots on each of those three characters, and then removing them when the spell is over. If the players are not tech savy that means you have one or two people making the changes for everybody. It really slows the game down. Jakob's chatsetattr script allows the gm to make the needed change to all three character sheet in seconds. But since the sheet worker isn't triggered when a script makes a change you still need to go into each sheet manually to trigger the recalc.&nbsp; This may not sound like an important update- but to the people using the scripts it would really make life easier.&nbsp; Please roll out the fix. If something is preventing you from doing so please explain the problem. And regardless, please come up with a way to make this happen! I agree with you on nearly every point, but from what I have heard and read the issue that is holding it up is how long it takes to process, how much it will slow the sheets down. &nbsp;Now, its my guess that if they think it is a big enough issue to mention as the hold up, then the slow down must be quite dramatic. &nbsp;Something like the 7-10 seconds delay that we had when nested inline rolls first made it to beta and a lesser extent when those were pushed to live.. Since they pushed that to live servers even WHEN they were still very slow to execute (and to some extent still are on some computers); this problem must be dramatic. Granted, I haven't seen this in action, so I could be totally wrong. &nbsp;If that is the case, then yes. &nbsp;I agree, this should be priority number one, get it out there and let people use it. &nbsp;But beyond that, I have faith that when it is time, when it has been perfected, it will be released.