Setting a watchdog to sheetworkers... Not worker on first sheet opened

Hello all, OK, my context is that I have a custom character sheet, that has a companion API. Because not everyone is pro, there is a flag attr_API with value 1 if the API is active, and 0 if not, which hides/shows some stuff through CSS. Initially we had it pretty simple, where the API was setting the attr_API to 1 on a add:character, but because the API is... unstable, sometimes we were missing some characters that had been character while the API was crashed... So we thought about a more "clever" communication between sheet and API where: in addition API==0 ( no API) and API == 1 (API active) there would be a API==2 (API has been previously detected, but can't be detected at the moment). Idea was that each time a sheet would be opened, on(sheet:opened) the sheetworker would set API==2, and API_max to a random value (to be sure to trigger a change) In the API, on(change:API) the API would write back 1, in attr_API... This ... works very well once sheet has been opened at least once... I even have a select set in the HTML where I can set attr_API value and I can see the API immediately setting it back to 1... Effectively everytime I open an already existing sheet, the API flag is set correctly BUT (and this is a big but), it doesn't initialize correctly on a new sheet. The crazy things is that all the logs show me that the API detects the new sheet, that the routines trigger the API response, and that the API actually tries to write in the sheet... But nothing happens on the sheet. When I try to change my attr_API manually, I can see from the logs that the APi detects the change, and thinks it has written it back to 1... I even have a get in the log to check back the value, and it sees it as written... But in the sheet the value has not changed back to 1... Not sure my explanation is clear, but is there anything in the first opening of a new character that would make the API unable to write an attribute (or the HTML unable to read the values written by API)...

I have a similar issue with my sheet trying to detect companion API. Not quite as sophisticated, but I observe the same difficulties.

Have you tried a setTimeout with a high value? Start stupidly high, like 10000ms - the mod sandbox might be jumping in too early.

I guess my question for this is "why?". The better user experience I think is to simply make the API features manually toggleable and put the option in the game defaults via the sheet.json. That said, the exact nature of your problem is going to depend a great deal on what exactly your code looks like.

@Laurent : good to know :( I am not alone @Oosh : yes, it is already part of the things tried already, I'll try to paste some code extracts @Scott : For me it is better experience if the user doesn't have to know that he has a manual setting to take care of... but the manual toggle and the option in the sheet.json is the current we have before trying to have this function... OK, some code  In the Sheetworker, on sheet opened, set the API toggle to 2 (I have pinged the API and waiting for a pong), and API_max to 12 or 2 (this ensures that there is some kind of heartbeat so that there is no lack of detection by the API due to having missed the changed value on API // Update the API status according to watchdog from the API var checkWatchdog = function fncheckWatchdog(){ 'use strict'; try{ getAttrs(["edition_max", "NPC", "API", "API_max"], function gacheckWatchdog(values) { 'use strict'; logvalues( values); log("API Watchdog " +values["API"] + " max "+ values["API_max"]) //API Watchdog - Set to NoAPI no matter what, if the API is active it will reset to API. //Temporarily disabled until API is released let vals={}; switch( values[ "API" ] ) { case undefined : vals["API"]="0"; case "0": case 0: if( values[ "API_max" ] == "10" ) vals[ "API_max" ] = "0"; else vals[ "API_max" ] = "10"; break; case "1": case 1: vals[ "API" ] = "2"; vals[ "API_max" ] = "2"; break; case "2": case 2: if( values[ "API_max" ] == "2" ) vals[ "API_max" ] = "12"; else vals[ "API_max" ] = "2"; } setAttrsLog( vals ); }); } catch(err) {log("Earthdawn:CheckWatchdog() error caught: " + err );} };//end CheckWatchdog/ // When a sheet is first opened, see if it needs to be updated from a previous version. on('sheet:opened',function fnOnSheetOpened(){ 'use strict'; try{ // edition_max is character sheet version getAttrs(["edition_max", "NPC"], function gaSheetOpened(values) { 'use strict'; logvalues( values); checkWatchdog(); var newSheetVersion = SheetVersion; // The Version number is set as a global variable at the beginning of the sheet-worker. It is changed with every revision of the sheet that needs to call this routine. Also updated twice above and in .js file. var oldSheetVersion = parseFloat( values[ "edition_max" ] ); if( oldSheetVersion != newSheetVersion ) { updateCharactersheet( newSheetVersion, oldSheetVersion, oldSheetVersion, 0, values, 0, "" ); } // End there are updates to perform }); recalc(); // recalc on sheet opened whether updated anything or not. } catch(err) {log("Earthdawn:OnSheetOpened() error caught: " + err );} });//end OnSheetOpened In the API, a routine checks for changes /addition of attributes, and triggers a response depending on which attribute it is... If it is API (i.e. the watchdog coming from the sheetworker), respond to it on("add:attribute", function (attr) { 'use strict'; Earthdawn.attribute( attr ); }); // change attribute. See if it needs some special processing. on("change:attribute", function (attr, prev) { 'use strict'; Earthdawn.attribute( attr, prev ); }); // end on("change:attribute" // An attribute for some character has changed. See if it is one that needs special processing and do it. Earthdawn.attribute = function ( attr, prev ) { 'use strict'; try { log( attr); // use attr.get("name") and attr.get("current"). // {"name":"Wounds","current":"1","max":8,"_id":"-MlqexKD2f4f744TgzlK","_type":"attribute","_characterid":"-MlqeuXlNO51-RYxmJv8"} //log( prev); // use prev["name"] and prev["current"] // {"name":"Wounds","current":0,"max":8,"_id":"-MlqexKD2f4f744TgzlK","_type":"attribute","_characterid":"-MlqeuXlNO51-RYxmJv8"} let sa = attr.get( "name" ), cID = attr.get( "_characterid" ); if( sa === "API" ) { // API or API_max has changed. If they have not changed to "1", set them to "1". if( attr.get( "current" ) !== "1" ) { Earthdawn.setWithWorker( attr, "current", "1", "1" ); Earthdawn.setWithWorker( attr, "max", "1", "1" ); log("set API watchdog from API change") } } Another routine checks for new characters, and after a timeout (to let the sheetworker do its stuff) sets some initializations   on("add:character", function( obj ) {     // Brand new character. Make sure that certain important attributes fully exist.     'use strict';     let ED = new Earthdawn.EDclass();     ED.newCharacter( obj.get( "_id" ) );   });         // This routine runs when a new character is added.         // It can be called from events "on character add" when one is created manually, or         // It can be called from WelcomePackage when it makes one.         // Note, it also seem to be triggered on character add, when a character is imported from the character vault (in which case it is not truely a new character!).   this.newCharacter = function( cID )  {     'use strict';     let edc = this;     try {       setTimeout(function() {       // When a character is imported from the character vault, the recalc caused by edition has been observed to cause a race condition. Delay this processig long enough for the import to have been done.         try {           Earthdawn.errorLog ( "newchar test ", this);           let npc = ( typeof WelcomePackage === 'undefined' ) ? Earthdawn.charType.pc : Earthdawn.charType.npc;           let plr = findObjs({ _type: "player", _online: true });     // If there is only one person on-line, that is the player.           if( plr && plr.length === 1 )             npc = playerIsGM( plr[0].get( "_id")) ? Earthdawn.charType.npc : Earthdawn.charType.pc;           let CharObj = getObj( "character", cID);     // See if we can put a default value in the player name.           if ( CharObj ) {             let lst = CharObj.get( "controlledby" );             let arr = _.without( lst.split( "," ), "all" );             if( arr && arr.length === 1 && arr[ 0 ] !== "" ) {     // If there is only one person who can control the character, use their name.               let pObj = getObj( "player", arr[ 0 ]);               if( pObj ) {                 Earthdawn.SetDefaultAttribute( cID, "player-name", pObj.get( "_displayname" ));                 let attribute = Earthdawn.findOrMakeObj({_type: 'attribute', _characterid: cID, name: "playerWho"});                 Earthdawn.set( attribute, "current", pObj.get( "_displayname" ));                 Earthdawn.set( attribute, "max", pObj.get( "_id" ));               }               npc = playerIsGM( arr[ 0 ] ) ? Earthdawn.charType.npc : Earthdawn.charType.pc;    // character was created by welcome package, if for GM, make it an NPC else PC.           } }           let attr = Earthdawn.findOrMakeObj({_type: 'attribute', _characterid: cID, name: "API"});           attr.set( "current", "1"); //set the API running           attr.set( "max", "1");           log("set API watchdog from newChar "+attr.get("current"));                     // If a character was created by Welcome package for a Player, or if Welcome Package is not installed, default to PC.                     // If a character was created by Welcome package for a GM, or if Welcome Package is installed, default to NPC.           Earthdawn.SetDefaultAttribute( cID, "NPC", npc ? npc : Earthdawn.charType.pc );           Earthdawn.SetDefaultAttribute( cID, "RollType", (state.Earthdawn.defRolltype & (( npc === Earthdawn.charType.pc ) ? 0x04 : 0x01)) ? "/w gm" : " " );         } catch(err) { Earthdawn.errorLog( "ED.newCharacter setTimeout() error caught: " + err, edc ); }       }, 10000);    // end delay 10 seconds.     } catch(err) { Earthdawn.errorLog( "newCharacter error caught: " + err, edc ); }   };     // End newCharacter; As you can see in this test code, I have added logs to try to understand, and I can actually see when creating a new character that the very last log is  set API watchdog from newChar 1 But still the value of the API is 2. I also have a manual toggle, just to see what the actual status of API is... When all works correctly, I can change this toggle to 2, and see instantaneously the API triggering and setting it back to 1... BUT when it is a brand new sheet (until I close and reopen it), I can set it to 2 and nothing happens. BUT I can see from the same Set API watchdog from newChar 1 log that not only was the event seen, not only did it try to set the attribute, but even when it reads back the attribute, with attr.get("current") it sees back the 1 it just wrote... But still on the sheet itself, the Attribute is 2, as if the communication between the API (which to my understanding runs on the servers) and the HTML (that to my understanding runs on my computer client) was somehow broken... Ideas ? 

Ok, you've abstracted out the part I actually needed to see into your earthdawn class. Not really a place I would have done that, but :shrug:. What does your earthDawn.setWithWorker function look like?

You have a limited number of functional paths regarding the way a sheet and a companion script can interact... and that is going to start reducing your returns from trying to artifice a solution, I think. If I was having trouble with the API setting an attribute on the character sheet, or if I was concerned that sheets were being added/edited while the API was down, I would build a function to call in the on('ready') event. I would build a library of characters-who-pass, and I would pass that library into the same function on an async timeout: const attrBank = {     API: 1,     sheet_version: 1.2,     someotherattribute: 'marmalade',     as_many_as_you_want_to_check: 'something else' }; const responseBank = {     sheet_version: (a, v) => { /* code here to upgrade the sheet */ },     default: (a, v) => a.set({ current: attrBank[v] }); }; const checkChars = (known = []) => {     charids = known.map(c => c.id);     let runagain = false;     findObjs({ type: 'character' }).reject(c => charids.includes(c.id)).forEach(c => {         let chartest = Object.keys(attrBank).reduce((m,v) => {             let a = findObjs({ type: 'attribute', name: v, characterid: c.id }) || creatObj({ type: 'attribute', name: v, characterid: c.id });             if (a.get('current') !== attrBank[v]) {                 m = true;                 responseBank.hasOwnProperty(v) ? responseBank[v](a, attrBank[v]) : responseBank['default'](a, attrBank[v]);             }             return m;         }, false);         if (!chartest) known.push(c);         runagain = runagain || chartest;     });     if (runagain) setTimeout(checkChars[known],250); }; on('ready', () => {     checkChars(); }); Basically I don't care about whether the sheet knows about the API until the sandbox is up. Then I start building my list of characters whose attributes test out (according to my attribute bank). If anything needs to be set, I run the associated function in the responseBank (to set and or do things like configure the sheet), and I trip a boolean that will launch the same function again after some timeout (here, about a quarter second). This will set every sheet in the game once the API starts up. The sheet can still do things on its own when it opens, but things like configuring the sheet version and detecting the API should be done from this side. (If you're going to wholesale upgrade the sheet, you can do that from outside the game, and not worry about automating a process within the game. This sort of upgrade would only make sense if the API were being updated, and there were sheet changes that were required because of it.) The problem with this approach is that if you transmog a character from a game where there is an API to one where there is *not*, all of the sheets will still see a value where the API attribute is set to "true". Your answer will probably be to reset the API attribute on the sheet when the sheet loads... but then that basically steps on the above (which runs whether or not the sheet is open -- at sandbox spin-up). You could call the above function from your listener for an attribute-change ping coming from the sheet, but it would only work if you were dynamically displaying content in response to the value of the attribute... Feels like a long way to go for a small reward that will never be perfect because of the interaction of the sheet/api and/or the transmog problem.

Ok, you've abstracted out the part I actually needed to see into your earthdawn class. Not really a place I would have done that, but :shrug:. What does your earthDawn.setWithWorker function look like? @Scott, the Earthdawn class holds our utility routine... Why ? I don't know, joined the project on the go :) We have plenty of functions that are designed to do the same as native function (like setwithworker), but with some extra safeties regarding undefined objects, NaN, etc. Earthdawn.setWithWorker = function( obj, att, val, dflt ) { 'use strict'; try { // log( "setww " + obj.get("name") + " val " + val); if(( val === undefined && dflt != undefined ) || (val !== val)) { // val !== val is the only way to test for it equaling NaN. Can't use isNan() because many values are not supposed to be numbers. But we do want to test for val having been set to NaN. log( Earthdawn.timeStamp() + "Warning!!! Earthdawn:setWithWorker() Attempting to set '" + att + "' to " + val + " setting to '" + dflt + "' instead. Object is ..."); log( obj ); obj.setWithWorker( att, (dflt === undefined) ? "" : dflt ); } else obj.setWithWorker( att, (val === undefined) ? "" : val ); } catch(err) { log( Earthdawn.timeStamp() + "Earthdawn:setWithWorker() error caught: " + err ); } } // end of setWithWorker()

If I was having trouble with the API setting an attribute on the character sheet, or if I was concerned that sheets were being added/edited while the API was down, I would build a function to call in the on('ready') event. In fact we have already this function, and it works well... BUT what it does is set all the characters at startup of the API... What it doesn't do is: Set the API setting for the new characters Warn the user if the API is down For the 1, I know that you can have it as a default parameter in the sheet.json... But it is one more manual setting the user has to set right in order to get things working... For the 2, I know that there are some watchdog API, but once trying to achieve the 1 (i.e. cheking if the API is active once opening a sheet), it was straightforward to do the 2. This also adresses the case you were giving of transmogging a sheet from an API game to a non API game... A message appears "No API detected, either it is down or forget that there ever has been one" Feels like a long way to go for a small reward that will never be perfect because of the interaction of the sheet/api and/or the transmog problem. It always end of feeling like this when the way is too long... Sometimes you think the way will be short, and so it's worth... I begin to consider to give up on this...

Jiboux said: What it doesn't do is: Set the API setting for the new characters Warn the user if the API is down For 1, you either have characters getting added while the API is down, or you have them added while the API is up. If they are added while the API is down, the script will fix that as soon as it starts up. If the characters are added while the sandbox is up, then you can call this function from your event listener for adding the character. Since it runs the timeout in a loop until the character is satisfactorily setup, it will just keep going until it gets the job done. As for 2... warning about whether the sandbox is down feels like you're introducing the "one more thing the user has to manually handle"-thing... while trying to avoid the "one more thing the user has to manually handle"-thing. Whether the sandbox is up or not really doesn't have any bearing on the sheet other than lost functionality... which is an argument for early-awareness on the part of a gm (or someone with the capability of rebooting the sandbox)... but which really isn't an argument, in my opinion, for solving this awareness issue in the sheet. The bottom line is that for many reasons -- loss of sheet functionality in your character sheet only one of them -- if the sandbox is down the gm is going to want to get that back up as soon as possible. The easiest way to do that is something like APIHeartbeat or some sort of a dead-man's switch arrangment -- send a message with a setTimeout that checks to see if the API is up or if there is a delay order sitting in its script (so long as the sandbox is up, the delay is there and pushes the message out). Something like that could give an immediate indication that the sandbox has crashed... and it is handled in the script. Your sheet doesn't have to worry about the sandbox going down any more than it worries in a game where there is no sandbox to begin with. It just needs to know whether the sandbox is there at all, so it knows what to display.

For 1, you either have characters getting added while the API is down, or you have them added while the API is up. If they are added while the API is down, the script will fix that as soon as it starts up. If the characters are added while the sandbox is up, then you can call this function from your event listener for adding the character. Since it runs the timeout in a loop until the character is satisfactorily setup, it will just keep going until it gets the job done. My biggest problem still, is that when creating a new character, despite the event listener on the API side, and the API seeming to actually write the attribute, the HTML doesn't seem to see it... This results, in new sheets that stay in NoAPI mode the first time they are opened, and turning back to the right mode once closed and reopened... Which is kind of ugly... I guess it leads me back to no choice than rely on the manual json default value, even if I don't like it

OK, while working on another bug, I think I found a common trend, that may explain what doesn't work with my watchdog... I made the following test code in the HTML         APIbug:         <input name="attr_testAPIbug" type="number" value="0" class="sheet-numShort" title="test1:   Worker Thread test - change this and see if the following one changes as well. If not the worker threads are not working, probably due to parser not being able to figure out javascript.">         max:         <input name="attr_testAPIbug_max" type="number" value="0" class="sheet-numShort" title="test1:   Worker Thread test - change this and see if the following one changes as well. If not the worker threads are not working, probably due to parser not being able to figure out javascript."> and in the API: on("ready", function() {   'use strict';   log("ready");   on("change:attribute", function (attr,prev) {     'use strict';   let sa = attr.get( "name" );   log("Value has change " + sa);   if(sa==="testAPIbug" && (attr.get("current") !== attr.get("max"))){     log("Pinged by the value " + attr.get("current"));     attr.setWithWorker(  "max", attr.get("current"));     log("Ponged with the value " + attr.get("max")); } }); // end  on("change:attribute" and in the sheetworker on("change:testapibug_max", function(eventInfo){ log("Pong was detected by sheetworker" + JSON.stringify(eventInfo)); }); Pretty straightforward: a test variable, set in the HTML, the change detected  by the API, that matches the change in another variable that is also displayed, with the logs to tell me where it seems to go wrong if anything goes wrong If now I try this test code with an existing character, all behaves as it should, i.e. when I change testAPIbug, testAPIbug_max changes to the same value and I receive the following log "Value has change testAPIbug" "Pinged by the value 1" "Ponged wth the value 1" "Pong was detected by sheetworker{\"sourceAttribute\":\"testAPIbug_max\",\"sourceType\":\"api\",\"triggerName\":\"testapibug_max\",\"previousValue\":\"0\",\"newValue\":\"1\"}" If now, I try the exact same thing with a new character sheet (that has just been created, never closed and reopened, I receive the exact same log, but in my browser, the value of testAPIbug_max is still 0 "Value has change testAPIbug" "Pinged by the value 10" "Ponged wth the value 10" "Pong was detected by sheetworker{\"sourceAttribute\":\"testAPIbug_max\",\"sourceType\":\"api\",\"triggerName\":\"testapibug_max\",\"newValue\":\"10\"}" So the APi has seen, the change, has written the value, is able to read back what it wrote, the sheetworker even saw what the API wrote... But it is not displayed... Now, if I close and reopen the sheet, the value is still wrong, and If I look in the Attributes of the character, there is no max value... So it looks like until the sheet has been closed and reopened a first time, some changes that the API makes to the attribute actually do happen, but are not sent to the sheet. What do you think ? It is maybe worth a new thread on the Bug forum ?

I don't have time to dig too deeply into it, but there's definitely something fishy going on there. Until there's some human interaction with the sheet - opening it (or @{character|attribute} in chat) the changes aren't persisting. I'm not sure what the sandbox setup looks like, but assuming it looks roughly like a player's browser instance, my guess is that the sandbox merely modifies its local copy of firebase (just as players do), but the changes aren't permitted to persist to the server until the sheet is 'unlocked' by a human. It might have been a precaution against the sandbox messing up new sheets, but somewhere along the line a timer or secondary condition to allow firebase saves has been broken. Or maybe it's always been like this. Either way, 'add:character' probably isn't a great event to be listening for. If the GM is adding a sheet using the UI, the event fires way before 'sheet:opened', so nothing in the character sheet sandbox initialisation will have happened yet. I haven't tested this, but if a player imports a character into the game while there's no one in it, I'm assuming no event at all will fire, since the sandbox is dead. It would all be much easier to handle with custom roll parsing - then again, if you used CRP you likely wouldn't need an API script anyway.

Your idea of human interaction made me think of something to test… instead of on(add:character I could test the value on(change:attribute and if it is incorrect set it at this moment. this way it would be triggered at the moment the user interacts with the sheet.  I ll test that later

OK popping up this subject again, and found another interesting fact that may be of interest... I have now in the API literally TONS of events that trigger to set the "API" value that tells my sheet if the Companion App is running or not... I know from the logs these events are triggering, but on the first opening of the sheet, the firebase database sees to be "offline" from the server, so it still shows me the default "NoAPI" value (i.e. the default value of my attr_API, that is NoAPI... Note that I agree that it could be solved by the JSON default value, but I am precisely testing a new version of my sheet on the live server, so no JSON, which is a good demonstration on how it can be annoying to a user)... I noticed one thing interesting thing though: The sequence : Create New Character > Save > Click Character Sheet : leaves me with an "unsynchronzied Firebase" The sequence Create New Character > Save > Click Attribute & Abilities  > Click Character Sheet :  I get the value I expect from the API Is there any way I could use that to my advantage to force an update of the firebase ?