Hello everyone,
I want to get some info out of a roll20 api I wrote myself. I saw this post from some years ago (https://app.roll20.net/forum/post/6503274/slug%7D) but there are few details in there so I sent a message to the author to see if he could give me some insight of how he did it. In the meantime, do you know a way to do it?
if you want to show data in a livestream, you can make a dedicated page/map for a ‘stream’ user that you capture in OBS (or another tool). ‘Invisibile’ tokens can be used to display hp bars and such. Also text items can be changed from an api script, tokens moved between layers efc. I posted the token trick in this forum, but cannot find it anymore.
If you just want data OUT of roll20, there are some ways of doing this, but all of them i would categorize as workarounds as there is no formal way of doing this.
a) adapt your api to update the data in a handout. Handouts can be opened in a seperate browser window (but need to be F5-ed constantly)
b) adapt your api to write to the api log. There are some (cumbersome) ways to read the api logs.
if you want to automate say a twitch chat INTO roll20, thats a whole other thing.
The tidy way to do it would be to hook straight in to firebase and get the same messages the API is seeing. That would require some knowledge of firebase, and writing an extension. That could be a lot to learn, if it's even possible...
As Martijn said, you can send it to the API log and read it from there. It's probably no going to be super reliable, but:
That's a Mutation Observer. It observes a node and reports any changes, which you can then sort through to find the event you're looking for. In this case, anything sent to the API log starting with "!export " will be grabbed and logged to the browser console.
There's a bunch of issues with this approach - it's only observing HTML, so if the API console stops scrolling, the Observer won't be seeing any events. The API console also updates every line each time you scroll, so if you scroll up and down the Observer sees a crapload of new lines and has no idea that they're 'old' lines.
You could probably add some jQuery in there to disable scrolling up, and ensure the log is always fully scrolled down so you don't miss any output.
You could also set the Observer to the actual R20 chat log, but you can only pick up HTML that's posted to that element, which means no !API calls.
All in all, there's no easy way to do it. Unless you're very familiar with the systems involved, in which case you probably wouldn't be asking :)
Thank you for your kind answers. I knew it wasnt going to be simple but I think I can understand it.
I just discovered the existence of the browser console and it seems to be the solution im looking for. From what I understand, the process would be the following: First I send my message to the roll20 api log (ex.: console.log(`!export msg`), then this message is reflected in the browser console and at last a mutation observer running in my side reads the message from the browser console.
Am I understanding it correctly? I will starting working on it and will post the updates. Once again, thank you.
Edit 1: console.log('msg') is not showing anything in the browser console, am I missing anything?
Master said:
I just discovered the existence of the browser console and it seems to be the solution im looking for. From what I understand, the process would be the following: First I send my message to the roll20 api log (ex.: console.log(`!export msg`), then this message is reflected in the browser console and at last a mutation observer running in my side reads the message from the browser console.
Am I understanding it correctly? I will starting working on it and will post the updates. Once again, thank you
Not quite - it's the move from the API log to the browser console that needs the mutation observer (or some other method). The API processes code in its sandbox, and sends the return back to the Campaign. It doesn't run locally on the DM's PC, so you can't get browser logging out of it. Think of it like an extra player with co-DM privileges - you ask them to do some calculations and whisper the result back, or move a token, etc. Because this processing is being done on "someone else's computer" it has no interaction with your browser instance or browser console. It returns data to two places: the Campaign's API Console (directly? not sure) or the Campaign's actual game instance (via firebase transactions), where it can take pretty much the same actions as a GM - post to chat, move a token - I believe it has unrestricted permissions on the database.
So the tricky part is getting from the API log (or the Campaign chat log) to your browser console - that's what the Mutation Observer is for. Once you have the data in your browser instance, you can use it in your own scripts.
API => log() to API Console => Mutation Observer reads HTML from API Console => Mutation Observer sends to browser console
If you write a browser extension, you have yet another scope: the background script - this cannot read data directly from the current page's Window, and would need a Window-scoped script injected to pass the data to the background. So, depending on what you actually want to do with this API data, you might be looking at passing it through more interfaces to do something useful with it.
This isn't a simple task to get working cleanly, as Roll20's API wasn't written as an external interface. Quite the opposite, in fact.
Have experimented last year with
Firebase.enableLogging((msg) => console.log(msg));
which is somewhat faster than the observer code of Oosh, but it is also a hassle as you need a second browser open and need go ensure the code is added to the api console page.
I used it to capture debug logging, but most of the time the lines that i really needed were lost.
Ok I think I understand it now, the api output console <div> id is "apiconsole" and the mutation observer is observing if there is a change in the childList of that element. When the change happens, you check wether the new child has the rxtrigger and if it does you send the msg to the console.
I dont quite understand how to work with firebase so I will first try the mutation observer.
Thank you all for your assistance. I will post my updates.
Edit 1: Just to make sure, the mutation observer runs in a program I run locally, right? But I do not understand why would I make console.log for the msg if I already have the information in my local script.
Yep, that's pretty much right. That observer I just wrote to run in the browser console. That's fine for quick testing, but obviously a page refresh will completely wipe it.
The final console.log() is there to see if the data is coming through correctly, nothing more. You're free to assign the value to a variable or send it straight to the next function, of course, but I would be leaving some form of logging there to make sure you're not getting broken or corrupted strings.
Ok, after a quick look at Ace, there's a much smarter way to do this than reading the HTML:
Paste that in your browser console (on the API log page) then send some chat commands from in game. You should get the full message object in the browser console (of the API log tab, not the in-game tab).
You can then do whatever you want with the msg object. It contains the whole shebang, including selected tokens, playerId etc.
Obviously you can throw in a regex check or what-have-you if you just want to grab any logs with a keyword, like in the Observer example.
This doesn't have any of the limitations of the HTML scanning - it doesn't matter if the scrolling falls off the API log, and it shouldn't miss any lines either.
Update 1: I made it work but I had to replace "new MutationObserver" with "new WebKitMutationObserver".
Now I will try to download the string as a text file so the program I run locally can read the information. Do you know of a better way to do that?
Oh, that looks like a more robust method. I will try it.
Edit 1: It works like a glove.
Oosh said:
Ok, after a quick look at Ace, there's a much smarter way to do this than reading the HTML:
const apiLogReader = (() => {let aceEditor = ace.edit('apiconsole');if (aceEditor) aceEditor.on('change', (ev) => checkOutput(ev));else console.warn(`API Console element not found, event handler not running.`)const checkOutput = (ev) => {if (/insert/i.test(ev.action) && ev.lines.length) {let msg;try { msg = JSON.parse(ev.lines[ev.lines.length-1]) }catch(e) { console.warn(e) }if (msg) console.info(msg);}}})();
Paste that in your browser console (on the API log page) then send some chat commands from in game. You should get the full message object in the browser console (of the API log tab, not the in-game tab).
You can then do whatever you want with the msg object. It contains the whole shebang, including selected tokens, playerId etc.
Obviously you can throw in a regex check or what-have-you if you just want to grab any logs with a keyword, like in the Observer example.
This doesn't have any of the limitations of the HTML scanning - it doesn't matter if the scrolling falls off the API log, and it shouldn't miss any lines either.
Master said:
Update 1: I made it work but I had to replace "new MutationObserver" with "new WebKitMutationObserver".
Now I will try to download the string as a text file so the program I run locally can read the information. Do you know of a better way to do that?
Following this website (https://techtalkbook.com/export-data-from-the-chrome-browser-console/) I created a snippet that runs a function that downloads a file with the information from the console. I think that is all I needed, if everything works I will make a final update with the result.
Saving from the console with this method has the drawback that AFAIK it creates a new file every time. Each file (after the first) is numbered and a receiving program needs to find the newest file and also cleanup is required.
Indeed that is a problem but I cannot imagine a different method to get that information inside the other node js script I'm running.
Where is the Node script running? As Martijn says, saving to local is a pretty inefficient way to get data out of the browser due to security measures. AFAIK there's no way around this - good thing too, otherwise our computers would melt with malware a millisecond after connecting to the internet.
If your script is running on a server, you should be able to POST the data as long as your server is set up to receive. The Fetch API should be enough to do this - whether or not you want ot worry about a security setup totally depends on what you're doing - you'll likely need to do something about CORS restrictions as well - MDN should have info on disabling CORS checks for the POST.
If your script is running in browser somewhere, you should be able to use an extension background script to pass data between tabs/sites. There's an API for posting messages between content scope and background scope for this purpose somewhere on MDN.
It's a bit hard to know without knowing what you're trying to do with the data.
Also, if you don't need it live, and you're just record-keeping, you can just keep stuffing the messages into an Array, then export the whole thing manually in bulk. This reduces the issues with browser security of having to manually initiate any operation on the user's local file system.
The node script is running in my pc and the server is a localhost port (maybe the terminology is not correct, I'm new to this). What I am exactly doing is using jquery to do some opacity animations in the website and capturing the website with obs. The animations are triggered by the output of roll20.
I had not considered to run a websocket in the roll20 browser console to send the information directly to the localhost port, is that possible?
It's doable, but it goes way beyond normal use of Roll20:
- your webserver should be https with a valid certificate
- this webserver should add the header Access-Control-Allow-Origin: https://app.roll20.net/
It will probably be much easier to combine a discord streamkit widget with this trick https://app.roll20.net/forum/permalink/9667080/
and then do some compositing in OBS.
Hello again guys, I just wanted to make a final update. Against all odds, the thing is working and works fine. Yes, it creates a lot of txt files that I have to clean between sessions but besides that there isnt any drawback. Here is a short video I created to share with my table, please do not mind the aesthetics nor the spanish. Thank you all for your invaluable assistance.
https://imgur.com/a/f6aWL9v
