Searching the Forum [tech]

Go here: <a href="http://menagerie.team/search.html" rel="nofollow">http://menagerie.team/search.html</a> - also has buttons for searching the main tags.

It only mattered for a couple of threads, but the search now correctly handles multi-page threads.

This thing is invaluable. I have been using it a lot. Thanks! (Are there any special formats or search terms? It doesn't seem so, but I haven't experimented exhaustively.)

Going to the page and not searching for anything will show you some typical uses (e.g. "keyword1 keyword2").&nbsp;<a href="https://lunrjs.com/guides/searching.html" rel="nofollow">https://lunrjs.com/guides/searching.html</a> has more specifics.

Ah. I knew I had seen that but couldn't figure out how to bring it back up: keyword1 keyword2 Find keyword1 OR keyword2. Favors matches that have more keywords. keyword* Wildcard (*) matches at the start, middle, or end of a search term. keyword1^10 keyword2 Boost the relative power of one keyword compared to others. chin~1 Fuzzy match keywords, number indicates edit distance (e.g. chin or chan).

Good News Everybody! This is something you could technically do before, but it was boring. Now, though, I added an author field! So you can do THIS: field:keyword1 field:keyword2 Find keyword1 OR keyword2 in specific fields. Valid fields are 'title', 'author', and 'body'. Example: 'author:dave body:Alycia' finds everything from Dave about Alycia. All the rest of the rules work as well. Example: 'author:Dave chin~1' gives you all the Chin or Chan matches from Dave. NOTE: you can filter on body: to omit title from consideration, e.g. "body:numina" pulls up everything naming her in the actual comment, but not each and every post in the "Jason and Numina" mega-thread. You probably want to use this for common terms. You can also do searches for all posts, e.g. 'author:james' pulls up his whole post collection.

That's fantastic, Bill. Thanks!

Very cool, thanks!

Occasionally I would mess up and use from: instead of author: and it would fail silently, so now there is an error message if you somehow trigger a search error. I also updated to a newer version of lunr.

I can't tell you how much I appreciate this tool. (Semi-related - I'm nudging the admin people, again, about expanding forum admin powers to players. It's all apparently linked to the game creator, and as near as I can tell, few admin abilities can be shared or transferred.)

The only thing we've needed admin for, as far as I know, is tagging posts after they've been created. If you look over in Post Tagging, I pitched (and Dave supported) an alternative thing which I'm implementing soon. Basically you can either [Tag] in post title, or put #Tag in the post body on its own line. Example: #Tech #Jalycia And the script will notice that and attach those tags to this post. Anyone who can edit their own post can then add tags after the fact.

Bill G. said: Anyone who can edit their own post can then add tags after the fact. Awesomesauce. Thank you. I use the search tool a lot, though the tags not as often. This is a nice expansion of that capability. Along these lines, Doyce has handed me forum admin rights through his GM fu, so I can break everything tweak titles if done in a timely fashion. (The titles do turn into the URLs, so if something gets linked to a post before it gets renamed, then it seems like those URLs would break. Unless it automatically generates a redirect.)

The URLs should still work, no matter what you do with the title. Look at this one for example, broken into its relevant components: app.roll20.net/forum/post/5815818/searching-whatever The roll20 app will look at the /forum/post/&lt;number&gt;/&lt;extra&gt; part. That '&lt;number&gt;' bit is the primary key in a database, representing this particular thread, so it goes and looks that up. Absolutely everything after that in the URL is garbage. Try this URL for example: <a href="https://app.roll20.net/forum/post/5815818/jalycia-" rel="nofollow">https://app.roll20.net/forum/post/5815818/jalycia-</a>...

The index builder needs a kick. Last update 2/24, and there has certainly be (cough) some activity over the last week on the Forums.

It's release day at work, I can do it tonight. This week has been pretty crazy.

No problem. Work definitely comes first.

Forum search has been unavailable for awhile due to either a permission change on our forums, or a bug in Roll20. Getting the forum page without being logged in gives a Ruby on Rails/CanCan error: NoMethodError: undefined method `role' for nil:NilClass Doyce, can you check that the forums are still set to be publicly visible?

I was able to search a couple of days ago ("purple"), though the indexing may have been down for a while.

Yeah sorry, I meant the indexer itself. The search is still available, it's just going off of old data.

YUCK. And... yes, it's still set public. I hadn't fiddled with any other settings, either. Grr.

So ... this is still broken? (Asking for a former character.)

It's still broken. I'm checking literally every day, no response from @Roll20App on twitter either. This weekend I might switch over to node-crawler or something, then write an authorization dingus with my username/password because at least that works.

Well, anything you can do would be appreciated, within the bounds of "Um, hey, this is a game, none of this is mission critical, only nifty." No heroic or life-consuming measures needed.

Progress is being made.

Still a few glitches. Haven't re-implemented pagination through long individual posts because Roll20 treats anonymous vs. logged in users COMPLETELY DIFFERENTLY for those, and my old way of doing it assumes anonymous users. Looks like it'll be done by tomorrow, with luck. I don't know how many people would care about this, but if I thought I could get away with it, I could start a Paypal/donation-based site that let any campaign in the world crawl their game forums, and feed that into a nice Elasticsearch instance. I lack the time to market that kind of solution to its intended audience, tho, and no idea how many people would pay real money for game stuff like this.

I think that a game only gets forums if the GM running the game is paying to be a member of roll20, so by definition I think you'd already be aiming it at people who are paying for their game and content.

I would sooner advise GMs that they should avoid these profoundly lame forums and use any number of alternatives. If we didn't have a massive amount of threaded material here already, that's what I would be encouraging for us, too.

*** Dave H. said: I would sooner advise GMs that they should avoid these profoundly lame forums and use any number of alternatives. If we didn't have a massive amount of threaded material here already, that's what I would be encouraging for us, too. I mean on the one hand, yeah, it would be rough to migrate to some other forum at this point. On the other hand, if it came down to that, I've literally written a crawler that downloads the entire contents of the forums in text form... It wouldn't be technically too difficult to port to some other regime, extract out to Wordpress, or whatever.

I have no doubt it's technically possible (for some of us). I'm not sure it's worth the effort for the constant stream of low-level irritations. On the other hand, if I woke up tomorrow (or some morning) and someone had actually ported everything to a forum setup (one of the free forums that most ISPs offer, or WP, or G+, or something else I'm not thinking of) that supported everything we have, was searchable, and which supported a richer text setup than what we can manage, I'd buy the responsible party a beer or three. If it were doable and transferrable, I'd be happy to host and administer a WP site -- or, via my ISP,&nbsp; phpBB, SMF, myBB, AEF, Vanilla, punBB, XMB, fluxBB, Phorum, bbPress, ElkArte, FuddForum, miniBB, LayerBB, Carbon Forum, My Little Forum, Beehive, or TangoBB.&nbsp; Or maybe I should just be satisfied with what we have (which, really, meets most of our needs). Because I don't want to add more work to anyone.

This got delayed from last week due to a variety of factors (the code is only on my home machine, and I haven't been there very often). I made some progress this week and it's very nearly done - I'm taking the opportunity to simplify the code as well. It's 100% Javascript now.

Woot!

I took this opportunity to roll tags together, so now [Cutscene] or #Cutscene will both work the same way. I still need to add back a "sort by date descending" option, so it's now harder to get (for example) the list of cutscene posts only, but you can now (for example) search for '#Recap body:Streak' and get those session recaps at the top of the page. I consider that a worthwhile trade for now, and I'll add sort-by-date once I get the attendant UI difficulties worked out.

Thanks for all the hard work, Bill!&nbsp; That forum search is an invalable resource and it is much appreciated.

Sort by date is in, and (JUST AS PLANNED) works for absolutely any sort you throw at it, not just the 'show me Cutscene' links. So (for example) I can find all recaps that mention Alycia and see them in posted order. It's extra work and I'm reluctant to ask you guys to do it, but I plan to. I'm going to start tagging individual posts in hashtag style, and go back and fix my old posts that are missing [Cutscene] and the like. I don't have to edit the titles any more, I can just add #Cutscene at the end of the post.

Note: tags are still case-sensitive, so #rp is not the same as #RP. Post tagging:&nbsp;<a href="https://app.roll20.net/forum/post/5815741/post-tagging" rel="nofollow">https://app.roll20.net/forum/post/5815741/post-tagging</a>

Bill G. said: Note: tags are still case-sensitive, so #rp is not the same as #RP. Post tagging:&nbsp; <a href="https://app.roll20.net/forum/post/5815741/post-tagging" rel="nofollow">https://app.roll20.net/forum/post/5815741/post-tagging</a> Of course they are. Certainly. I mean, everyone should know that, amirite? (Scurries off to fix poorly-cased hashtags ...)

Basically if the title already has the right tag, it's ok to leave it be. Also, only tag the first relevant post, e.g. I added Cutscene to some of the threads that turned into Summer RP.

Well, I like the tagless titles, but I won't go back any further than I have. :-)

Some changes to the forum code are out now, most of which are around speed and performance on the backend. But a few front-end items: Fixed a bug where multiple tag searches wouldn't work (e.g. #Cutscene #RP) Clicking on a tag from the right side now adds it to an existing search, if there is one (e.g. type 'Jason', click '#Cutscene', new search becomes 'Jason #Cutscene') Sort-by dropdown now correctly disabled when the index isn't loaded yet Laid the groundwork for keyword highlighting and intelligent truncation (e.g. search for 'keynome' and the text summary for a match shows you truncated text around a highlighted 'keynome')

Remarkable. Thanks!

This process also gives us some data to work with. Since I work at Tableau, I decided to try creating a viz. I'm missing a few fields of interest (like post date), but I use post ID as a proxy for that, since it can be assumed to increment at a steady pace over time as people post stuff. Tags are also not authoritative (I'm making it easier on search results, at the expense of accuracy). So with those caveats, here we go! Unsurprisingly, Dave and I are churning out the most forum content, a combination of teen drama shenanigans and palaver. Even including Jason/Alycia's Tale recap posts (marked with the Recap tag), Cutscene outnumbers Recap like 3 to 1. I also took a crack at analyzing post length (overall - not filtered by tag, so this includes the one-sentence replies to our behemoth YA novels). Setting aside anything below 2700 characters, we've only written a handful of doorstoppers.

Clearly I need to write more in order to beat Bill... But, in all seriousness, this is interesting stuff. Thanks.&nbsp;

*** Dave H. said: Clearly I need to write more in order to beat Bill... Too late! Total number of characters in all posts by author. Seriously, this is why I love working at a company like this.

So ... you write more posts, but I run on at the mouth more. Makes sense. :-)

I am completely unsurprising by any this info (I am the most laconic of all the regular posters?&nbsp; Unthinkable! :P), but is it definitely cool to see.&nbsp; Thanks Bill! (Now if there was only something that tracked how many times I started typing a post only to stop and say, "eh, no one wants to read this" and stopped.&nbsp; Actually, I'd rather not like to think about how big that number is.)

Mike said: (Now if there was only something that tracked how many times I started typing a post only to stop and say, "eh, no one wants to read this" and stopped.&nbsp; Actually, I'd rather not like to think about how big that number is.) Well, FWIW, there hasn't been anything you have written that I didn't enjoy reading. So either your judgment in sorting out what others want to read is uncannily accurate, or you need to post more of those writings. :-)