Another Dice Roller Thread (I know, but I have actual evidence)

Even with a large sample set, dice roller does not match expected Averages: Sample Size: 1867 clicks Metric Expected Recieved Results Standard Deviation 5.77 (+/- 0.005) 5.7935283 Fail Average 10.5 (+/- 0.005) 10.4866071 Pass Repeats (Same Number twice within set within 2above and 2 below) .05 (+/- 0.005) .18877551 Fail Spread: 1 0.05 (+/- 0.005) .054 Pass 2 0.05 (+/- 0.005) .048 Pass 3 0.05 (+/- 0.005) .045 Pass 4 0.05 (+/- 0.005) .049 Pass 5 0.05 (+/- 0.005) .048 Pass 6 0.05 (+/- 0.005) .052 Pass 7 0.05 (+/- 0.005) .048 Pass 8 0.05 (+/- 0.005) .048 Pass 9 0.05 (+/- 0.005) .048 Pass 10 0.05 (+/- 0.005) .054 Pass 11 0.05 (+/- 0.005) .046 Pass 12 0.05 (+/- 0.005) .048 Pass 13 0.05 (+/- 0.005) .056 Pass 14 0.05 (+/- 0.005) .042 Fail 15 0.05 (+/- 0.005) .049 Pass 16 0.05 (+/- 0.005) .057 Fail 17 0.05 (+/- 0.005) .051 Pass 18 0.05 (+/- 0.005) .058 Fail 19 0.05 (+/- 0.005) .048 Pass 20 0.05 (+/- 0.005) .051 Pass How Was this test performed:  1.  I used the D20 Character Sheet. 2.  I set up an action on an NPC to roll a d20 as an Attack Roll.  3.  As a GM, I clicked that button as fast as I could, getting a sample set of 3734 numbers 4.  I used MS Excel to Analyze the STDV, Average, Number of Duplicates found in a set (of 5), and number of times a number occurs within the set FINDINGS: The Expected STDEV of the set outside of tolerance (+ 0.0235283) The Expected AVERAGE of the set is outside of tolerance (+ 0.1108731) The Expected number of duplicates within a set of five rolls was EXTREMELY OUT OF TOLERANCE !   The Expected Spread of Numbers is mostly within tolerance of (0.005), with thee (3) numbers being out of tolerance.  The average was correct.  EXPECTED ERROR AND ROOT CAUSE Based on standard probability; the number of times you can expect a duplicate within a set of 5 rolls is .05 for a d20.  In the experienced case, what we find is that the same Advantage | Disadvantage roll shows up as duplicates when the GM clicks a button twice in a row.  An order of magnitude = 3.79 x more likely than the set should support.  This is likely due to the server recognizing the web service call as the same call, and therefore returning the previous values as the same numbers. RAW DATA IN NEXT POST

RAW DATA HERE: STARTS AT:&nbsp; NEXT RUN OF NUMBERS <a href="https://app.roll20.net/campaigns/chatarchive/3390728" rel="nofollow">https://app.roll20.net/campaigns/chatarchive/3390728</a>

So, it may help your case if your pass/fail markings match the actual +/- tolerance that you show. Examples of discrepancies in your table are the average of the data set and frequency of a result of 1. Additionally, I would point out that a data set of 1867 rolls (the amount listed at the top of your table) is actually not terribly large for an investigation like this, that equates to an expected number of each result of less than 100. EDIT: Additionally, note that no one (other than devs) can access your raw data. EDIT the 2nd: Also, where did you figure your expected stDEV from?

+/- 1 seems like an extremely high tolerance for 1800 clicks. 1800 clicks should equal 3600 individual dice rolls The RAW Data I am referring to is the chat results of the test. I got the STD Dev number from this thread.&nbsp;&nbsp; <a href="https://forum.rpg.net/index.php?threads/why-is-standard-deviation-important.710190/" rel="nofollow">https://forum.rpg.net/index.php?threads/why-is-standard-deviation-important.710190/</a> The simple fact is that my players and I notice that the GM when clicking attack for several monsters during a session see that those monsters get the same Advantage / Disadvantage rolls several times in a row, which is outside of the statistical probability of it happening.&nbsp; The data set seems to support that observation.

Faight &nbsp;said: +/- 1 seems like an extremely high tolerance for 1800 clicks. 1800 clicks should equal 3600 individual dice rolls The RAW Data I am referring to is the chat results of the test. I got the STD Dev number from this thread.&nbsp;&nbsp; <a href="https://forum.rpg.net/index.php?threads/why-is-standard-deviation-important.710190/" rel="nofollow">https://forum.rpg.net/index.php?threads/why-is-standard-deviation-important.710190/</a> The simple fact is that my players and I notice that the GM when clicking attack for several monsters during a session see that those monsters get the same Advantage / Disadvantage rolls several times in a row, which is outside of the statistical probability of it happening.&nbsp; The data set seems to support that observation. Ah, from the way you described your methodology, it sounded like you were making a single d20 roll. But, I've run your data through a student's two-tailed t-test to analyze the average. I back calculated how many of each result you got based on your stated frequencies, and then had excel generate that many (rounded to the nearest whole number) of each number (I wound up with 1 less total roll than your sample) . Then I fed those into prism and compared them to the perfect distribution of 0.05 frequency of each. The p-value for the difference between the observed and the expected was 0.401, which is a long way from the significance threshold of 0.05. I can't analyze the frequency of a number within a given number of rolls since the raw data isn't available to a non-Dev, but the average at least is well within the threshold of being considered the same as a perfectly distributed sample.

Ok, I think we are saying the same thing.&nbsp; I observed that the spread (number of times a specific number occurs over the dataset) is near to 1/20 time.&nbsp; I don't have access to the tools you are using for analysis; however, I can't doubt their veracity. I do see that you haven't addressed the number of times you find a repeating number or set of numbers within the dataset.&nbsp; Standard Probablility indicates that within a number of rolls (5), we should expect to see a duplicate number according to the following formula (m^(n-1)) / m^n&nbsp; (m = die size, n = number of rolls).&nbsp; For any 5 rolls, we should expect 1/20 chance that a duplicate would appear.&nbsp; In my dataset, we find 708 duplicates, often repeating the same two numbers.&nbsp; 708/3734 = 0.189608998, or 3.79 times more likely to occur.

Yes, I didn't address the duplication because I can't access your raw data, and I don't really want to roll a few thousand times. But, the duplication is actually something that proves that the number set is random. Truly random (to the extent we can measure true randomness) numbers actually cluster a great deal. If the numbers were perfectly distributed where there were no or few repeats the numbers would actually not be random, they would instead be shuffled (aka put through something like a CD player's shuffle algorithm). Examples of this clustering can be seen in things like the birthday paradox , sections of repeat DNA code, and any other number of real world applications. While I can't find a truly comprehensive discussion of the matter, here's a stack exchange discussion of the issue. Note that I am not claiming that quantum roll is perfectly random (if such a thing exists outside of theoretical math), only that it is not predictable and that it is more random than standard random number generators and probably far more random than we truly need in a dice rolling application for games.

I think there is a difference between seeing clusters of numbers, and clusters of the same " Advantage | Disadvantage" numbers showing up within a single click of the others.&nbsp; As a web service developer, I've seen this kind of thing happen all too often, where the Web Service looks at a request as a duplicate and responds back with the previous response, even if the web service request is in fact not a duplicate, it's just very similar. I really think they have a uniqueness issue within their web service call that is causing players who click the same button often to get the same results, even when we would expect a different set of numbers. here is the raw data, in csv format: 16,10,6,13,10,8,17,1,2,10,14,9,12,13,16,4,6,4,14,7,5,3,20,18,13,5,2,13,4,9,11,15,5,2,20,1,6,6,9,6,6,18,18,17,7,20,6,13,11,17,4,2,11,10,7,7,14,5,6,12,17,16,6,15,7,18,16,17,18,6,1,12,7,3,15,18,14,2,10,5,17,18,3,16,3,12,6,19,2,4,15,1,18,7,14,6,15,7,1,9,5,18,17,8,13,17,11,7,9,3,1,13,20,12,5,13,11,16,7,9,20,18,1,2,14,19,12,9,13,16,2,4,16,4,5,20,9,6,4,10,20,3,10,1,1,1,15,18,6,17,1,13,18,8,10,4,8,3,13,2,17,2,13,1,7,11,3,4,17,10,14,19,20,13,2,10,18,6,12,18,3,2,10,14,5,11,16,19,18,3,15,19,14,13,4,18,13,3,18,1,15,4,9,6,18,2,5,5,13,6,7,19,6,19,16,2,12,10,8,10,6,17,18,13,11,8,12,1,6,12,12,4,19,9,6,20,13,20,14,12,20,2,20,7,1,12,16,8,15,20,6,20,5,1,3,19,5,18,2,13,13,16,14,11,17,15,15,3,19,11,9,10,14,10,17,7,9,14,7,10,15,10,13,17,19,3,14,2,1,6,3,6,13,19,11,1,16,19,7,10,13,12,13,3,9,1,8,20,8,11,13,8,4,12,8,7,6,12,11,7,14,15,4,19,3,3,19,12,4,17,13,11,19,6,13,2,11,14,13,18,10,10,10,14,20,1,19,2,9,6,6,2,17,15,19,15,5,7,7,5,8,2,16,2,5,17,11,7,19,9,15,13,4,1,9,5,20,15,10,16,12,7,2,10,9,19,2,14,20,10,20,19,11,6,15,4,17,2,7,20,10,13,18,4,9,7,9,6,11,18,18,7,12,9,5,17,3,9,16,3,7,15,18,6,15,11,16,15,17,20,11,11,5,13,14,17,8,10,11,9,17,8,4,9,17,5,5,3,4,1,14,7,20,20,17,18,19,1,1,16,3,8,12,9,7,20,18,1,1,3,13,9,13,8,1,11,6,15,14,20,1,12,13,3,1,16,12,5,9,11,6,3,15,12,1,6,15,3,15,10,14,1,17,5,1,15,17,14,11,5,18,6,13,18,10,4,13,8,5,3,9,7,11,19,20,13,8,1,16,18,8,6,15,4,3,15,19,7,18,18,6,2,16,13,10,14,14,7,3,4,11,9,7,6,9,14,5,10,4,6,11,20,13,13,14,14,15,12,18,16,2,10,19,1,16,17,3,19,13,12,18,11,4,4,13,13,10,19,16,19,10,1,6,16,15,14,6,3,15,15,18,12,15,17,19,7,13,13,16,11,13,2,11,20,5,9,14,4,15,3,6,3,17,20,5,2,6,4,7,11,9,16,17,13,15,1,4,2,19,14,9,16,18,15,14,4,17,9,5,7,17,20,1,10,13,8,17,18,1,14,8,9,16,8,8,8,9,14,2,17,18,1,10,4,8,13,18,18,17,15,19,6,9,1,5,13,20,17,9,5,9,12,16,6,19,11,11,1,20,15,2,4,10,15,15,3,4,8,1,2,3,2,ROLL START ANOTHER RUN,10,3,7,11,6,19,12,11,17,1,2,5,6,3,11,14,12,1,5,17,14,13,19,20,18,3,15,18,14,15,10,5,9,10,9,19,16,18,9,10,13,16,5,16,16,13,17,18,4,5,9,4,1,18,3,5,1,1,4,19,16,13,16,19,9,9,13,5,12,5,4,9,19,15,6,2,7,18,15,12,3,16,13,16,7,17,20,1,8,20,5,2,7,5,4,17,1,8,6,17,11,5,4,14,13,1,16,9,9,14,6,12,18,16,18,13,9,13,4,15,4,13,19,18,11,5,19,6,16,10,19,20,20,5,7,16,6,5,15,4,2,10,12,20,13,20,4,8,6,17,11,12,15,6,2,11,17,1,15,4,13,8,15,7,20,3,9,10,11,8,19,20,4,9,15,8,20,13,20,1,18,20,17,16,7,18,16,16,19,9,16,10,15,13,16,7,9,20,1,14,3,9,18,7,1,17,1,16,9,16,8,7,9,13,1,16,13,11,12,8,2,14,7,18,1,17,7,5,3,15,16,7,15,7,10,13,10,20,17,8,5,11,6,17,1,3,7,12,6,15,4,10,10,6,6,17,20,16,14,8,18,8,7,18,15,4,11,16,18,3,10,19,11,7,10,8,19,2,18,9,18,17,2,20,17,9,20,10,12,17,17,15,17,11,19,18,9,1,8,13,17,16,6,4,19,3,19,19,15,1,12,10,11,1,19,16,16,7,11,16,11,15,2,9,17,5,18,6,8,17,18,20,6,9,19,11,1,15,11,13,10,11,4,6,10,1,19,7,4,11,6,8,19,14,17,3,12,8,8,20,1,12,18,8,6,10,19,19,7,18,1,17,3,9,12,20,7,11,14,15,18,3,9,18,20,5,15,12,17,7,13,11,20,13,18,15,13,4,11,4,13,9,4,11,4,16,17,15,5,18,20,1,3,6,10,18,14,5,3,6,8,8,11,17,12,18,13,11,15,11,14,9,1,2,17,13,11,19,6,9,14,8,20,11,19,17,20,7,6,3,13,7,12,19,19,16,17,9,14,2,15,10,3,12,10,3,15,4,18,2,12,19,5,3,15,6,2,19,18,20,3,8,13,10,16,8,2,20,5,16,19,20,17,8,10,7,17,6,6,2,6,14,1,17,11,13,11,18,1,9,9,17,13,10,12,5,20,11,6,1,20,11,17,1,10,20,5,20,8,5,4,4,12,9,15,12,3,18,4,5,12,9,13,13,10,19,2,11,16,9,7,20,13,13,4,18,3,3,20,19,6,10,7,9,2,3,7,12,3,18,12,20,16,5,16,8,2,4,1,8,15,13,13,6,9,13,14,9,13,13,5,11,2,20,14,6,2,10,16,11,12,15,20,10,6,10,9,10,1,6,8,8,18,5,16,16,12,13,2,10,6,7,5,6,4,18,4,2,13,11,5,16,18,3,6,6,15,5,16,8,16,12,4,17,7,3,4,20,12,12,6,12,15,10,19,6,8,19,4,11,2,12,14,9,18,4,12,7,3,14,15,1,13,17,16,17,7,4,7,5,20,11,18,10,14,19,5,2,17,1,11,16,17,5,12,16,20,15,10,8,11,10,11,6,12,8,9,13,9,10,19,10,9,9,2,11,4,20,7,2,20,18,16,6,7,6,6,11,20,18,6,10,12,3,7,6,19,17,4,9,2,13,19,19,4,5,15,7,8,19,15,14,14,9,14,13,5,20,20,4,8,18,2,2,13,19,19,5,5,19,14,17,12,5,2,10,16,12,5,17,16,17,7,7,11,8,2,13,13,20,6,11,5,3,1,1,8,7,12,15,16,1,14,18,1,3,6,2,4,19,1,3,3,19,7,18,2,19,17,5,19,20,6,9,9,10,13,10,18,16,18,2,3,1,10,20,2,20,10,6,19,20,6,18,6,10,3,12,18,17,3,3,18,8,6,7,10,11,10,7,1,4,8,14,10,7,12,11,16,8,5,5,6,4,2,19,9,6,20,11,12,19,8,11,14,11,3,4,10,11,15,16,13,18,18,19,13,4,17,8,7,8,4,10,7,5,12,11,1,15,6,3,13,16,11,15,1,4,1,10,2,17,17,8,1,3,9,7,12,6,17,15,16,11,14,6,12,12,8,11,10,10,2,4,14,8,4,11,8,13,6,4,6,20,13,4,1,1,15,3,9,6,7,7,19,3,18,11,12,7,7,12,1,11,7,13,2,20,6,3,9,18,16,15,13,8,14,19,19,19,16,9,20,7,13,4,2,1,13,13,5,14,7,17,8,17,10,3,17,6,20,9,9,16,14,16,18,16,6,9,14,6,10,1,11,17,7,13,7,5,10,19,15,18,11,12,15,19,14,10,17,13,4,5,18,16,20,10,17,5,19,13,1,19,14,11,14,2,19,11,1,9,20,2,11,16,16,11,5,9,20,4,4,3,14,7,1,15,6,20,13,19,11,4,7,20,19,18,7,9,12,20,16,5,4,4,18,20,20,9,7,13,3,3,6,17,6,18,20,11,19,4,17,6,10,7,11,16,8,2,13,9,5,19,16,18,12,5,14,14,2,8,13,1,7,18,8,18,10,18,2,3,6,1,18,7,7,11,10,15,9,4,9,11,17,2,18,4,20,3,5,18,11,1,7,19,18,10,12,1,10,1,20,9,8,16,13,4,19,8,2,19,6,15,7,12,5,10,10,6,11,2,10,13,16,4,16,17,2,9,7,17,4,12,8,18,19,3,17,15,7,12,5,4,2,2,3,13,12,19,14,12,20,16,8,13,15,4,9,16,1,14,12,17,14,6,12,16,16,1,3,8,7,8,1,4,4,9,16,12,2,1,17,11,20,8,10,5,3,4,16,11,15,17,13,4,17,5,8,11,7,20,16,1,13,8,15,6,7,3,4,1,18,17,16,6,20,18,3,12,13,4,12,20,10,15,12,16,1,3,16,10,1,16,12,6,7,17,18,12,19,19,16,10,18,6,18,17,17,1,20,3,14,3,10,5,19,7,4,2,5,4,18,13,12,12,13,10,11,15,1,17,18,2,11,9,9,3,18,11,18,6,4,8,13,6,11,10,10,20,9,9,19,11,20,5,14,13,12,15,15,4,19,10,15,7,17,12,18,20,4,4,6,7,13,8,5,8,18,2,11,5,12,16,4,5,6,11,7,6,1,12,19,16,15,12,6,18,13,18,18,5,10,20,1,17,2,4,6,14,11,12,19,12,7,18,1,8,20,2,1,5,11,3,18,2,17,20,9,14,10,1,19,20,18,6,12,15,10,11,1,8,10,5,9,9,16,11,17,12,15,1,4,7,7,14,5,7,4,2,16,6,10,18,5,18,5,1,14,13,5,5,3,13,1,20,1,19,4,19,5,15,19,5,14,6,11,11,11,19,1,8,20,9,18,20,17,5,3,16,4,3,20,8,3,12,13,17,5,16,15,11,16,10,10,20,15,5,10,6,11,13,16,9,12,2,1,14,7,13,2,4,10,3,20,16,4,19,4,8,19,5,20,9,4,17,19,10,5,10,20,16,2,18,12,1,7,16,18,19,16,16,2,19,14,3,7,14,10,6,17,2,10,17,8,5,19,11,3,16,4,14,8,12,10,8,17,16,13,6,15,1,15,13,12,4,5,2,12,6,20,7,3,10,4,10,15,19,18,7,19,10,13,20,17,10,8,17,6,1,1,8,16,2,15,8,5,1,16,3,20,9,19,14,13,16,7,19,13,13,8,6,1,9,19,5,20,7,14,16,9,12,11,2,14,7,20,8,11,8,2,17,9,3,8,2,2,17,3,16,1,12,14,16,14,17,15,15,10,1,10,15,7,4,6,11,8,16,7,5,12,7,7,2,19,5,17,8,16,4,16,8,20,2,6,17,3,5,18,9,10,10,17,17,20,1,18,5,3,20,1,18,11,16,7,14,12,15,5,19,18,13,18,1,15,5,17,20,4,6,18,20,1,12,20,17,5,12,16,10,14,18,17,2,19,12,16,15,17,2,17,8,4,2,19,15,15,7,17,16,7,20,8,18,5,19,2,5,12,11,4,13,14,7,16,1,2,1,16,16,15,18,18,19,2,20,12,8,3,12,12,14,13,13,17,2,12,13,20,16,15,14,19,3,8,15,16,18,11,18,9,20,13,10,14,8,14,5,8,18,6,10,1,5,15,13,16,12,11,13,16,20,11,13,16,19,20,2,19,10,4,16,5,2,1,15,10,18,11,17,11,4,5,8,17,17,20,2,4,2,20,13,3,18,14,7,11,13,16,13,14,4,9,8,15,10,19,18,15,12,20,10,10,4,18,1,20,20,9,7,18,6,12,12,1,14,8,13,9,18,3,18,11,15,15,18,9,10,5,13,11,15,18,17,9,20,12,15,17,12,20,4,9,14,17,6,4,13,12,11,20,3,3,17,19,14,1,2,1,15,17,3,16,9,5,16,11,5,13,20,6,18,4,9,6,4,12,17,2,8,16,1,4,8,12,18,17,14,18,3,16,1,3,14,14,5,17,20,15,16,18,6,5,2,7,18,15,14,9,5,2,12,20,19,10,6,11,18,1,13,12,11,5,15,5,16,7,6,1,7,5,18,18,8,13,2,11,8,14,12,17,16,1,8,4,5,2,6,10,16,11,3,17,16,8,16,6,17,6,13,2,4,8,13,3,17,9,16,15,7,10,10,17,10,19,2,18,10,13,8,19,2,18,6,3,16,3,16,8,7,10,2,1,7,1,2,20,6,14,14,19,6,4,7,19,1,13,14,5,10,1,9,8,11,8,8,10,16,13,13,8,20,4,17,7,3,12,10,6,9,19,18,6,8,7,6,2,17,3,20,16,14,19,8,17,18,1,1,1,9,13,1,1,12,7,3,5,1,9,13,1,12,7,1,20,5