Progress on Image upload

Isn't there supposed to be some form of indication on the progress of uploading images? I've had that issue for quite some time, and for a lot of the images it didn't bother me all that much. But atm I'm uploading some larger maps, and I can't tell if it's done or not. There is no progress bar etc. I think my uploads should be done, but when I pull them on the vtt they come out all low res and pixelated and absolutly unusable.

I tried with firefox and a blank brand new chrome, no difference etc.

This happens with me too. It jumps back in the progress indicator as well.

I got it partially to work by cutting the map into pieces where possible, and uploading the pieces seperately. Still I got 1 larger piece (1000*1100px, jpg about 500kb) that absolutly doesn't want to upload correctly. The screenshot shows what it looks like on the right vs what it is supposed to look like on the left. As you can see it's absolutly ugly, and not really usable. This is just a small part, of an overlap these 2 pieces had.

I tried different formats (jpg/png), I verified that the pieces don't look like that on my HDD etc. For some reason I can't get this to upload...
It gets even worse, when I try to upload the map as a whole. The right part looks high res, to what I get when I try to upload the whole map...

One thing to check is to load the page in a different browser, to ensure the problem is not serving of cached imagery.

keithcurtis said:

One thing to check is to load the page in a different browser, to ensure the problem is not serving of cached imagery.

Okay, I tried both Firefox and Chrome with the full map, and couldn't get it to work. I also tried ctrl+f5, which should be the same as manually deleting the cache and then reloading the page. For some reason this didn't work in Firefox.

After checking with Chrome and manually deleting the cache in Firefox the pictures do now correctly load, at least for the cut up map. I also tried again with the full map, and both opening it in Chrome and using ctrl+f5 or manually deleting the cache does nothing in regards to the full map, which still is annoying, but at least I can work around it, even though it means a lot more work, for each map.

Still I got no progress bars, when uploading images, so that problem still remains, and leaves me guessing when my uploads are actually done.

This sounds wrong.

If I drag and drop an image from desktop to VTT, I get a progress bar. If I use the upload button, I get a dialog box with the space for a progress bar (though no animation), but there is a notification on that dialog box when the upload has finished. You get neither?

keithcurtis said:

This sounds wrong.

If I drag and drop an image from desktop to VTT, I get a progress bar. If I use the upload button, I get a dialog box with the space for a progress bar (though no animation), but there is a notification on that dialog box when the upload has finished. You get neither?

I only used the Upload Dialog so far. not dragging it directly to the vtt. I can either drag an Image into that dialog Box or use the file dialog Button, both ending with an empty progress bar in the dialog and no notification whatsoever in both firefox and Chrome.

alright, I did a little bit of testing with dragging it directly to the vtt. in Firefox I get No progress bars for jpg, but it does upload in the Background. pngs have a progress bar in that case. In Chrome I get a progress bar for both png and jpg. both Images are the Same pixelsize. about 1600x2200.

the jpg is about 1.6m, the png is about 8.5m

in both firefox and Chrome the Uploads for these Images seem to Stop at around 30-35% for both the jpg and the png for some reason. I Just didn't notice it before, because I didn't get any progress feedback.

the Images are created in Gimp from a much Higher ress source, so I am pretty certain they are okay and Not corrupted.

Obviously, this bears more testing...

Noticed this too.

My guess there is a relation to the ‘jpg becomes png’ bug. When uploading large maps, Roll20 receives the file, does some identifier assignment, converts the image to a number of sizes and uploads each size as a seperate image to s3 and finally updates the art library.

(Still my guess) The progress bar uses above steps to show progress, the first one-third being the upload from your pc to the roll20 server. But as the image size increases drastically (like 4-5 times at least), the progress bar becomes erratic as the other steps have a way longer duration.

regardless, if left uninterrupted ( no quick F5 to reload roll20) the map uploads correctly.

What i do: ignore the progress bar, get myself a coffee and use the art library to check whether the upload is finished correctly.

Martijn S. said:

Noticed this too.

My guess there is a relation to the ‘jpg becomes png’ bug. When uploading large maps, Roll20 receives the file, does some identifier assignment, converts the image to a number of sizes and uploads each size as a seperate image to s3 and finally updates the art library.

(Still my guess) The progress bar uses above steps to show progress, the first one-third being the upload from your pc to the roll20 server. But as the image size increases drastically (like 4-5 times at least), the progress bar becomes erratic as the other steps have a way longer duration.

regardless, if left uninterrupted ( no quick F5 to reload roll20) the map uploads correctly.

What i do: ignore the progress bar, get myself a coffee and use the art library to check whether the upload is finished correctly.

Well, this kinda makes sense. I left an upload running for about 3h, and it got to about 50-55%. It is still not really usable, but this makes sense in regards to your guess. The Image still wasn't usable after 3h, but I think it got a little better. I might try to leave it running over night to see if I get some usable maps.

I was thinking about minutes... not hours. 

a 8mb jpeg can become a 40mb png, and unless your internet is atrociously bad, hours are way too long. Will do some testing and recording tomorrow.

Martijn S. said:

I was thinking about minutes... not hours. 

a 8mb jpeg can become a 40mb png, and unless your internet is atrociously bad, hours are way too long. Will do some testing and recording tomorrow.

I don't think it's the Upload from my Computer that's the Problem, but the processing on the Server. I got 10mbit upstream, that shouldn't be the Problem, and small files get done pretty fast. even the cut up map got processed in a few minutes, even though the pieces itself were pretty much the same size combined. I got the impression this happens only with images that are consideravly over something like 1000x1100px.

okay, left it over night, tried again today, but no luck. I cant get any image larger than 1000x1100px uploaded correctly.

This is so infuriating....Now I got a map that I downscaled to 1500x960-ish----I cut out some fluff, and reduced it to 2 parts each 600x800, jpg under 250kb each, and I can't get them to finish uploading...Waiting for 10 minutes now for the first part, and it seems to be stuck on 50%...ffs

I tested various other ways of uploading things, including dropbox etc, I did some internet speed test, all say my upstream is fine, so I am pretty sure this is not on my end...

I couldn't get any prepwork done the whole weekend, as I need these maps...

Edit it took me until now to finish uploading those 2 files. 50 minutes for 2 600x800 jpg with under 250kb each. This Screenshot took literally seconds to upload. Whatever is happening, it makes roll20 almost unusable for me atm...

Did some testing and learned something new. Roll20 is more decentralized than i thought. Images are directly uploaded from the client/browser to Amazon s3. For each image 5 versions are uploaded to s3: mini, thumb, med, max and original.

Two 9.9MB sized assets "SDT Cathedral 1 Roof Snow 22x16.png" and "CTBR Dead Forest Map Section 5 30x30.jpg" were uploaded and when the asset became visible on the map Z was pressed to load the original image. Assets where selected by size from my sizable Gabriel Pickard collection.

Ran the test multiple times and performance is very, very unpredictable. One moment it's under a minute, and another moment it takes over 15 minutes. Test were run in Chrome and Firefox with same results. Some of the tests where captured with OBS and are available for analysis by Roll20 on request. 

One run the png was uploaded within 20 seconds, it became visible on the map and Z was pressed to load the original image. 11 MB of data was sent to the network and 11 MB of data was received. Which is kind of half of what i expected (maybe the result of http gzip compression?).

Then the jpg was uploaded and it took 18 minutes to upload (and Z was again pressed). 

This time 56 MB of data was sent and 56 MB of data was received. The jpg that was yielded by pressing Z has the jpg extension, but is in fact a png of 42 MB large. When there are network issues, more data exacerbates the effects.

I reran the test at least 20 times... even with chrome and firefox next to each other. Was it only the first image that uploads fastafter a fresh Roll20 start? No. Sometimes the jpg upload in chrome overtook the png upload in firefox (and vice versa). Somehow it looks like s3 decides to throttle certain uploads.
This is a very complex problem.  Due to the decentralized nature of the upload, it is very hard for the developers to trace and test. But it is a customer experience disaster that affects paying customers most (as these probably GM most and upload most).

For now my solution would be (when you have a sufficient fast ISP): keep the network monitor open while uploading large images. If the progress bar stalls halfway, the network speed on the sysmonitor drops to a low line for 10 seconds. Press F5 and start again.

Well, for me it just started working again. I just now uploaded both maps I talked earlier about without any problems in a few seconds.

Both were jpg 1 was 1653x2273 the other one 1500x923. Both took less then 30 seconds to upload suddenly. I don't know what exactly changed, I changed pretty much nothing on my System, I just wanted to check what Martijn S. was talking about.

Still this was an extremly irritating experience. I hope this won't repeat. I will report back, if this issue comes back.

Wow, solid research Martijn.

alright, I uploaded 7 or 8 maps for the campaign in the last 2 days, all between 1900x1900 - 3750x2400 and all jpg.

No more stuck uploads, No more problems with cached Images all finished in unser 30 seconds.

I really don't know what the issue was this weekend, I didn't really change anything, but for now it seems to be fixes except for no Animations for the progress bar when using the Upload dialog box.

Martijn S. said:

Then the jpg was uploaded and it took 18 minutes to upload (and Z was again pressed). 

This time 56 MB of data was sent and 56 MB of data was received. The jpg that was yielded by pressing Z has the jpg extension, but is in fact a png of 42 MB large. When there are network issues, more data exacerbates the effects.

I've reported this problem in the past (JPEGs being converted to PNGs).  Roll20 said they were investigating but nothing has been done about it.

https://app.roll20.net/forum/permalink/8569852/