Weekend update to GS1 Resolver means faster data uploads and better Excel compatibility

[Nick Lansley]

Dear GS1 Digital Link Developer and Triallists

Development work has continued apace since the last update to GS1 Resolver at resolver-dv1.gs1.org and we have some more weekend work planned for Sunday 7 June.

As well as a myriad of hidden code updates, two visible updates will be available:

Super-fast data uploads

Uploading large amounts of data will take seconds rather than minutes. This is because Resolver's Data Entry API will queue data into the database rather than make you wait in realtime. We fully expect the production service to have to cope with many thousands of data entries being uploaded at one time and we can't keep clients waiting. The production version of Resolver causes the API to perform some validation checks before publishing your uploaded data into the live system. In the triallist version at resolver-dv1.gs1.org this is bypassed but you still get the benefits of a speedy upload.

Here's what will happen from the end of Sunday within the JavaScript that powers the upload page:

1. The upload web page JavaScript will cause the data to be POSTed by the upload web page to the API in batches of 100 entries as it does today.
2. The API will respond almost immediately with a 9-digit 'batchId' code number which the web page saves in a list.
3. The web page will continue with the next batch upload, and receive another (different) batchId code to add to its list, and so on until all the data is uploaded.
4. The web page will then loop through its batchIds, calling an API 'batch' command to see if the processing is completed. if not, it will wait 10 seconds then ask again.
5. You can exit the web page if you wish as soon as step 4 starts, although if there are any issues the API will reveal them yet you will miss them if you exit the page.

If you are a JavaScript fan then please take a look at what is happening, by reading the (fully commented!) code in the upload.js file.

Better Excel compatibility

A first version of the Excel reading capability is already at resolver-dv1.gs1.org but there are a couple of issues that happen in spreadsheets with blank cells in the data which confuses the conversion code. This is because empty cells are not actually saved in xlsx format, so the JavaScript parser that reads the file has been updated to become a 'ghosthunter' that detects cells that aren't there so that data remain in their correct columns and don't get 'shifted'. Like CSV files, Excel file reading all happens within your web browser; the file is not actually sent to our servers,

Where's my data?! Oh there it is...!

Once I've pushed the code to resolver-dv1.gs1.org I may need to reupload all the data so please don't be surprised if, for a few minutes, you can't see your data if you try and download it on Sunday. This won't affect that resolving side, though, which remains on-air with 99.999% uptime.

I would like feedback on your web browser

Since the Data Entry service has been built as a REST API, this means that your web browser does all the work reading your files, checking them, and uploading their contained data to the API. That's over 1000 lines of JavaScript code running in your web browser just to upload your data. I've been developing and testing the JavaScript in the latest versions of Firefox and Chrome. 

I am about to test it on Apple Mac with Safari, and the latest version of Microsoft's Edge browser. However, is it working for you or do you get any strange anomalies? I'd like to get some feedback on your experience along with the make and version of your web browser. Should I be adding more browsers to the test? Bear in mind that the main use case is a corporate setting, are you still using Internet Explorer 11 or 9 (don't say 6!)? Feedback appreciated.

That's my Sunday sorted then!

You have a good weekend too.

Cheers

Nick