Data upload fails sometimes

[Christina Lubinus]

Hi Jatos-experts, 

currently I am piloting my study (using JsPsych, Jatos and Prolific) and am facing some issues, one of which is related to saving/uploading the data generated during an experimental session. For some participants, the experiment runs smoothly and data is saved just as I want/would expect it to. But for some subjects files are either not saved at all or just the first few. 

For instance, one participant reported that they completed all the trials, got to the final page saying "you finished and will be redirected to Prolific" but then redirection failed. For this participant only the first 3 files (out of 12) were saved. 

Data is saved in 2 different ways (because there are different tasks):

1. after 30 seconds of audio recordings, the audio file is saved (as base64 string) before the next trial starts.

on_finish: function(data){

        var outname = "whisper" + filecount_whisp + ".json"

        var resultCsv = jsPsych.data.get().json();

        jatos.uploadResultFile(resultCsv,outname)

        filecount_whisp = filecount_whisp +1;

      }

2. data are saved at the very end of the component: 

on_finish: function() {

          var interaction_data = jsPsych.data.getInteractionData().json();

            jsPsych.data.addProperties({

            interactData:interaction_data

          });

          jatos.uploadResultFile(trial_info_rand, 'randomized_trial_info.json')

            var resultData = jsPsych.data.get().json();

            jatos.uploadResultFile(resultData, "task1.json") / save in file

            jatos.submitResultData(resultData, jatos.startComponentByPos(8)) 

}

In case of failure, the state reported for the component was RESULTDATA_POSTED. Is it possible that I use the wrong jatos.js function to save the data and this causes the problem? I have read somewhere in the thread that it may be better not to save during the experiment but rather at the very end? 

I would be very happy about any help solving this issue. 

All the best,

Christina 

[Kristian Lange]

Hi Christina,

I see a tricky bug in your code: You use

jatos.submitResultData(resultData, jatos.startComponentByPos(8))

This calls jatos.startComponentByPos right away without waiting for jatos.submitResultData to be finished. It's just coincidence that some of your study runs actually worked. If you'd put

jatos.submitResultData(resultData, () => { jatos.startComponentByPos(8) });

it should work. The difference here is tricky and a common error in JavaScript. jatos.submitResultData takes a function as the second parameter (also called callback function).  In your version you call the function already yourself and only the result is handed over. In the second version a function is defined with () => {} and given to jatos.submitResultData to be called when all result data are submitted. 

Btw. in newer versions of JATOS (v3.3.1) you can write it simpler with just

jatos.startComponentByPos(8, resultData);

Another thing that I noticed is that you used only one '/' in your comment (JavaScript uses '//'). Some browsers might break over this.

Best,

Kristian

[Elisa Filevich]

..And, just to add, it's not necessary to convert audio files to base64, JATOS allows you to upload files in any format: 

http://www.jatos.org/Submit-and-upload-data-to-the-server.html#upload-and-download-result-files-since-jatos-version-351

Best

Elisa 

-- 
You received this message because you are subscribed to the Google Groups "JATOS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jatos+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jatos/3c2f31f5-5b3d-42ec-b080-ba8fc6868e50n%40googlegroups.com.

[Christina Lubinus]

Hi Kristian, 

Thanks a lot! First thing tomorrow, I will see if that small change in the code will solve the issue. 

A little follow-up question - can I understand your comment to say that there is no issue with saving the audios right after each trial? No need to save at the very end of the component? 

The "/" I actually just included for this post, it is not included in the actual script and shouldn't be the source of my issues.

Best, 
Christina 

[Christina Lubinus]

Hi Elisa, 

Thanks for pointing this out. I did not add the conversion to base64 because of Jatos. This is part of the jspsych-plugin which makes audio recording possible. Since it worked, I did not look into how to change the format it was saved in. 

Best, 

Christina 

[Kristian Lange]

> A little follow-up question - can I understand your comment to say that there is no issue with saving the audios right after each trial? No need to save at the very end of the component? 

After each trail sounds like too often to me. It depends on how long one of your trails is. The thing is that each sending of result data with jatos.submitResultData or jatos.appendResultData puts load on the server and if you are doing it too often with large data you might overload your server. The rule of thumb I have is once a minute is fine. I've seen the extreme several times a second which is definitely too often. The other extreme, only at the end of the component, saves the server but might lead to lost data if the participant does not completely finish the component until the end.

Best,

Kristian