Track time spent in each page and focused time with debug mode

[Juanfran]

Hello all,

I am new on oTree, and I wondered how I could track the time spent on each page of my study (composed by several apps). This time should be calculated as the difference between entering in each page and typing the next button (once the forms requested are properly completed).

Some colleagues told me about otree tools, but I found that this option is no longer available in new versions of oTree (I currently have version 3.3.9). If there is no way now, how can I download an older oTree version while maintaining my current one?

I also found a post here about the file pagetimes.py in the oTree documentation (https://otree.readthedocs.io/en/latest/admin.html#export-data).

However, although I understand this file should be run after extracting the data in the study (I suppose in the folder of the .csv data file), I would like to know whether it can be used with data in debug mode (since I have no timing data in the .csv so I don't know how pagetimes could work).

I ask about it since I would like to test my study before running my experiment to be sure about its functioning . Also the debug point is important, because I don't manage to run my experiment with debug = False getting always the following error (which I don't know how to solve): 

"devserver & zipserver cannot be used in production mode. Ensure that your settings.py does not contain a DEBUG setting, and that the OTREE_PRODUCTION env var is not set".

Thank you so much in advanced.

Juanfran

[Rok]

You can use ```epoch = int(time.time())``` in python to get current time in seconds.

So you can record starting and ending time of every page in an appropriate function. For the starting time you could use a function like vars_for_template (this is just off the top of my head, there might be a better function for it) and for the ending time the function before_next_page. Then subtract your starting time from your ending time, and you get time spent on the page in seconds.

Rok

[Juanfran]

Many thanks for your fast reply Rok. I managed to implement what you suggested and it helps me to track the total time spent on a page in seconds, but is it possible to track it more accurately (like in milliseconds)? 

With this, I manage to save in a variable the time elapsed in the pages, but is there a way that also helps to save the time until a form is filled or a button (even different to the next one) is clicked?

Moreover, I have seen an answer to a previous post by Chris (https://groups.google.com/g/otree/c/pJPbKMd5bUg/m/Sf0gCNPzBQAJ) which mentions that timing-sensitive things should be done in javascript (string it in hidden form fields), not server-side python, but I don't know how to do so. I suppose that using JS, the tracked time is more precise, avoiding differences due to loading pages and other server issues. How can I track time with JS? 

In general, I am quite confused about how to integrate JS code into the HTML files I wrote for my experiment, so I did not manage to do it by myself with the ways I found by googling. I would really appreciate it if anybody could show me a text example on how to do this to solve the problem and start learning how to integrate JS in the HTML files of the oTree project).

Kind regards.

Juanfran

[Chris @ oTree]

see here: https://otree.readthedocs.io/en/latest/forms.html#raw-html-example-custom-user-interface-with-javascript

[Rok]

Oh I did not realize you need to measure time with such precision.

First of all, you can put your JS code anywhere in your html inside the <script> </script> tags.

To record starting time you could use something like window.onload function which runs once the page is fully loaded (displayed).

window.onload = function() { var startTime = Date.now(); };

To get the end time, you could probably just make the next page button also call a JS function which records it (and subtracts both, and saves the result to a hidden field like its shown in the example in the docs that Chris linked above).

<button onclick="doTimeStuff()">Next Page</button>

<script>

function doTimeStuff(){

var timeSpent = Date.now(); - startTime;

// save the number in the hidden field

document.getElementById('my_hidden_field_id').value = timeSpent;

}

</script>

Although I am not completely sure what happens when you click a button that takes you to the next page. Does the redirect happen before the javascript? Is it a simultaneous thing where you just hope your browser will execute a small JS script before oTree redirects the browser window to a new page? Maybe Chris can comment on that.

Rok

[Chris @ oTree]

On Wednesday, March 31, 2021 at 5:48:23 AM UTC+8 Rok wrote:

Although I am not completely sure what happens when you click a button that takes you to the next page. Does the redirect happen before the javascript? Is it a simultaneous thing where you just hope your browser will execute a small JS script before oTree redirects the browser window to a new page? Maybe Chris can comment on that.

Sure. That should be fine. The click handler will execute its own javascript, then do the default action which is to submit the form. However, there are other cases where this will result in a race condition, such as doing a liveSend when the next button is clicked, in which case I recommend the technique here: https://otree.readthedocs.io/en/latest/live.html#troubleshooting

[Juanfran]

Many thanks for both replies.

I tried to use directly Rok's one, but I don't know why but it seems that the JS code I write in my files is not being executed (in fact, it does not have any color when I try to add JS code to my files in Pycharm, whereas other parts of the code have). I say that because the variable I create to save the time (timeJS, defined in my models.py) remains blank.

The simplest example I tried to develop following your advice is the following, just trying to track the time in a page where the subject has to enter an ID (my idea is going further and being able to track multiple times, like the time until clicking in several fields of the same page). 

HTML code:

{% extends "global/Page.html" %}
{% load otree %}

{% block scripts %}

<script>
function doTimeStuff(){
var timeSpent = Date.now(); - startTime; 

document.getElementById('timeJS').value = timeSpent;
}
</script>
{% endblock %}

{% block title %}
<center>WELCOME!</center>
{% endblock %}

{% block content %}
<p>
Please, before starting the study, enter <strong>your ID</strong>.
</p>

{% formfield player.ID label="ID" %}
<p style="margin:4% 0;"></p>

<p> Press the button below to continue.</p>

<button class="otree-btn-next btn btn-primary" onclick="doTimeStuff()">Next</button>
<input type="hidden" name="timeJS" id="timeJS" />

{% endblock %}

In my models.py code, I included in the Player class both timeJS = models.FloatField() and ID = models.StringField(). First, I thought the problem could be because I didn't add timeJS to the page in pages.py, but after trying to do so, the page raised the error:

The form field timeJS has errors, but its error message is not being displayed, possibly because you did not include the field in the page. There are 2 ways to fix this:

1. Include the field with the formfield tag, e.g. {% formfield player.timeJS %}
2. If you are not using formfield but are instead writing the raw HTML for the form input, remember to include {{ form.timeJS.errors }} somewhere in your page's HTML.

And after entering in my HTML file {%  form.timeJS.errors  %}, an error asking for timeJS raised.

I also tried to do it as written by Rok, directly adding the block I write above in the block scripts right after the button, and at the end of the file, but none of them worked.

What is the problem in this case? shouldn't it work any JS code just having it between the <script> tags?

My apologies if this question is too dummy, but I don't manage to make JS code work with my HTML, so I cannot track the time by this method.

I also tried to solve this having at least the time by page with the data section, but when I try to run the pagetimes.py in the same folder than the csv times file, I get the following error:

Traceback (most recent call last):

  File "pagetimes.py", line 25, in <module>

    pcode = row['participant_code']

KeyError: 'participant_code'

I suppose the troubleshooting problem mentioned by Chris is a harder thing, so I would prefer to stay knowing this first before trying to understand that (I don't get what a liveSend is).

[Juanfran]

Attached there is the HTML example I am trying to develop in case this can help someone to solve my problem.

The only thing needed in models.py to run it should be to add timeJS = models.FloatField() in the player class.

I don't understand why, although it enters into the JS function (I checked it using alert(), thank you Tomasso for your suggestion), it does not save anything in the field timeJS (it remains blank).

I followed Rok and Chris suggestions of the documentation, but it seems that document.getElementById('timeJS').value is not working. 

Does anybody know why?

Thank you in advance

Kind regards,

Juanfran

PS: I also attach a screenshot of the code in case it can help or fasten the process.

.

[Chris @ oTree]

Did you put timeJS in your page’s form_fields and submit the page?

Sent from my phone

On Apr 5, 2021, at 7:16 PM, 'Juanfran' via oTree help & discussion <ot...@googlegroups.com> wrote:

Attached there is the HTML example I am trying to develop in case this can help someone to solve my problem.

The only thing needed in models.py to run it should be to add timeJS = models.FloatField() in the player class.

I don't understand why, although it enters into the JS function (I checked it using alert(), thank you Tomasso for your suggestion), it does not save anything in the field timeJS (it remains blank).

I followed Rok and Chris suggestions of the documentation, but it seems that document.getElementById('timeJS').value is not working. 

Does anybody know why?

Thank you in advance

Kind regards,

Juanfran

PS: I also attach a screenshot of the code in case it can help or fasten the process.

--
You received this message because you are subscribed to the Google Groups "oTree help & discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to otree+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/otree/a3964967-abd2-400c-96e9-85ef7ff9b943n%40googlegroups.com.

<GettingID.html>

<htmlfile.png>

[Chris @ oTree]

By the way I see an extra semicolon in your javascript code. 

Sent from my phone

On Apr 5, 2021, at 7:19 PM, Chris @ oTree <ch...@otree.org> wrote:

Did you put timeJS in your page’s form_fields and submit the page?

[Juanfran]

Is the extra semicolon right after Date.now()?

I fixed the {{ typo. However, it does not solve the problem, it asks to fix the error since it requires the field to be filled (although it should be filled by the JS code).

Here you have a screenshot of the page.

[Katy Tabero]

Hi,

I am not sure if you have managed to solve your problem, but I tried the same method as you and got the same error so I found an alternative . Here are the relevant code segments I am using, you will need to edit it to fit your page/variable names, I have added comments after '//' and the code itself is in blue:

models.py, in Player :

// Create a field to store the data

timeSpent = models.FloatField()

pages.py: 

// Add your variable name to your form_fields in the relevant page

class Task(Page):

    form_model = 'player'

    form_fields = ['task_answers','timeSpent',]

    timer_text = 'Time remaining:'

page.html:

// Add this to the html portion of your page, it will be invisible so it doesn't matter where - mine is just above my next button

<input type="hidden" name="timeSpent" id="timeSpent"/> 

// Add this script with the rest of the javascript for your page

<script>

//This bit creates 2 variables, one will function as a timer, the other will hold the timer output. They are set to 0 and -1 here to initialise them, these will not be the final outputs.

var pageTimeElapsed = 0;

var pageTimerID = -1;

//This function captures the timer output. When the timer starts it will increase 'pageTimeElapsed' every second and send this value to the 'timeSpent' variable that will be saved

//When a person moves to the next page, or the page times out, the final value that the timer was on will be saved. The value resets each round/page.

function pageTick() {

  pageTimeElapsed++

  document.getElementById("timeSpent").value = pageTimeElapsed;

}

//This function automatically starts the timer when the page loads. In this case I want seconds so the interval is set to 1000, but I believe if you set it to 1 then it will count milliseconds - you will need to test this

window.onload = function() { 

  if(pageTimerID == -1){

    pageTimerID = setInterval(pageTick, 1000);

  } 

};

</script>

I am only just learning code so it may not be the best method, but it is working for me. 

Best wishes, 

Katy

[Rok]

Hello,

Sorry, like I've written in another thread, it seems i forget about oTree related forums when i work on other projects.

It seems your problem essentially is that some part of code in the doTimeStuff() function fails before it gets to the last line where you write a value into your hidden input. (Easiest way to test that would be to remove the timer stuff for now: Only leave the last line in your doTimeStuff(), and change it to something like document.getElementById('timeJS').value = 42; and see if your problem goes away.)

The first problem would definitely be the extra semicolon after  Date.now(); . It was my typo when i gave you the example code. 

If you fixed that and it still doesn't work, I don't see window.onload = function() { var startTime = Date.now(); }; anywhere in there. Are you running that anywhere? If not, then startTime variable does not exist, and the code in yout function will fail again. Just add that line above your function declaration like:

<script>

window.onload = function() { var startTime = Date.now(); };

function doTimeStuff(){

var timeSpent = Date.now() - startTime;

// save the number in the hidden field

document.getElementById('my_hidden_field_id').value = timeSpent;

}

</script>

Your javascript code not getting colored in PyCharm is a PyCharm thing, I looked into it myself because it was annoying me when i first started with PyCharm, and it seems to be a premium only feature, meaning you gotta pay to get your javascript colored. :)

---------------------------

On a more general note, since you had problems with javascript code starting, but not doing what you think it should, here's how you could check what goes wrong:

https://stackoverflow.com/a/13490378

as stated above, you can type console.log(); anywhere in javascript code, and it will print what you write in it to the browser console. The above link also explains how to find the consile in firefox and chrome, or you can google it yourself for other browsers. With console.log() you could do something like this:

<script>

window.onload = function() { var startTime = Date.now(); };

function doTimeStuff(){

console.log("dotimestuff started");

var timeSpent = Date.now() - startTime;

console.log("timeSpent calculated:");
console.log(timeSpent);

// save the number in the hidden field

console.log("writing to hidden field");

console.log("hidden field value before: " + document.getElementById('my_hidden_field_id').value);

document.getElementById('my_hidden_field_id').value = timeSpent;

console.log("hidden field value after: " + document.getElementById('my_hidden_field_id').value);

}

</script>

Then you could check your browser console for details and errors, similarly to how you can debug and check the python console for otree (python) issues.

Here's a bit more about console.log, and then some more advanced methods like using debugger too:

https://www.w3schools.com/js/js_debugging.asp

Good luck and I hope this is not too late and it still helps you. :)

Best Regards,

Rok

[Juanfran]

Hi all,

Many thanks for your detailed replies and the detailed explanation on the JS issue Rok, I would check it in detail to learn more about the usage of JS.

I used a method with the same type of code proposed by Rok but based on JQuery (thanks to Tomasso Batistoni, who helped me a lot).

Sorry for my late reply but I was also busy with other projects.

Kind regards,

Juanfran

[sahar sangi]

Hi all, 

I have the same problem as what Juanfran had. I went over all the suggestions in the discussion, but it still does not work (same errors).  Could you please share your solution if you have one?

Kind regards,

Sahar