incomplete sleep diary - unequal data & a single night analysis

[Markus Brügger]

Dear Google Group,

I have three questions:

My sleep log is missing values for individual nights, resulting in those nights not being listed in the Part 4 output. I have tried manipulating these missing data points with spaces and NA in Excel, each resulting in the loss of the nights in the output. Is there any way to adjust the guider for these nights with the missing values so that they are not completely lost?

Some subjects in my dataset have received the Actiwatch earlier and thus generated more than seven days of data. The goal would be to include these subjects in the analysis from only the last seven days of available data to compare to subjects who have exactly one week of data. Would it be the best way to use a cleaningfile or are there different possibilities to achieve my goal?
Specifically, I have subjects with 6, 8, and 9 nights of data that I need to truncate to 7 days.

Furthermore, I have another night to analyze, where the actiwatch was only worn between 7pm and 8am. Which settings of GGIR are most useful here?

Thanks for your help

Markus

[Vincent van Hees]

Hi Markus,

1. In folder results/QC you will find part4_nightsummary_sleep_full.csv. This file includes the estimates from all nights, including the nights with missing sleeplog. In the guider column you should find a clarification of what guider was used for that night. Note that the data in this file is not cleaned at night level. For example it does not exclude nights where the accelerometer was hardly worn. Therefore, you will have to do that yourself. The column cleaningcode (as documented in the package vignette) should be helpful with this. On the topic of missing values in sleeplog: Leave the cells empty and do not insert NA values.

2. The extra days of data in some of your participants will help to get a more accurate estimate of their sleep. I would try to keep them by working with a weighted average of weekend and weekdays, this is provided in the person level aggregates as stored in part4_summary_sleep... file. However, you could also calculate them yourself from the nightsummary file.

However, if you are concerned that all data after the 7th day is inaccurate or if you insist on excluding those days for any other reason then you could use argument maxdur = 7, which labels all data after the 7th day as invalid and ignores them for all analyses. These 7 days are relative to the start of the recording, there is no similar option for the end of the recording. Although, one thing you could is move all 9 day recordings to a separate dataset folder and process them with hrs.del.end =48. This makes that the last 48 hours of those recordings are labelled as invalid. And you would do the same for all 8 day recordings with hrs.del.end=24.

3. To process nights like this you will have to reduce the includenightcrit argument, which tells GGIR how many hours of valid data are needed from a night. By default it is 16, so maybe you want to change it to 8 hours if you think that it is reasonable to expect 8 valid hours between 8pm and 7am.

Hope this answers your questions,

Vincent

Dr. Vincent van Hees | Independent consultant | https://accelting.com/

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

--
You received this message because you are subscribed to the Google Groups "R package GGIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to RpackageGGIR...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/25b7da02-04be-4cdd-b29e-9ea531c03e33n%40googlegroups.com.

[Markus Brügger]

Dear Vincent,

Thank you very much for the extensive support.

I noticed that in the QC Sleep Full Report the Guider is not set according to the Sleeplog. In the last nights the entry L512 is in the guider instead of Sleeplog and also the cleaning code switches to 1.

The entries at guider_inbedStart_ts and guider_inbedEnd_ts are empty, but I get the guider sleeplog in the report.

Then the other way around on the last nights, where the sleeplog is not empty. 

This also happens when the sleeplog entry is missing on another day.

Thanks for your help

[Vincent van Hees]

Thanks for reporting this Markus,

I think I found the cause of this issue: GGIR assumes that the sleeplog is used as soon as a non NA-value is found in the sleeplog, while it would make more sense for it to wait with that assumption until it has been verified that the value can be evaluated as a timestamp.

I have now fixed this in a development version of GGIR. Can you please try it out and let me know whether this addresses the issue?

You can install this specific version with following command (in RStudio console):

install.packages("remotes")

remotes::install_github("wadpac/GGIR", ref= "issue504_handling_invalid_sleeplog_entries")

library(GGIR)

Thanks,

Vincent

Dr. Vincent van Hees | Independent consultant | https://accelting.com/

------- Original Message -------

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/1c7c71ee-9109-4663-95b7-00173f8561fdn%40googlegroups.com.

[Markus Brügger]

Dear Vincent,

thank you very much for your help, I can now start my evaluations and hope to get the corresponding results ; ).
I still noticed that e.g. in the night summary sleep cleaned output something is still wrong with the sleeplog_used.
I get the value 0 although it says next to it that the guider is sleeplog.

Kind regards
Markus

[Vincent van Hees]

Dear Markus,

This should be fixed with the update I made 5 days ago.

Let me know if the problem persists.

Regards,

Vincent

Dr. Vincent van Hees | Independent consultant | https://accelting.com/

------- Original Message -------

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/e29dbe0f-0851-490e-adf0-3659d66e660cn%40googlegroups.com.

[Zoë Hawks]

Hi Vincent and Markus,

Adding to this thread because I'm running into a similar issue with the development version of GGIR. When guiders aren't available for the first night of an experiment (coded as empty cells in the sleep log), I'm finding that the sleep log is ignored for the entirety of the experiment. E.g., in a simple two-night experiment with missing guiders for night 1, I'm getting the following output in the part4_night summary: cleaningcode = 1, sleeplog_used = 1, guider = HDCZA for night 1, cleaningcode = 1, sleeplog_used = 0, guider = HDCZA for night 2. In contrast, when guiders are available for night 1 but missing for night 2, part4_night summary output is as expected: cleaningcode = 0, sleeplog_used = 1, guider = sleeplog for night 1, cleaningcode = 1, sleeplog_used = 0, guider = HDCZA for night 2. I've attached a simple sleep log for reference, and arguments relevant to sleep estimation are pasted below. Any advice you might be able to provide would be much appreciated! Thank you.

Zoë

__________________________ 

    ignorenonwear = TRUE, 
    loglocation = c(paste0(path, "Raw/ID_", as.character(patID), "/sleepLog", as.character(patID), ".csv")),
    do.visual = TRUE,
    outliers.only = TRUE, 
    criterror = .5,
    excludefirstlast = FALSE,
    colid = 1, coln1 = 2,
    sleepwindowType = "SPT", 
    nnights = nights_info,
    def.noc.sleep = c(1)

[Vincent van Hees]

Thanks for reporting this Zoë! Amazing the code only check whether sleep log entry was available for first night to decide whether to use the sleeplog. I have no changed this such that it checks whether sleeplog entry is available for any night.

You can try out my fix to the issue as follows:

In R consulte:

install.packages("remotes")

library(remotes)

remotes::install_github("wadpac/GGIR", ref = "issue531_sleeplogignored")

library(GGIR)

Next, use GGIR as you previously did.

I will integrate this in the next GGIR release.

Best,

Vincent

Dr. Vincent van Hees | Independent consultant | https://accelting.com/

------- Original Message -------

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/09730156-5584-4703-8c15-eed76d31bb4cn%40googlegroups.com.

[Zoë Hawks]

Thanks Vincent! Your changes effectively fixed the "first night" issue. I'm still seeing some unexpected behavior for subsequent nights (bolded below), possibly because we have so much missing sleep log data. It seems that the sleeplog variable gets switched "off" at a certain point, and HDCZA is used thereafter. 

Night 1: sleep NA, cleaning code = 1, sleeplog_used = 1, guider = HDCZA

Night 2: sleep log present, cleaning code = 0, sleeplog_used = 1, guider = sleeplog

Night 3: sleep NA, cleaning code = 1, sleeplog_used = 1, guider = HDCZA

Night 4: sleep log present, cleaning code = 0, sleeplog_used = 1, guider = sleeplog

Night 5: sleep NA, cleaning code = 1, sleeplog_used = 0, guider = HDCZA

Night 6: sleep log present, cleaning code = 1, sleeplog_used = 0, guider = HDCZA

Night 7: sleep log present, cleaning code = 1, sleeplog_used = 0, guider = HDCZA

[Vincent van Hees]

Thanks, I have now made an additional fix that should address the problem. At least in my test file everything looks good now.

To try it out, please use same installation steps as in my email below.

Best, Vincent

------- Original Message -------

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/070b1552-b5ea-42f6-a5ef-73e0579b455fn%40googlegroups.com.

[Zoë Hawks]

Hi Vincent, 

Thank you so much for your help working through this! Very grateful for continued efforts and package updates to address the issue. In part4_nightsummary_sleep_full.csv, relevant variables (cleaningcode, sleeplog_used, and guider) are now aligned, such that cleaningcode = 0 when sleeplog_used = 1 and guider = sleeplog. However, part4_nightsummary_sleep_full.csv indicates that the HDCZA algorithm was used to estimate nights 6 and 7, despite the fact that sleep log data are present. I've attached sleep log and part4_nightsummary files in case they may be useful. It seems as though, after multiple alternations between sleep log and HDCZA, the guider ceases to update.

Best,

Zoë

[Joe Winer]

Hi Vincent,

I think you may have this issue on the to-do list but I wanted to reply to this thread with a note that I am experiencing the same issues with both the basic and advanced sleep logs. Once there is a blank entry, part 4 stops. I have attached the two logs and the part4 night summary for each. 

Let me know if I can help in terms of installing a fix and trying on my data. 

Thank you,

Joe

[Vincent van Hees]

This issues should be fixed now. I fixed various issues in relation to this in recent weeks, so hopefully it is more robust now.

To try it out please install main GGIR GitHub branch:

library(remotes)

remotes::install_github("wadpac/GGIR")

library(GGIR)

Thanks,

Vincent

------- Original Message -------

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/9ef5be29-318b-4676-8693-7c670318d990n%40googlegroups.com.

[Joe Winer]

Hi Vincent, I've tried it out on the same files and the fix appears to be working perfectly. 

Thank you!

Joe

[Xinting Yu]

Hi Vincent,

Hope you are doing well. 

I am running GGIR in batches and have a similar issue with missing value in the diary. It seems GGIR opts out from some valid data points from the diary if there are missing values for other days. I am not quite sure if I need to correct the nnights? Also do I need to feed GGIR with the nights for every subject? I am running data for 2 subjects for example with 15 days and 11 days (missing day1, day8, day11, and day13) respectively. I put nnights=15. There is no problem for the first one with 15 days but for the second subject diary is not used by GGIR for day12-15 (sleeplog_used=0)  Look forward to your reply.

Thank you!

Best,

Xinting

[Vincent van Hees]

Hi Xinting,

Which GGIR version are you using?

Did you try installing the development version?

library(remotes)

remotes::install_github("wadpac/GGIR")

library(GGIR)

​

My plan is to make a new CRAN release in May.

If not, are you able to single out the file/recording that is causing the issue?

Can you process that file without diary? (just to help identify source of issue)

I will be travelling the upcoming days, so may not reply before Friday.

Vincent

Dr. Vincent van Hees | Independent consultant | https://accelting.com/

------- Original Message -------

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/b6c3980d-1756-49c9-964e-4b10df17398fn%40googlegroups.com.

[Xinting Yu]

Yeah, it works! I am using 2.6.0. After I install the development version the diary is loaded correctly and GGIR runs well. Thank you so much for your quick reply, Vincent! Have a nice weekend.

Best,

Xinting