Part 2 wrongly identifies measurement start time as midnight

Andy Whale

unread,

Nov 22, 2023, 7:34:54 AM11/22/23

to R package GGIR

Hi All,

I'm using GGIR to process .bin files from GENEActiv. I recently updated the package from (I think) 2.8-0 to 2.10-1 after an R upgrade, and I'm getting this issue for new files, and files I've previously processed with the old version without issue.

I'll paste my code below, but I'm getting a message in part 2- "measurement starts at midnight or there is no midnight". The recordings are all ~14 days long, and none of them start at midnight- if I convert the .bin file to a 60-second epoch csv using the GENEActiv PC software, the csv file correctly has the first timestamp at ~08:40:00 in the morning.

After the analysis runs, if I load m$metashort, the first record is indeed at 00:00:00. I've loaded the .bin file in the GENEActiv PC software- this also has the corect start time noted in the summary information, it only seems to be an issue when loading the .bin file into GGIR.

The practical upshot is that none of the sleep reports make a great deal of sense- the automated sleep time detections are out of line by the difference between midnight and the actual start time, and the raw data doesn't line up with the sleep diary. Any idea what might be causing this, and is there a way in GGIR to manually specifiy the recording start time?

Many thanks

Andy

###########

GGIR(#-------------------------------
# General parameters
#-------------------------------
mode= 1:5,
datadir=datadir,
outputdir=outputdir,
studyname=studyname,
f0=f0,
f1=f1,
overwrite = FALSE,
do.imp=TRUE,
idloc=1,
print.filename=FALSE,
storefolderstructure = FALSE,
#-------------------------------
# Part 1 parameters:
#-------------------------------
windowsizes = c(5,900,3600),
do.enmo = TRUE,
do.anglez=TRUE,
chunksize=1,
printsummary=TRUE,
#-------------------------------
# Part 2 parameters:
#-------------------------------
strategy = 1,
ndayswindow=7,
hrs.del.start = 1,
hrs.del.end = 1,
maxdur = 15,
includedaycrit = 6,
L5M5window = c(0,24),
M5L5res = 10,
winhr = c(5,10),
qlevels = c(c(1380/1440),c(1410/1440)),
qwindow=c(0,24),
ilevels = c(seq(0,400,by=50),8000),
mvpathreshold =c(100,120),
#-------------------------------
# Part 3 parameters:
#-------------------------------
timethreshold= c(5),
anglethreshold=5,
ignorenonwear = TRUE,
#-------------------------------
# Part 4 parameters:
#-------------------------------
excludefirstlast = FALSE,
includenightcrit = 6,
def.noc.sleep = 1,
sleepwindowType = "TimeInBed",
loglocation= c(filepath_to_sleeplog),
# loglocation = c(),
outliers.only = FALSE,
criterror = 4,
relyonguider = FALSE,
sleeplogidnum = FALSE,
colid=1,
coln1=2,
do.visual = TRUE,
nnights = 14,
#-------------------------------
# Part 5 parameters:
#-------------------------------
# Key functions: Merging physical activity with sleep analyses
threshold.lig = c(30),
threshold.mod = c(100),
threshold.vig = c(400),
boutcriter = 0.8,
boutcriter.in = 0.9,
boutcriter.lig = 0.8,
boutcriter.mvpa = 0.8,
boutdur.in = c(10,20,30),
boutdur.lig = c(1,5,10),
boutdur.mvpa = c(1,5,10),
timewindow = c("WW"),
save_ms5rawlevels = TRUE,
save_ms5raw_format = "RData",
#-----------------------------------
# Report generation
#-------------------------------
do.report=c(2, 4, 5))

}

Andy Whale

unread,

Nov 22, 2023, 8:02:20 AM11/22/23

to R package GGIR

For what it's worth I've been doing some digging;

The contents of /meta/basic/I$header contains the correct start time, however /meta/basic/M$metalong still has the first record timestamped at 00:00:00. /meta/ms2.out/IMP$metashort also starts at midnight....

Jairo Hidalgo Migueles

unread,

Nov 27, 2023, 6:03:27 AM11/27/23

to R package GGIR

Hi Andy,

Were your data collected in the same time zone where you are processing the data? If it was a different time zone, you would need to define the argument desiredtz, please see the GGIR documentation on this. Additionally, if the device was configured in a different timezone, you would also need to provide configtz.

Best,
Jairo

Alex Dallaway

unread,

Dec 12, 2023, 10:41:29 AM12/12/23

to R package GGIR

Hi Andy,

Did you ever manage to sort this out? I am having the same issue where sleep time starts at 12 noon (see below).

I didn't have this problem previously, but using the same code and same laptop, it has started to do this. I also had this issue on a separate laptop, which has always had this issue.

Jairo Migueles - any help on this would be appreciated as it feels like a random error at the moment. One minute it worked, the next it didn't.

Kind regards,

Alex

Alex Dallaway

unread,

Dec 12, 2023, 10:47:09 AM12/12/23

to R package GGIR

I have included the code below. The message I receive from running the code is: " Part 2 1 measurement starts at midnight or there is no midnight"

GGIR (#-------------------------------

# General parameters

#-------------------------------

mode=mode,

datadir=datadir,

outputdir=outputdir,

studyname=studyname,

f0=1,

f1=1,

overwrite = FALSE,

do.imp = TRUE,

idloc = 1,

print.filename = TRUE,

storefolderstructure = TRUE,

#-------------------------------

# Part 1 parameters:

#-------------------------------

windowsizes = c(5,900,3600),

do.cal=TRUE,

do.enmo = TRUE,

do.anglez=TRUE,

acc.metric = "ENMO",

chunksize=1,

printsummary=TRUE,

#-------------------------------

# Part 2 parameters:

#-------------------------------

strategy = 2,

maxdur = 7,

includedaycrit = 10,

qwindow = c(0,24),

L5M5window = c(0,24),

qM5L5 = c(0.1,0.25,0.5,0.75,0.9),

M5L5res = 10,

winhr = c(1,6),

qlevels = c(

960/1440,

1320/1440,

1380/1440,

1410/1440,

1425/1440,

1435/1440,

1438/1440),

ilevels = c(seq(0,400,by=50),700, 8000),

iglevels = TRUE,

MX.ig.min.dur = 1,

mvpathreshold = c(93.2),

mpvadur = c(10),

excludefirstlast = FALSE,

do.part2.pdf = TRUE,

#-------------------------------

# Part 3 (sleep) parameters:

#-------------------------------

includenightcrit = 4,

timethreshold= 5,

anglethreshold=5,

ignorenonwear = TRUE,

do.part3.pdf = TRUE,

#-------------------------------

# Part 4 (sleep) parameters:

#-------------------------------

def.noc.sleep = 1,

outliers.only = FALSE,

do.visual = TRUE,

#-------------------------------

# Part 5 parameters:

#-------------------------------

threshold.lig = c(45.8), threshold.mod = c(93.2), threshold.vig = c(418.3),

boutcriter = 0.8, boutcriter.in = 0.9, boutcriter.lig = 0.8,

boutcriter.mvpa = 0.8,

boutdur.in = c(1, 10, 30), boutdur.lig = c(1, 10), boutdur.mvpa = c(1),

includedaycrit.part5 = 2/3,

timewindow = c("WW"),

part5_agg2_60seconds = TRUE,

#-------------------------------------------

# Report generation

#-------------------------------------------

do.report=c(2,4,5),

visualreport = TRUE,

dofirstpage = TRUE,

epochvalues2csv = TRUE,

viewingwindow = 1)

Vincent van Hees

unread,

Dec 12, 2023, 11:08:40 AM12/12/23

to Alex Dallaway, R package GGIR

Alex,

You are not specifying the timezone and you show that sleep is happening in the middle of the day. Are you sure your local time zone settings have not changed?
How long is this recording? Just this one day or is it followed by more data?
Can you share the same visualisation for your old output?
What accelerometer brand are you using?
What GGIR version are you using now and what GGIR version where you using before? It sounds like you updated GGIR or the timezone settings of your machine because GGIR does not change itself.
Further, can you share the corresponding RData file form the meta/basic folder? If it is a real bug then this should help us to reproduce it in order to investigate. Please check that you can reproduce the error by processing only that file with GGIR parts 2, 3, 4 and 5.

Vincent

--
You received this message because you are subscribed to the Google Groups "R package GGIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to RpackageGGIR...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/0554e09c-7f82-4689-a388-9f6b20ea6d1cn%40googlegroups.com.

Andy Whale

unread,

Dec 12, 2023, 11:22:23 AM12/12/23

to R package GGIR

Apologies, I'm just coming back this.

I did come across the timezone issue, some of my recordings include the seasonal clock change, however I tried various combinations of configtz and desiredtz with no luck. In addition a re-ran the analysis using an old recording that worked on a previous version of GGIR, that does NOT have a timezone transition, and that was producing the same error. I also tried rolling back GGIR to an older version, with the same results, so it looks to me like it's some interaction of GGIR and the system it's running on. I think I've ruled out conflicts with other R packages by only loading GGIR.

If it helps, my 'workaround' was to do the analysis on an old laptop with the old version of GGIR, so I could run the same file using the same commands on both machines, and compare the output?

Many thanks

Andy

Alex Dallaway

unread,

Dec 12, 2023, 11:46:55 AM12/12/23

to R package GGIR

Thanks Andy for the quick response. I'll have a go at that and report back.

Pretty frustrating as it seems to be a random error. If it is a bug in the update then I hope an older version will fix it.

Vincent van Hees

unread,

Dec 12, 2023, 11:54:50 AM12/12/23

to Alex Dallaway, R package GGIR

Alex,

Pretty frustrating as it seems to be a random error. If it is a bug in the update then I hope an older version will fix it.

If it is a bug then please help fix it. If everyone who encounters a bug in GGIR just continuous using older GGIR versions (that also have bugs btw) then it will be impossible to ever improve GGIR.

So, please respond to the questions I asked you earlier and help fix the bug (if it is one) for all GGIR users.

Thanks,

Vincent

--

You received this message because you are subscribed to the Google Groups "R package GGIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to RpackageGGIR...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/614be894-7b8d-4089-a541-4a3be174ca70n%40googlegroups.com.

Andy Whale

unread,

Dec 12, 2023, 3:42:41 PM12/12/23

to R package GGIR

Hi Vincent/Jairo,

Appreciate your efforts on this. If it's useful, I've run the same code on the same data across my 2 machines- I'll paste the code in a new message, this one is too long. For what it's worth both the code and the data are saved in cloud storage, so the code and data are identical in both cases. I've attached the report files that are generated from each machine, (I tried to attach the raw .bin file, but it's too big). The start time in meta/basic/I$header is correct in both recordings

The recording is 14 days, starting at around 2023-11-06 08:40:00. Uses the GENEActiv monitor. There's some non-wear during the day, but it was worn for most nights during the recording. The recording started and ended in the Europe/Dublin timezone, and there was no change in the local timezone during the recording.

New machine is running R 4.3.2 and GGIR 2.10-1, and produces the 'measurement starts at midnight or there is no midnight' warning.

Old machine is running R 4.2.1 and GGIR 2.8-2, and looks like it works as expected,

Please come back to me if I can provide anything further that might help you- I have several dozen recordings I can repeat this process on it it helps.

Many thanks

Andy

# do.part3.pdf = FALSE,
#-------------------------------

# Part 4 parameters:
#-------------------------------
excludefirstlast = FALSE,

includenightcrit = 16,
def.noc.sleep = 1,
sleepwindowType = "SPT",
# loglocation= c("~/IRFU HPU Dropbox/Andy Whale/Sleep Dropbox/2 - Data/sleeplog.csv"),

loglocation = c(),
outliers.only = FALSE,
criterror = 4,
relyonguider = FALSE,
sleeplogidnum = FALSE,
colid=1,
coln1=2,
do.visual = TRUE,
nnights = 14,

# desiredtz = "Europe/Dublin",
# configtz = "Europe/Dublin",

#-------------------------------
# Part 5 parameters:
#-------------------------------
# Key functions: Merging physical activity with sleep analyses
threshold.lig = c(30),
threshold.mod = c(100),
threshold.vig = c(400),
boutcriter = 0.8,
boutcriter.in = 0.9,
boutcriter.lig = 0.8,
boutcriter.mvpa = 0.8,
boutdur.in = c(10,20,30),
boutdur.lig = c(1,5,10),
boutdur.mvpa = c(1,5,10),
timewindow = c("WW"),

Report - Old Machine.bin.pdf

Report - New Machine.pdf

Andy Whale

unread,

Dec 12, 2023, 3:43:46 PM12/12/23

to R package GGIR

Attached here is the contents of meta/basic/ from both machines- it was too big to all fit in one message

ATB

Andy

Meta-Basic Old Machine.RData

Meta-Basic New Machine.RData

Alex Dallaway

unread,

Dec 13, 2023, 6:27:25 AM12/13/23

to R package GGIR

Hi Vincent,

Thank you for your responses, and apologies if I offended you, it was not intentional and I wish to help the academic community in any way possible.

I had already checked the time zone on my system and used the arguments to specify the correct time zone.

The recording was about 3 - 4 days.

Below is the old output, which appears to be correct.

I was using GENEActiv.

I have been using the same GGIR version for all analyses (3.0-0).

I have been able to reproduce the error and also run GGIR with the same output as the older one below. The solution was to change the R version back to 4.2.3 from 4.3.0. I have ran GGIR on the same data a few times to make sure and it is this change causing the issue.

Thank you again for your support Vincent and Andy.

Andy - try changing the R version in the global options and re-run GGIR on your new machine.

Lena Kushleyeva

unread,

Dec 14, 2023, 11:25:59 AM12/14/23

to R package GGIR

I'd give a shot trying to debug this, if someone could share a .bin file that they get this error with. I can't reproduce this issue with the couple of files that I have, using GGIR 3.0-0 and R 4.3.0

-Lena

Andy Whale

unread,

Dec 14, 2023, 11:28:30 AM12/14/23

to R package GGIR

Hi Lena,

Many thanks- I have several bin files, however the smallest is over 150MB so I think it's too big to attach here- do you have another way I might share the file with you?

ATB

Andy

Lena Kushleyeva

unread,

Dec 14, 2023, 11:36:52 AM12/14/23

to R package GGIR

Andy,

Maybe add the file to your google drive, and share the link to the uploaded file here? (If you have a gmail account, you also have access to google drive)

-Lena

Andy Whale

unread,

Dec 14, 2023, 11:41:53 AM12/14/23

to R package GGIR

Not sure about Google, but will a DropBox link work?

https://www.dropbox.com/scl/fi/oflla296fu3nisv7xtpd4/AWL_left-wrist_069602_2023-01-30-09-09-48.bin?rlkey=4b4yf91q8ix6z05o5eeszpbeg&dl=0

It's a 3 or so day recording I did on myself as a test. Let me know if I can provide any details, but the overall picture is as described above- working on my old machine with R 4.2.1 and GGIR 2.8-2, not working on a different machine with R 4.3.2 and GGIR 2.10-1, returns the no midnight warning in part 2 and won't create the visual reports;

Error in 1:nplots : result would be too long a vector
In addition: Warning message:
In max(P2daysummary_tmp$measurementday) :
no non-missing arguments to max; returning -Inf

Many thanks

Andy

Lena Kushleyeva

unread,

Dec 14, 2023, 11:47:25 AM12/14/23

to R package GGIR

Got the file, will try and report back.

Andy Whale

unread,

Dec 14, 2023, 2:13:06 PM12/14/23

to R package GGIR

Thank you so much, let me know if you need anything further.

Lena Kushleyeva

unread,

Dec 15, 2023, 2:26:53 AM12/15/23

to R package GGIR

Andy, Alex,

this looks like a very cool bug, thank you for catching it! You are right, it's related to the R version upgrade. I couldn't reproduce it at first because it only happens if your machine is in the GMT timezone!

I'll work on a fix on Friday, and i'll get back to you.

As for this warning:

In max(P2daysummary_tmp$measurementday) :
no non-missing arguments to max; returning -Inf

that'll go away once you upgrade to the latest version of GGIR.

-Lena

Andy Whale

unread,

Dec 15, 2023, 3:14:58 AM12/15/23

to R package GGIR

Outstanding, thank you! That is a very niche bug.

A

Alex Dallaway

unread,

Dec 15, 2023, 5:10:22 AM12/15/23

to R package GGIR

Hi Lena,

That is great. Glad to be of some help.

Kind regards,

Alex

Vincent van Hees

unread,

Dec 18, 2023, 7:38:35 AM12/18/23

to R package GGIR

Update:

To install the development version of GGIR that includes Lena's fix to the bug do:

install.packages("remotes')

library("remotes")

remotes::install_github("wadpac/GGIR")

This will become GGIR version 3.0-3 sometime in the upcoming weeks, even though it currently says that it is version 3.0-2.

Great job Lena for spotting and fixing this and thanks Andy and Alex for your cooperation!

Kind regards,

Vincent

To view this discussion on the web, visit https://groups.google.com/d/msgid/RpackageGGIR/88cdcf4d-4d31-49f7-a41e-4f3e0e762b1en%40googlegroups.com.

Andy Whale

unread,

Dec 18, 2023, 8:48:24 AM12/18/23

to R package GGIR

Perfect, thanks all

Andy

Alex Dallaway

unread,

Dec 19, 2023, 5:05:07 AM12/19/23

to R package GGIR

Thanks for sorting this out Lena and Vincent. And thanks Andy for helping. It's great using a package with such dedicated support.

Reply all

Reply to author

Forward