Timezone for :new_records_only

[Yves-Eric Martin]

Hi all,

I am having an issue with the :new_records_only option of database extractions: In my transactional system, all database-side timestamps are in UTC (Rails does its magic to convert everything to local time on the frontend). But since the ETL process runs separately from Rails, the automagic timezone conversion does not apply, and the :new_records_only option generates a query with a local timestamps instead of UTC. As a result, the extraction fails to pick up the latest changes (at GMT+9, I loose 9 hours worth of data).

How can I make the database extraction generate the right query, with a UTC timestamp? Things I tried but did not work so far:

- Time.zone = 'Asia/Tokyo'

- ENV['TZ'] = 'Asia/Tokyo'

- ActiveRecord::Base.default_timezone = 'Asia/Tokyo'

Thank you,

-- 

Yves-Eric

[Yves-Eric Martin]

Sorry for replying to myself but I just cracked it. In case someone else stumbles upon this post, the solution was simply:

    ActiveRecord::Base.default_timezone = :utc

I put that in a shared config file that all control files require, and now everything works fine.

One warning: this will also change the ETL Execution database from local to UTC timestamps. 

-- 

Yves-Eric

[Thibaut Barrère]

Hi Yves-Eric,

glad you found out, and thanks for sharing too, this will definitely help someone at some point.

thanks!

Thibaut

--

https://www.wisecashhq.com

http://www.logeek.fr

--
You received this message because you are subscribed to the Google Groups "ActiveWarehouse Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to activewarehouse-d...@googlegroups.com.
To post to this group, send email to activewareh...@googlegroups.com.
Visit this group at http://groups.google.com/group/activewarehouse-discuss.
For more options, visit https://groups.google.com/d/optout.

[Yves-Eric Martin]

Hi Thibaut ^^/

I am kind of surprised nobody ran into this before (or at least, googling the issue turned up nothing)... Is it supposed to work properly by default (so the issue is just with my setup)? If not, then can I start a pull request to either:

1) add that information to the documentation

or 2) change the default behavior of the etl script to default to UTC

Thank you,

-- 

Yves-Eric

[Thibaut Barrère]

Hello!

I am kind of surprised nobody ran into this before (or at least, googling the issue turned up nothing)... Is it supposed to work properly by default (so the issue is just with my setup)?

Actually I have no idea, because I didn't use this specific options yet - and also, the usage of ETL in general is a bit confidential in the Ruby world (the latest release of aw-etl has been downloaded only 8400 times).

So since this is not a "high traffic" gem, you can definitely stumble upon rough edges.

 

If not, then can I start a pull request to either:

1) add that information to the documentation

or 2) change the default behavior of the etl script to default to UTC

I'd vote to tweak the documentation, because I'm pretty sure some people depend on the current behaviour, and also that there will be other places in the code (I suspect) where a similar change would be needed.

thanks!

-- Thibaut

 

[Yves-Eric Martin]

For the record: doc updated. Thanks!

-- 

Yves-Eric

[Thibaut Barrère]

Thank you :-)

-- Thibaut