BTC minute data ingest error

[Alex Jiang]

Hi zipliner,

I am trying to ingest custom btc minute data. I searched around here and was able to rebuild the calendar and exception files, but I still get the keyerror as attached.

BTW, I was able to ingest btc daily data.

any help would be appreciated.

[Ed Bartosh]

Hi Alex,

Your calendar uses US/Eastern time zone, but your data is in UTC. You can try to

return timezone("US/Eastern")to return timezone("UTC")

in exchange_calendar_gdax.py and ingest your data again.

Regards,

Ed

--
You received this message because you are subscribed to the Google Groups "Zipline Python Opensource Backtester" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Alex Jiang]

Hi Ed,

first of all, thank you for your contribution to this brilliant package. I tried your suggestion but still get the same error.

Thanks,

Alex

[Bryan Yu]

Alex,

You've got gaps in your data. If you review the following timestamps. You'll need to backfill your data or write a module to handle it.

1/16/18 23:33

1/16/18 23:39

1/16/18 23:40

2/1/18 4:58

2/1/18 4:59

2/1/18 8:09

2/28/18 8:31

2/28/18 8:32

2/28/18 8:34

[Alex Jiang]

Hi Bryan,

Thanks, I tried the data attached and I checked no gaps in the file, still the same error...

Best,

Alex

[Ed Bartosh]

Hi Alex,

I was able to ingest your latest BTC.csv with below settings. I used latest zipline master.

extension.py:

from zipline.utils.calendars import register_calendar

from zipline.data.bundles import register

from zipline.data.bundles.csvdir import csvdir_equities

from zipline.utils.calendars.exchange_calendar_gdax import GDAXExchangeCalendar

import pandas as pd

register_calendar('GDAX', GDAXExchangeCalendar(start=pd.Timestamp('2016-04-01', tz='UTC'),

                                               end=pd.Timestamp('2018-03-08', tz='UTC')))

register('gdax-bundle', csvdir_equities(['minute']), calendar_name = 'GDAX', minutes_per_day=24*60)

exchange_calendar_gdax.py

from datetime import time

from zipline.utils.memoize import lazyval

from .trading_calendar import TradingCalendar

from pytz import utc

from pandas.tseries.offsets import Day

class GDAXExchangeCalendar(TradingCalendar):

    """

    Exchange calendar for GDAX

    Open Time: 00:00 am, UTC

    Close Time: 11:59 pm, UTC

    """

    @property

    def name(self):

        return "GDAX"

    @property

    def tz(self):

        return utc

    @property

    def open_time(self):

        return time(0, 0)

    @property

    def close_time(self):

        return time(23,59)

    @lazyval

    def day(self):

        return Day()

Directory structure:

$ find ./csv/

./csv/

./csv//minute

./csv//minute/BTC.csv

Ingesting:

$ CSVDIR=./csv zipline ingest -b gdax-bundle

Loading custom pricing data:   [####################################]  100% | BTC: sid 0

Loading custom pricing data:   [####################################]  100%

Merging minute equity files:  [####################################] 

Hope it helps

Regards,

Ed

--

You received this message because you are subscribed to the Google Groups "Zipline Python Opensource Backtester" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Alex Jiang]

Hi Ed,

Thanks, I will test it in a while.

Best,

Alex

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Alex Jiang]

It magically works, thank you Ed!

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Dragan Milosevic]