Custom bundle using local CSVs

[C I]

I've looked at the documentation on writing a new bundle and had a look at the Yahoo bundle in Github, but was hoping someone has created a tutorial targeted at a slightly more novice level.  Is there such a thing?  Has anyone already written an investor that takes a CSV as input?

If not, what exactly is the ingest function supposed to do?  If seems like it needs to yield a security id and a data frame – is that roughly correct?

[Ed Bartosh]

Hi,

You can look at my work in progress: https://github.com/bartosh/zipline/blob/master/zipline/data/bundles/csvdir.py and ask questions if any. So far I was able to load minute data from CSV files and run simple algo in minute mode. However, if algo uses daily history it needs also daily data and dividends&splits data, so I'm going to combine Yahoo bundle that loads daily data and dividends&splits data with what I did.

Regards,

Ed

2016-09-27 7:21 GMT+03:00 C I <ci2...@gmail.com>:

I've looked at the documentation on writing a new bundle and had a look at the Yahoo bundle in Github, but was hoping someone has created a tutorial targeted at a slightly more novice level.  Is there such a thing?  Has anyone already written an investor that takes a CSV as input?

If not, what exactly is the ingest function supposed to do?  If seems like it needs to yield a security id and a data frame – is that roughly correct?

--
You received this message because you are subscribed to the Google Groups "Zipline Python Opensource Backtester" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Amit Patil]

I have csv files for each stock with ohlcv + some other custom columns. I can import few stocks and run algorithm. But if I try get say all stocks in s&p 500, it gives me memory error. so thought I need to create custom data source. I want to follow directions on zip line github wicki. But there is no data_source to "from zipline.sources.data_source import DataSource. What is current proper way to get data from 500 different files and run some algorithm or to convert these files to a data bundle.
I appreciate any help. Thank you.

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Richard P]

Following along in Ed's footsteps, here is another basic example of creating a custom data bundle from local csv files and the steps needed to both test and run it:

http://www.prokopyshen.com/create-custom-zipline-data-bundle

Richard

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+u...@googlegroups.com.

[Amit Patil]

Thank you Ed and Richard,
          So far what I find is Record function is not recording price right. Data in csv and record don't match. if I  print (data.current(symbol('AAPL'), 'SMA50')). it just prints Nan.
Can some one please help me?
    

''store aapl,msft,spy csv files in one directory ( I had it in Work_Data directory). hard code path on line #87 in test_Own_Data_zipline.py.

add code in extension file to your extention file. ingest my_custom_bundle.

Attached png is the result I get. aglo is basic algo. just record and order 10 aapl stock from zipline examples.

I appreciate your help.''

On Friday, October 7, 2016 at 9:52:24 AM UTC-4, Richard Prokopyshen wrote:

Following along in Ed's footsteps, here is another basic example of creating a custom data bundle from local csv files and the steps needed to both test and run it:

http://www.prokopyshen.com/create-custom-zipline-data-bundle

Richard

On 9/27/2016 3:54 AM, Ed Bartosh wrote:

Hi,

You can look at my work in progress: https://github.com/bartosh/zipline/blob/master/zipline/data/bundles/csvdir.py and ask questions if any. So far I was able to load minute data from CSV files and run simple algo in minute mode. However, if algo uses daily history it needs also daily data and dividends&splits data, so I'm going to combine Yahoo bundle that loads daily data and dividends&splits data with what I did.

Regards,

Ed

2016-09-27 7:21 GMT+03:00 C I <ci2...@gmail.com>:

I've looked at the documentation on writing a new bundle and had a look at the Yahoo bundle in Github, but was hoping someone has created a tutorial targeted at a slightly more novice level.  Is there such a thing?  Has anyone already written an investor that takes a CSV as input?

If not, what exactly is the ingest function supposed to do?  If seems like it needs to yield a security id and a data frame – is that roughly correct?

--
You received this message because you are subscribed to the Google Groups "Zipline Python Opensource Backtester" group.

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Ed Bartosh]

Hi Amit,

Your code in extension.py uses my_custom_data bundle. I don't have it in my zipline sources and can't reproduce this.

I'd suggest you to use latest zipline release 1.0.2 and my csvdir bundle loader.

Regards,

Ed

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Amit Patil]

Hi Ed,
   I am getting ERROR: zipline.data.bundles.csvdir: CSVDIR environment variable is not set
   I did setup env variable in System from the Control Panel, selecting Advanced system settings, and clicking Environment Variables.
   Set CSVDIR as C:\data . This data folder has folders daily and minute. daily folder contains in my csv's
   Please help.
   Thank you.

[Ed Bartosh]

Hi Amit,

>  I am getting ERROR: zipline.data.bundles.csvdir: CSVDIR environment variable is not set

Can you explain how do you run zipline ingest?

Can you run command prompt(cmd) and type 'set' in there? Do you see CSVDIR variable in the output?

If you do - just run zipline ingest there to see if it works.

Regards,

Ed

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[hamed shafiee]

Hi Ed,

I thinks its been a long time after the last post here. I need some help.

Is there any way we can use both minute data and daily data at the same time in our back tests? I really appreciate your help. If yes, how can I use both bundles in my backtest?

Thanks,

To unsubscribe from this group and stop receiving emails from it, send an email to zip...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed