Help with finding a bundle that has the SP500?

[Dave S]

Hi All,

Thanks to all who help on this forum!

My two noob questions:  

   Is there an existing bundle I can ingest that has the S&P500?  (I checked, and the quantopian-quandl package doesn't have it.)

   Is there a symbol for the default benchmark data that I can use?  SPX and GSPC and ^GSPC fail. 

The simple algo needs to trade the same symbol as the benchmark to see if it beats buy and hold.

Thanks!

Optional reading:

I'm learning zipline by migrating my simple 50/200 MA crossover algo (from Quantopian).  Really simple.  I've enjoyed the migration and learning python and the tools. I've read lots of examples.

At this point, I think creating registering a custom yahoo bundle is too advanced for me right now.  I can't find the ".zipline" directory, and with anaconda I can't find out the zipline root, and thus I can't find the right place to put extension.py.  It's been a rabbit hole!)

[Ed Bartosh]

Hi Dave,

There is no existing bundle for S&P 500 as far as I know. However, you can ingest it using yahoo bundle if you put list of S&P 500 symbols to extensition.py

As for default benchmark data, I was able to ingest GSPC and SPY from using yahoo bundle just fine. And as far as I know you don't even have to load it as zipline does it for you when you run your algo.

I'm afraid you'll have to learn where .zipline/ directory is to progress further. On Windows you'll be in your home directory when you run 'Command prompt' application. It's C:/users/<your username> on Windows as far as I know. On Mac it's /Users/<your username>. I guess you're not using Unix :)

Hope it helps

Regards,

Ed

--
You received this message because you are subscribed to the Google Groups "Zipline Python Opensource Backtester" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Ed Bartosh]

Sorry, forgot to mention. You can still use load_bars_from_yahoo if you're afraid to use bundles.

It's deprecated, but still in the zipline codebase and may be even working. Not sure it will last long though.

--

BR,
Ed

[Dave S]

Hey Ed,

Thanks for offering a hand.  Really appreciate it.  Good to know I need to make yahoo bundle work.  That saves me time hunting elsewhere.

That extension.py is really beating me.  Are there any other files that should be in that same directory I can search for?

I'm running Mac OSX (which is Linux), and I can't find any doc between Anaconda and Zipline where exactly it should go.  

I have sprinkled extension.py files all around the place with no luck.  I have no directories starting with a period. 

As you know, Anaconda can create several different environments, and I'm running a Python 3.4 environment.  

This has been brutal.

Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Joe Jevnik]

The zipline root is based on your home. If it doesn't exist, you just need to create it. On the command like, you can do that by writing: `mkdir ~/.zipline`. Then you can create your extensions.py file in that directory.

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.

[Dave S]

Ed, Thanks again for all the help.  I know this will work now!  

[Ed Bartosh]

Hi Dave,

Yeah, I understand. It's not easy to even find .zipline directory without a bit of command line unix experience.

Let me try to explain you how to start in a bit more detailed way:

If you didn't run zipline you might not even have .zipline directory in your home directory.

So, first you need to launch Terminal application (press Cmd+Space and enter Terminal in the prompt). When terminal window opens you'll see shell prompt. Something like this:

$ 

Type 'zipline bundles' and press enter. You should see something like this:

$ zipline bundles

quandl <no ingestions>

quantopian-quandl <no ingestions>

$

This should make zipline to create .zipline directory in your home directory /Users/<your username> with empty extension.py in it. You can list directory content using 'ls' command:

$ ls -la ~/.zipline

total 0

drwxr-xr-x   3 ed  staff   102B Oct 16 11:57 ./

drwx------  53 ed  staff   1.8K Oct 16 11:57 ../

-rw-r--r--   1 ed  staff     0B Oct 16 11:57 extension.py 

$

Now you can prepare your ~/.zipline/extension.py to use yahoo bundle. Open it with nano text editor:

$ nano ~/.zipline/extension.py

and add this content:

from zipline.data.bundles import register, yahoo_equities

# these are the tickers you would like data for
equities = {
    'AAPL',
    'MSFT',
    'GOOG',
}
register(
    'my-yahoo-equities-bundle',  # name this whatever you like
    yahoo_equities(equities),
)

as suggested here: http://www.zipline.io/bundles.html#yahoo-bundle-factories.

You can replace equities list with S&P 500 equities later. At this point I'd suggest to stick with 3 suggested tickers to make sure zipline can ingest them.

Now you should be ready to run zipline ingest for your bundle:

$ zipline ingest -b my-yahoo-equities-bundle

Merging daily equity files:  [-------------#----------------------]  2#]  100%

Merging daily equity files:  [####################################]   

Downloading Yahoo adjustment data:   [####################################]  100%

$

If all goes well you should be able to see your bundle in 'zipline bundles' output:

$ zipline bundles

my-yahoo-equities-bundle 2016-10-16 09:16:25.095616

quandl <no ingestions>

quantopian-quandl <no ingestions>

And the last step is to run test algo with your bundle. If you're lucky you'll see something like that:

$ zipline run --algofile ./zipline/examples/buyapple.py --start 2000-01-01 --end 2016-01-01 --bundle my-yahoo-equities-bundle

[2016-10-16 09:24:49.095224] INFO: Loader: Cache at /Users/ebartosh/.zipline/data/^GSPC_benchmark.csv does not have data from 1990-01-02 00:00:00+00:00 to 2016-10-12 00:00:00+00:00.

Downloading benchmark data for '^GSPC'.

[2016-10-16 09:24:54.471509] INFO: Performance: after split: sid: Equity(1 [AAPL]), amount: 2340.0, cost_basis: 55.65, last_sale_price: 101.25

[2016-10-16 09:24:54.471665] INFO: Performance: returning cash: 0.0

[2016-10-16 09:24:57.333482] INFO: Performance: after split: sid: Equity(1 [AAPL]), amount: 28220.0, cost_basis: 15.77, last_sale_price: 88.99

[2016-10-16 09:24:57.333620] INFO: Performance: returning cash: 0.0

[2016-10-16 09:25:03.676233] INFO: Performance: after split: sid: Equity(1 [AAPL]), amount: 361060.0, cost_basis: 18.23, last_sale_price: 645.57

[2016-10-16 09:25:03.676354] INFO: Performance: returning cash: 0.0

[2016-10-16 09:25:04.865082] INFO: Performance: Simulated 4025 trading days out of 4025.

[2016-10-16 09:25:04.865228] INFO: Performance: first open: 2000-01-03 14:31:00+00:00

[2016-10-16 09:25:04.865365] INFO: Performance: last close: 2015-12-31 21:00:00+00:00

...

Good luck with exploring power of command line tools :) !

Ed

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed

[Misc]

Hi, 

I've ingested the bundle successfully and it appears in the command line and i can see it the file manager.

for some reason when i try to access the bundle from jupyter notebook it doesnt recognize the bundle and throws the following error: UnknownBundle: No bundle registered with the name 'my-yahoo-equities-bundle2'

It was actually working earlier until my PC did some updates, and now it doesnt work which makes no sense.

thanks,

[Pierre villenave]

Hi, when following the steps and trying to ingest the data with the command: zipline ingest b- my-yahoo-equities-bundle, I get the following error:

Error: Got unexpected extra arguments (b- my-yahoo-equities-bundle)

PierreAsMacBook:~ Pierre$ zipline ingest b- my-yahoo-equities-bundle

/Users/Pierre/anaconda/lib/python2.7/site-packages/zipline/__main__.py:50: UserWarning: Failed to load extension: '/Users/Pierre/.zipline/extension.py'

cannot import name yahoo_equities

  os.environ,

Usage: zipline ingest [OPTIONS]

I'm new to programming and to zipline and I don't really understand how to handle this.. 

[Ed Bartosh]

Hi Pierre,

Yahoo bundle is not going to be supported as Yahoo finance API was discontinued: https://forums.yahoo.net/t5/Yahoo-Finance-help/Is-Yahoo-Finance-API-broken/td-p/250503

You can still use quandl bundle if it suits your needs.

Regards,

Ed

To unsubscribe from this group and stop receiving emails from it, send an email to zipline+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

BR,
Ed