Yahoo Bundle Not Working

[Vishal Gandhi]

I am experiencing problem with 'zipline ingest'. In my case, it does download quantopian-quandl , but it doesn't do anything with yahoo or at-least doesn't show any message on the screen.  I've added following code to my extension.py file:

symbols = (
        'AMD',
        'AEG',
        'ARC',
)
from zipline.data.bundles import yahoo_equities, register
register('my_yahoo_bundle', yahoo_equities(symbols,'2016-07-21','2016-07-29'))

I see progress-bar for quantopian-quandl only.  No errors or warnings.
After some trouble-shooting, I am thinking that the ingest function for yahoo bundle is not being called!
 Any thoughts?

[Scott Sanderson]

How are you invoking `zipline ingest`?  Registering a bundle makes it so that you can do `zipline ingest --bundle my-bundle`, but the default bundle is still quantopian-quandl.

In your case, it looks like you'd want to do `zipline ingest --bundle my_yahoo_bundle`.

- Scott

[Dario UK]

Hi Vishal,

I'd like to create a custom yahoo bundle as ell, but I don't find extension.py file. Where it should be located?

many thanks

Dario

[Dario Dallefrate]

No worries, I found it and I created my own bundle but, when I try to ingest, I get the following SQL error:

Sqlite3.OperationError: unable to open database file

I use anaconda2 64 bit on Windows 7

Any suggestion to solve the issue?

Regards
Dario

[Rich Frank]

Hi Dario,

Sorry for the trouble - likely it's this bug that is now fixed on master.  You might install the latest zipline from github (or the anaconda quantopian ci channel).  That fix will be included in the next release, if you'd rather wait.

-Rich

[Dario Dallefrate]

Hi Rich
Thank you for your reply.
Is it possible to install the latest master build using conda instruction in the command line?

Best regards
Dario

[Rich Frank]

Yep, you can add our ci channel to your install command with:

conda install -c Quantopian -c https://conda.anaconda.org/Quantopian/label/ci zipline

[Dario]

Thanks Rich

I did that but I still get the following issue

File "C:\Users\Public\AppData\Local\Continuum\Anaconda2\lib\site-packages\zipine\data\us_equity_pricing.py", line 824, in __init__

  self.conn = sqlite3.connect(conn_or_path)

lite3.OperationalError: unable to open database file

Best regards

Dario

[Rich Frank]

Hmm, is that still on ingest?  What zipline version does conda say you have installed?

-Rich

[Dario]

Hi Rich,

it still happens when I ingest my bundle

here is the list of the package I have

# packages in environment at C:\Users\Admin\AppData\Local\Continuum\Miniconda2:

#

bottleneck                1.0.0               np110py27_1  

click                     6.6                      py27_0  

conda                     4.1.11                   py27_0  

conda-env                 2.5.2                    py27_0  

console_shortcut          0.1.1                    py27_1  

contextlib2               0.5.3                    py27_0  

cython                    0.24.1                   py27_0  

decorator                 4.0.10                   py27_0  

menuinst                  1.4.1                    py27_0  

mkl                       11.3.3                        1  

multipledispatch          0.4.8                    py27_0  

networkx                  1.11                     py27_0  

numexpr                   2.6.0               np110py27_0  

numpy                     1.10.4                   py27_2  

pandas                    0.17.1              np110py27_0  

pandas-datareader         0.2.1                    py27_0  

patsy                     0.4.1                    py27_0  

pip                       8.1.2                    py27_0  

pycosat                   0.6.1                    py27_1  

pycrypto                  2.6.1                    py27_4  

python                    2.7.12                        0  

python-dateutil           2.5.3                    py27_0  

pytz                      2016.6.1                 py27_0  

pyyaml                    3.11                     py27_4  

bcolz                     0.12.1              np110py27_0    Quantopian

cachetools                1.1.6                    py27_0    Quantopian

cyordereddict             0.2.2                    py27_0    Quantopian

intervaltree              2.1.0                    py27_0    Quantopian

logbook                   0.12.5                   py27_0    Quantopian

sortedcontainers          1.4.4                    py27_0    Quantopian

zipline                   1.0.1           np110py27_2_g85327d8    Quantopian

requests                  2.10.0                   py27_0  

requests-file             1.4                      py27_0  

ruamel_yaml               0.11.14                  py27_0  

scipy                     0.17.1              np110py27_1  

setuptools                23.0.0                   py27_0  

six                       1.10.0                   py27_0  

sqlalchemy                1.0.13                   py27_0  

statsmodels               0.6.1               np110py27_1  

toolz                     0.8.0                    py27_0  

vs2008_runtime            9.00.30729.1                  2  

wheel                     0.29.0                   py27_0  

Many thanks for your support

Regards

Dario

[Rich Frank]

Hmm, looks like you've got commit 85327d8 installed, which is from May, but that bug wasn't fixed until July.  Does it fail for you with a more recent commit?

[Dario]

Hi Rich

what it seems strange to me is that  the command 

Enter code here...
conda install -c Quantopian --override-channels -c https://conda.anaconda.org/Quantopian/label/ci zipline

doesn't seem to install the latest commit

Please see here the output of unistalling my current zipline and installing a new one with the above command. The versions are always the same: g85327d8

Best regards

Dario

Enter code here...

C:\Users\DALLDA01\conda uninstall zipline

Fetching package metadata .........

Solving package specifications: ..........

Package plan for package removal in environment C:\Users\DALLDA01\AppData\Local\

Continuum\Miniconda2:

The following packages will be REMOVED:

    zipline: 1.0.1-np110py27_2_g85327d8 Quantopian

Proceed ([y]/n)? y

Unlinking packages ...

[      COMPLETE      ]|##################################################| 100%

C:\Users\DALLDA01>conda install -c Quantopian --override-channels -c https://conda.anaconda.org/Quantopian/label/ci zipline

Fetching package metadata .......

Solving package specifications: ..........

Package plan for installation in environment C:\Users\DALLDA01\AppData\Local\Con

tinuum\Miniconda2:

The following NEW packages will be INSTALLED:

    zipline: 1.0.1-np110py27_2_g85327d8 Quantopian

Proceed ([y]/n)?

[Rich Frank]

I don't know exactly how conda resolves packages to install, but looks like the channel order matters.  I specified the ci channel first, and it installed the latest ci package successfully:

(zp) C:\Users\Administrator>conda install -c https://conda.anaconda.org/Quantopian/label/ci -c quantopian zipline
Fetching package metadata .............
Solving package specifications: ..........

Package plan for installation in environment C:\Users\Administrator\Miniconda2\envs\zp:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    mkl-11.3.3                 |                1       110.0 MB
    cachetools-1.1.6           |           py27_0          23 KB  quantopian
    click-6.6                  |           py27_0          98 KB
    contextlib2-0.5.3          |           py27_0          12 KB
    cyordereddict-0.2.2        |           py27_0          36 KB  quantopian
    cython-0.24.1              |           py27_0         2.1 MB
    decorator-4.0.10           |           py27_0          12 KB
    logbook-0.12.5             |           py27_0         114 KB  quantopian
    multipledispatch-0.4.8     |           py27_0          11 KB
    numpy-1.10.4               |           py27_2         2.9 MB
    pytz-2016.6.1              |           py27_0         171 KB
    requests-2.11.1            |           py27_0         624 KB
    six-1.10.0                 |           py27_0          16 KB
    sortedcontainers-1.4.4     |           py27_0          61 KB  quantopian
    sqlalchemy-1.0.13          |           py27_0         1.3 MB
    toolz-0.8.0                |           py27_0          78 KB
    bcolz-0.12.1               |      np110py27_0         331 KB  quantopian
    bottleneck-1.0.0           |      np110py27_1         145 KB
    intervaltree-2.1.0         |           py27_0          45 KB  quantopian
    networkx-1.11              |           py27_0         1.1 MB
    numexpr-2.6.0              |      np110py27_0         139 KB
    patsy-0.4.1                |           py27_0         329 KB
    python-dateutil-2.5.3      |           py27_0         236 KB
    requests-file-1.4          |           py27_0           4 KB
    scipy-0.17.1               |      np110py27_1        11.5 MB
    pandas-0.17.1              |      np110py27_0         5.8 MB
    pandas-datareader-0.2.1    |           py27_0          48 KB
    statsmodels-0.6.1          |      np110py27_1         4.6 MB
    zipline-1.0.1              |np110py27_426_g7f575c5        1007 KB  Quantopian/label/ci

[Dario]

Thank you Rich,

You are right, I see the difference

Best regards

Dario

[Dario Biasini]

HI Dario,

I cant find the extension.py (or extensions.py ? ) file in my zipline directory... how did you find it?

best,

Dario

[Walter Kissling]

It should be in "C:\Users\[your_username]\.zipline"