Installing a requirement before setup.py

[Francesc Alted]

Hi,

I am in the process of implementing support for Blosc2 (see direct-chunking-blosc2 branch).  The plan is to use the wheels for python-blosc2 so as to avoid the costly task of compiling Blosc2 inside PyTables; this is also a much cleaner procedure.  However, I need to install the packages in requirements.txt prior to execute setup.py, but apparently this is not possible (yet).

I propose to modify the ci.yaml file to add a line similar to:

https://github.com/PyTables/PyTables/blob/master/.github/workflows/wheels.yml#L40

Do you think there should be any issue in doing that?

Also, I tried to modify the files in .github/workflows (in the branch) but I am getting this:

"""

To https://github.com/PyTables/PyTables.git
 ! [remote rejected]   direct-chunking-blosc2 -> direct-chunking-blosc2 (refusing to allow a Personal Access Token to create or update workflow `.github/workflows/ci.yml` without `workflow` scope)
error: failed to push some refs to 'https://github.com/PyTables/PyTables.git'

"""

Hints?

Thanks in advance!

--

Francesc Alted

[Antonio Valentino]

Der Francesc,

Il 21/08/22 13:59, Francesc Alted ha scritto:

> Hi,
>
> I am in the process of implementing support for Blosc2 (see
> direct-chunking-blosc2 branch). The plan is to use the wheels for
> python-blosc2 so as to avoid the costly task of compiling Blosc2 inside
> PyTables; this is also a much cleaner procedure. However, I need to
> install the packages in requirements.txt prior to execute setup.py, but
> apparently this is not possible (yet).
>
> I propose to modify the ci.yaml file to add a line similar to:
>
> https://github.com/PyTables/PyTables/blob/master/.github/workflows/wheels.yml#L40
>
> Do you think there should be any issue in doing that?

the only problem that I see is that ci.yaml uses a conda environment do,
in principle, we should use the c-blosc2 package available in conda-forge.
Is that an issue for you?

In any case as a temporary solution IMHO using a mixed environment with
python-blosc2 installed with pip is not a big issue.

> Also, I tried to modify the files in .github/workflows (in the branch) but
> I am getting this:
>
> """
> To https://github.com/PyTables/PyTables.git
> ! [remote rejected] direct-chunking-blosc2 -> direct-chunking-blosc2
> (refusing to allow a Personal Access Token to create or update workflow
> `.github/workflows/ci.yml` without `workflow` scope)
> error: failed to push some refs to 'https://github.com/PyTables/PyTables.git
> '
> """
>
> Hints?

Yes you should use an access token with the appropriate authorization
scopes. In the specific case it seems that the one that you are using
now is missing the "workflow" one.


kind regards
--
Antonio Valentino

[Francesc Alted]

Dear Antonio,

On Sun, Aug 21, 2022 at 5:35 PM Antonio Valentino <antonio....@tiscali.it> wrote:

Der Francesc,

Il 21/08/22 13:59, Francesc Alted ha scritto:
> Hi,
>
> I am in the process of implementing support for Blosc2 (see
> direct-chunking-blosc2 branch).  The plan is to use the wheels for
> python-blosc2 so as to avoid the costly task of compiling Blosc2 inside
> PyTables; this is also a much cleaner procedure.  However, I need to
> install the packages in requirements.txt prior to execute setup.py, but
> apparently this is not possible (yet).
>
> I propose to modify the ci.yaml file to add a line similar to:
>
> https://github.com/PyTables/PyTables/blob/master/.github/workflows/wheels.yml#L40
>
> Do you think there should be any issue in doing that?

the only problem that I see is that ci.yaml uses a conda environment do,
in principle, we should use the c-blosc2 package available in conda-forge.
Is that an issue for you?

Ok.  It took a while (damn, I had forget how difficult is to deal with the intricacies of supporting binary libs on different platforms), but now I have a solution that should support both the c-blosc2 in conda-forge and the blosc2 wheel in PyPI.  For now, I am using just the blosc2 wheel which is a more uniform solution for all the environments (all tests in CI are green now :-).

 

In any case as a temporary solution IMHO using a mixed environment with
python-blosc2 installed with pip is not a big issue.

To be clear, only either c-blosc2 in conda-forge or blosc2 wheel are necessary, not both.  I am choosing the later just for consistency.

 

Thanks,

Francesc