Task for pip-compile requirements.in

[Marcin Raczyński]

I have a task compile_requirements which compiles requirements.in file into requirements.txt (see https://github.com/jazzband/pip-tools):

def task_compile_requirements():
    return {
        'actions': ['pip-compile --output-file %(targets)s %(dependencies)s'],
        'file_dep': ['requirements.in'],
        'targets': ['requirements.txt'],
    }

In my repository I commited both requirements.in and requirements.txt files.

When new user clones this repository and runs doit, task compile_requirements is not "uptodate" and it compiles requirements.in into requirements.txt.

This not what I want.

I would like to run compilation only if requirements.in file changed, not in the first doit run.

Do you know how can I achieve my goal?

[Eduardo Schettino]

I guess you should write a custom `uptodate` function [1].

Actually there is a  command that do just what you want, but it would

require every user to use the command `reset-dep` once after a clone. [2]

If the users can handle a more complex workflow you could use

[1] http://pydoit.org/dependencies.html#attr-uptodate
[2] http://pydoit.org/cmd_other.html#reset-dep

cheers

[Marcin Raczyński]

Thanks schettino72!

The command `reset-dep` is close to my goal but:

1. It must be called by hand
2. It resets all tasks

I also think that custom `uptodate` function is what I should write. I would like to use code of `file_dep` implementation from `doit` as much as possible without copy&pasting it. 

It seems not possible of reusing it now. The code which checks if the file is modified is hidden in `Dependency.get_status` method and I don't have access to `state` object to reuse `MD5Checker` class.

Cheers,

Marcin

[Marcin Raczyński]

I found simple solution for this problem:

from doit.tools import run_once

from doit.action import CmdAction

first_run = False

def task__set_first_run():

    def action():

        global first_run

        first_run = True

    return {

        'actions': [action],

        'uptodate': [run_once]

    }

def task_compile_requirements():

    def create_cmd():

        if first_run:

            return ''

        else:

            return 'pip-compile --output-file %(targets)s %(dependencies)s'

    return {

        'actions': [CmdAction(create_cmd)],

        'file_dep': ['requirements.in'],

        'targets': ['requirements.txt'],

        'task_dep': ['_set_first_run'],

    }

[Marcin Raczyński]

My solutions works, preventing compiling requirements.in when user runs `doit` command for the first time. But this solutions has some problems in context of pip-compile:

1. It does not work when user first modify `requirement.in` than run `doit` for the first time (compilation will not run).
2. It compiles `requirements.in` (and - as a side effect - upgrade packages version in `requirements.txt`) even if user modify comment only in `requirements.in`.
3. If user what to upgrade packages versions in `requirements.txt`, simple `doit` does not work if requirements.in is not modified. The user must provide '-a' option with explicit task name in this situation (doit run -a compile_requirements).

Solutions for these problems:

Ad. 1. Removing checking if this is first run of doit.

Ad. 2. Removing auto running of pip-compilation (removing this task from `DOIT_CONFIG.default_tasks` and from other tasks dependencies) .

Ad. 3. Removing `requirements.in` dependency (file_dep).

I ended up with this simple code below. It requires conscious user's decision to generate `requirements.txt`. The user must run `doit compile_requirements` every time he wants to upgrade packages in `requirements.txt` using pip-compile.

Modifying `requirements.in` is not enough nor necessary.

DOIT_CONFIG = {

    'default_tasks': [],

}

def task_compile_requirements():

    return {

        'actions': ['pip-compile --output-file %(targets)s requirements.in'],

        'targets': ['requirements.txt'],

    }