Making command-line args available to deeply-nested functions

[Loris Bennett]

Hi,

TL;DR:

If I have a command-line argument for a program, what is the best way
of making this available to a deeply-nested[1] function call without
passing the parameter through every intermediate function?

Long version:

If I have, say, a command-line program to send an email with a
personalised salutation, a naive approach to the function calls might
look like the following

create_email(..., args.salutation_server_credentials)
create_body(..., args.salutation_server_credentials)
create_salutation(..., args.salutation_server_credentials)

where args.salutation_server_credentials could be given on the
command-line or read from a configuration file by the top-level, but is
only ever actually needed by the create_salutation function.

I can see that the top-level could just create an object from a class
which encapsulates everything, but what if I want to keep the salutation
generation separate, so that I can have a separate program which just
generates the salutation and print it to the terminal?

I guess I am really asking how to avoid "passing through" arguments to
functions which only need them to call other functions, so maybe the
answer is just to avoid nesting.

Cheers,

Loris


Footnotes:

[1] Is a TL;DR allowed to have footnotes? Probably not, but just to
clarify, I would consider the third of three levels as being
already "deeply-nested".

--
This signature is currently under construction.

[Julio Di Egidio]

On Friday, 20 August 2021 at 11:54:00 UTC+2, Loris Bennett wrote:
> Hi,
>
> TL;DR:
>
> If I have a command-line argument for a program, what is the best way
> of making this available to a deeply-nested[1] function call without
> passing the parameter through every intermediate function?

To not pass arguments you need shared state ("global variables"): and options in shard state, unless you have very good reasons to do otherwise, is simply a no no.

<snip>

> I can see that the top-level could just create an object from a class
> which encapsulates everything, but what if I want to keep the salutation
> generation separate, so that I can have a separate program which just
> generates the salutation and print it to the terminal?

Yes, that's basically the way to go: parse arguments into a structure (an object) that contains all options/parameters then pass that down. Next level: some sections of your code may require a certain subset of those options, some may require some other, so you would structure your options object in sub-objects for the various sets of correlated options, then rather pass just the sub-object(s) that are relevant to the section of code you are calling. Variations are of course possible, anyway that's the basic idea.

Also have a look at the "argparse" library, it does all the heavy lifting for the parsing and creation of those objects, definitely advised for in non trivial cases: <https://docs.python.org/3/library/argparse.html>.

> I guess I am really asking how to avoid "passing through" arguments to
> functions which only need them to call other functions, so maybe the
> answer is just to avoid nesting.

No, you don't get rid of code structure just not to pass arguments to a function... Code may be poorly structured, but that's another story.

HTH,

Julio

[Loris Bennett]

Julio Di Egidio <ju...@diegidio.name> writes:

> On Friday, 20 August 2021 at 11:54:00 UTC+2, Loris Bennett wrote:
>> Hi,
>>
>> TL;DR:
>>
>> If I have a command-line argument for a program, what is the best way
>> of making this available to a deeply-nested[1] function call without
>> passing the parameter through every intermediate function?
>
> To not pass arguments you need shared state ("global variables"): and
> options in shard state, unless you have very good reasons to do
> otherwise, is simply a no no.

Doesn't that slightly depend on the size of your "globe"? If a program
does a few, in some sense unrelated things, and, say, only runs for a
few minutes, could you not decide to make a particular parameter global,
even though only one function needs it? In general, however, I would
also avoid this.

> <snip>
>> I can see that the top-level could just create an object from a class
>> which encapsulates everything, but what if I want to keep the salutation
>> generation separate, so that I can have a separate program which just
>> generates the salutation and print it to the terminal?
>
> Yes, that's basically the way to go: parse arguments into a structure (an
> object) that contains all options/parameters then pass that down. Next level:
> some sections of your code may require a certain subset of those options, some
> may require some other, so you would structure your options object in
> sub-objects for the various sets of correlated options, then rather pass just
> the sub-object(s) that are relevant to the section of code you are calling.
> Variations are of course possible, anyway that's the basic idea.
>
> Also have a look at the "argparse" library, it does all the heavy lifting for
> the parsing and creation of those objects, definitely advised for in non trivial
> cases: <https://docs.python.org/3/library/argparse.html>.

I am already using 'argparse' ('configargparse' actually). What aspect
should I be looking at in order to produce "sub-objects"?

>> I guess I am really asking how to avoid "passing through" arguments to
>> functions which only need them to call other functions, so maybe the
>> answer is just to avoid nesting.
>
> No, you don't get rid of code structure just not to pass arguments to
> a function... Code may be poorly structured, but that's another
> story.

As I am writing new code it is more a question of imposing structure,
rather than getting rid of structure. Unwritten code for a given
purpose obviously has some sort of structure with regards to, say, loops
and conditions, but I am less sure about the implications for how the
code should be nested. Another argument against deeply-nested functions
is the increased complexity of testing.

Cheers,

Loris

[Julio Di Egidio]

On Friday, 20 August 2021 at 13:15:21 UTC+2, Loris Bennett wrote:
> Julio Di Egidio <ju...@diegidio.name> writes:
> > On Friday, 20 August 2021 at 11:54:00 UTC+2, Loris Bennett wrote:

> >> If I have a command-line argument for a program, what is the best way
> >> of making this available to a deeply-nested[1] function call without
> >> passing the parameter through every intermediate function?
> >
> > To not pass arguments you need shared state ("global variables"): and
> > options in shard state, unless you have very good reasons to do
> > otherwise, is simply a no no.
>
> Doesn't that slightly depend on the size of your "globe"? If a program
> does a few, in some sense unrelated things, and, say, only runs for a
> few minutes, could you not decide to make a particular parameter global,
> even though only one function needs it?

I just wouldn't: there is no point in not passing a parameter to a function in a 30 liner, either. What I may very well do is e.g. initialize instance variables in a class constructor that are available to all instance methods (which is another example of shared state, but this is quite what objects do), though even there one should be careful (if things with the state get tough, the state is to be abstracted away and in turn be properly encapsulated...): but we were talking of command line arguments and nested functions, and I explicitly talked of "options" which hints at a common pattern: and in that specific case I can think of no reason not to just follow the best practice.

Anyway, my 2c.

Julio

[George Fischhof]

Loris Bennett <loris....@fu-berlin.de> ezt írta (időpont: 2021. aug.
20., P 17:54):

> --
> https://mail.python.org/mailman/listinfo/python-list
>

>
>
>


Hi,

Also you can give a try to click and / or typer packages.
Putting args into environment variables can be a solution too
All of these depends on several things: personal preferences, colleagues /
firm standards, the program, readability, variable accessibility (IDE
support, auto completition) (env vars not supported by IDEs as they are not
part of code)

BR,
George

[Loris Bennett]

[snip (15 lines)]>

> Hi,
>
> Also you can give a try to click and / or typer packages.
> Putting args into environment variables can be a solution too
> All of these depends on several things: personal preferences, colleagues /
> firm standards, the program, readability, variable accessibility (IDE
> support, auto completition) (env vars not supported by IDEs as they are not
> part of code)

Thanks for the pointers, although I have only just got my head around
argparse/configargparse, so click is something I might have a look at
for future project.

However, the question of how to parse the arguments is somewhat separate
from that of how to pass (or not pass) the arguments around within a
program.

[George Fischhof]

Loris Bennett <loris....@fu-berlin.de> ezt írta (időpont: 2021. aug.

23., H 19:26):

> --
> https://mail.python.org/mailman/listinfo/python-list
>

>
>
>



Hi,
I thought not just parsing, but the usage method: you add a decorator to
the function where you want to use the parameters. This way you do not have
to pass the value through the calling hierarchy.

Note: typer is a newer package, it contains click and leverages command
line parsing even more.


BR,
George

[Loris Bennett]

George Fischhof <geo...@fischhof.hu> writes:

[snip (79 lines)]

>> > Hi,
>> >
>> > Also you can give a try to click and / or typer packages.
>> > Putting args into environment variables can be a solution too
>> > All of these depends on several things: personal preferences, colleagues
>> /
>> > firm standards, the program, readability, variable accessibility (IDE
>> > support, auto completition) (env vars not supported by IDEs as they are
>> not
>> > part of code)
>>
>> Thanks for the pointers, although I have only just got my head around
>> argparse/configargparse, so click is something I might have a look at
>> for future project.
>>
>> However, the question of how to parse the arguments is somewhat separate
>> from that of how to pass (or not pass) the arguments around within a
>> program.

[snip (16 lines)]

>
> Hi,
> I thought not just parsing, but the usage method: you add a decorator to
> the function where you want to use the parameters. This way you do not have
> to pass the value through the calling hierarchy.
>
> Note: typer is a newer package, it contains click and leverages command
> line parsing even more.

Do you have an example of how this is done? From a cursory reading of
the documentation, it didn't seem obvious to me how to do this, but then
I don't have much understanding of how decorators work.

[George Fischhof]

Loris Bennett <loris....@fu-berlin.de> ezt írta (időpont: 2021. aug.

26., Cs, 16:02):

> --
> https://mail.python.org/mailman/listinfo/python-list


Hi,

will create a sample code on Monday - Tuesday

BR,
George

[George Fischhof]

George Fischhof <geo...@fischhof.hu> ezt írta (időpont: 2021. aug. 29., V,
21:27):

>
>
> Loris Bennett <loris....@fu-berlin.de> ezt írta (időpont: 2021. aug.

> 26., Cs, 16:02):

>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Hi,
>
> will create a sample code on Monday - Tuesday
>
> BR,
> George
>

Hi,

here is the program ;-) (see below)
typer does not uses decorators, to solve this problem they advice to use
click's decorators, mixing typer and click.
Practically I prefer not to mix them, also the parts for easiest way to do
this just available in latest click, which is not supported in typer.

So I created all the stuff in click, 8.x should be used

BR,
George


import click


# read command line parameters
@click.command()
@click.option('--level_1', help='Level 1')
@click.option('--level_2', help='Level 2')
def main(level_1, level_2):
# put command line parameters into global context
ctx = click.get_current_context()
ctx.meta['level_1'] = level_1
ctx.meta['level_2'] = level_2

level_1_function()


# pass / inject level_1 parameter to this function
@click.decorators.pass_meta_key('level_1')
def level_1_function(level_1):
print(f'level 1 variable: {level_1}')
level_2_function()


# pass / inject level_2 parameter to this function
@click.decorators.pass_meta_key('level_2')
def level_2_function(level_2):
print(f'level 2 variable: {level_2}')


if __name__ == "__main__":
main()

[Loris Bennett]

Thanks for the example - that's very interesting. However, after a bit
of reflection I think I am going to stick to explicit argument passing,
so that I can have more generic modules that can be used by other
programs. I'll then just encapsulate the argument parsing in a single
function corresponding to the command line tool.