How to preprocess parameters and continue with picocli type detection?

Sven K

unread,

Apr 27, 2024, 2:39:16 AMApr 27

to picocli

Hi everyone,

First of all, thanks a lot for all the work that you've put in this great library. I discovered picocli only recently and I'm impressed about the capabilities and the simplicity of use! Especially in combination with native-image this is surely a game-changer to build great cli applications!

I have a specific use-case that I wanted to discuss.

Use-Case

I'm running my cli application in docker and there is a REST facade to create new containers with specific parameters. All parameters can also be set via env-variables which fits perfectly because I can just create a docker env-file where I specify all parameters that then get injected to the java fields with picocli.

However there is a limitation with env-files: They don't support multi-line values and I have some parameters that might contain multiple values (e.g. a String template that is used by the application). Of course I could use files here that I would link inside the container but I want to keep it simple (e.g. because I'm not sure if I want to migrate in future e.g. to another container runtime or to K8s).

My idea would be to encode such values with Base64, put them in the docker env files and then decode them again inside the application.

How to handle it properly with picocli?

If I wanted to just handover all values in Base64 format and then decode them before they are injected with picocli, how would be the right approach for this?

I've seen in the documentation that I could use IParameterConsumer (simple) or IParameterPreprocessor (more flexible). I tested this for example with a boolean field and I've seen that it works if I do it like this:

boolean field:

```

@Option(names = "--test-flag", defaultValue = "dHJ1ZQ==" /* base64-encoded 'true' */, parameterConsumer = Base64BooleanConsumer.class)

private boolean testFlag;

```

Consumer class:

```

@Override

public void consumeParameters(final Stack<String> args, final CommandLine.Model.ArgSpec argSpec, final CommandLine.Model.CommandSpec commandSpec)

{

if (!args.isEmpty())

{

String result = new String(Base64.getDecoder().decode(args.pop()));

argSpec.setValue(Boolean.parseBoolean(result));

}

```

However I see 2 issues here:

* #1: I need to have type-specific Consumer-classes. Would I would like to have would be a consumer that just decodes the String and then hands it over to picocli for type detection. Is this possible somehow?

* #2: the default-value has to be base64-encoded as well which is not so nice to read (this is not nice but probably acceptable)

Issue #1 is maybe a litte bit artificial since I currently only really need the base64 encoding/decoding for Strings, but I was curious how the best way to handle it with picocli would look like and if that there is a way to just have some global preprocessing for all parameters?

Thanks a lot for any comments and help!

Sven

picocli

unread,

Apr 27, 2024, 7:18:54 AMApr 27

to picocli

Hi Sven,

I hope I understand your use case correctly;

if all input values (command line arguments) are base64-encoded,

I would use type converters rather than parameter consumer or parameter preprocessors
(those last 2 are plugins into the parser for cases where normal parsing is somehow insufficient, this is not your use case).

One advantage of type converters is that you can register them globally, so they apply to all options/positional parameters of that type.

(You can also register type converters on a per-option basis if necessary.)

For example, to register a Base64BooleanTypeConverter for all boolean options:

class Base64BooleanTypeConverter implements ITypeConverter<Boolean> { public Boolean convert(String value) throws Exception { return Boolean.parseBoolean( new String(Base64.getDecoder().decode(args.pop())) ); } }

You would register your custom type converter for all boolean options as follows:

App app = new App(); CommandLine commandLine = new CommandLine(app) .registerConverter(Boolean.class, new Base64BooleanTypeConverter ()) // for java.lang.Boolean options .registerConverter(Boolean.TYPE, new Base64BooleanTypeConverter ()); // for primitive boolean options

commandLine.execute(args);

picocli

unread,

Apr 27, 2024, 7:21:47 AMApr 27

to picocli

small correction: (hopefully formatting/spacing is preserved this time...)

class Base64BooleanTypeConverter implements ITypeConverter<Boolean> {
public Boolean convert(String value) throws Exception {

return Boolean.parseBoolean(new String(Base64.getDecoder().decode(value)));
}
}

Sven K

unread,

Apr 27, 2024, 11:14:54 AMApr 27

to picocli

Thanks, TypeConverters in deed look like a better approach for my use-case!

I tried it in my IDE with Azul JDK 17.0.10 and picocli 4.7.5. It works perfectly with primitive boolean types (using the registerConverter-Method with "Boolean.class", "Boolean.TYPE" somehow doesn't work and the converter never gets called). However with an option that has Boolean object type it seems that the converter is called twice trying to decode the decoded value once again. Is that somehow explainable or am I doing something wrong? Of course I could check in the converter if the value that I get looks like a Base64-Value but it seems like a strange behavior...

Remko Popma

unread,

Apr 27, 2024, 7:12:32 PMApr 27

to picocli

Behaviour being different between options with primitive boolean type vs Boolean object... that sounds strange.
Not sure what could cause the type converter to be called twice.

You could try with tracing set to DEBUG (-Dpicocli.trace=DEBUG),
that will print what picocli is doing to the console.

Also, I don't understand the problem you mentioned with the registerConverter method:

"using the registerConverter-Method with "Boolean.class", "Boolean.TYPE" somehow doesn't work and the converter never gets called)"

If you can provide minimal example(s) that reproduce these issues I can take a closer look.

--
You received this message because you are subscribed to the Google Groups "picocli" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picocli+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/picocli/96ef3bd3-e04e-4f83-b41b-f659da7ab1a4n%40googlegroups.com.

Sven K

unread,

Apr 28, 2024, 3:24:24 AMApr 28

to picocli

Added an example project: https://codeberg.org/sven.k/picocli-reproducer (main class with the code is here: https://codeberg.org/sven.k/picocli-reproducer/src/branch/master/app/src/main/java/org/example/ConverterDemo.java)

```

Output with the mentioned debug option:

[picocli DEBUG] Creating CommandSpec for org.example.ConverterDemo@68bbe345 with factory picocli.CommandLine$DefaultFactory
[picocli INFO] Picocli version: 4.7.5, JVM: 22.0.1 (Eclipse Adoptium OpenJDK 64-Bit Server VM 22.0.1+8), OS: Mac OS X 14.4.1 aarch64
[picocli INFO] Parsing 0 command line args []
[picocli DEBUG] Parser configuration: optionsCaseInsensitive=false, subcommandsCaseInsensitive=false, abbreviatedOptionsAllowed=false, abbreviatedSubcommandsAllowed=false, allowOptionsAsOptionParameters=false, allowSubcommandsAsOptionParameters=false, aritySatisfiedByAttachedOptionParam=false, atFileCommentChar=#, caseInsensitiveEnumValuesAllowed=false, collectErrors=false, endOfOptionsDelimiter=--, expandAtFiles=true, limitSplit=false, overwrittenOptionsAllowed=false, posixClusteredShortOptionsAllowed=true, separator=null, splitQuotedStrings=false, stopAtPositional=false, stopAtUnmatched=false, toggleBooleanFlags=false, trimQuotes=false, unmatchedArgumentsAllowed=false, unmatchedOptionsAllowedAsOptionParameters=true, unmatchedOptionsArePositionalParams=false, useSimplifiedAtFiles=false
[picocli DEBUG] (ANSI is disabled by default: systemproperty[picocli.ansi]=null, isatty=false, TERM=null, OSTYPE=null, isWindows=false, JansiConsoleInstalled=false, ANSICON=null, ConEmuANSI=null, NO_COLOR=null, CLICOLOR=null, CLICOLOR_FORCE=null)
[picocli DEBUG] Initializing command 'null' (user object: org.example.ConverterDemo@68bbe345): 1 options, 0 positional parameters, 0 required, 0 groups, 0 subcommands.
[picocli DEBUG] Set initial value for field Boolean org.example.ConverterDemo.testOption of type class java.lang.Boolean to null.
[picocli DEBUG] Applying default values for command '<main class>'
[picocli DEBUG] Applying defaultValue (*****(masked)) to field Boolean org.example.ConverterDemo.testOption on ConverterDemo@68bbe345
[picocli DEBUG] 'dHJ1ZQ==' doesn't resemble an option: 0 matching prefix chars out of 1 option names
[picocli INFO] Setting field Boolean org.example.ConverterDemo.testOption to 'false' (was 'null') for field Boolean org.example.ConverterDemo.testOption on ConverterDemo@68bbe345
[picocli DEBUG] Help was not requested. Continuing to process ParseResult...
[picocli DEBUG] RunLast: handling ParseResult...
[picocli DEBUG] RunLast: executing user object for '<main class>'...
[picocli DEBUG] Invoking Runnable::run on user object org.example.ConverterDemo@68bbe345...
[picocli DEBUG] RunLast: ParseResult has 0 exit code generators
[picocli DEBUG] resolveExitCode: exit code generators resulted in exit code=0
[picocli DEBUG] resolveExitCode: execution results resulted in exit code=0
[picocli DEBUG] resolveExitCode: returning exit code=0

converter invocation 1: called with value dHJ1ZQ==
converter invocation 2: called with value true
testOption was set to: false

```

Remko Popma

unread,

Apr 28, 2024, 8:50:54 AMApr 28

to picocli

Thank you for the reproducer project.

The converter is called twice when the default value is assigned;

essentially there is special logic for boolean values that determines the boolean value of the String once,

then converts the boolean back to a string and a few lines later calls the converter again to turn the string back into a boolean.

Not sure why I did that... 😅

I suspect the best workaround is to allow your type converter to deal both with base64-encoded boolean values, as well as with "true" and "false" String values.

I also see that trying to register the type converter with Boolean.TYPE does not succeed, and the built-in type converter is still in place.
I will look at that next.

To view this discussion on the web visit https://groups.google.com/d/msgid/picocli/d8aadab6-f601-40d7-99ff-91f8814efcban%40googlegroups.com.

Message has been deleted

Sven

unread,

Apr 28, 2024, 9:42:42 PMApr 28

to Remko Popma, picocli

Great, thanks for all the quick support, I can properly handle my use-case now! =)

You received this message because you are subscribed to a topic in the Google Groups "picocli" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/picocli/kNjVIrFOqlY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to picocli+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/picocli/CAAFrFhT8S9ZtHGSRFyPJW8yCs5_k9qp-2Tf%3DuyjAEntJ_qJgTg%40mail.gmail.com.

Message has been deleted

Remko Popma

unread,

May 1, 2024, 4:36:21 AMMay 1

to picocli

I created a GitHub ticket for this:

https://github.com/remkop/picocli/issues/2270

Message has been deleted

picocli

unread,

May 3, 2024, 8:12:33 PMMay 3

to picocli

FYI
I have banned Psychedelics Haven
from this group now.

On Friday, May 3, 2024 at 11:32:16 PM UTC+9 Psychedelics Haven wrote:

Is 2024 Buy 25I-NBOMe 500g Online Blotter for Sale Online:
https://psychedelicshaven.com/product/25i-nbome-500g/
https://telegram.me/Nextdoormarket
Do you want to buy 25I-NBOMe 500g blotter online? We make your way easier this is the Best Place to Buy 25i-NBOMe. Also, we are concerned with all this product's essential and complete traits. We help you to improve your experience. We are presenting all the best quality products that come in a cheap price range. It would help if you appreciated our servers. This is the exact spot and most reputational website. We are selling 25I-NBOMe 500g for Sale online. These products contain the best quality.
All about: 25i Nbome Blotter
https://telegram.me/Nextdoormarket
This powerful psychedelic drug was first synthesized in 2003 by German chemist Ralph Heim. It is typically sold as a liquid, although it can also be found in tablet or gel form. These effects are unpredictable and can vary significantly from person to person. Despite its illegal status, likewise, it is widely available on the internet.
Types of blotter:
There are multiple types available in this paper 25i-NBOMe blotter is one the well-known. This hallucinate drug made by the 2-(4-Iodo-2,5-dimethoxyphenyl)-N-[(2-methoxyphenyl)methyl]ethanamine, or 25I-NBOMe. Also, it was found to be a powerful serotonin 2A (5-HT2A) receptor agonist
Buy 25i-NBOMe Blotter USA:
https://psychedelicshaven.com/product/25i-nbome-500g/
This is a very potent drug, with just a tiny amount producing powerful effects. The effects of this drug are unpredictable and vary depending on the individual, the dose, the environment, and other factors. It can cause visual and auditory hallucinations, changes in perception, feelings of euphoria or anxiety, and changes in mood and behaviour.
How to Order 25i Nbome Online?
25i-NBOMe is a powerful hallucinogenic drug not legal in many countries. It is often made in illegal laboratories and can be very dangerous. However, there are websites where you can order 25i income online. These websites are not always safe, and you should be careful if you use them.
25i-NBOMe Blotter Dosage :
https://psychedelicshaven.com/product/25i-nbome-500g/
A normal dose of the hydrochloride salt is 600–1,200 μg. The UK Council on the Misuse of Drugs states that a common dose is between 49 and 99 μg. It's no secret that this is one of the most popular illegal drugs on the market.
Though it's illegal, that doesn't stop people from trying to buy it online. While many websites claim to sell 25i-NBOMe, being careful is essential. Reviews show how powerful this powder is.
Positive Effects of Include:
It is considered a Schedule I drug in the United States, meaning it has a high potential for abuse and is not considered safe for medical use.
Increased sense of well-being
Increased creativity
Improved mood
Improved mental clarity
Hallucinations
Short-Term side effects
anxiety
paranoia
hallucinations
flashbacks
psychosis
https://psychedelicshaven.com/product/25i-nbome-500g/
https://psychedelicshaven.com/product/buy-ketamine-powder-online/
https://psychedelicshaven.com/product/yg-og-marijuana-rapper-strain/
https://psychedelicshaven.com/product/buy-3-cmc-crystal-online/
https://psychedelicshaven.com/product/3-pack-mix-and-match-thc-chocolates/
https://psychedelicshaven.com/product/purchase-200ug-1p-lsd/
https://psychedelicshaven.com/product-category/ayahuasca/
https://psychedelicshaven.com/product-category/psilocybin-mushroom/
https://psychedelicshaven.com/product-category/other-psychedelics/
https://psychedelicshaven.com/product-category/marijuana-flowers/

https://psychedelicshaven.com/product/buy-aura-extract-mini-thc-bar/
https://psychedelicshaven.com/product/gelato-strain/
https://psychedelicshaven.com/product/ak-47-strain/
https://psychedelicshaven.com/product/purchase-best-2024-1p-lsd/
https://psychedelicshaven.com/product/buy-tresiba-flextouch-pens/
https://psychedelicshaven.com/product/iboga-ta-extract/
https://psychedelicshaven.com/product/shop-aura-extract-mini-thc-bar/

https://psychedelicshaven.com/product/skywalker-og-strain/
https://psychedelicshaven.com/product/chemdawg/
https://psychedelicshaven.com/product/ketamine-for-sale-bulk-delivery/
https://psychedelicshaven.com/product/purchase-xanax-xr-3-mg/
https://psychedelicshaven.com/product/yellow-xanax-bar-2-mg/
https://psychedelicshaven.com/product/iboga-pta-extract/
https://psychedelicshaven.com/product/green-xanax-bars-2mg/
https://psychedelicshaven.com/product/2024-best-white-xanax-bars/
https://psychedelicshaven.com/product/you-can-purchase-xanax/
https://psychedelicshaven.com/product/new-wonder-psilocybin-gummies/
https://psychedelicshaven.com/product/purchase-integral-alchemist/

https://psychedelicshaven.com/product/ethereal-essence-4aco-dmt-tincture/
https://psychedelicshaven.com/product/bright-future-milk-chocolate/
https://psychedelicshaven.com/product/2024-dancing-tiger-magic-mushrooms/
https://psychedelicshaven.com/product/best-2024-golden-teacher-mushrooms/
https://psychedelicshaven.com/product/best-2024-penis-envy-mushrooms/
https://psychedelicshaven.com/product/buy-shafaa-heal-macrodose/
https://psychedelicshaven.com/product/best-2024-mazatapec-mushrooms/
https://psychedelicshaven.com/product/purchase-the-best-mastermind/
https://psychedelicshaven.com/product/buy-the-best-room-920-hot/

https://psychedelicshaven.com/product/2024-best-puff-boyz/
https://psychedelicshaven.com/product/2024-best-5ml-dmt-vape/
https://psychedelicshaven.com/product/buy-2024-best-fentanyl-for-sale-now-available/
https://psychedelicshaven.com/product/is-2024-buy-oxycontin-op/
https://psychedelicshaven.com/product/is-2024-buy-opana-er/
https://psychedelicshaven.com/product/purchase-codeine-phosphate/

https://psychedelicshaven.com/product/dank-vapes/
https://psychedelicshaven.com/product/is-2024-buy-valium-is-now-available/
https://psychedelicshaven.com/product/25i-nbome-500g/

Reply all

Reply to author

Forward