Config for my ConfigSource.

[Phillip Krüger]

Hi All.

I have a potential silly question. I am writing a ConfigSource that finds config values in a external datasource (in my case etcd).

Is it possible to use the Config API to configure my ConfigSource ?

So I want to configure the URL for the etcd server. I want to set the Ordinal for my config source higher than the default once, and want to be able to find the value for the URL in one of those sources.

Is this at all possible ? And if not, is this a good idea ?

Thanks

[Rudy De Busscher]

Hi Philip,

Yes, this is possible, I have created something similar where values are retrieved from a JAX-RS endpoint. (maybe there is a better solution, but it works)

I did the configuration of the URL the first time a config value is requested. You only have to take care that you don't end up with a loop because your ConfigSource is also consulted for the URL.

regards

Rudy

[Phillip Krüger]

Hi Rudy.

Thanks for your reply.

Can you perhaps share some code ? An in what impl of MicroProfile did you get this to work ?

Here are my code that runs on the latest Para Micro:

public class EtcdConfigSource implements ConfigSource {
       
    //private Client client;
   
    @Inject @ConfigProperty(name = "configsource.etcd.scheme", defaultValue = "http")
    private String scheme;
   
    @Inject @ConfigProperty(name = "configsource.etcd.host", defaultValue = "localhost")
    private String host;
   
    @Inject @ConfigProperty(name = "configsource.etcd.port", defaultValue = "2379")
    private String port;
   
    public EtcdConfigSource(){
        //this.client = getClient();
    }
   
    @Override
    public int getOrdinal() {
        return 450;
    }
   
    @Override
    public Map<String, String> getProperties() {
       
        // removed for space
    }

    @Override
    public String getValue(String key) {
        ByteSequence bsKey = ByteSequence.fromString(key);
        CompletableFuture<GetResponse> getFuture = getClient().getKVClient().get(bsKey);
        try {
            GetResponse response = getFuture.get();
            return toString(response);
        } catch (InterruptedException | ExecutionException ex) {
            log.log(Level.WARNING, "Can not get config value for [{0}] from etcd Config source: {1}", new Object[]{key, ex.getMessage()});
        }
       
        return null;
    }

    @Override
    public String getName() {
        return "EtdcConfigSource";
    }
   
    private String toString(GetResponse response){
        if(response.getCount()>0){
            return response.getKvs().get(0).getValue().toStringUtf8();
        }
        return null;
    }
   
    private Client getClient(){
        log.info("Loading [etcd] MicroProfile ConfigSource");
        String endpoint = String.format("%s://%s:%s",scheme,host,port);
        log.log(Level.INFO, "Using [{0}] as etcd server endpoint", endpoint);
        return Client.builder().endpoints(endpoint).build();
    }
}

It does not matter if I put the "getClient" in the constructor or change it to first use, I always get null for all injected values. (so my url end up being null://null:null)

It seems like it never inject, hence the null (and not the default value)

I have also tried adding the value in the META-INF/microprofile-config.properties and also by passing the property in when starting (-D) but it still returns null.

I have also played around with the Ordinal, making it the highest and lowest.

I even injected a property that comes from System.getProperties and that to returns null:

    @Inject @ConfigProperty(name = "java.version")
    private String javaVersion;

Please any hints or code samples would be appreciated.

Thanks

[Craig McClanahan]

Probably stupid peanut gallery comment, but just in case ... is the value returned by

  @Override
    public String getName() {
        return "EtdcConfigSource";
    }




what you really want?  All the other spellings of things start with "Etcd", not "Etdc".




Craig McClanahan




--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To post to this group, send email to microp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/0cfecf89-262f-4d87-be8a-0ba2d26d1030%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Phillip Krüger]

(*blush*) - yes, thank you, typing error.

[Alasdair Nottingham]

Based on what you are seeing I would suspect that EtcdConfigSource isn’t a CDI bean and thus CDI injection isn’t happening. I honestly don’t know if a ConfigSource can be a CDI bean, but that is the most obvious reason why you’d be getting null even for system properties.

Alasdair

[Bob McWhirter]

You can use the Config API without injection and do manual lookups in that case. 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/F039EF34-8D2F-4F75-B0D6-D80C98EAD974%40gmail.com.

[Rudy De Busscher]

Phillip,

As other already mentioned, ConfigSource instances are not CDI beans. You can retrieve the values with

ConfigProvider.getConfig().getValue(...) 

Rudy 

You received this message because you are subscribed to a topic in the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/rkkZJGJpxMU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To post to this group, send email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CA%2B45JvEChhQ4gcYWp3zZ79czzaHVnzFMSUQGJRu-TTus2d6OyA%40mail.gmail.com.

[Phillip Krüger]

Hi Bob, Alasdair and Rudy,

Thank you very much for your help. I did not know you can get to the Config values like this:

ConfigProvider.getConfig().getValue(...)

This is exactly what I was looking for. Thanks again.

Cheers

[Gordon Hutchison]

Hi Phillip,

Interesting to read this, I have done something similar in the past:

Last July I played around with some code to inject values straight out a database

into Java using a CDI annotation. This app had a ConfigSource (cloudant db) that used a ConfigSource to configure itself (db credentials) and

the programatic API was not only useful but necessary to prevent a cycle.

Here is what I wrote then:

"In our example code we use the Java service loader pattern support of the Config API to to allow our CloudantConfigSource source to be discovered and used to provide values for config CDI injection of values from an IBM cloud database, even before any application code is run:

The 
   src/main/webapp/META-INF/services/ directory 
contains the file
   org.eclipse.microprofile.config.spi.ConfigSource 
which contains
   application.config.CloudantConfigSource
 

We do not wish the CredentialsVcapConfigSource to be part of the default set of sources loaded (as it's implementation uses 'addDefaults' and we do not wish to create a cycle as the service loaded pattern is used as as part of the default sources setup of CDI injectable sources) so we access it in the CloudantConfigSource using the configuration builder API to add it manually to a config."

and here is some code:

 

package application.rest.v1;

import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import org.eclipse.microprofile.config.inject.ConfigProperty;

@Path("v1/cloudant")
public class CloudantData {


        @Inject
        @ConfigProperty(name = "Ireland", defaultValue="Punt" )  //<<< These values are stored in an IBM Bluemix database
        String irelandCurrency;

        @Inject
        @ConfigProperty(name = "UK", defaultValue="Groats" )
        String ukCurrency;
        
        @GET
        @Produces(MediaType.TEXT_PLAIN)
        @Path("data")
        public Response exampleCloudant() {
                String msg = "";
                msg = msg + "UK==" + ukCurrency + "\n";
                msg = msg + "Ireland==" + irelandCurrency + "\n";

                return Response.ok(msg).build();
        }

}

package application.config;

import java.net.MalformedURLException;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;

import org.eclipse.microprofile.config.Config;
import org.eclipse.microprofile.config.spi.ConfigProviderResolver;
import org.eclipse.microprofile.config.spi.ConfigSource;

import com.cloudant.client.api.ClientBuilder;
import com.cloudant.client.api.CloudantClient;
import com.cloudant.client.api.Database;

import application.bluemix.InvalidCredentialsException;

public class CloudantConfigSource implements ConfigSource {

        Map<String, String> props = null;

        @Override
        public String getName() {
                return "Cloundant/config/properties";

        }

        @Override
        public Map<String, String> getProperties() {

                if (props == null) {
                        props = fetchProps();
                }
                return props;

        }

        @Override
        public String getValue(String key) {

                if (props == null) {
                        props = fetchProps();
                }
                return props.get(key);
        }

        private Map<String, String> fetchProps() {

                Map<String, String> map = null;
                
                // We want to source the credentials information from different sources
                // depending on what deployment stage or environment we are running in.
                
                
                try {
                        CredentailsVcapConfigSource configSrc = new CredentailsVcapConfigSource();
                        Config c = ConfigProviderResolver.instance().getBuilder().addDefaultSources().withSources(configSrc).build();
                        String user = sanitiseString(c.getValue("CLOUDANT_USERNAME", String.class));
                        String password = sanitiseString(c.getValue("CLOUDANT_PASSWORD", String.class));
                        URL url = new URL(sanitiseString(c.getValue("CLOUDANT_URL", String.class)));
                        String dbName = c.getValue("CLOUDANT_DB", String.class);
                        String sampleData = c.getValue("CLOUDANT_DOC", String.class);

                        CloudantClient client = ClientBuilder.url(url).username(user).password(password).build();
                        Database db = client.database(dbName, false);
                        map = (db.find(HashMap.class, sampleData));
                        map.remove("_id");
                        map.remove("_rev");
                } catch (InvalidCredentialsException e) {
                        e.printStackTrace();
                        map = new HashMap<>();
                } catch (MalformedURLException e) {
                        e.printStackTrace();
                        map = new HashMap<>();
                }
                return map;
        }

        protected String sanitiseString(String data) throws InvalidCredentialsException {
                if (data == null || data.isEmpty()) {
                        throw new InvalidCredentialsException("Invalid string.");
                }
                char first = data.charAt(0);
                char last = data.charAt(data.length() - 1);
                if ((first == '"' || first == '\'') && (first == last)) {
                        String result = data.substring(1, data.length() - 1);
                        return result;
                }
                return data;
        }

}

more...


package application.config;

import java.io.StringReader;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import javax.json.Json;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.json.JsonValue;

import org.eclipse.microprofile.config.Config;
import org.eclipse.microprofile.config.ConfigProvider;
import org.eclipse.microprofile.config.spi.ConfigBuilder;
import org.eclipse.microprofile.config.spi.ConfigProviderResolver;
import org.eclipse.microprofile.config.spi.ConfigSource;

import application.bluemix.InvalidCredentialsException;

public class CredentailsVcapConfigSource implements ConfigSource {

        Map<String, String> properties = null;

        @Override
        public int getOrdinal() {
                return 1000;

        }

        @Override
        public Map<String, String> getProperties() {

                if (properties == null) {
                        try {
                                
                                properties = new HashMap<String, String>();                             
                                JsonObject vcapCreds = getCredentialsObject();
                                                
                                if (vcapCreds != null) {
                                        String username = vcapCreds.getJsonString("username").getString();
                                        String password = vcapCreds.getJsonString("password").getString();
                                        String url = vcapCreds.getJsonString("url").getString();

                                        properties.put("CLOUDANT_URL", url);
                                        properties.put("CLOUDANT_USERNAME", username);
                                        properties.put("CLOUDANT_PASSWORD", password);
                                }

                        } catch (InvalidCredentialsException e) {
                                e.printStackTrace();
                        }
                }
                return properties;
        }

        @Override
        public String getValue(String propertyName) {
                return getProperties().get(propertyName);
        }

        @Override
        public String getName() {
                return this.getClass().getName();
        }

        public JsonObject getCredentialsObject() throws InvalidCredentialsException {

                Config c = ConfigProviderResolver.instance().getBuilder().addDefaultSources().build();
                
                String serviceType = c.getOptionalValue("CLOUDANT_SERVICE_TYPE", String.class).orElse(null);
                String serviceName = c.getOptionalValue("CLOUDANT_SERVICE_NAME", String.class).orElse(null);
                String vcapServicesEnv = c.getOptionalValue("VCAP_SERVICES", String.class).orElse(null);

                JsonObject credentials = null;
                try {
                        if (vcapServicesEnv != null) {
                                JsonObject vcapServices = Json.createReader(new StringReader(vcapServicesEnv)).readObject();
                                JsonArray serviceObjectArray = vcapServices.getJsonArray(serviceType);
                                JsonObject serviceObject = null;
                                if (serviceName == null || serviceName.isEmpty()) {
                                        serviceObject = serviceObjectArray.getJsonObject(0);
                                } else {
                                        Iterator<JsonValue> itr = serviceObjectArray.iterator();
                                        while (itr.hasNext()) {
                                                JsonObject object = (JsonObject) itr.next();
                                                if (serviceName.equals(object.getJsonString("name").getString())) {
                                                        serviceObject = object;
                                                }
                                        }
                                }
                                credentials = serviceObject.getJsonObject("credentials");
                        }
                } catch (NullPointerException npe) {
                        throw new InvalidCredentialsException("Unable to parse VCAP_SERVICES", npe);
                }
                return credentials;
        }

}

Apologies for the rawness of the code.

Gordon.

[Phillip Krüger]

Hi Gordon.

Thank you very much for your detailed reply. I think (at least conceptually) it's the same as the answers proposed before ?

I wrote a Post on my eventual solution:

https://www.phillip-kruger.com/post/microprofile_config/

It also seems like there is another why all together to get this done, as per Mark Struberg -  "You can also do it lazily with an init flag."

Once I know the details I'll update the post.

Thanks again for all the help !

Cheers

[Gordon Hutchison]

(Apologies for late reply due to vacation)

Hi Phillip - yes you are right, I liked reading your article as it explains how to

'do-it-youself' very clearly.

What re-sparked my interest this time was the reminder that I had to use the programmatic API

to avoid picking up the default config sources... A CDI injectable source is

going to be on the default path (or service loaded) and it can't rely on itself as a config source

without creating a loop. I read your use of a KEY_PREFIX with interest.

Always interesting to add to ones examples of usage patterns and tips and tricks

so thanks for sharing :-)

Gordon.