New Features proposal to enable Low-Code Connectors

[Tony Tkacik]

Dear ConnId community,

at Evolveum, we are exploring and designing a low-code base framework to ease the creation of connectors, and we have identified the following new feature set, which can improve code reuse and ease development of low-code (scripted) or no-code (declarative) connectors:

1. Connector loading from bundled libraries - connectors are currently detected only by scanning only direct classes

2. Configuration model overrides - configuration model is created using reflection, and the override feature allows declarative hiding of unused properties

3. Complex configuration properties - support for complex properties for connector configuration

We at Evolveum are very interested in these features, and would like to develop it, if it makes sense to you. What do you think?

Below you can find more detailed proposals for each of these features.

Your opinion and feedback on these proposed features is welcomed.

1. Connector loading from bundled libraries

Support for detecting & loading Connector class from bundled libraries when the immediate JAR doesn’t contain one.

Rationale: Current ConnId Framework implementation requires connector bundle to contain at least one Java class outside of bundle libraries - Connector class.
This does not allow for connectors implemented in no-code or low-code frameworks, which do not provide Java classes, but only a set of libraries (where one contains connector implementation) and configuration files.

Examples:

• A SCIM 2.0 connector implemented in a no-code framework, which wraps a base SCIM connector and contains only application-specific static configuration (constants).

• A REST connector implemented in Groovy, which also bundles a base framework for wiring and executing bundled scripts.

Design and Implementation

The new optional property ConnectorBundle-ConnectorClass in connector’s MANIFEST.MF allows developer to declare a class name of connector to be used.

If the specified class exists in bundle classloader and it is a valid connector, it will be added to the list of detected connectors.

LocalConnectorInfoManagerImpl and related manifest parsing classes will be extended to process newly introduced optional ConnectorBundle-ConnectorClass property and algorithm for scanning will be updated as follows:

1. (Original behaviour) Scan all bundle’s direct classes for @ConnectorClass annotation, add to list of detected connectors

2. If manifest has non-empty ConnectorClass property, the implementation will try to load the class using bundle class loader (this allows loading from bundled JARs). If the class is a connector, it will be added to the connector list.

Note that the referenced connector class should be responsible for detecting the scripts and no-code configuration inside connector bundle.

2. Configuration properties visibility override

Support for declarative override of available configuration properties in the connector configuration.

Rationale: Current development trends and practices in the ConnId ecosystem often use an additional base connector framework to ease the implementation of connectors.
These frameworks typically provide a set of base connector classes and configuration properties that are common across multiple connectors.
However since the connector configuration model is derived by Java reflection, it may discover configuration properties that are not relevant for the specific connector implementation (eg. unsupported authorization types).
There should be an easy way for downstream connector implementations to hide irrelevant properties (such as authorization mechanisms unsupported by the target system) without the need to reimplement configuration classes.

Example: A SCIM 2.0 connector for a concrete application based on the existing SCIM 2.0 connector framework, but hiding the username and password configuration properties, since the target system uses only Bearer token for authorization.

Design and Implementation:

Extension of the @ConfigurationClass annotation to allow specifying the name of a properties file that may contain the list of configuration properties to be overridden.

• Update of the implementation of JavaClassProperties classes to support this feature.

  • @ConfigurationClass is checked for the presence of overrideFile attribute.

  • If the attribute is present and the properties file exists on the connector classpath, the configuration properties declared in the file with ignore flag will be ignored and not part of configuration.

The example properties file is in the format:

username=ignore
password=ignore

Using ignore keyword allows for further future extensions of this property file to support changing mandatory/optional status, maybe others.

3. Complex configuration properties

Support for complex configuration properties in the connector configuration model, allowing for structured data representation.

Rationale: Current configuration systems lack support for complex attributes, forcing connectors to use workarounds like custom syntax or paired lists (e.g., separate key and value lists) to represent structured data.
Examples include mapping between custom resources and URLs, or LDAP object class relationships.

Examples

• List of resource mappings for SCIM 2.0 application.

  • Object Class (string)

  • API Endpoint (string)

• List of relationships between LDAP object classes:

  • subjectClass (eg. InetOrgPerson)

  • subjectAttribute (eg. memberOf)

  • objectClass

  • objectAttribute (eg. member)

  
  

These examples were taken from existing connectors which use custom semantics/syntax for capturing complex configuration properties.

Design and Implementation

The JavaClassProperties, serialization and deserialization will be extended to support configuration beans for structured configuration properties.

The structured configuration properties will be Java Beans (in a similar fashion as top-level configuration), and properties will be discovered using the same mechanism.

The affected annotations may need to be extended.

As mentioned earlier, we at Evolveum are very interested in these features and would like to develop them if it makes sense to you. What do you think?

Thanks for your feedback.

---
Anton Tkáčik
Backend Technical Leader and Developer
evolveum.com

[Francesco Chicchiriccò]

Hi again Tony,
the proposals below look interesting but I wonder how much code and maintenance we would save if we could embed at least some aspects of Spring Boot's properties management.

I would also recommend to switch to a new major version for these: since we are currently at 1.6, it would then be 1.7 or even 2.0.

In the new major version we could also require Java 21 (we're at Java 11 in 1.6) and also take the chance to modernize the code and deprecate a few methods / classes.

Regards.

--
You received this message because you are subscribed to the Google Groups "connid-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to connid-dev+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/connid-dev/252647321.2312895.1753268296606.JavaMail.zimbra%40evolveum.com.

-- 
Francesco Chicchiriccò

Tirasa - Open Source Excellence
http://www.tirasa.net/

Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA
https://about.me/ilgrosso