Infinispan serialization

[ta...@transdata.net]

I want to migrate from Wildfly 17 to 21. Now I am facing the problem of serialization with Infinispan in a clustered environment. I get errors like 'No marshaller registered for Java type XYZ'. I know that this is because of the new infinispan version but I don't know how to set up infinispan correctly to work as expected.
For the time being I am working like this: In the wildfly config xml I define a cache which I then pick up in code as a default configuration for all the caches needed.
If I annotate my cache item classes I succeed in generating a proto schema file and a marshaller class. And I provide a SerializationContextInitializer. But I think it is never picked up by the application.
Searching for help I find many examples using GlobalConfigurationBuilder. But is it the correct way to do so if I have already a configuration in the wildfly xml config? Is there something I have to add to the wildfly xml config? Or can I modify the wildfly config in code?

Any help appreciated!

-- Heiko

[Paul Ferraro]

Are you creating your cache manager programmatically?  Or are you using configurations defined in the Infinispan subsystem?

If the former, can you post this code?  If the latter, can you post both your Infinispan subsystem configuration and the code that references this configuration in your application?

[dt pham]

a note here you have to make sure the old caches removed before run with new version just to avoid reload old serialization cache back.

[ta...@transdata.net]

Hi Paul!

I use a default configuration (default-replicated-cache) defined in the Infinispan subsystem and copy it in code for each cache that is needed.

Configuration rc = cacheManager.getCacheConfiguration(DEFAULT_REPLICATED_CACHE);
ConfigurationBuilder cb = new ConfigurationBuilder().read(rc);
Configuration config = cb.build();
cacheManager.defineConfiguration(name, config);

        <subsystem xmlns="urn:jboss:domain:infinispan:11.0">
            <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
                <transport lock-timeout="60000"/>
                <replicated-cache name="sso">
                    <locking isolation="REPEATABLE_READ"/>
                    <transaction mode="BATCH"/>
                </replicated-cache>
                <replicated-cache name="routing"/>
                <distributed-cache name="dist">
                    <locking isolation="REPEATABLE_READ"/>
                    <transaction mode="BATCH"/>
                    <file-store/>
                </distributed-cache>
            </cache-container>
            <cache-container name="ejb" aliases="sfsb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan">
                <transport lock-timeout="60000"/>
                <distributed-cache name="dist">
                    <locking isolation="REPEATABLE_READ"/>
                    <transaction mode="BATCH"/>
                    <file-store/>
                </distributed-cache>
            </cache-container>
            <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server">
                <transport lock-timeout="60000"/>
                <invalidation-cache name="default-invalidation-cache"/>
                <replicated-cache name="default">
                    <transaction mode="BATCH"/>
                </replicated-cache>
                <replicated-cache name="default-replicated-cache"/>
            </cache-container>
            <cache-container name="hibernate" module="org.infinispan.hibernate-cache">
                <transport lock-timeout="60000"/>
                <local-cache name="local-query">
                    <heap-memory size="10000"/>
                    <expiration max-idle="100000"/>
                </local-cache>
                <local-cache name="pending-puts">
                    <expiration max-idle="60000"/>
                </local-cache>
                <invalidation-cache name="entity">
                    <heap-memory size="10000"/>
                    <expiration max-idle="100000"/>
                </invalidation-cache>
                <replicated-cache name="timestamps"/>
            </cache-container>
        </subsystem>

[Paul Ferraro]

Thanks for the details.  Another question - which cache-container are you using?  How did you obtain the reference to "cacheManager"?

[ta...@transdata.net]

I obtain the reference to "cacheManager" like this:

    private static final String DEFAULT_CACHE_CONTAINER = "java:jboss/infinispan/container/server";

    @Resource(lookup = DEFAULT_CACHE_CONTAINER)
    private EmbeddedCacheManager cacheManager;

[Paul Ferraro]

If you are not using the "server" cache-container for anything else
(e.g. singleton deployments/service), you can remove its module
attribute.
Otherwise, create a separate cache-container without a defined module.
This will force Infinispan to use JBoss Marshallling for marshalling
cache entries (as opposed to ProtoStream).
In general, to determine its marshaller, the Infinispan subsystem
tries to load services implementing
org.infinispan.protostream.SerializationContextInitializer from the
module associated with the cache-container. If no implementations are
found, the marshaller defaults to JBoss Marshalling - otherwise it
will use ProtoStream.
In a subsequent release, I plan to make this an explicit configuration
attribute, rather than auto-configured.

Let me know if you have any other questions.

Paul

> --
> You received this message because you are subscribed to a topic in the Google Groups "WildFly" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/wildfly/ScLhNZ9_S1g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to wildfly+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/c6046341-f309-4ea2-a6ad-f1245634370cn%40googlegroups.com.

[ta...@transdata.net]

I created a separate cache-container without a defined module. I can successfully access the CacheManager in code. But when I now try to read the cache configuration for "default-recplicated-cache" which is definded in this new cache-container I get 'null'!

Any idea?

[Paul Ferraro]

Clustering services are started on-demand.  This include the registration of cache configurations into a cache manager.

Injecting the cache configuration into your class will ensure that a cache configuration defined in the subsystem is available when your application needs it (and removed when it undeploys).

The same is true for cache instances.

e.g.

@Resource(lookup = "java:jboss/infinispan/container/server")

private EmbeddedCacheManager cacheManager;

@Resource(lookup = "java:jboss/infinispan/configuration/server/default-replicated-cache")

private Configuration configuration;

@PostConstruct

public void init() {

    // You can skip this line, as the configuration is already injected.

    // Configuration rc = cacheManager.getCacheConfiguration(DEFAULT_REPLICATED_CACHE);

    ConfigurationBuilder cb = new ConfigurationBuilder().read(this.configuration);

    Configuration config = cb.build();
    cacheManager.defineConfiguration(name, config);

    // ...

}

Paul

[ta...@transdata.net]

"/configuration/" ?

java:jboss/infinispan/configuration/myserver/default-replicated-cache => NameNotFoundException: infinispan/configuration/myserver/default-replicated-cache

java:jboss/infinispan/container/myserver/default-replicated-cache => NotContextException: infinispan/container/myserver

[Paul Ferraro]

It sounds like you added a new "myserver" cache-container.  Can you post your infinispan subsystem configuration of this cache-container?

Note, if you want to reference the configuration of the default cache of a given container, you would use:

@Resource(lookup = "java:jboss/infinispan/configuration/container-name/default")

private Configuration configuration;

Paul

> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/9cfc00be-f51e-4643-acb2-2cff3416f515n%40googlegroups.com.

[ta...@transdata.net]

Sure. Here we go:

           <cache-container name="myserver">

                <transport lock-timeout="60000"/>
                <invalidation-cache name="default-invalidation-cache"/>

                <replicated-cache name="default-replicated-cache"/>
            </cache-container>

[ta...@transdata.net]

I thought it might be related to a missing "resource-ref" in the web.xml

But I had no luck in adding that either (if I did it correctly).

[Paul Ferraro]

OK - those jndi names should work.  How are you looking up the configuration?

To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/9b909689-6242-4314-b012-18e1387af2a0n%40googlegroups.com.

[ta...@transdata.net]

I am not sure what you mean. Like this:

@Resource(lookup = "java:jboss/infinispan/configuration/myserver/default-replicated-cache")

private Configuration configuration;

[ta...@transdata.net]

Oh, I forgot to mention the "root  cause" of the exception. I don't know wether this is normal or not:

Caused by: javax.naming.NameNotFoundException: infinispan/configuration/myserver/default-replicated-cache [Root exception is java.lang.IllegalStateException]
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:153)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:83)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.NamingContext.lookup(NamingContext.java:207)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:239)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.NamingContext.lookup(NamingContext.java:193)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.NamingContext.lookup(NamingContext.java:189)
    at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
    at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
    at org.jbos...@21.0.2.Final//org.jboss.as.weld.services.bootstrap.WeldResourceInjectionServices.resolveResource(WeldResourceInjectionServices.java:240)
    ... 155 more
Caused by: java.lang.IllegalStateException
    at org.jb...@1.4.12.Final//org.jboss.msc.value.InjectedValue.getValue(InjectedValue.java:50)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.service.BinderService.getValue(BinderService.java:148)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.service.BinderService.getValue(BinderService.java:46)
    at org.jb...@1.4.12.Final//org.jboss.msc.service.ServiceControllerImpl.getValue(ServiceControllerImpl.java:1109)
    at org.jboss...@21.0.2.Final//org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)
    ... 164 more

[Paul Ferraro]

Ah - OK.  Does the class that uses @Resource use any CDI annotations?

If so, then CDI is used to resolve the @Resource annotation, which (due to a long standing issue) does not setup the correct dependencies (hence the need for a <resource-ref/>).  If not, then the @Resource annotation is processed by the "ee" subsystem, which does properly setup the correct dependencies for the resource.

Assuming that your bean is CDI-enabled, you will need to setup resource-refs or resource-env-refs in web.xml:

<resource-ref>
    <res-ref-name>mycontainer</res-ref-name>
    <lookup-name>java:jboss/infinispan/container/myserver</lookup-name>
</resource-ref>

<resource-ref>
    <res-ref-name>myconfig</res-ref-name>
    <lookup-name>java:jboss/infinispan/configuration/myserver/default-replicated-cache</lookup-name>
</resource-ref>

Now CDI will be able to resolve your @Resource annotations using either the referenced name, the global jndi name (within the java:jboss namespace), or the context-specific jndi name within the java:comp/env/ namespace:

@Resource(name = "mycontainer")

private EmeddedCacheManager manager;

@Resource(name = "myconfig")

private Configuration configuration;

To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/bcd03b2a-3160-4581-8204-176bc8c679afn%40googlegroups.com.

[ta...@transdata.net]

Yes. I use CDI annotations. That's why I already tried to use resource-refs. Without success so far.

javax.naming.NameNotFoundException: default-replicated-cache -- service jboss.naming.context.java.default-replicated-cache

after having added to my web.xml:

    <resource-ref>
        <res-ref-name>default-replicated-cache</res-ref-name>

        <lookup-name>java:jboss/infinispan/configuration/myserver/default-replicated-cache</lookup-name>
    </resource-ref>

and referencing it with:

@Resource(name = "default-replicated-cache")

private Configuration configuration;

[Paul Ferraro]

Hmm.  This should be working.  How is the bean in question actually packaged?  I assume this is a WAR, yes?  Is the bean class reside in /WEB-INF/classes?  Or is it part of an EAR?

To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/c0e0a38e-18a3-46e8-91f9-6e5495eb8d27n%40googlegroups.com.

[ta...@transdata.net]

Yes. It's a WAR and not part of an EAR.

[ta...@transdata.net]

Interestingly I can get the container in the CDI bean by saying:

    @Resource(lookup = "java:jboss/infinispan/container/myserver")

    private EmbeddedCacheManager cacheManager;

But it does not work with the cache (configuration) by using its lookup name.

And using the resource-ref in web.xml I have the impression that they are totally ignored. For both: cache and container.

[ta...@transdata.net]

As for the resource-ref entries: Actually they are not totally ignored. If I define incorrect lookup names I run into deployment errors:

"WFLYCTL0412: Required services that are not installed:" => ["jboss.naming.context.java.jboss.infinispan.container.myserverxxx"]

[Paul Ferraro]

Hmm.  I'm not exactly sure what's going on.  We definitely have test cases that cover this behavior, including weld-managed injection, but I want to rule out any buggy behavior.

Would you put together a simplified deployment (with it's source) that reproduces the problem and file a bug report here?  https://issues.redhat.com/projects/WFLY/issues/

I'll take a deeper look as soon as possible.

Thanks,

Paul

[ta...@transdata.net]

Sure. Give me some time. I need to purge a lot of unnecessary code first.

Thanks for you help so far!