Chapter 28. Infinispan integration

eXo JCR can rely on distributed cache such as Infinispan. This article describes the required configuration.

28.1. Components configuration requirements

<component>
    <key>org.infinispan.transaction.lookup.TransactionManagerLookup</key>
    <type>org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup</type>
</component>

<component profiles="ispn">
    <key>org.exoplatform.services.transaction.TransactionService</key>
    <type>org.exoplatform.services.transaction.infinispan.JBossTransactionsService</type>
    <init-params>
        <value-param>
            <name>timeout</name>
            <value>3000</value>
        </value-param>
    </init-params>
</component>

<component profiles="ispn">
    <key>org.exoplatform.services.rpc.RPCService</key>
    <type>org.exoplatform.services.rpc.jgv3.RPCServiceImpl</type>
    <init-params>
        <value-param>
            <name>jgroups-configuration</name>
            <value>jar:/conf/udp-mux-v3.xml</value>
        </value-param>
        <value-param>
            <name>jgroups-cluster-name</name>
            <value>RPCService-Cluster</value>
        </value-param>
        <value-param>
            <name>jgroups-default-timeout</name>
            <value>0</value>
        </value-param>
    </init-params>
</component>

28.2. Workspaces configuration requirements

Each mentioned below components uses instances of Infinispan Cache product for caching in clustered environment. So every element has it's own transport and has to be configured in a proper way. As usual, workspaces have similar configuration. The simplest way to configure them is to define their own configuration files for each component in each workspace. There are several commons parameters.

"infinispan-configuration" defines path to template based configuration for Infinispan Cache instance.

JGroups is used by Infinispan Cache for network communications and transport in a clustered environment. If property "jgroups-configuration" is defined in component configuration, it will be injected into the Infinispan Cache instance on startup.

The another parameter is "infinispan-cluster-name". This defines the name of the cluster. Needs to be the same for all nodes in a cluster in order to find each other.

Cache configuration:

<cache enabled="true" class="org.exoplatform.services.jcr.impl.dataflow.persistent.infinispan.ISPNCacheWorkspaceStorageCache">
    <properties>
        <property name="infinispan-configuration" value="jar:/conf/portal/cluster/infinispan-data.xml" />
        <property name="jgroups-configuration" value="jar:/conf/udp-mux-v3.xml" />
        <property name="infinispan-cluster-name" value="JCR-cluster" />
    </properties>
</cache>

Indexer configuration

<query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex">
    <properties>
        <property name="index-dir" value="${exo.jcr.parent.dir:..}/temp/jcrlucenedb/production" />
        <property name="changesfilter-class" value="org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter" />
        <property name="infinispan-configuration" value="jar:/conf/portal/cluster/infinispan-indexer.xml" />
        <property name="jgroups-configuration" value="jar:/conf/udp-mux-v3.xml" />
        <property name="infinispan-cluster-name" value="JCR-cluster" />
        <property name="max-volatile-time" value="60" />
    </properties>
</query-handler>

changesfilter-class - defines cluster-ready index strategy based on Infinispan Cache, it can be either org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter (for shared index) or org.exoplatform.services.jcr.impl.core.query.ispn.LocalIndexChangesFilter (for local index)

Lock Manager configuration

<lock-manager class="org.exoplatform.services.jcr.impl.core.lock.infinispan.ISPNCacheableLockManagerImpl">
    <properties>
        <property name="time-out" value="15m" />
        <property name="infinispan-configuration" value="jar:/conf/portal/cluster/infinispan-lock.xml" />
        <property name="jgroups-configuration" value="jar:/conf/udp-mux-v3.xml" />
        <property name="infinispan-cluster-name" value="JCR-cluster" />
        <property name="infinispan-cl-cache.jdbc.table.name" value="lk" />
        <property name="infinispan-cl-cache.jdbc.table.create" value="true" />
        <property name="infinispan-cl-cache.jdbc.table.drop" value="false" />
        <property name="infinispan-cl-cache.jdbc.id.column" value="id" />
        <property name="infinispan-cl-cache.jdbc.data.column" value="data" />
        <property name="infinispan-cl-cache.jdbc.timestamp.column" value="timestamp" />
        <property name="infinispan-cl-cache.jdbc.datasource" value="jdbcjcr" />
        <property name="infinispan-cl-cache.jdbc.connectionFactory" value="org.infinispan.loaders.jdbc.connectionfactory.ManagedConnectionFactory" />
    </properties>
</lock-manager>^

infinispan-cl-cache.jdbc.table.name - table name
infinispan-cl-cache.jdbc.table.create - is true or false. Indicates whether to create table at start phase. If true, the table is created if it does not already exist.
infinispan-cl-cache.jdbc.table.drop - is true or false. Indicates whether to drop the table at stop phase.
infinispan-cl-cache.jdbc.id.column - id column name
infinispan-cl-cache.jdbc.data.column - data column name
infinispan-cl-cache.jdbc.timestamp.column - timestamp column name
infinispan-cl-cache.jdbc.datasource - name of configured in Container datasource, where you want to store locks. The best idea is to use the same as for workspace.
infinispan-cl-cache.jdbc.connectionFactory - connection factories for use with the JDBC Cache Store.

28.3. Shipped Infinispan Cache configuration templates

eXo JCR implementation is shipped with ready-to-use Infinispan Cache configuration templates for JCR's components.

28.3.1. Data container template

Data container template is "infinispan-data.xml":

<infinispan 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". 
      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd". 
      xmlns="urn:infinispan:config:5.1">

    <global>
      <evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
        <properties>
          <property name="threadNamePrefix" value="EvictionThread"/>
        </properties>
      </evictionScheduledExecutor>

      <globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/>

      <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout=
        <properties>
          <property name="configurationFile" value="${jgroups-configuration}"/>
        </properties>
      </transport>
    </global>

    <default>
      <clustering mode="replication">
        <stateTransfer timeout="20000" fetchInMemoryState="false" />
        <sync replTimeout="20000"/>
      </clustering>

      <locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="true"/>
      <transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" s
      <jmxStatistics enabled="true"/>
      <eviction strategy="LRU" threadPolicy="DEFAULT" maxEntries="1000000"/>
      <expiration wakeUpInterval="5000"/>
   </default>
</infinispan>

Table 28.1. Template variables

infinispan-cluster-name

jgroups-configuration

28.3.2. Lock manager template

Its template name is "infinispan-lock.xml"

<infinispan 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". 
      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd". 
      xmlns="urn:infinispan:config:5.1">

    <global>
      <evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
        <properties>
          <property name="threadNamePrefix" value="EvictionThread"/>
        </properties>
      </evictionScheduledExecutor>

      <globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/>

      <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout=
        <properties>
          <property name="configurationFile" value="${jgroups-configuration}"/>
        </properties>
      </transport>
    </global>

    <default>
      <clustering mode="replication">
        <stateTransfer timeout="20000" fetchInMemoryState="false" />
        <sync replTimeout="20000"/>
      </clustering>

      <locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="false"/>
      <transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" s
      <jmxStatistics enabled="true"/>
      <eviction strategy="NONE"/>

      <loaders passivation="false" shared="true" preload="true">
        <loader class="org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore" fetchPersistentState="true" ignoreModifications="false" purgeOnStar
          <properties>
             <property name="stringsTableNamePrefix" value="${infinispan-cl-cache.jdbc.table.name}"/>
             <property name="idColumnName" value="${infinispan-cl-cache.jdbc.id.column}"/>
             <property name="dataColumnName" value="${infinispan-cl-cache.jdbc.data.column}"/>
             <property name="timestampColumnName" value="${infinispan-cl-cache.jdbc.timestamp.column}"/>
             <property name="idColumnType" value="${infinispan-cl-cache.jdbc.id.type}"/>
             <property name="dataColumnType" value="${infinispan-cl-cache.jdbc.data.type}"/>
             <property name="timestampColumnType" value="${infinispan-cl-cache.jdbc.timestamp.type}"/>
             <property name="dropTableOnExit" value="${infinispan-cl-cache.jdbc.table.drop}"/>
             <property name="createTableOnStart" value="${infinispan-cl-cache.jdbc.table.create}"/>
             <property name="connectionFactoryClass" value="${infinispan-cl-cache.jdbc.connectionFactory}"/>
             <property name="datasourceJndiLocation" value="${infinispan-cl-cache.jdbc.datasource}"/>
          </properties>
          <async enabled="false"/>
        </loader>
      </loaders>
   </default>

</infinispan>

Table 28.2. Template variables

infinispan-cluster-name

jgroups-configuration

infinispan-cl-cache.jdbc.table.name

infinispan-cl-cache.jdbc.id.column

infinispan-cl-cache.jdbc.data.column

infinispan-cl-cache.jdbc.timestamp.column

infinispan-cl-cache.jdbc.id.type

infinispan-cl-cache.jdbc.data.type

infinispan-cl-cache.jdbc.timestamp.type

infinispan-cl-cache.jdbc.table.drop

infinispan-cl-cache.jdbc.table.create

infinispan-cl-cache.jdbc.connectionFactory

infinispan-cl-cache.jdbc.datasource

28.3.3. Query handler (indexer) template

Have a look at "infinispan-indexer.xml"

<infinispan 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". 
      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd". 
      xmlns="urn:infinispan:config:5.1">

    <global>
      <evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
        <properties>
          <property name="threadNamePrefix" value="EvictionThread"/>
        </properties>
      </evictionScheduledExecutor>

      <globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/>

      <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout=
        <properties>
          <property name="configurationFile" value="${jgroups-configuration}"/>
        </properties>
      </transport>
    </global>

    <default>
      <clustering mode="replication">
        <stateTransfer timeout="20000" fetchInMemoryState="false" />
        <sync replTimeout="20000"/>
      </clustering>

      <locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="false"/>
      <transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" s
      <jmxStatistics enabled="true"/>
      <eviction strategy="NONE"/>

      <loaders passivation="false" shared="false" preload="false">
        <loader class="${infinispan-cachestore-classname}" fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false">
          <async enabled="false"/>
        </loader>
      </loaders>
   </default>
</infinispan>

Table 28.3. Template variable

infinispan-cluster-name

jgroups-configuration