Infinispan Subsystem - Latest WildFly Documentation

Purpose

The Infinispan subsystem provides caching support for HA services in the form of Infinispan caches: high-performance, transactional caches which can operate in both non-distributed and distributed scenarios. Distributed caching support is used in the provision of many key HA services. For example, the failover of a session-oriented client HTTP request from a failing node to a new (failover) node depends on session data for the client being available on the new node. In other words, the client session data needs to be replicated across nodes in the cluster. This is effectively achieved via a distributed Infinispan cache. This approach to providing fail-over also applies to EJB SFSB sessions. Over and above providing support for fail-over, an underlying cache is also required when providing second-level caching for entity beans using Hibernate, and this case is also handled through the use of an Infinispan cache.

The Infinispan subsystem provides the following features:

allows definition and configuration of named cache containers and caches
view run-time metrics associated with cache container and cache instances

In the following sections, we describe the Infinispan subsystem.

Infiispan cache containers and caches are created transparently as part of the clustering functionality (e.g. on clustered application deployment, cache containers and their associated caches will be created behind the scenes to support clustered features such as session replication or caching of entities around the cluster).

Configuration Example

In this section, we provide an example XML configuration of the infinispan subsystem and review the configuration elements and attributes.

The schema for the subsystem, describing all valid elements and attributes, can be found in the Wildfly distribution, in the docs/schema directory.

<subsystem xmlns="urn:jboss:domain:infinispan:4.0">
  <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server">
      <transport lock-timeout="60000"/>
      <replicated-cache name="default" mode="SYNC">
          <transaction mode="BATCH"/>      </replicated-cache>
  </cache-container>
  <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
      <transport lock-timeout="60000"/>
      <replicated-cache name="repl" mode="SYNC">
          <transaction mode="BATCH"/>          <file-store/>
      </replicated-cache>
      <distributed-cache name="dist" mode="SYNC">
          <transaction mode="BATCH"/>
          <file-store/>
      </distributed-cache>
  </cache-container>
  <cache-container name="ejb" aliases="sfsb" default-cache="dist" module="org.wildfly.clustering.ejb.infinispan">
      <transport lock-timeout="60000"/>
      <replicated-cache name="repl" mode="SYNC">
          <transaction mode="BATCH"/>
          <file-store/>
      </replicated-cache>
      <distributed-cache name="dist" mode="SYNC" l1-lifespan="0">
          <transaction mode="BATCH"/>
          <file-store/>
      </distributed-cache>
  </cache-container>
  <cache-container name="hibernate" module="org.hibernate.infinispan">
      <transport lock-timeout="60000"/>
      <local-cache name="local-query">
          <transaction mode="NONE"/>
          <eviction strategy="LRU" max-entries="10000"/>
          <expiration max-idle="100000"/>
      </local-cache>
      <invalidation-cache name="entity" mode="SYNC">
          <transaction mode="NON_XA"/>
          <eviction strategy="LRU" max-entries="10000"/>
          <expiration max-idle="100000"/>
      </invalidation-cache>
      <replicated-cache name="timestamps" mode="ASYNC">
          <transaction mode="NONE"/>
          <eviction strategy="NONE"/>
       </replicated-cache>
  </cache-container>
</subsystem>

<cache-container>

This element is used to configure a cache container.

name This attribute is used to specify the name of the cache container.
default-cache This attribute configures the default cache to be used, when no cache is otherwise specified.
listener-executor This attribute references a defined thread pool executor in the threads subsystem. It governs the allocation and execution of runnable tasks in the replication queue.
eviction-executor This attribute references a defined thread pool executor in the threads subsystem. It governs the allocation and execution of runnable tasks to handle evictions.
replication-queue-executor This attribute references a defined thread pool executor in the threads subsystem. It governs the allocation and execution of runnable tasks to handle asynchronous cache operations.
jndi-name This attribute is used to assign a name for the cache container in the JNDI name service.
module This attribute configures the module whose class loader should be used when building this cache container's configuration.
start This attribute configured the cache container start mode and has since been deprecated, the only supported and the default value is LAZY (on-demand start).
aliases This attribute is used to define aliases for the cache container name.

This element has the following child elements: <transport>, <local-cache>, <invalidation-cache>, <replicated-cache>, and <distributed-cache>.

<transport>

This element is used to configure the JGroups transport used by the cache container, when required.

channel This attribute configures the JGroups channel to be used for the transport. If none is specified, the default channel as defined by the JGroups subsystem is used.
cluster This attribute configures the name of the group communication cluster. This is the name which will be seen in debugging logs.
executor This attribute references a defined thread pool executor in the threads subsystem. It governs the allocation and execution of runnable tasks to handle ? <fill me in >?.
lock-timeout This attribute configures the time-out to be used when obtaining locks for the transport.
site This attribute configures the site id of the cache container.
rack This attribute configures the rack id of the cache container.
machine This attribute configures the machine id of the cache container.

The presence of the transport element is required when operating in clustered mode

The remaining child elements of <cache-container>, namely <local-cache>, <invalidation-cache>, <replicated-cache> and <distributed-cache>, each configures one of four key cache types or classifications.

These cache-related elements are actually part of an xsd hierarchy with abstract complexTypes cache, clustered-cache, and shared-cache. In order to simplify the presentation, we notate these as pseudo-elements <abstract cache>, <abstract clustered-cache> and <abstract shared-cache>. In what follows, we first describe the extension hierarchy of base elements, and then show how the cache type elements relate to them.

<abstract cache>

This abstract base element defines the attributes and child elements common to all non-clustered caches.

name This attribute configures the name of the cache. This name may be referenced by other subsystems.
start This attribute configured the cache container start mode and has since been deprecated, the only supported and the default value is LAZY (on-demand start).
batching This attribute configures batching. If enabled, the invocation batching API will be made available for this cache.
indexing This attribute configures indexing. If enabled, entries will be indexed when they are added to the cache. Indexes will be updated as entries change or are removed.
jndi-name This attribute is used to assign a name for the cache in the JNDI name service.
module This attribute configures the module whose class loader should be used when building this cache container's configuration.

The <abstract cache> abstract base element has the following child elements: <indexing-properties>, <locking>, <transaction>, <eviction>, <expiration>, <store>, <file-store>, <string-keyed-jdbc-store>, <binary-keyed-jdbc-store>, <mixed-keyed-jdbc-store>, <remote-store>.

<indexing-properties>

This child element defines properties to control indexing behaviour.

<locking>

This child element configures the locking behaviour of the cache.

isolation This attribute the cache locking isolation level. Allowable values are NONE, SERIALIZABLE, REPEATABLE_READ, READ_COMMITTED, READ_UNCOMMITTED.
striping If true, a pool of shared locks is maintained for all entries that need to be locked. Otherwise, a lock is created per entry in the cache. Lock striping helps control memory footprint but may reduce concurrency in the system.
acquire-timeout This attribute configures the maximum time to attempt a particular lock acquisition.
concurrency-level This attribute is used to configure the concurrency level. Adjust this value according to the number of concurrent threads interacting with Infinispan.

<transaction>

This child element configures the transactional behaviour of the cache.

mode This attribute configures the transaction mode, setting the cache transaction mode to one of NONE, NON_XA, NON_DURABLE_XA, FULL_XA.
stop-timeout If there are any ongoing transactions when a cache is stopped, Infinispan waits for ongoing remote and local transactions to finish. The amount of time to wait for is defined by the cache stop timeout.
locking This attribute configures the locking mode for this cache, one of OPTIMISTIC or PESSIMISTIC.

<eviction>

This child element configures the eviction behaviour of the cache.

strategy This attribute configures the cache eviction strategy. Available options are 'UNORDERED', 'FIFO', 'LRU', 'LIRS' and 'NONE' (to disable eviction).
max-entries This attribute configures the maximum number of entries in a cache instance. If selected value is not a power of two the actual value will default to the least power of two larger than selected value. -1 means no limit.

<expiration>

This child element configures the expiration behaviour of the cache.

max-idle This attribute configures the maximum idle time a cache entry will be maintained in the cache, in milliseconds. If the idle time is exceeded, the entry will be expired cluster-wide. -1 means the entries never expire.
lifespan This attribute configures the maximum lifespan of a cache entry, after which the entry is expired cluster-wide, in milliseconds. -1 means the entries never expire.
interval This attribute specifies the interval (in ms) between subsequent runs to purge expired entries from memory and any cache stores. If you wish to disable the periodic eviction process altogether, set wakeupInterval to -1.

The remaining child elements of the abstract base element <cache>, namely <store>, <file-store>, <remote-store>, <string-keyed-jdbc-store>, <binary-keyed-jdbc-store> and <mixed-keyed-jdbc-store>, each configures one of six key cache store types.

These cache store-related elements are actually part of an xsd extension hierarchy with abstract complexTypes base-store and base-jdbc-store. As before, in order to simplify the presentation, we notate these as pseudo-elements <abstract base-store> and <abstract base-jdbc-store>. In what follows, we first describe the extension hierarchy of base elements, and then show how the cache store elements relate to them.

<abstract base-store>

This abstract base element defines the attributes and child elements common to all cache stores.

shared This attribute should be set to true when multiple cache instances share the same cache store (e.g. multiple nodes in a cluster using a JDBC-based CacheStore pointing to the same, shared database) Setting this to true avoids multiple cache instances writing the same modification multiple times. If enabled, only the node where the modification originated will write to the cache store. If disabled, each individual cache reacts to a potential remote update by storing the data to the cache store.
preload This attribute configures whether or not, when the cache starts, data stored in the cache loader will be pre-loaded into memory. This is particularly useful when data in the cache loader is needed immediately after start-up and you want to avoid cache operations being delayed as a result of loading this data lazily. Can be used to provide a 'warm-cache' on start-up, however there is a performance penalty as start-up time is affected by this process. Note that pre-loading is done in a local fashion, so any data loaded is only stored locally in the node. No replication or distribution of the preloaded data happens. Also, Infinispan only pre-loads up to the maximum configured number of entries in eviction.
passivation If true, data is only written to the cache store when it is evicted from memory, a phenomenon known as passivation. Next time the data is requested, it will be 'activated' which means that data will be brought back to memory and removed from the persistent store. If false, the cache store contains a copy of the cache contents in memory, so writes to cache result in cache store writes. This essentially gives you a 'write-through' configuration.
fetch-state This attribute, if true, causes persistent state to be fetched when joining a cluster. If multiple cache stores are chained, only one of them can have this property enabled.
purge This attribute configures whether the cache store is purged upon start-up.
singleton This attribute configures whether or not the singleton store cache store is enabled. SingletonStore is a delegating cache store used for situations when only one instance in a cluster should interact with the underlying store.
class This attribute configures a custom store implementation class to use for this cache store.
properties This attribute is used to configure a list of cache store properties.

The abstract base element has one child element: <write-behind>

<write-behind>

This element is used to configure a cache store as write-behind instead of write-through. In write-through mode, writes to the cache are also synchronously written to the cache store, whereas in write-behind mode, writes to the cache are followed by asynchronous writes to the cache store.

flush-lock-timeout This attribute configures the time-out for acquiring the lock which guards the state to be flushed to the cache store periodically.
modification-queue-size This attribute configures the maximum number of entries in the asynchronous queue. When the queue is full, the store becomes write-through until it can accept new entries.
shutdown-timeout This attribute configures the time-out (in ms) to stop the cache store.
thread-pool This attribute is used to configure the size of the thread pool whose threads are responsible for applying the modifications to the cache store.

<abstract base-jdbc-store> extends <abstract base-store>

This abstract base element defines the attributes and child elements common to all JDBC-based cache stores.

datasource This attribute configures the datasource for the JDBC-based cache store.
entry-table This attribute configures the database table used to store cache entries.
bucket-table This attribute configures the database table used to store binary cache entries.

<file-store> extends <abstract base-store>

This child element is used to configure a file-based cache store. This requires specifying the name of the file to be used as backing storage for the cache store.

relative-to This attribute optionally configures a relative path prefix for the file store path. Can be null.
path This attribute configures an absolute path to a file if relative-to is null; configures a relative path to the file, in relation to the value for relative-to, otherwise.

<remote-store> extends <abstract base-store>

This child element of cache is used to configure a remote cache store. It has a child <remote-servers>.

cache This attribute configures the name of the remote cache to use for this remote store.
tcp-nodelay This attribute configures a TCP_NODELAY value for communication with the remote cache.
socket-timeout This attribute configures a socket time-out for communication with the remote cache.

<remote-servers>

This child element of cache configures a list of remote servers for this cache store.

<remote-server>

This element configures a remote server. A remote server is defined completely by a locally defined outbound socket binding, through which communication is made with the server.

outbound-socket-binding This attribute configures an outbound socket binding for a remote server.

<local-cache> extends <abstract cache>

This element configures a local cache.

<abstract clustered-cache> extends <abstract cache>

This abstract base element defines the attributes and child elements common to all clustered caches. A clustered cache is a cache which spans multiple nodes in a cluster. It inherits from <cache>, so that all attributes and elements of <cache> are also defined for <clustered-cache>.

async-marshalling This attribute configures async marshalling. If enabled, this will cause marshalling of entries to be performed asynchronously.
mode This attribute configures the clustered cache mode, ASYNC for asynchronous operation, or SYNC for synchronous operation.
queue-size In ASYNC mode, this attribute can be used to trigger flushing of the queue when it reaches a specific threshold.
queue-flush-interval In ASYNC mode, this attribute controls how often the asynchronous thread used to flush the replication queue runs. This should be a positive integer which represents thread wakeup time in milliseconds.
remote-timeout In SYNC mode, this attribute (in ms) used to wait for an acknowledgement when making a remote call, after which the call is aborted and an exception is thrown.

<invalidation-cache> extends <abstract clustered-cache>

This element configures an invalidation cache.

<abstract shared-cache> extends <abstract clustered-cache>

This abstract base element defines the attributes and child elements common to all shared caches. A shared cache is a clustered cache which shares state with its cache peers in the cluster. It inherits from <clustered-cache>, so that all attributes and elements of <clustered-cache> are also defined for <shared-cache>.

<state-transfer>

enabled If enabled, this will cause the cache to ask neighbouring caches for state when it starts up, so the cache starts 'warm', although it will impact start-up time.
timeout This attribute configures the maximum amount of time (ms) to wait for state from neighbouring caches, before throwing an exception and aborting start-up.
chunk-size This attribute configures the size, in bytes, in which to batch the transfer of cache entries.

<backups>

<backup>

strategy This attribute configures the backup strategy for this cache. Allowable values are SYNC, ASYNC.
failure-policy This attribute configures the policy to follow when connectivity to the backup site fails. Allowable values are IGNORE, WARN, FAIL, CUSTOM.
enabled This attribute configures whether or not this backup is enabled. If enabled, data will be sent to the backup site; otherwise, the backup site will be effectively ignored.
timeout This attribute configures the time-out for replicating to the backup site.
after-failures This attribute configures the number of failures after which this backup site should go off-line.
min-wait This attribute configures the minimum time (in milliseconds) to wait after the max number of failures is reached, after which this backup site should go off-line.

<backup-for>

remote-cache This attribute configures the name of the remote cache for which this cache acts as a backup.
remote-site This attribute configures the site of the remote cache for which this cache acts as a backup.

<replicated-cache> extends <abstract shared-cache>

This element configures a replicated cache. With a replicated cache, all contents (key-value pairs) of the cache are replicated on all nodes in the cluster.

<distributed-cache> extends <abstract shared-cache>

This element configures a distributed cache. With a distributed cache, contents of the cache are selectively replicated on nodes in the cluster, according to the number of owners specified.

owners This attribute configures the number of cluster-wide replicas for each cache entry.
segments This attribute configures the number of hash space segments which is the granularity for key distribution in the cluster. Value must be strictly positive.
l1-lifespan This attribute configures the maximum lifespan of an entry placed in the L1 cache. Configures the L1 cache behaviour in 'distributed' caches instances. In any other cache modes, this element is ignored.

Use Cases

In many cases, cache containers and caches will be configured via XML as in the example above, so that they will be available upon server start-up. However, cache containers and caches may also be added, removed or have their configurations changed in a running server by making use of the Wildfly management API command-line interface (CLI). In this section, we present some key use cases for the Infinispan management API.

The key use cases covered are:

adding a cache container
adding a cache to an existing cache container
configuring the transaction subsystem of a cache

The Wildfly management API command-line interface (CLI) can be used to provide extensive information on the attributes and commands available in the Infinispan subsystem interface used in these examples.

Add a cache container

/subsystem=infinispan/cache-container=mycontainer:add(default-cache=<default-cache-name>)
/subsystem=infinispan/cache-container=mycontainer/transport=jgroups:add(lock-timeout=<timeout>)

Add a cache

/subsystem=infinispan/cache-container=mycontainer/local-cache=mylocalcache:add()

Configure the transaction component of a cache

/subsystem=infinispan/cache-container=mycontainer/local-cache=mylocalcache/component=transaction:add(mode=<transaction-mode>)