The main purpose of content repository is to maintain the data. The heart of CR is the data model:

Figure 1.1. Item hierarchy

JCR (Java Content Repository) is a java interface used to access contents that are not only web contents, but also other hierarchically stored data. The content is stored in a repository. The repository can be a file system, a relational database or an XML document. The internal structure of JCR data looks similar to an XML document, that means a document tree with nodes and data, but with a small difference, in JCR the data are stored in "property items".

Or better to cite the specification of JCR: "A content repository is a high-level information management system that is a superset of traditional data repositories."

How do you know the data of your website are stored? The images are probably in a file system, the meta data are in some dedicated files - maybe in XML - the text documents and pdfs are stored in different folders with the meta data in an other place (a database?) and in a proprietary structure. How do you manage to update these data and how do you manage the access rights? If your boss asks you to manage different versions of each document or not? The larger your website is, the more you need a Content Management Systems (CMS) which tackles all these issues.

These CMS solutions are sold by different vendors and each vendor provides its own API for interfacing the proprietary content repository. The developers have to deal with this and need to learn the vendor-specific API. If in the future you wish to switch to a different vendor, everything will be different and you will have a new implementation, a new interface, etc.

JCR provides a unique java interface for interacting with both text and binary data, for dealing with any kind and amount of meta data your documents might have. JCR supplies methods for storing, updating, deleting and retrieving your data, independent of the fact if this data is stored in a RDBMS, in a file system or as an XML document - you just don't need to care about. The JCR interface is also defined as classes and methods for searching, versioning, access control, locking, and observation.

Furthermore, an export and import functionality is specified so that a switch to a different vendor is always possible.

eXo fully complies a JCR standard JSR 170; therefore with eXo JCR you can use a vendor-independent API. It means that you could switch any time to a different vendor. Using the standard lowers your lifecycle cost and reduces your long term risk.

Of course eXo does not only offer JCR, but also the complete solution for ECM (Enterprise Content Management) and for WCM (Web Content Management).

In order to further understand the theory of JCR and the API, please refer to some external documents about this standard:

Access Control Configuration, Export Import Implementation, External Value Storages, JDBC Data Container config, Locking, Multilanguage support, Node types and Namespaces, Repository and Workspace management, Repository container life cycle, Workspace, Persistence Storage Workspace, SimpleDB storage

eXo Repository Service is a standard eXo service and is a registered IoC component, i.e. can be deployed in some eXo Containers (see Service configuration for details). The relationships between components are shown in the picture below:

eXo Container: some subclasses of org.exoplatform.container.ExoContainer (usually org.exoplatform.container.StandaloneContainer or org.exoplatform.container.PortalContainer) that holds a reference to Repository Service.

Usual JCR application use case includes two initial steps:

The following diagram explains which components of eXo JCR implementation are used in a data flow to perform operations specified in JCR API

The Workspace Data Model can be split into 4 levels by data isolation and value from the JCR model point of view.

An extension to the API exists as we can see in the following layer schema.

Level 1 includes read-only functionality for very simple repositories. It is useful to port an existing data repository and convert it to a more advanced form step by step. JCR uses a well-known Session abstraction to access the repository data (similar to the sessions we have in OS, web, etc).

The features of level 1:

JCR level 2 includes reading/ writing content functionality, importing other sources and managing content definition and structuring using extensible node types.

In addition to the features of the Level 1, it also supports the following major features:

On the top of Level 1 or Level 2, a number of optional features are defined for a more advanced repository functionality. This includes functions such as Versioning, (JTA) Transactions, Query using SQL, Explicit Locking and Content Observation. eXo JCR supports all optional features.

A javax.jcr.Repository object can be obtained by:

            RepositoryService repositoryService = (RepositoryService) container.getComponentInstanceOfType(RepositoryService.class);
  Repository repository = repositoryService.getRepository("repositoryName"); 
            

  // set current repository at initial time
  RepositoryService repositoryService = (RepositoryService) container.getComponentInstanceOfType(RepositoryService.class);
  repositoryService.setCurrentRepositoryName("repositoryName");
  ....
  // retrieve and use this repository
  Repository repository = repositoryService.getCurrentRepository();
            

  Context ctx = new InitialContext();
  Repository repository =(Repository) ctx.lookup("repositoryName");

eXo JCR supports observation (JSR-170 8.3), which enables applications to register interest in events that describe changes to a workspace, and then monitor and respond to those events. The standard observation feature allows dispatching events when persistent change to the workspace is made.

eXo JCR also offers a proprietary Extension Action which dispatches and fires an event upon each transient session level change, performed by a client. In other words, the event is triggered when a client's program invokes some updating methods in a session or a workspace (such as: Session.addNode(), Session.setProperty(), Workspace.move() etc.

By default when an action fails, the related exception is simply logged. In case you would like to change the default exception handling, you can implement the interface AdvancedAction. In case the JCR detects that your action is of type AdvancedAction, it will call the method onError instead of simply logging it. A default implementation of the onError method is available in the abstract class AbstractAdvancedAction. It reverts all pending changes of the current JCR session for any kind of event corresponding to a write operation. Then in case the provided exception is an instance of type AdvancedActionException, it will throw it otherwise it will log simply it. An AdvancedActionException will be thrown in case the changes could not be reverted.

One important recommendation should be applied for an extension action implementation. Each action will add its own execution time to standard JCR methods (Session.addNode(), Session.setProperty(), Workspace.move() etc.) execution time. As a consequence, it's necessary to minimize Action.execute(Context) body execution time.

To make the rule, you can use the dedicated Thread in Action.execute(Context) body for a custom logic. But if your application logic requires the action to add items to a created/updated item and you save these changes immediately after the JCR API method call is returned, the suggestion with Thread is not applicable for you in this case.

Interceptor framework class diagram

Add a SessionActionCatalog service and an appropriate AddActionsPlugin (see the example below) configuration to your eXo Container configuration. As usual, the plugin can be configured as in-component-place, which is the case for a Standalone Container or externally, which is a usual case for Root/Portal Container configuration).

Each Action entry is exposed as org.exoplatform.services.jcr.impl.ext.action. ActionConfiguration of actions collection of org.exoplatform.services.jcr.impl.ext.action.AddActionsPlugin$ActionsConfig (see an example below). The mandatory field named actionClassName is the fully qualified name of org.exoplatform.services.command.action.Action implementation - the command will be launched in case the current event matches the criteria. All other fields are criteria. The criteria are *AND*ed together. In other words, for a particular item to be listened to, it must meet ALL the criteria:

* workspace: the comma delimited (ORed) list of workspaces

* eventTypes: a comma delimited (ORed) list of event names (see below) to be listened to. This is the only mandatory field, others are optional and if they are missing they are interpreted as ANY.

* path - a comma delimited (ORed) list of item absolute paths (or within its subtree if isDeep is true, which is the default value)

* nodeTypes - a comma delimited (ORed) list of the current NodeType. Since version 1.6.1 JCR supports the functionalities of nodeType and parentNodeType. This parameter has different semantics, depending on the type of the current item and the operation performed. If the current item is a property it means the parent node type. If the current item is a node, the semantic depends on the event type: ** add node event: the node type of the newly added node. ** add mixin event: the newly added mixing node type of the current node. ** remove mixin event the removed mixin type of the current node. ** other events: the already assigned NodeType(s) of the current node (can be both primary and mixin).

The list of supported Event names: addNode, addProperty, changeProperty, removeProperty, removeNode, addMixin, removeMixin, lock, unlock, checkin, checkout, read.

<component>
   <type>org.exoplatform.services.jcr.impl.ext.action.SessionActionCatalog</type>
   <component-plugins>
      <component-plugin>
         <name>addActions</name>
         <set-method>addPlugin</set-method>
         <type>org.exoplatform.services.jcr.impl.ext.action.AddActionsPlugin</type>
         <description>add actions plugin</description>
         <init-params>
            <object-param>
               <name>actions</name>
               <object type="org.exoplatform.services.jcr.impl.ext.action.AddActionsPlugin$ActionsConfig">
               <field  name="actions">
                  <collection type="java.util.ArrayList">
                     <value>
                        <object type="org.exoplatform.services.jcr.impl.ext.action.ActionConfiguration">
                          <field  name="eventTypes"><string>addNode,removeNode</string></field>
                          <field  name="path"><string>/test,/exo:test</string></field>       
                          <field  name="isDeep"><boolean>true</boolean></field>       
                          <field  name="nodeTypes"><string>nt:file,nt:folder,mix:lockable</string></field>       
                          <!-- field  name="workspace"><string>backup</string></field -->
                          <field  name="actionClassName"><string>org.exoplatform.services.jcr.ext.DummyAction</string></field>       
                        </object>
                     </value>
                  </collection>
               </field>
            </object>
          </object-param>
        </init-params>
      </component-plugin>
    </component-plugins>
</component>

Access Control extension

Registry Service

REST Groovy services

eXo JCR implementation supports various methods of the node-type registration.

ExtendedNodeTypeManager nodeTypeManager = (ExtendedNodeTypeManager) session.getWorkspace()
                                                             .getNodeTypeManager();
InputStream is = MyClass.class.getResourceAsStream("mynodetypes.xml");
nodeTypeManager.registerNodeTypes(is,ExtendedNodeTypeManager.IGNORE_IF_EXISTS );

ExtendedNodeTypeManager nodeTypeManager = (ExtendedNodeTypeManager) session.getWorkspace()
                                                             .getNodeTypeManager();
NodeTypeValue testNValue = new NodeTypeValue();

List<String> superType = new ArrayList<String>();
superType.add("nt:base");
testNValue.setName("exo:myNodeType");
testNValue.setPrimaryItemName("");
testNValue.setDeclaredSupertypeNames(superType);
List<PropertyDefinitionValue> props = new ArrayList<PropertyDefinitionValue>();
props.add(new PropertyDefinitionValue("*",
                                      false,
                                      false,
                                      1,
                                      false,
                                      new ArrayList<String>(),
                                      false,
                                      0,
                                      new ArrayList<String>()));
testNValue.setDeclaredPropertyDefinitionValues(props);

nodeTypeManager.registerNodeType(testNValue, ExtendedNodeTypeManager.FAIL_IF_EXISTS);

If you want to replace existing node type definition, you should pass ExtendedNodeTypeManager.REPLACE_IF_EXISTS as a second parameter for the method ExtendedNodeTypeManager.registerNodeType.

ExtendedNodeTypeManager nodeTypeManager = (ExtendedNodeTypeManager) session.getWorkspace()
                                                                           .getNodeTypeManager();
InputStream is = MyClass.class.getResourceAsStream("mynodetypes.xml");
.....
nodeTypeManager.registerNodeTypes(is,ExtendedNodeTypeManager.REPLACE_IF_EXISTS );

nodeTypeManager.unregisterNodeType("myNodeType");

The Registry Service is one of the key parts of the infrastructure built around eXo JCR. Each JCR that is based on service, applications, etc may have its own configuration, settings data and other data that have to be stored persistently and used by the approptiate service or application. ( We call it "Consumer").

The service acts as a centralized collector (Registry) for such data. Naturally, a registry storage is JCR based i.e. stored in some JCR workspace (one per Repository) as an Item tree under /exo:registry node.

Despite the fact that the structure of the tree is well defined (see the scheme below), it is not recommended for other services to manipulate data using JCR API directly for better flexibility. So the Registry Service acts as a mediator between a Consumer and its settings.

The proposed structure of the Registry Service storage is divided into 3 logical groups: services, applications and users:

 exo:registry/          <-- registry "root" (exo:registry)
   exo:services/        <-- service data storage (exo:registryGroup)
     service1/
       Consumer data    (exo:registryEntry)
     ...
   exo:applications/    <-- application data storage (exo:registryGroup)
     app1/
       Consumer data    (exo:registryEntry)
     ...
   exo:users/           <-- user personal data storage (exo:registryGroup)
     user1/
       Consumer data    (exo:registryEntry)
     ...

Each upper level eXo Service may store its configuration in eXo Registry. At first, start from xml-config (in jar etc) and then from Registry. In configuration file, you can add force-xml-configuration parameter to component to ignore reading parameters initialization from RegistryService and to use file instead:

<value-param>
  <name>force-xml-configuration</name>
  <value>true</value>
</value-param>

The main functionality of the Registry Service is pretty simple and straightforward, it is described in the Registry abstract class as the following:

public abstract class Registry
{

   /**
    * Returns Registry node object which wraps Node of "exo:registry" type (the whole registry tree)
    */
   public abstract RegistryNode getRegistry(SessionProvider sessionProvider) throws RepositoryConfigurationException,
      RepositoryException;

   /**
    * Returns existed RegistryEntry which wraps Node of "exo:registryEntry" type
    */
   public abstract RegistryEntry getEntry(SessionProvider sessionProvider, String entryPath)
      throws PathNotFoundException, RepositoryException;

   /**
    * creates an entry in the group. In a case if the group does not exist it will be silently
    * created as well
    */
   public abstract void createEntry(SessionProvider sessionProvider, String groupPath, RegistryEntry entry)
      throws RepositoryException;

   /**
    * updates an entry in the group
    */
   public abstract void recreateEntry(SessionProvider sessionProvider, String groupPath, RegistryEntry entry)
      throws RepositoryException;

   /**
    * removes entry located on entryPath (concatenation of group path / entry name)
    */
   public abstract void removeEntry(SessionProvider sessionProvider, String entryPath) throws RepositoryException;

}

As you can see it looks like a simple CRUD interface for the RegistryEntry object which wraps registry data for some Consumer as a Registry Entry. The Registry Service itself knows nothing about the wrapping data, it is Consumer's responsibility to manage and use its data in its own way.

To create an Entity Consumer you should know how to serialize the data to some XML structure and then create a RegistryEntry from these data at once or populate them in a RegistryEntry object (using RegistryEntry(String entryName) constructor and then obtain and fill a DOM document).

Example of RegistryService using:

    RegistryService regService = (RegistryService) container
    .getComponentInstanceOfType(RegistryService.class);

    RegistryEntry registryEntry = regService.getEntry(sessionProvider,
            RegistryService.EXO_SERVICES + "/my-service");

    Document doc = registryEntry.getDocument();
    
    String mySetting = getElementsByTagName("tagname").item(index).getTextContent();
     .....

RegistryService has two optional params: value parameter mixin-names and properties parameter locations. The mixin-names is used for adding additional mixins to nodes exo:registry, exo:applications, exo:services, exo:users and exo:groups of RegistryService. This allows the top level applications to manage these nodes in special way. Locations is used to mention where exo:registry is placed for each repository. The name of each property is interpreted as a repository name and its value as a workspace name (a system workspace by default).

<component>
   <type>org.exoplatform.services.jcr.ext.registry.RegistryService</type>
   <init-params>
      <values-param>
         <name>mixin-names</name>
         <value>exo:hideable</value>      
      </values-param>
      <properties-param>         
      <name>locations</name>
         <property name="db1" value="ws2"/>
      </properties-param>
   </init-params>
</component>

ExtendedNamespaceRegistry namespaceRegistry = (ExtendedNamespaceRegistry) workspace.getNamespaceRegistry();
namespaceRegistry.registerNamespace("newMapping", "http://dumb.uri/jcr");

ExtendedNamespaceRegistry namespaceRegistry = (ExtendedNamespaceRegistry) workspace.getNamespaceRegistry();
namespaceRegistry.registerNamespace("newMapping", "http://dumb.uri/jcr");
namespaceRegistry.registerNamespace("newMapping2", "http://dumb.uri/jcr");

ExtendedNamespaceRegistry namespaceRegistry = (ExtendedNamespaceRegistry) workspace.getNamespaceRegistry();
namespaceRegistry.registerNamespace("newMapping", "http://dumb.uri/jcr");
namespaceRegistry.unregisterNamespace("newMapping");

Node type registration extension is declared in org.exoplatform.services.jcr.core.nodetype.ExtendedNodeTypeManager interface

Your custom service can register some neccessary predefined node types at the start-up time. The node definition should be placed in a special XML file (see DTD below) and declared in the service's configuration file thanks to eXo component plugin mechanism, described as follows:

<external-component-plugins>
  <target-component>org.exoplatform.services.jcr.RepositoryService</target-component>
      <component-plugin>
        <name>add.nodeType</name>
        <set-method>addPlugin</set-method>
        <type>org.exoplatform.services.jcr.impl.AddNodeTypePlugin</type>
        <init-params>
          <values-param>
            <name>autoCreatedInNewRepository</name>
            <description>Node types configuration file</description>
            <value>jar:/conf/test/nodetypes-tck.xml</value>
            <value>jar:/conf/test/nodetypes-impl.xml</value>
          </values-param>
    <values-param> 
            <name>repo1</name> 
            <description>Node types configuration file for repository with name repo1</description> 
            <value>jar:/conf/test/nodetypes-test.xml</value> 
          </values-param>
    <values-param> 
            <name>repo2</name> 
            <description>Node types configuration file for repository with name repo2</description> 
            <value>jar:/conf/test/nodetypes-test2.xml</value> 
          </values-param>
        </init-params>
      </component-plugin>

There are two types of registration. The first type is the registration of node types in all created repositories, it is configured in values-param with the name autoCreatedInNewRepository. The second type is registration of node types in specified repository and it is configured in values-param with the name of repository.

Node type definition file format:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE nodeTypes [
   <!ELEMENT nodeTypes (nodeType)*>
      <!ELEMENT nodeType (supertypes?|propertyDefinitions?|childNodeDefinitions?)>

      <!ATTLIST nodeType
         name CDATA #REQUIRED
         isMixin (true|false) #REQUIRED
         hasOrderableChildNodes (true|false)
         primaryItemName CDATA
      >
      <!ELEMENT supertypes (supertype*)>
      <!ELEMENT supertype (CDATA)>
   
      <!ELEMENT propertyDefinitions (propertyDefinition*)>

      <!ELEMENT propertyDefinition (valueConstraints?|defaultValues?)>
      <!ATTLIST propertyDefinition
         name CDATA #REQUIRED
         requiredType (String|Date|Path|Name|Reference|Binary|Double|Long|Boolean|undefined) #REQUIRED
         autoCreated (true|false) #REQUIRED
         mandatory (true|false) #REQUIRED
         onParentVersion (COPY|VERSION|INITIALIZE|COMPUTE|IGNORE|ABORT) #REQUIRED
         protected (true|false) #REQUIRED
         multiple  (true|false) #REQUIRED
      >    
    <!-- For example if you need to set ValueConstraints [], 
      you have to add an empty element <valueConstraints/>. 
      The same order is for other properties like defaultValues, requiredPrimaryTypes etc.
      -->  
      <!ELEMENT valueConstraints (valueConstraint*)>
      <!ELEMENT valueConstraint (CDATA)>
      <!ELEMENT defaultValues (defaultValue*)>
      <!ELEMENT defaultValue (CDATA)>

      <!ELEMENT childNodeDefinitions (childNodeDefinition*)>

      <!ELEMENT childNodeDefinition (requiredPrimaryTypes)>
      <!ATTLIST childNodeDefinition
         name CDATA #REQUIRED
         defaultPrimaryType  CDATA #REQUIRED
         autoCreated (true|false) #REQUIRED
         mandatory (true|false) #REQUIRED
         onParentVersion (COPY|VERSION|INITIALIZE|COMPUTE|IGNORE|ABORT) #REQUIRED
         protected (true|false) #REQUIRED
         sameNameSiblings (true|false) #REQUIRED
      >
      <!ELEMENT requiredPrimaryTypes (requiredPrimaryType+)>
      <!ELEMENT requiredPrimaryType (CDATA)>  
]>

Default namespaces are registered by repository at the start-up time

Your custom service can extend a set of namespaces with some application specific ones, declaring it in service's configuration file thanks to eXo component plugin mechanism, described as follows:

      <component-plugin> 
          <name>add.namespaces</name>
          <set-method>addPlugin</set-method>
          <type>org.exoplatform.services.jcr.impl.AddNamespacesPlugin</type>
          <init-params>
            <properties-param>
              <name>namespaces</name>
              <property name="test" value="http://www.test.org/test"/>
            </properties-param>      
          </init-params>                  
      </component-plugin>

Like other eXo services, eXo JCR can be configured and used in the portal or embedded mode (as a service embedded in GateIn) and in standalone mode.

In Embedded mode, JCR services are registered in the Portal container and the second option is to use a Standalone container. The main difference between these container types is that the first one is intended to be used in a Portal (Web) environment, while the second one can be used standalone (see the comprehensive page Service Configuration for Beginners for more details).

The following setup procedure is used to obtain a Standalone configuration (see more in Container configuration):

JCR service configuration looks like:

<component>
  <key>org.exoplatform.services.jcr.RepositoryService</key>
  <type>org.exoplatform.services.jcr.impl.RepositoryServiceImpl</type>
</component>
<component>
  <key>org.exoplatform.services.jcr.config.RepositoryServiceConfiguration</key>
  <type>org.exoplatform.services.jcr.impl.config.RepositoryServiceConfigurationImpl</type>
  <init-params>
    <value-param>
      <name>conf-path</name>
      <description>JCR repositories configuration file</description>
      <value>jar:/conf/standalone/exo-jcr-config.xml</value>
    </value-param>
    <value-param>
      <name>max-backup-files</name>
      <value>5</value>
    </value-param>
    <properties-param>
      <name>working-conf</name>
      <description>working-conf</description>
      <property name="source-name" value="jdbcjcr" />
      <property name="dialect" value="hsqldb" />
      <property name="persister-class-name" value="org.exoplatform.services.jcr.impl.config.JDBCConfigurationPersister" />
    </properties-param>
  </init-params>
</component>

conf-path : a path to a RepositoryService JCR Configuration.

max-backup-files : max number of backup files. This option lets you specify the number of stored backups. Number of backups can't exceed this value. File which will exceed the limit will replace the oldest file.

working-conf : optional; JCR configuration persister configuration. If there isn't a working-conf, the persister will be disabled.

The Configuration is defined in an XML file (see DTD below).

JCR Service can use multiple Repositories and each repository can have multiple Workspaces.

From v.1.9 JCR, repositories configuration parameters support human-readable formats of values. They are all case-insensitive:

Service configuration may be placed in jar:/conf/standalone/exo-jcr-config.xml for standalone mode. For portal mode, it is located in the portal web application portal/WEB-INF/conf/jcr/repository-configuration.xml.

default-repository: The name of a default repository (one returned by RepositoryService.getRepository()).

repositories: The list of repositories.

name: The name of a repository.

default-workspace: The name of a workspace obtained using Session's login() or login(Credentials) methods (ones without an explicit workspace name).

system-workspace: The name of workspace where /jcr:system node is placed.

security-domain: The name of a security domain for JAAS authentication.

access-control: The name of an access control policy. There can be 3 types: optional - ACL is created on-demand(default), disable - no access control, mandatory - an ACL is created for each added node(not supported yet).

authentication-policy: The name of an authentication policy class.

workspaces: The list of workspaces.

session-max-age: The time after which an idle session will be removed (called logout). If session-max-age is not set up, idle session will never be removed.

lock-remover-max-threads: Number of threads that can serve LockRemover tasks. Default value is 1. Repository may have many workspaces, each workspace have own LockManager. JCR supports Locks with defined lifetime. Such a lock must be removed is it become expired. That is what LockRemovers does. But LockRemovers is not an independent timer-threads, its a task that executed each 30 seconds. Such a task is served by ThreadPoolExecutor which may use different number of threads.

name: The name of a workspace

container: Workspace data container (physical storage) configuration.

initializer: Workspace initializer configuration.

cache: Workspace storage cache configuration.

query-handler: Query handler configuration.

auto-init-permissions: DEPRECATED in JCR 1.9 (use initializer). Default permissions of the root node. It is defined as a set of semicolon-delimited permissions containing a group of space-delimited identities (user, group, etc, see Organization service documentation for details) and the type of permission. For example, any read; :/admin read;:/admin add_node; :/admin set_property;:/admin remove means that users from group admin have all permissions and other users have only a 'read' permission.

value-storage: Optional value Storage plugin definition.

class: A value storage plugin class name (attribute).

properties: The list of properties (name-value pairs) for a concrete Value Storage plugin.

filters: The list of filters defining conditions when this plugin is applicable.

class: Initializer implementation class.

properties: The list of properties (name-value pairs). Properties are supported.

root-nodetype: The node type for root node initialization.

root-permissions: Default permissions of the root node. It is defined as a set of semicolon-delimited permissions containing a group of space-delimited identities (user, group etc, see Organization service documentation for details) and the type of permission. For example any read; :/admin read;:/admin add_node; :/admin set_property;:/admin remove means that users from group admin have all permissions and other users have only a 'read' permission.

Configurable initializer adds a capability to override workspace initial startup procedure (used for Clustering).

enabled: If workspace cache is enabled or not.

class: Cache implementation class, optional from 1.9. Default value is. org.exoplatform.services.jcr.impl.dataflow.persistent.LinkedWorkspaceStorageCacheImpl.

Cache can be configured to use concrete implementation of WorkspaceStorageCache interface. JCR core has two implementation to use:

properties: The list of properties (name-value pairs) for Workspace cache.

max-size: Cache maximum size (maxSize prior to v.1.9).

live-time: Cached item live time (liveTime prior to v.1.9).

From 1.9 LinkedWorkspaceStorageCacheImpl supports additional optional parameters.

statistic-period: Period (time format) of cache statistic thread execution, 5 minutes by default.

statistic-log: If true cache statistic will be printed to default logger (log.info), false by default or not.

statistic-clean: If true cache statistic will be cleaned after was gathered, false by default or not.

cleaner-period: Period of the eldest items remover execution, 20 minutes by default.

blocking-users-count: Number of concurrent users allowed to read cache storage, 0 - unlimited by default.

class: A Query Handler class name.

properties: The list of properties (name-value pairs) for a Query Handler (indexDir).

Properties and advanced features described in Search Configuration.

time-out: Time after which the unused global lock will be removed.

persister: A class for storing lock information for future use. For example, remove lock after jcr restart.

path: A lock folder. Each workspace has its own one.

<!ELEMENT repository-service (repositories)>
<!ATTLIST repository-service default-repository NMTOKEN #REQUIRED>
<!ELEMENT repositories (repository)>
<!ELEMENT repository (security-domain,access-control,session-max-age,authentication-policy,workspaces)>
<!ATTLIST repository
  default-workspace NMTOKEN #REQUIRED
  name NMTOKEN #REQUIRED
  system-workspace NMTOKEN #REQUIRED
>
<!ELEMENT security-domain (#PCDATA)>
<!ELEMENT access-control (#PCDATA)>
<!ELEMENT session-max-age (#PCDATA)>
<!ELEMENT authentication-policy (#PCDATA)>
<!ELEMENT workspaces (workspace+)>
<!ELEMENT workspace (container,initializer,cache,query-handler)>
<!ATTLIST workspace name NMTOKEN #REQUIRED>
<!ELEMENT container (properties,value-storages)>
<!ATTLIST container class NMTOKEN #REQUIRED>
<!ELEMENT value-storages (value-storage+)>
<!ELEMENT value-storage (properties,filters)>
<!ATTLIST value-storage class NMTOKEN #REQUIRED>
<!ELEMENT filters (filter+)>
<!ELEMENT filter EMPTY>
<!ATTLIST filter property-type NMTOKEN #REQUIRED>
<!ELEMENT initializer (properties)>
<!ATTLIST initializer class NMTOKEN #REQUIRED>
<!ELEMENT cache (properties)>
<!ATTLIST cache 
  enabled NMTOKEN #REQUIRED
  class NMTOKEN #REQUIRED
>
<!ELEMENT query-handler (properties)>
<!ATTLIST query-handler class NMTOKEN #REQUIRED>
<!ELEMENT access-manager (properties)>
<!ATTLIST access-manager class NMTOKEN #REQUIRED>
<!ELEMENT lock-manager (time-out,persister)>
<!ELEMENT time-out (#PCDATA)>
<!ELEMENT persister (properties)>
<!ELEMENT properties (property+)>
<!ELEMENT property EMPTY>

Products that use eXo JCR, sometimes missuse it since they continue to use a session that has been closed through a method call on a node, a property or even the session itself. To prevent bad practices we propose three modes which are the folllowing:

Since usage of closed session affects usage of closed datasource we propose three ways to resolve such kind of isses:

The effective configuration of all the repositories and their workspaces can be known thanks to the method getConfigurationXML() that is exposed through JMX at the RepositoryServiceConfiguration level in case of a PortalContainer the name of the related MBean will be of type exo:portal=${portal-container-name},service=RepositoryServiceConfiguration. This method will give you the effective configuration in XML format that has been really interpreted by the the JCR core. This could be helpful to understand how your repositories/workspaces are configured especially if you would like to overwrite the configuration for some reasons.

You can configure values of properties defined in the file repository-configuration.xml using System Properties. This is quite helpful especially when you want to change the default configuration of all the workspaces for example if we want to disable the rdms indexing for all the workspace without this kind of improvement it is very error prone. For all components that can be configured thanks to properties such as container, value-storage, workspace-initializer, cache, query-handler, lock-manager, access-manager and persister the logic for example for the component 'container' and the property called 'foo' will be the following:

To turn on this feature you need to define a component called SystemParametersPersistenceConfigurator. A simple example:

  <component>
    <key>org.exoplatform.services.jcr.config.SystemParametersPersistenceConfigurator</key>
    <type>org.exoplatform.services.jcr.config.SystemParametersPersistenceConfigurator</type>
    <init-params>
      <value-param>
        <name>file-path</name>
        <value>target/temp</value>
      </value-param>
      <values-param>
        <name>unmodifiable</name>
        <value>cache.test-parameter-I</value>
      </values-param>
      <values-param>
        <name>before-initialize</name>
        <value>value-storage.enabled</value>
      </values-param>
    </init-params>
  </component>

To make the configuration process easier here you can define thee parameters.

The parameter in the list have the following format: {component-name}.{parameter-name}. This takes affect for every workspace component called {component-name}.

Please take into account that if this component is not defined in the configuration, the workspace configuration overriding using system properties mechanism will be disabled. In other words: if you don't configure SystemParametersPersistenceConfigurator, the system properties are ignored.

In order to run multilanguage JCR on an Oracle backend Unicode encoding for characters set should be applied to the database. Other Oracle globalization parameters don't make any impact. The only property to modify is NLS_CHARACTERSET.

We have tested NLS_CHARACTERSET = AL32UTF8 and it works well for many European and Asian languages.

Example of database configuration (used for JCR testing):

NLS_LANGUAGE             AMERICAN
NLS_TERRITORY            AMERICA
NLS_CURRENCY             $
NLS_ISO_CURRENCY         AMERICA
NLS_NUMERIC_CHARACTERS   .,
NLS_CHARACTERSET         AL32UTF8
NLS_CALENDAR             GREGORIAN
NLS_DATE_FORMAT          DD-MON-RR
NLS_DATE_LANGUAGE        AMERICAN
NLS_SORT                 BINARY
NLS_TIME_FORMAT          HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT     DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT       HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT  DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY        $
NLS_COMP                 BINARY
NLS_LENGTH_SEMANTICS     BYTE
NLS_NCHAR_CONV_EXCP      FALSE
NLS_NCHAR_CHARACTERSET   AL16UTF16

Create database with Unicode encoding and use Oracle dialect for the Workspace Container:

<workspace name="collaboration">
          <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
            <properties>
              <property name="source-name" value="jdbcjcr" />
              <property name="dialect" value="oracle" />
              <property name="multi-db" value="false" />
              <property name="max-buffer-size" value="200k" />
              <property name="swap-directory" value="target/temp/swap/ws" />
            </properties>
          .....

DB2 Universal Database (DB2 UDB) supports UTF-8 and UTF-16/UCS-2. When a Unicode database is created, CHAR, VARCHAR, LONG VARCHAR data are stored in UTF-8 form. It's enough for JCR multi-lingual support.

Example of UTF-8 database creation:

DB2 CREATE DATABASE dbname USING CODESET UTF-8 TERRITORY US

Create database with UTF-8 encoding and use db2 dialect for Workspace Container on DB2 v.9 and higher:

<workspace name="collaboration">
          <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
            <properties>
              <property name="source-name" value="jdbcjcr" />
              <property name="dialect" value="db2" />
              <property name="multi-db" value="false" />
              <property name="max-buffer-size" value="200k" />
              <property name="swap-directory" value="target/temp/swap/ws" />
            </properties>
          .....

JCR MySQL-backend requires special dialect MySQL-UTF8 to be used for internationalization support. But the database default charset should be latin1 to use limited index space effectively (1000 bytes for MyISAM engine, 767 for InnoDB). If database default charset is multibyte, a JCR database initialization error is thrown concerning index creation failure. In other words, JCR can work on any singlebyte default charset of database, with UTF8 supported by MySQL server. But we have tested it only on latin1 database default charset.

Repository configuration, workspace container entry example:

<workspace name="collaboration">
          <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
            <properties>
              <property name="source-name" value="jdbcjcr" />
              <property name="dialect" value="mysql-utf8" />
              <property name="multi-db" value="false" />
              <property name="max-buffer-size" value="200k" />
              <property name="swap-directory" value="target/temp/swap/ws" />
            </properties>
          .....

You will need also to indicate the charset name either at server level using the server parameter --character-set-server (find more details there ) or at datasource configuration level by adding a new property as below:

          <property name="connectionProperties" value="useUnicode=yes;characterEncoding=utf8;characterSetResults=UTF-8;" />
     

On PostgreSQL/PostgrePlus-backend, multilingual support can be enabled in different ways:

Create database with UTF-8 encoding and use a PgSQL dialect for Workspace Container:

<workspace name="collaboration">
          <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
            <properties>
              <property name="source-name" value="jdbcjcr" />
              <property name="dialect" value="pgsql" />
              <property name="multi-db" value="false" />
              <property name="max-buffer-size" value="200k" />
              <property name="swap-directory" value="target/temp/swap/ws" />
            </properties>
          .....

To enable this feature you need to replace org.jboss.cache.loader.JDBCCacheLoader with org.exoplatform.services.jcr.impl.core.lock.jbosscache.JDBCCacheLoader in JBossCache configuration file.

Here is an example of this very part of the configuration:

<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.1">

   <locking useLockStriping="false" concurrencyLevel="500" lockParentForChildInsertRemove="false"
      lockAcquisitionTimeout="20000" />

   <clustering mode="replication" clusterName="${jbosscache-cluster-name}">
      <stateRetrieval timeout="20000" fetchInMemoryState="false" />
      <sync />
   </clustering>

   <loaders passivation="false" shared="true">
      <!-- All the data of the JCR locks needs to be loaded at startup -->
      <preload>
         <node fqn="/" />
      </preload>  
      <!--
      For another cache-loader class you should use another template with
      cache-loader specific parameters
      -->
      <loader class="org.exoplatform.services.jcr.impl.core.lock.jbosscache.JDBCCacheLoader" async="false" fetchPersistentState="false"
         ignoreModifications="false" purgeOnStartup="false">
         <properties>
            cache.jdbc.table.name=${jbosscache-cl-cache.jdbc.table.name}
            cache.jdbc.table.create=${jbosscache-cl-cache.jdbc.table.create}
            cache.jdbc.table.drop=${jbosscache-cl-cache.jdbc.table.drop}
            cache.jdbc.table.primarykey=${jbosscache-cl-cache.jdbc.table.primarykey}
            cache.jdbc.fqn.column=${jbosscache-cl-cache.jdbc.fqn.column}
            cache.jdbc.fqn.type=${jbosscache-cl-cache.jdbc.fqn.type}
            cache.jdbc.node.column=${jbosscache-cl-cache.jdbc.node.column}
            cache.jdbc.node.type=${jbosscache-cl-cache.jdbc.node.type}
            cache.jdbc.parent.column=${jbosscache-cl-cache.jdbc.parent.column}
            cache.jdbc.datasource=${jbosscache-cl-cache.jdbc.datasource}
         </properties>
      </loader>
   </loaders>
</jbosscache>

You can also obtain file example from GitHub.

If you use HibernateService for JDBC connections management you will need to specify explicitly the default schema by setting "hibernate.default_schema" property in the configuration of HibernateService.

Here is an example:

<component>
    <key>org.exoplatform.services.database.HibernateService</key>
    <jmx-name>database:type=HibernateService</jmx-name>
    <type>org.exoplatform.services.database.impl.HibernateServiceImpl</type>
    <init-params>
      <properties-param>
        <name>hibernate.properties</name>
        <description>Default Hibernate Service</description>
        ...........
        <property name="hibernate.default_schema" value="${gatein.idm.datasource.schema:}"/>
      </properties-param>
    </init-params>
</component>

JCR index configuration. You can find this file here: .../portal/WEB-INF/conf/jcr/repository-configuration.xml

<repository-service default-repository="db1">
  <repositories>
    <repository name="db1" system-workspace="ws" default-workspace="ws">
       ....
      <workspaces>
        <workspace name="ws">
       ....
          <query-handler class="org.exoplatform.services.jcr.impl.core.query.lucene.SearchIndex">
            <properties>
              <property name="index-dir" value="${java.io.tmpdir}/temp/index/db1/ws" />
              <property name="synonymprovider-class" value="org.exoplatform.services.jcr.impl.core.query.lucene.PropertiesSynonymProvider" />
              <property name="synonymprovider-config-path" value="/synonyms.properties" />
              <property name="indexing-configuration-path" value="/indexing-configuration.xml" />
              <property name="query-class" value="org.exoplatform.services.jcr.impl.core.query.QueryImpl" />
            </properties>
          </query-handler>
        ... 
        </workspace>
     </workspaces>
    </repository>        
  </repositories>
</repository-service>

JCR Repository Service uses org.exoplatform.services.jcr.config.RepositoryServiceConfiguration component to read its configuration.

<component>
    <key>org.exoplatform.services.jcr.config.RepositoryServiceConfiguration</key>
    <type>org.exoplatform.services.jcr.impl.config.RepositoryServiceConfigurationImpl</type>
    <init-params>
      <value-param>
        <name>conf-path</name>
        <description>JCR configuration file</description>
        <value>/conf/standalone/exo-jcr-config.xml</value>
      </value-param>
    </init-params>
  </component>

In the example, Repository Service will read the configuration from the file /conf/standalone/exo-jcr-config.xml.

But in some cases, it's required to change the configuration on the fly. And know that the new one will be used. Additionally we wish not to modify the original file.

In this case, we have to use the configuration persister feature which allows to store the configuration in different locations.

On startup RepositoryServiceConfiguration component checks if a configuration persister was configured. In that case, it uses the provided ConfigurationPersister implementation class to instantiate the persister object.

Configuration with persister:

<component>
    <key>org.exoplatform.services.jcr.config.RepositoryServiceConfiguration</key>
    <type>org.exoplatform.services.jcr.impl.config.RepositoryServiceConfigurationImpl</type>
    <init-params>
      <value-param>
        <name>conf-path</name>
        <description>JCR configuration file</description>
        <value>/conf/standalone/exo-jcr-config.xml</value>
      </value-param>
      <properties-param>
        <name>working-conf</name>
        <description>working-conf</description>
        <property name="source-name" value="jdbcjcr" />
        <property name="dialect" value="mysql" />
        <property name="persister-class-name" value="org.exoplatform.services.jcr.impl.config.JDBCConfigurationPersister" />
      </properties-param>
    </init-params>
  </component>
  

Where:

ConfigurationPersister interface:

/**
   * Init persister.
   * Used by RepositoryServiceConfiguration on init. 
   * @return - config data stream
   */
  void init(PropertiesParam params) throws RepositoryConfigurationException;
  
  /**
   * Read config data.
   * @return - config data stream
   */
  InputStream read() throws RepositoryConfigurationException;
  
  /**
   * Create table, write data.
   * @param confData - config data stream
   */
  void write(InputStream confData) throws RepositoryConfigurationException;
  
  /**
   * Tell if the config exists.
   * @return - flag
   */
  boolean hasConfig() throws RepositoryConfigurationException;
  

JCR Core implementation contains a persister which stores the repository configuration in the relational database using JDBC calls - org.exoplatform.services.jcr.impl.config.JDBCConfigurationPersister.

The implementation will crate and use table JCR_CONFIG in the provided database.

But the developer can implement his own persister for his particular usecase.

Isolated-database configuration allows to configure single database for repository but separate database tables for each workspace. First step is to configure the data container in the org.exoplatform.services.naming.InitialContextInitializer service. It's the JNDI context initializer, which registers (binds) naming resources (DataSources) for data containers.

For example:

 <external-component-plugins>
    <target-component>org.exoplatform.services.naming.InitialContextInitializer</target-component>
    <component-plugin>
      <name>bind.datasource</name>
      <set-method>addPlugin</set-method>
      <type>org.exoplatform.services.naming.BindReferencePlugin</type>
      <init-params>
        <value-param>
          <name>bind-name</name>
          <value>jdbcjcr</value>
        </value-param>
        <value-param>
          <name>class-name</name>
          <value>javax.sql.DataSource</value>
        </value-param>
        <value-param>
          <name>factory</name>
          <value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
        </value-param>
          <properties-param>
            <name>ref-addresses</name>
            <description>ref-addresses</description>
            <property name="driverClassName" value="org.postgresql.Driver"/>
            <property name="url" value="jdbc:postgresql://exoua.dnsalias.net/portal"/>
            <property name="username" value="exoadmin"/>
            <property name="password" value="exo12321"/>
          </properties-param>
      </init-params>
    </component-plugin>
  </external-component-plugins>

We configure the database connection parameters:

When the data container configuration is done, we can configure the repository service. Each workspace will be configured for the same data container.

For example:

<workspaces>
   <workspace name="ws">
      <!-- for system storage -->
      <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
         <properties>
            <property name="source-name" value="jdbcjcr" />
            <property name="db-structure-type" value="isolated" />
            ...
         </properties>
         ...
      </container>
      ...
   </workspace>

   <workspace name="ws1">
      <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
         <properties>
            <property name="source-name" value="jdbcjcr" />
            <property name="db-structure-type" value="isolated" />
            ...
         </properties>
         ...
      </container>
      ...
   </workspace>
</workspaces>

In this way, we have configured two workspace which will be persisted in different database tables.

You need to configure each workspace in a repository. You may have each one on different remote servers as far as you need.

First of all configure the data containers in the org.exoplatform.services.naming.InitialContextInitializer service. It's the JNDI context initializer which registers (binds) naming resources (DataSources) for data containers.

For example:

<component>
   <key>org.exoplatform.services.naming.InitialContextInitializer</key>
   <type>org.exoplatform.services.naming.InitialContextInitializer</type>
   <component-plugins>
      <component-plugin>
         <name>bind.datasource</name>
         <set-method>addPlugin</set-method>
         <type>org.exoplatform.services.naming.BindReferencePlugin</type>
         <init-params>
            <value-param>
               <name>bind-name</name>
               <value>jdbcjcr</value>
            </value-param>
            <value-param>
               <name>class-name</name>
               <value>javax.sql.DataSource</value>
            </value-param>
            <value-param>
               <name>factory</name>
               <value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
            </value-param>
            <properties-param>
               <name>ref-addresses</name>
               <description>ref-addresses</description>
               <property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
               <property name="url" value="jdbc:hsqldb:file:target/temp/data/portal"/>
               <property name="username" value="sa"/>
               <property name="password" value=""/>
            </properties-param>
         </init-params>
      </component-plugin>
      <component-plugin>
         <name>bind.datasource</name>
         <set-method>addPlugin</set-method>
         <type>org.exoplatform.services.naming.BindReferencePlugin</type>
         <init-params>
            <value-param>
               <name>bind-name</name>
               <value>jdbcjcr1</value>
            </value-param>
            <value-param>
               <name>class-name</name>
               <value>javax.sql.DataSource</value>
            </value-param>
            <value-param>
               <name>factory</name>
               <value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
            </value-param>
            <properties-param>
               <name>ref-addresses</name>
               <description>ref-addresses</description>
               <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
               <property name="url" value="jdbc:mysql://exoua.dnsalias.net/jcr"/>
               <property name="username" value="exoadmin"/>
               <property name="password" value="exo12321"/>
               <property name="maxActive" value="50"/>
               <property name="maxIdle" value="5"/>
               <property name="initialSize" value="5"/>
            </properties-param>
         </init-params>
      </component-plugin>
   <component-plugins>
</component>
                    

When the data container configuration is done, we can configure the repository service. Each workspace will be configured for its own data container.

For example:

<workspaces>
   <workspace name="ws">
      <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
         <properties>
            <property name="source-name" value="jdbcjcr"/>
            <property name="db-structure-type" value="multi"/>
            ...
         </properties>
      </container>
      ...
   </workspace>

   <workspace name="ws1">
      <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
         <properties>
            <property name="source-name" value="jdbcjcr1"/>
            <property name="db-structure-type" value="multi"/>
            ...
         </properties>
      </container>
      ...
   </workspace>
</workspaces>                                     

In this way, we have configured two workspace which will be persisted in two different databases (ws in HSQLDB, ws1 in MySQL).

It's simplier to configure a single-database data container. We have to configure one naming resource.

For example:

<external-component-plugins>
    <target-component>org.exoplatform.services.naming.InitialContextInitializer</target-component>
    <component-plugin>
        <name>bind.datasource</name>
        <set-method>addPlugin</set-method>
        <type>org.exoplatform.services.naming.BindReferencePlugin</type>
        <init-params>
          <value-param>
            <name>bind-name</name>
            <value>jdbcjcr</value>
          </value-param>
          <value-param>
            <name>class-name</name>
            <value>javax.sql.DataSource</value>
          </value-param>
          <value-param>
            <name>factory</name>
            <value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
          </value-param>
          <properties-param>
            <name>ref-addresses</name>
            <description>ref-addresses</description>
            <property name="driverClassName" value="org.postgresql.Driver"/>
            <property name="url" value="jdbc:postgresql://exoua.dnsalias.net/portal"/>
            <property name="username" value="exoadmin"/>
            <property name="password" value="exo12321"/>
            <property name="maxActive" value="50"/>
            <property name="maxIdle" value="5"/>
            <property name="initialSize" value="5"/>
          </properties-param>
        </init-params>
    </component-plugin>
  </external-component-plugins>
  

And configure repository workspaces in repositories configuration with this one database.

For example:

<workspaces>
  <workspace name="ws">
    <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
      <properties>
        <property name="source-name" value="jdbcjcr"/>
        <property name="db-structure-type" value="single" />
        ...
      </properties>
    </container>
    ...
  </workspace>

  <workspace name="ws1">
    <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
    <properties>
      <property name="source-name" value="jdbcjcr"/>
      <property name="db-structure-type" value="single" />
      ...
    </properties>
    ...
  </workspace>
</workspaces>

In this way, we have configured two workspaces which will be persisted in one database (PostgreSQL).

eXo JCR provides two ways for interact with Database - JDBCStorageConnection that uses simple queries and CQJDBCStorageConection that uses complex queries for reducing amount of database callings.

Simple queries will be used if you chose org.exoplatform.services.jcr.impl.storage.jdbc.JDBCWorkspaceDataContainer:

<workspaces>
  <workspace name="ws">
    <container class="org.exoplatform.services.jcr.impl.storage.jdbc.JDBCWorkspaceDataContainer">
    ...
  </workspace>
</worksapces>

Complex queries will be used if you chose org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer:

<workspaces>
  <workspace name="ws">
    <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
    ...
  </workspace>
</worksapces>

Why we should use a Complex Queries?

Why we should use a Simple Queries?

Some databases supports hints to increase query performance (like Oracle, MySQL, etc). eXo JCR have separate Complex Query implementation for Orcale dialect, that uses query hints to increase performance for few important queries.

To enable this option put next configuration property:

<workspace name="ws">
  <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
    <properties>
      <property name="dialect" value="oracle"/>
      <property name="force.query.hints" value="true" />
      ......

Query hints enabled by default.

eXo JCR uses query hints only for Complex Query Oracle dialect. For all other dialects this parameter is ignored.

The current configuration of eXo JCR uses Apache DBCP connection pool (org.apache.commons.dbcp.BasicDataSourceFactory). It's possible to set a big value for maxActive parameter in configuration.xml. That means usage of lots of TCP/IP ports from a client machine inside the pool (i.e. JDBC driver). As a result, the data container can throw exceptions like "Address already in use". To solve this problem, you have to configure the client's machine networking software for the usage of shorter timeouts for opened TCP/IP ports.

Microsoft Windows has MaxUserPort, TcpTimedWaitDelay registry keys in the node HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters, by default these keys are unset, set each one with values like these:

A sample registry file is below:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"MaxUserPort"=dword:00001b58
"TcpTimedWaitDelay"=dword:0000001e

Holds Values in tree-like FileSystem files. path property points to the root directory to store the files.

This is a recommended type of external storage, it can contain large amount of files limited only by disk/volume free space.

A disadvantage is that it's a higher time on Value deletion due to unused tree-nodes remove.

<value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.fs.TreeFileValueStorage">
     <properties>
       <property name="path" value="data/values"/>
     </properties>
     <filters>
       <filter property-type="Binary" min-value-size="1M"/>
     </filters>

Where :

Each file value storage can have the filter(s) for incoming values. A filter can match values by property type (property-type), property name (property-name), ancestor path (ancestor-path) and/or size of values stored (min-value-size, in bytes). In code sample, we use a filter with property-type and min-value-size only. I.e. storage for binary values with size greater of 1MB. It's recommended to store properties with large values in file value storage only.

Another example shows a value storage with different locations for large files (min-value-size a 20Mb-sized filter). A value storage uses ORed logic in the process of filter selection. That means the first filter in the list will be asked first and if not matched the next will be called etc. Here a value matches the 20 MB-sized filter min-value-size and will be stored in the path "data/20Mvalues", all other in "data/values".

<value-storages>
  <value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.fs.TreeFileValueStorage">
    <properties>
      <property name="path" value="data/20Mvalues"/>
    </properties>
    <filters>
      <filter property-type="Binary" min-value-size="20M"/>
    </filters>
  <value-storage>
  <value-storage id="Storage #2" class="org.exoplatform.services.jcr.impl.storage.value.fs.TreeFileValueStorage">
    <properties>
      <property name="path" value="data/values"/>
    </properties>
    <filters>
      <filter property-type="Binary" min-value-size="1M"/>
    </filters>
  <value-storage>
<value-storages>

Hold Values in flat FileSystem files. path property points to root directory in order to store files

<value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.fs.SimpleFileValueStorage">
     <properties>
       <property name="path" value="data/values"/>
     </properties>
     <filters>
       <filter property-type="Binary" min-value-size="1M"/>
     </filters>

eXo JCR supports Content-addressable storage feature for Values storing.

Content Addressable Value storage stores unique content once. Different properties (values) with same content will be stored as one data file shared between those values. We can tell the Value content will be shared across some Values in storage and will be stored on one physical file.

Storage size will be decreased for application which governs potentially same data in the content.

If property Value changes, it is stored in an additional file. Alternatively the file is shared with other values, pointing to the same content.

The storage calculates Value content address each time the property was changed. CAS write operations are much more expensive compared to the non-CAS storages.

Content address calculation based on java.security.MessageDigest hash computation and tested with MD5 and SHA1 algorithms.

CAS support can be enabled for Tree and Simple File Value Storage types.

To enable CAS support, just configure it in JCR Repositories configuration as we do for other Value Storages.

<workspaces>
        <workspace name="ws">
          <container class="org.exoplatform.services.jcr.impl.storage.jdbc.optimisation.CQJDBCWorkspaceDataContainer">
            <properties>
              <property name="source-name" value="jdbcjcr"/>
              <property name="dialect" value="oracle"/>
              <property name="multi-db" value="false"/>
              <property name="max-buffer-size" value="200k"/>
              <property name="swap-directory" value="target/temp/swap/ws"/>
            </properties>
            <value-storages>
<!------------------- here ----------------------->
              <value-storage id="ws" class="org.exoplatform.services.jcr.impl.storage.value.fs.CASableTreeFileValueStorage">
                <properties>
                  <property name="path" value="target/temp/values/ws"/>
                  <property name="digest-algo" value="MD5"/>
                  <property name="vcas-type" value="org.exoplatform.services.jcr.impl.storage.value.cas.JDBCValueContentAddressStorageImpl"/>
                  <property name="jdbc-source-name" value="jdbcjcr"/>
                  <property name="jdbc-dialect" value="oracle"/>
                </properties>
                <filters>
                  <filter property-type="Binary"/>
                </filters>
              </value-storage>
            </value-storages>

Properties:

JCR allows to disable value storage by adding property into configuration. For interal usage and testing purpose only.

<property name="enabled" value="false" />

Be careful, all stored values will be unaccessible.

Scripts should extend ResourceContainer and should be stored in JCR as a node of type exo:groovyResourceContainer.

Detailed REST services step-by-step implementation check there Create REST service step by step.

Component configuration enables Groovy services loader:

<component>
  <type>org.exoplatform.services.jcr.ext.script.groovy.GroovyScript2RestLoader</type>
  <init-params>
    <object-param>
      <name>observation.config</name>
      <object type="org.exoplatform.services.jcr.ext.script.groovy.GroovyScript2RestLoader$ObservationListenerConfiguration">
        <field name="repository">
          <string>repository</string>
        </field>
        <field name="workspaces">
          <collection type="java.util.ArrayList">
            <value>
              <string>collaboration</string>
            </value>
          </collection>
        </field>
      </object>
    </object-param>
  </init-params>
</component>

Each mentioned components uses instances of JBoss Cache product for caching in clustered environment. So every element has its own transport and has to be configured in a proper way. As usual, workspaces have similar configuration but with different cluster-names and may-be some other parameters. The simplest way to configure them is to define their own configuration files for each component in each workspace:

<property name="jbosscache-configuration" value="conf/standalone/test-jbosscache-lock-db1-ws1.xml" />

But if there are few workspaces, configuring them in such a way can be painful and hard-manageable. eXo JCR offers a template-based configuration for JBoss Cache instances. You can have one template for Lock Manager, one for Indexer and one for data container and use them in all the workspaces, defining the map of substitution parameters in a main configuration file. Just simply define ${jbosscache-<parameter name>} inside xml-template and list correct value in JCR configuration file just below "jbosscache-configuration", as shown:

Template:

...
<clustering mode="replication" clusterName="${jbosscache-cluster-name}">
  <stateRetrieval timeout="20000" fetchInMemoryState="false" />
...

and JCR configuration file:

...
<property name="jbosscache-configuration" value="jar:/conf/portal/jbosscache-lock.xml" />
<property name="jbosscache-cluster-name" value="JCR-cluster-locks-db1-ws" />
...

JGroups is used by JBoss Cache for network communications and transport in a clustered environment. If property "jgroups-configuration" is defined in component configuration, it will be injected into the JBoss Cache instance on startup.

<property name="jgroups-configuration" value="your/path/to/modified-udp.xml" />

As mentioned above, each component (lock manager, data container and query handler) for each workspace requires its own clustered environment. In other words, they have their own clusters with unique names. By default, each cluster should perform multi-casts on a separate port. This configuration leads to much unnecessary overhead on cluster. That's why JGroups offers multiplexer feature, providing ability to use one single channel for set of clusters. This feature reduces network overheads and increase performance and stability of application. To enable multiplexer stack, you should define appropriate configuration file (upd-mux.xml is pre-shipped one with eXo JCR) and set "jgroups-multiplexer-stack" into "true".

<property name="jgroups-configuration" value="jar:/conf/portal/udp-mux.xml" />
<property name="jgroups-multiplexer-stack" value="true" />

It is now highly recommended to use the shared transport instead of the multiplexer, to do so simply disable the multiplexer stack in the configuration of each component then set the property singleton_name of your JGroups configuration to a unique name.

<property name="jgroups-configuration" value="jar:/conf/portal/udp-mux.xml" />
<property name="jgroups-multiplexer-stack" value="false" />

A JBoss Cache instance is quite resource consuming and by default we will have 3 JBoss Cache instances (one instance for the indexer, one for the lock manager and one for the data container) for each workspace, so if you intend to have a lot of workspaces it could make sense to decide to share one JBoss Cache instance with several cache instances of the same type (i.e. indexer, lock manager or data container). This feature is disabled by default and can be enabled at component configuration level (i.e. indexer configuration, lock manager configuration and/or data container configuration) by setting the property "jbosscache-shareable" to true as below:

<property name="jbosscache-shareable" value="true" />

Once enabled this feature will allow the JBoss Cache instance used by the component to be re-used by another components of the same type (i.e. indexer, lock manager or data container) with the exact same JBoss Cache configuration (except the eviction configuration that cans be different), which means that all the parameters of type ${jbosscache-<parameter name>} must be identical between the components of same type of different workspaces. In other words, if we use the same values for the parameters of type ${jbosscache-<parameter name>} in each workspace, we will have only 3 JBoss Cache instances (one instance for the indexer, one for the lock manager and one for the data container) used whatever the total amount of workspaces defined.

In clustered environment, jbosscache-max-invalidations parameter is used by JBossCacheWorkspaceStorageCache to indicate the maximum total amount of invalidation operations that can be done outside the current transaction in auto commit mode to prevent potential deadlocks. This parameter is dedicated to cluster mode only and is needed to ensure that even with big transactions that do a lot of invalidations, we won't have too many JGroups calls which would dramatically affect the scalability and the performance of the whole platform. The value of this parameter must be big enough to prevent deadlocks on operations that add and/or remove concurrently JCR nodes to/from the same parent node but also it should not be too big to avoid affecting too much the scalability and the performance of the whole platform. Default value is 10.

This parameter can be configured at cache configuration as below:

<property name="jbosscache-max-invalidations" value="20" />

eXo JCR implementation is shipped with ready-to-use JBoss Cache configuration templates for JCR's components. They are situated in application package in /conf/porta/ folder.

<?xml version="1.0" encoding="UTF-8"?>
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.1">

   <locking useLockStriping="false" concurrencyLevel="500" lockParentForChildInsertRemove="false"
      lockAcquisitionTimeout="20000" />

   <clustering mode="replication" clusterName="${jbosscache-cluster-name}">
      <stateRetrieval timeout="20000" fetchInMemoryState="false" />
      <sync />
   </clustering>

   <!-- Eviction configuration -->
   <eviction wakeUpInterval="5000">
      <default algorithmClass="org.jboss.cache.eviction.ExpirationAlgorithm"
         actionPolicyClass="org.exoplatform.services.jcr.impl.dataflow.persistent.jbosscache.ParentNodeEvictionActionPolicy"
         eventQueueSize="1000000">
         <property name="maxNodes" value="1000000" />
         <property name="warnNoExpirationKey" value="false" />
      </default>
   </eviction>
</jbosscache>

<?xml version="1.0" encoding="UTF-8"?>
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.1">

   <locking useLockStriping="false" concurrencyLevel="500" lockParentForChildInsertRemove="false"
      lockAcquisitionTimeout="20000" />
   <clustering mode="replication" clusterName="${jbosscache-cluster-name}">
      <stateRetrieval timeout="20000" fetchInMemoryState="false" />
      <sync />
   </clustering>
   <loaders passivation="false" shared="true">
      <preload>
         <node fqn="/" />
      </preload>
      <loader class="org.exoplatform.services.jcr.impl.core.lock.jbosscache.JDBCCacheLoader" async="false" fetchPersistentState="false"
         ignoreModifications="false" purgeOnStartup="false">
         <properties>
            cache.jdbc.table.name=${jbosscache-cl-cache.jdbc.table.name}
            cache.jdbc.table.create=${jbosscache-cl-cache.jdbc.table.create}
            cache.jdbc.table.drop=${jbosscache-cl-cache.jdbc.table.drop}
            cache.jdbc.table.primarykey=${jbosscache-cl-cache.jdbc.table.primarykey}
            cache.jdbc.fqn.column=${jbosscache-cl-cache.jdbc.fqn.column}
            cache.jdbc.fqn.type=${jbosscache-cl-cache.jdbc.fqn.type}
            cache.jdbc.node.column=${jbosscache-cl-cache.jdbc.node.column}
            cache.jdbc.node.type=${jbosscache-cl-cache.jdbc.node.type}
            cache.jdbc.parent.column=${jbosscache-cl-cache.jdbc.parent.column}
            cache.jdbc.datasource=${jbosscache-cl-cache.jdbc.datasource}
         </properties>
      </loader>
   </loaders>
</jbosscache>

<?xml version="1.0" encoding="UTF-8"?>
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.1">
   <locking useLockStriping="false" concurrencyLevel="500" lockParentForChildInsertRemove="false"
      lockAcquisitionTimeout="20000" />
   <clustering mode="replication" clusterName="${jbosscache-cluster-name}">
      <stateRetrieval timeout="20000" fetchInMemoryState="false" />
      <sync />
   </clustering>
</jbosscache>

CacheableLockManagerImpl stores Lock objects in JBoss-cache, so Locks are replicable and affect on cluster, not only a single node. Also, JBoss-cache has JDBCCacheLoader, so Locks will be stored to the database.

You can enable LockManager by adding lock-manager-configuration to workspace-configuration.

For example:

<workspace name="ws">
   ...
   <lock-manager class="org.exoplatform.services.jcr.impl.core.lock.jbosscache.CacheableLockManagerImpl">
      <properties>
         <property name="time-out" value="15m" />
         ...
      </properties>
   </lock-manager>               
   ...
</workspace>

Wher time-out parameter represents interval to remove Expired Locks. LockRemover separates threads, that periodically ask LockManager to remove Locks that live so long.

<?xml version="1.0" encoding="UTF-8"?>
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.1">

   <locking useLockStriping="false" concurrencyLevel="500" lockParentForChildInsertRemove="false"
      lockAcquisitionTimeout="20000" />

   <clustering mode="replication" clusterName="${jbosscache-cluster-name}">
      <stateRetrieval timeout="20000" fetchInMemoryState="false" />
      <sync />
   </clustering>

   <loaders passivation="false" shared="true">
      <!-- All the data of the JCR locks needs to be loaded at startup -->
      <preload>
         <node fqn="/" />
      </preload>  
      <!--
      For another cache-loader class you should use another template with
      cache-loader specific parameters
      ->
      <loader class="org.exoplatform.services.jcr.impl.core.lock.jbosscache.JDBCCacheLoader" async="false" fetchPersistentState="false"
         ignoreModifications="false" purgeOnStartup="false">
         <properties>
            cache.jdbc.table.name=${jbosscache-cl-cache.jdbc.table.name}
            cache.jdbc.table.create=${jbosscache-cl-cache.jdbc.table.create}
            cache.jdbc.table.drop=${jbosscache-cl-cache.jdbc.table.drop}
            cache.jdbc.table.primarykey=${jbosscache-cl-cache.jdbc.table.primarykey}
            cache.jdbc.fqn.column=${jbosscache-cl-cache.jdbc.fqn.column}
            cache.jdbc.fqn.type=${jbosscache-cl-cache.jdbc.fqn.type}
            cache.jdbc.node.column=${jbosscache-cl-cache.jdbc.node.column}
            cache.jdbc.node.type=${jbosscache-cl-cache.jdbc.node.type}
            cache.jdbc.parent.column=${jbosscache-cl-cache.jdbc.parent.column}
            cache.jdbc.datasource=${jbosscache-cl-cache.jdbc.datasource}
         </properties>
      </loader>
   </loaders>
</jbosscache>

<lock-manager class="org.exoplatform.services.jcr.impl.core.lock.jbosscache.CacheableLockManagerImpl">
   <properties>
      <property name="time-out" value="15m" />
      <property name="jbosscache-configuration" value="test-jbosscache-lock.xml" />
      <property name="jgroups-configuration" value="udp-mux.xml" />
      <property name="jgroups-multiplexer-stack" value="true" />
      <property name="jbosscache-cluster-name" value="JCR-cluster-locks-ws" />
      <property name="jbosscache-cl-cache.jdbc.table.name" value="jcrlocks_ws" />
      <property name="jbosscache-cl-cache.jdbc.table.create" value="true" />
      <property name="jbosscache-cl-cache.jdbc.table.drop" value="false" />
      <property name="jbosscache-cl-cache.jdbc.table.primarykey" value="jcrlocks_ws_pk" />
      <property name="jbosscache-cl-cache.jdbc.fqn.column" value="fqn" />
      <property name="jbosscache-cl-cache.jdbc.fqn.type" value="AUTO"/>
      <property name="jbosscache-cl-cache.jdbc.node.column" value="node" />
      <property name="jbosscache-cl-cache.jdbc.node.type" value="AUTO"/>
      <property name="jbosscache-cl-cache.jdbc.parent.column" value="parent" />
      <property name="jbosscache-cl-cache.jdbc.datasource" value="jdbcjcr" />
      <property name="jbosscache-cl-cache.jdbc.dialect" value="${dialect}" />
      <property name="jbosscache-shareable" value="true" />
   </properties>
</lock-manager>

<config>
    <UDP
         singleton_name="JCR-cluster" 
         mcast_addr="${jgroups.udp.mcast_addr:228.10.10.10}"
         mcast_port="${jgroups.udp.mcast_port:45588}"
         tos="8" 
         ucast_recv_buf_size="20000000"
         ucast_send_buf_size="640000" 
         mcast_recv_buf_size="25000000" 
         mcast_send_buf_size="640000" 
         loopback="false"
         discard_incompatible_packets="true" 
         max_bundle_size="64000" 
         max_bundle_timeout="30"
         use_incoming_packet_handler="true" 
         ip_ttl="${jgroups.udp.ip_ttl:2}"
         enable_bundling="false" 
         enable_diagnostics="true"
         thread_naming_pattern="cl" 

         use_concurrent_stack="true" 

         thread_pool.enabled="true" 
         thread_pool.min_threads="2"
         thread_pool.max_threads="8" 
         thread_pool.keep_alive_time="5000" 
         thread_pool.queue_enabled="true"
         thread_pool.queue_max_size="1000"
         thread_pool.rejection_policy="discard"

         oob_thread_pool.enabled="true"
         oob_thread_pool.min_threads="1"
         oob_thread_pool.max_threads="8"
         oob_thread_pool.keep_alive_time="5000"
         oob_thread_pool.queue_enabled="false" 
         oob_thread_pool.queue_max_size="100" 
         oob_thread_pool.rejection_policy="Run" />

    <PING timeout="2000"
            num_initial_members="3"/>
    <MERGE2 max_interval="30000"
            min_interval="10000"/>
   <FD_SOCK />
   <FD timeout="10000" max_tries="5" shun="true" />
   <VERIFY_SUSPECT timeout="1500" />
   <BARRIER />
    <pbcast.NAKACK use_stats_for_retransmission="false"
                   exponential_backoff="150"
                   use_mcast_xmit="true" gc_lag="0"
                   retransmit_timeout="50,300,600,1200"
                   discard_delivered_msgs="true"/>
   <UNICAST timeout="300,600,1200" />
    <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
                   max_bytes="1000000"/>
   <VIEW_SYNC avg_send_interval="60000" />
    <pbcast.GMS print_local_addr="true" join_timeout="3000"
                shun="false"
                view_bundling="true"/>
    <FC max_credits="500000"
                    min_threshold="0.20"/>
   <FRAG2 frag_size="60000" />
   <!--pbcast.STREAMING_STATE_TRANSFER /-->
   <pbcast.STATE_TRANSFER />
   <pbcast.FLUSH />
</config>

JCR offers multiple indexing strategies. They include both for standalone and clustered environments using the advantages of running in a single JVM or doing the best to use all resources available in cluster. JCR uses Lucene library as underlying search and indexing engine, but it has several limitations that greatly reduce possibilities and limits the usage of cluster advantages. That's why eXo JCR offers three strategies that are suitable for it's own usecases. They are standalone, clustered with shared index, clustered with local indexes and RSync-based. Each one has it's pros and cons.

Stanadlone strategy provides a stack of indexes to achieve greater performance within single JVM.

It combines in-memory buffer index directory with delayed file-system flushing. This index is called "Volatile" and it is invoked in searches also. Within some conditions volatile index is flushed to the persistent storage (file system) as new index directory. This allows to achieve great results for write operations.

Clustered implementation with local indexes is built upon same strategy with volatile in-memory index buffer along with delayed flushing on persistent storage.

As this implementation designed for clustered environment it has additional mechanisms for data delivery within cluster. Actual text extraction jobs done on the same node that does content operations (i.e. write operation). Prepared "documents" (Lucene term that means block of data ready for indexing) are replicated withing cluster nodes and processed by local indexes. So each cluster instance has the same index content. When new node joins the cluster it has no initial index, so it must be created. There are some supported ways of doing this operation. The simplest is to simply copy the index manually but this is not intended for use. If no initial index found JCR uses automated sceneries. They are controlled via configuration (see "index-recovery-mode" parameter) offering full re-indexing from database or copying from another cluster node.

For some reasons having a multiple index copies on each instance can be costly. So shared index can be used instead (see diagram below).

This indexing strategy combines advantages of in-memory index along with shared persistent index offering "near" real time search capabilities. This means that newly added content is accessible via search practically immediately. This strategy allows nodes to index data in their own volatile (in-memory) indexes, but persistent indexes are managed by single "coordinator" node only. Each cluster instance has a read access for shared index to perform queries combining search results found in own in-memory index also. Take in account that shared folder must be configured in your system environment (i.e. mounted NFS folder). But this strategy in some extremely rare cases can have a bit different volatile indexes within cluster instances for a while. In a few seconds they will be up2date.

Shared index is consistent and stable enough, but slow, while local index is fast, but requires much time for re-synchronization, when cluster node is leaving a cluster for a small period of time. RSync-based index solves this problem along with local file system advantages in term of speed.

This strategy is the same shared index, but stores actual data on local file system, instead of shared. Eventually triggering a synchronization job, that woks on the level of file blocks, synchronizing only modified data. Diagram shows it in action. Only single node in the cluster is responsible for modifying index files, this is the Coordinator node. When data persisted, corresponding command fired, starting synchronization jobs all over the cluster.

See more about Search Configuration.

Managing a big set of data using JCR in production environment sometimes requires special operations with Indexes, stored on File System. One of those maintenance operations is a recreation of it. Also called "re-indexing". There are various usecases when it's important to do. They include hardware faults, hard restarts, data-corruption, migrations and JCR updates that brings new features related to index. Usually index re-creation requested on server's startup or in runtime.

boolean online = ((QueryManagerImpl)Workspace.getQueryManager()).getQueryHandeler().isOnline();

[INFO] Setting index OFFLINE (repository/production[system]).

[INFO] Created initial index for 143018 nodes (repository/production[system]).
[INFO] Setting index ONLINE (repository/production[system]).

1.25.3.2. Hot Asynchronous Workspace Reindexing via JMX

Some hard system faults, error during upgrades, migration issues and some other factors may corrupt the index. Most likely end customers would like the production systems to fix index issues in run-time, without delays and restarts. Current versions of JCR supports "Hot Asynchronous Workspace Reindexing" feature. It allows end-user (Service Administrator) to launch the process in background without stopping or blocking whole application by using any JMX-compatible console (see screenshot below, "JConsole in action").

Server can continue working as expected while index is recreated. This depends on the flag "allow queries", passed via JMX interface to reindex operation invocation. If the flag set, then application continues working. But there is one critical limitation the end-users must be aware. Index is frozen while background task is running. It meant that queries are performed on index present on the moment of task startup and data written into repository after startup won't be available through the search until process finished. Data added during re-indexation is also indexed, but will be available only when task is done. Briefly, JCR makes the snapshot of indexes on asynch task startup and uses it for searches. When operation finished, stale indexes replaced by newly created including newly added data. If flag "allow queries" is set to false, then all queries will throw an exception while task is running. Current state can be acquired using the following JMX operation:

• getHotReindexingState() - returns information about latest invocation: start time, if in progress or finish time if done.

Example configuration:

  <component>
    <key>org.exoplatform.services.transaction.TransactionService</key>
    <type>org.exoplatform.services.transaction.jbosscache.JBossTransactionsService</type>
    <init-params>
      <value-param>
        <name>timeout</name>
        <value>3000</value>
      </value-param>
    </init-params>   
  </component>

timeout - XA transaction timeout in seconds

<component>
    <key>org.infinispan.transaction.lookup.TransactionManagerLookup</key>
    <type>org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup</type>
</component>

<component profiles="ispn">
    <key>org.exoplatform.services.transaction.TransactionService</key>
    <type>org.exoplatform.services.transaction.infinispan.JBossTransactionsService</type>
    <init-params>
        <value-param>
            <name>timeout</name>
            <value>3000</value>
        </value-param>
    </init-params>
</component>

<component profiles="ispn">
    <key>org.exoplatform.services.rpc.RPCService</key>
    <type>org.exoplatform.services.rpc.jgv3.RPCServiceImpl</type>
    <init-params>
        <value-param>
            <name>jgroups-configuration</name>
            <value>jar:/conf/udp-mux-v3.xml</value>
        </value-param>
        <value-param>
            <name>jgroups-cluster-name</name>
            <value>RPCService-Cluster</value>
        </value-param>
        <value-param>
            <name>jgroups-default-timeout</name>
            <value>0</value>
        </value-param>
    </init-params>
</component>

Each mentioned below components uses instances of Infinispan Cache product for caching in clustered environment. So every element has it's own transport and has to be configured in a proper way. As usual, workspaces have similar configuration. The simplest way to configure them is to define their own configuration files for each component in each workspace. There are several commons parameters.

"infinispan-configuration" defines path to template based configuration for Infinispan Cache instance.

JGroups is used by Infinispan Cache for network communications and transport in a clustered environment. If property "jgroups-configuration" is defined in component configuration, it will be injected into the Infinispan Cache instance on startup.

The another parameter is "infinispan-cluster-name". This defines the name of the cluster. Needs to be the same for all nodes in a cluster in order to find each other.

eXo JCR implementation is shipped with ready-to-use Infinispan Cache configuration templates for JCR's components.

<infinispan 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". 
      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd". 
      xmlns="urn:infinispan:config:5.1">

    <global>
      <evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
        <properties>
          <property name="threadNamePrefix" value="EvictionThread"/>
        </properties>
      </evictionScheduledExecutor>

      <globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/>

      <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout=
        <properties>
          <property name="configurationFile" value="${jgroups-configuration}"/>
        </properties>
      </transport>
    </global>

    <default>
      <clustering mode="replication">
        <stateTransfer timeout="20000" fetchInMemoryState="false" />
        <sync replTimeout="20000"/>
      </clustering>

      <locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="true"/>
      <transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" s
      <jmxStatistics enabled="true"/>
      <eviction strategy="LRU" threadPolicy="DEFAULT" maxEntries="1000000"/>
      <expiration wakeUpInterval="5000"/>
   </default>
</infinispan>

<infinispan 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". 
      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd". 
      xmlns="urn:infinispan:config:5.1">

    <global>
      <evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
        <properties>
          <property name="threadNamePrefix" value="EvictionThread"/>
        </properties>
      </evictionScheduledExecutor>

      <globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/>

      <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout=
        <properties>
          <property name="configurationFile" value="${jgroups-configuration}"/>
        </properties>
      </transport>
    </global>

    <default>
      <clustering mode="replication">
        <stateTransfer timeout="20000" fetchInMemoryState="false" />
        <sync replTimeout="20000"/>
      </clustering>

      <locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="false"/>
      <transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" s
      <jmxStatistics enabled="true"/>
      <eviction strategy="NONE"/>

      <loaders passivation="false" shared="true" preload="true">
        <loader class="org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore" fetchPersistentState="true" ignoreModifications="false" purgeOnStar
          <properties>
             <property name="stringsTableNamePrefix" value="${infinispan-cl-cache.jdbc.table.name}"/>
             <property name="idColumnName" value="${infinispan-cl-cache.jdbc.id.column}"/>
             <property name="dataColumnName" value="${infinispan-cl-cache.jdbc.data.column}"/>
             <property name="timestampColumnName" value="${infinispan-cl-cache.jdbc.timestamp.column}"/>
             <property name="idColumnType" value="${infinispan-cl-cache.jdbc.id.type}"/>
             <property name="dataColumnType" value="${infinispan-cl-cache.jdbc.data.type}"/>
             <property name="timestampColumnType" value="${infinispan-cl-cache.jdbc.timestamp.type}"/>
             <property name="dropTableOnExit" value="${infinispan-cl-cache.jdbc.table.drop}"/>
             <property name="createTableOnStart" value="${infinispan-cl-cache.jdbc.table.create}"/>
             <property name="connectionFactoryClass" value="${infinispan-cl-cache.jdbc.connectionFactory}"/>
             <property name="datasourceJndiLocation" value="${infinispan-cl-cache.jdbc.datasource}"/>
          </properties>
          <async enabled="false"/>
        </loader>
      </loaders>
   </default>

</infinispan>

<infinispan 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance". 
      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd". 
      xmlns="urn:infinispan:config:5.1">

    <global>
      <evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
        <properties>
          <property name="threadNamePrefix" value="EvictionThread"/>
        </properties>
      </evictionScheduledExecutor>

      <globalJmxStatistics jmxDomain="exo" enabled="true" allowDuplicateDomains="true"/>

      <transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="${infinispan-cluster-name}" distributedSyncTimeout=
        <properties>
          <property name="configurationFile" value="${jgroups-configuration}"/>
        </properties>
      </transport>
    </global>

    <default>
      <clustering mode="replication">
        <stateTransfer timeout="20000" fetchInMemoryState="false" />
        <sync replTimeout="20000"/>
      </clustering>

      <locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="false"/>
      <transaction transactionManagerLookupClass="org.exoplatform.services.transaction.infinispan.JBossStandaloneJTAManagerLookup" syncRollbackPhase="true" s
      <jmxStatistics enabled="true"/>
      <eviction strategy="NONE"/>

      <loaders passivation="false" shared="false" preload="false">
        <loader class="${infinispan-cachestore-classname}" fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false">
          <async enabled="false"/>
        </loader>
      </loaders>
   </default>
</infinispan>

RepositoryConfigurationService depends to next components:

There is two ways to create repositry: make it in single step - just call createRepository(String backupId, RepositoryEntry); or reserve repositoryName at first (reserveRepositoryName(String repositoryName)), than create reserved repository (createRepository(String backupId, RepositoryEntry rEntry, String token)).

RepositoryCreationService configuration

<component>
   <key>org.exoplatform.services.jcr.ext.backup.BackupManager</key>
   <type>org.exoplatform.services.jcr.ext.backup.impl.BackupManagerImpl</type>
   <init-params>
      <properties-param>
         <name>backup-properties</name>
         <property name="backup-dir" value="target/backup" />
      </properties-param>
   </init-params>
</component>

<component>
   <key>org.exoplatform.services.database.creator.DBCreator</key>
   <type>org.exoplatform.services.database.creator.DBCreator</type>
   <init-params>
      <properties-param>
         <name>db-connection</name>
         <description>database connection properties</description>
         <property name="driverClassName" value="org.hsqldb.jdbcDriver" />
         <property name="url" value="jdbc:hsqldb:file:target/temp/data/" />
         <property name="username" value="sa" />
         <property name="password" value="" />
      </properties-param>
      <properties-param>
         <name>db-creation</name>
         <description>database creation properties</description>
         <property name="scriptPath" value="src/test/resources/test.sql" />
         <property name="username" value="sa" />
         <property name="password" value="" />
      </properties-param>
   </init-params>
</component>

<component>
   <key>org.exoplatform.services.rpc.RPCService</key>
   <type>org.exoplatform.services.rpc.impl.RPCServiceImpl</type>
   <init-params>
      <value-param>
         <name>jgroups-configuration</name>
         <value>jar:/conf/standalone/udp-mux.xml</value>
      </value-param>
      <value-param>
         <name>jgroups-cluster-name</name>
         <value>RPCService-Cluster</value>
      </value-param>
      <value-param>
         <name>jgroups-default-timeout</name>
         <value>0</value>
      </value-param>
   </init-params>
</component>  

<component>
   <key>org.exoplatform.services.jcr.ext.repository.creation.RepositoryCreationService</key>
   <type>
      org.exoplatform.services.jcr.ext.repository.creation.RepositoryCreationServiceImpl
   </type>
     <init-params> 
         <value-param> 
            <name>factory-class-name</name> 
            <value>org.apache.commons.dbcp.BasicDataSourceFactory</value> 
         </value-param> 
      </init-params>
</component>

public interface RepositoryCreationService
{
   /**
    * Reserves, validates and creates repository in a simplified form.
    * 
    * @param rEntry - repository Entry - note that datasource must not exist.
    * @param backupId - backup id
    * @param creationProps - storage creation properties 
    * @throws RepositoryConfigurationException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    * @throws RepositoryCreationServiceException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    */
   void createRepository(String backupId, RepositoryEntry rEntry, StorageCreationProperties creationProps)
      throws RepositoryConfigurationException, RepositoryCreationException;

   /**
    * Reserves, validates and creates repository in a simplified form. 
    * 
    * @param rEntry - repository Entry - note that datasource must not exist.
    * @param backupId - backup id
    * @throws RepositoryConfigurationException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    * @throws RepositoryCreationServiceException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    */
   void createRepository(String backupId, RepositoryEntry rEntry) throws RepositoryConfigurationException,
      RepositoryCreationException;

   /**
    * Reserve repository name to prevent repository creation with same name from other place in same time
    * via this service.
    * 
    * @param repositoryName - repositoryName
    * @return repository token. Anyone obtaining a token can later create a repository of reserved name.
    * @throws RepositoryCreationServiceException if can't reserve name
    */
   String reserveRepositoryName(String repositoryName) throws RepositoryCreationException;

   /**
    * Creates repository, using token of already reserved repository name. 
    * Good for cases, when repository creation should be delayed or made asynchronously in dedicated thread. 
    * 
    * @param rEntry - repository entry - note, that datasource must not exist
    * @param backupId - backup id
    * @param rToken - token
    * @param creationProps - storage creation properties
    * @throws RepositoryConfigurationException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    * @throws RepositoryCreationServiceException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    */
   void createRepository(String backupId, RepositoryEntry rEntry, String rToken, StorageCreationProperties creationProps)
      throws RepositoryConfigurationException, RepositoryCreationException;

   /**
    * Creates  repository, using token of already reserved repository name. Good for cases, when repository creation should be delayed or 
    * made asynchronously in dedicated thread. 
    * 
    * @param rEntry - repository entry - note, that datasource must not exist
    * @param backupId - backup id
    * @param rToken - token
    * @throws RepositoryConfigurationException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    * @throws RepositoryCreationServiceException
    *          if some exception occurred during repository creation or repository name is absent in reserved list
    */
   void createRepository(String backupId, RepositoryEntry rEntry, String rToken)
      throws RepositoryConfigurationException, RepositoryCreationException;

   /**
    * Remove previously created repository. 
    * 
    * @param repositoryName - the repository name to delete
    * @param forceRemove - force close all opened sessions 
    * @throws RepositoryCreationServiceException
    *          if some exception occurred during repository removing occurred
    */
   void removeRepository(String repositoryName, boolean forceRemove) throws RepositoryCreationException;

}