Hibernate.orgCommunity Documentation

Hibernate OGM Reference Guide

Logo


Hibernate Object/Grid Mapper (OGM) is a persistence engine providing Java Persistence (JPA) support for NoSQL datastores. It reuses Hibernate ORM’s object life cycle management and (de)hydration engine but persists entities into a NoSQL store (key/value, document, column-oriented, etc) instead of a relational database. It reuses the Java Persistence Query Language (JP-QL) as an interface to querying stored data.

The project is still very young and very ambitious at the same time. Many things are on the roadmap (more NoSQL, query, denormalization engine, etc). If you wish to help, please check Chapter 1, How to get help and contribute on Hibernate OGM.

Hibernate OGM is released under the LGPL open source license.

Warning

This documentation and this project are work in progress. Please give us feedback on

  • what you like
  • what you don’t like
  • what is confusing

Check Section 1.2, “How to contribute” on how to contact us.

Hibernate OGM:

NoSQL can be very disconcerting as it is composed of many disparate solutions with different benefits and drawbacks. Speaking only of the main ones, NoSQL is at least categorized in four families:


Each have different benefits and drawbacks and one solution might fit a use case better than an other. However access patterns and APIs are different from one product to the other.

Hibernate OGM is not expected to be the Rosetta stone used to interact with all NoSQL solution in all use cases. But for people modeling their data as a domain model, it provides distinctive advantages over raw APIs and has the benefit of providing an API and semantic known to Java developers. Reusing the same programmatic model and trying different (No)SQL engines will hopefully help people to explore alternative datastores.

Hibernate OGM also aims at helping people scale traditional relational databases by providing a NoSQL front-end and keeping the same JPA APIs and domain model.

Hibernate OGM is a young project. The code, the direction and the documentation are all in flux and being built by the community. Join and help us shape it!

First of all, make sure to read this reference documentation. This is the most comprehensive formal source of information. Of course, it is not perfect: feel free to come and ask for help, comment or propose improvements in our Hibernate OGM forum.

You can also:

  • open bug reports in JIRA
  • propose improvements on the development mailing list
  • join us on IRC to discuss developments and improvements (#hibernate-dev on freenode.net; you need to be registered on freenode: the room does not accept "anonymous" users).

Welcome!

There are many ways to contribute:

  • report bugs in JIRA
  • give feedback in the forum, IRC or the development mailing list
  • improve the documentation
  • fix bugs or contribute new features
  • propose and code a datastore dialect for your favorite NoSQL engine

Hibernate OGM’s code is available on GitHub at https://github.com/hibernate/hibernate-ogm.

If you are familiar with JPA, you are almost good to go :-) We will nevertheless walk you through the first few steps of persisting and retrieving an entity using Hibernate OGM.

Before we can start, make sure you have the following tools configured:

Hibernate OGM is published in the JBoss hosted Maven repository. Adjust your ~/.m2/settings.xml file according to the guidelines found on this webpage. In this example we will use Infinispan as the targeted datastore.

Add org.hibernate.ogm:hibernate-ogm-infinispan:4.0.0.Beta4 to your project dependencies.


<dependency>
    <groupId>org.hibernate.ogm</groupId>
    <artifactId>hibernate-ogm-infinispan</artifactId>
    <version>4.0.0.Beta4</version>
</dependency>
<dependency>
    <groupId>org.hibernate.javax.persistence</groupId>
    <artifactId>hibernate-jpa-2.0-api</artifactId>
    <version>1.0.1.Final</version>
</dependency>

Note

While Hibernate OGM depends on JPA 2.0, it is marked as provided in the Maven POM file. If you run outside a Java EE container, make sure to explicitly add the dependency.

We will use the JPA APIs in this tutorial.

Let’s now map our first Hibernate OGM entity.

@Entity

public class Dog {
   @Id @GeneratedValue(strategy = GenerationType.TABLE, generator = "dog")
   @TableGenerator(
      name = "dog",
      table = "sequences",
      pkColumnName = "key",
      pkColumnValue = "dog",
      valueColumnName = "seed"
   )
   public Long getId() { return id; }
   public void setId(Long id) { this.id = id; }
   private Long id;
   public String getName() { return name; }
   public void setName(String name) { this.name = name; }
   private String name;
   @ManyToOne
   public Breed getBreed() { return breed; }
   public void setBreed(Breed breed) { this.breed = breed; }
   private Breed breed;
}
@Entity
public class Breed {
   @Id @GeneratedValue(generator = "uuid")
   @GenericGenerator(name="uuid", strategy="uuid2")
   public String getId() { return id; }
   public void setId(String id) { this.id = id; }
   private String id;
   public String getName() { return name; }
   public void setName(String name) { this.name = name; }
   private String name;
}

I lied to you, we have already mapped two entities! If you are familiar with JPA, you can see that there is nothing specific to Hibernate OGM in our mapping.

In this tutorial, we will use JBoss Transactions for our JTA transaction manager. The final list of dependencies should look like this:


<dependencies>
    <!-- Hibernate OGM dependency -->
    <dependency>
        <groupId>org.hibernate.ogm</groupId>
        <artifactId>hibernate-ogm-core</artifactId>
        <version>4.0.0.Beta4</version>
    </dependency>

    <!-- standard APIs dependencies - provided in a Java EE container -->
    <dependency>
        <groupId>org.hibernate.javax.persistence</groupId>
        <artifactId>hibernate-jpa-2.0-api</artifactId>
        <version>1.0.1.Final</version>
    </dependency>
    <dependency>
        <groupId>org.jboss.spec.javax.transaction</groupId>
        <artifactId>jboss-transaction-api_1.1_spec</artifactId>
        <version>1.0.0.Final</version>
        <scope>provided</scope>
    </dependency>

    <!-- JBoss Transactions dependency -->
    <dependency>
        <groupId>org.jboss.jbossts</groupId>
        <artifactId>jbossjta</artifactId>
        <version>4.16.4.Final</version>
    </dependency>
</dependencies>

Next we need to define the persistence unit. Create a META-INF/persistence.xml file.


<?xml version="1.0"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
             version="2.0">

    <persistence-unit name="ogm-jpa-tutorial" transaction-type="JTA">
        <!-- Use Hibernate OGM provider: configuration will be transparent -->
        <provider>org.hibernate.ogm.jpa.HibernateOgmPersistence</provider>
        <properties>
            <!-- property optional if you plan and use Infinispan, otherwise adjust to your favorite
                NoSQL Datastore provider.
            <property name="hibernate.ogm.datastore.provider"
                      value="org.hibernate.ogm.datastore.infinispan.impl.InfinispanDatastoreProvider"/>
            -->
            <!-- defines which JTA Transaction we plan to use -->
            <property name="hibernate.transaction.jta.platform"
                      value="org.hibernate.service.jta.platform.internal.JBossStandAloneJtaPlatform"/>
        </properties>
    </persistence-unit>
</persistence>

Let’s now persist a set of entities and retrieve them.

//accessing JBoss's Transaction can be done differently but this one works nicely

TransactionManager tm = getTransactionManager();
//build the EntityManagerFactory as you would build in in Hibernate ORM
EntityManagerFactory emf = Persistence.createEntityManagerFactory(
    "ogm-jpa-tutorial");
final Logger logger = LoggerFactory.getLogger(DogBreedRunner.class);
[..]
//Persist entities the way you are used to in plain JPA
tm.begin();
logger.infof("About to store dog and breed");
EntityManager em = emf.createEntityManager();
Breed collie = new Breed();
collie.setName("Collie");
em.persist(collie);
Dog dina = new Dog();
dina.setName("Dina");
dina.setBreed(collie);
em.persist(dina);
Long dinaId = dina.getId();
em.flush();
em.close();
tm.commit();
[..]
//Retrieve your entities the way you are used to in plain JPA
tm.begin();
logger.infof("About to retrieve dog and breed");
em = emf.createEntityManager();
dina = em.find(Dog.class, dinaId);
logger.infof("Found dog %s of breed %s", dina.getName(), dina.getBreed().getName());
em.flush();
em.close();
tm.commit();
[..]
emf.close();
public static TransactionManager getTransactionManager() throws Exception
    Class<?> tmClass = Main.class.getClassLoader().loadClass(JBOSS_TM_CLASS_NAME);
    return (TransactionManager) tmClass.getMethod("transactionManager").invoke(null);
}

Note

Some JVM do not handle mixed IPv4/IPv6 stacks properly (older Mac OS X JDK in particular), if you experience trouble starting the Infinispan cluster, pass the following property: -Djava.net.preferIPv4Stack=true to your JVM or upgrade to a recent JDK version. jdk7u6 (b22) is known to work on Max OS X.

A working example can be found in Hibernate OGM’s distribution under hibernate-ogm-documentation/examples/gettingstarted.

What have we seen?

  • Hibernate OGM is a JPA implementation and is used as such both for mapping and in API usage
  • It is configured as a specific JPA provider: org.hibernate.ogm.jpa.HibernateOgmPersistence

Let’s explore more in the next chapters.

Note

Hibernate OGM defines an abstraction layer represented by DatastoreProvider and GridDialect to separate the OGM engine from the datastores interaction. It has successfully abstracted various key/value stores and MongoDB. We are working on testing it on other NoSQL families.

In this chapter we will will explore:

  • the general architecture
  • how the data is persisted in the NoSQL datastore
  • how we support JP-QL queries

Let’s start with the general architecture.

Hibernate OGM is really made possible by the reuse of a few key components:


Hibernate OGM reuses as much as possible from the Hibernate ORM infrastructure. There is no need to rewrite an entirely new JPA engine. The Persisters and the Loaders (two interfaces used by Hibernate ORM) have been rewritten to persist data in the NoSQL store. These implementations are the core of Hibernate OGM. We will see in Section 3.2, “How is data persisted” how the data is structured.

The particularities between NoSQL stores are abstracted by the notion of a DatastoreProvider and a GridDialect.

  • DatastoreProvider abstracts how to start and maintain a connection between Hibernate OGM and the datastore.
  • GridDialect abstracts how data itself including association is persisted.

Think of them as the JDBC layer for our NoSQL stores.

Other than these, all the Create/Read/Update/Delete (CRUD) operations are implemented by the Hibernate ORM engine (object hydration and dehydration, cascading, lifecycle etc).

As of today, we have implemented four datastore providers:

  • a Map based datastore provider (for testing)
  • an Infinispan based datastore provider to persist your entities in Infinispan
  • a Ehcache based datastore provider to persist your entities in Ehcache
  • a MongoDB based datastore provider to persist data in a MongoDB database

To implement JP-QL queries, Hibernate OGM parses the JP-QL string and calls the appropriate translator functions to build a native query. If the query is too complex for the native capabilities of the NoSQL store, the Teiid query engine is used as an intermediary engine to implement the missing features (typically joins between entities, aggregation). Finally, if the underlying engine does not have any query support, we use Hibernate Search as an external query engine.

Reality is a bit more nuanced, we will discuss the subject of querying in more details in Section 3.3, “How is data queried”.

Hibernate OGM best works in a JTA environment. The easiest solution is to deploy it on a Java EE container. Alternatively, you can use a standalone JTA TransactionManager. We explain how to in Section 4.2.2, “In a standalone JTA environment”.

Let’s now see how and in which structure data is persisted in the NoSQL data store.

Hibernate OGM tries to reuse as much as possible the relational model concepts, at least when they are practical and make sense in OGM’s case. For very good reasons, the relational model brought peace in the database landscape over 30 years ago. In particular, Hibernate OGM inherits the following traits:

If the application data model is too tightly coupled with your persistent data model, a few issues arise including:

Entities are stored as tuples of values by Hibernate OGM. More specifically, each entity is conceptually represented by a Map<String,Object> where the key represents the column name (often the property name but not always) and the value represents the column value as a basic type. We favor basic types over complex ones to increase portability (across platforms and across type / class schema evolution over time). For example a URL object is stored as its String representation.

The key identifying a given entity instance is composed of:


The GridDialect specific to the NoSQL datastore you target is then responsible to convert this map into the most natural model:

  • for a key/value store or a data grid, we use the logical key as the key in the grid and we store the map as the value. Note that it’s an approximation and some key/value providers will use more tailored approaches.
  • for a document oriented store, the map is represented by a document and each entry in the map corresponds to a property in a document.

Associations are also stored as tuple as well or more specifically as a set of tuples. Hibernate OGM stores the information necessary to navigate from an entity to its associations. This is a departure from the pure relational model but it ensures that association data is reachable via key lookups based on the information contained in the entity tuple we want to navigate from. Note that this leads to some level of duplication as information has to be stored for both sides of the association.

The key in which association data are stored is composed of:

  • the table name
  • the column name(s) representing the foreign key to the entity we come from
  • the column value(s) representing the foreign key to the entity we come from

Using this approach, we favor fast read and (slightly) slower writes.


Note that this approach has benefits and drawbacks:

  • it ensures that all CRUD operations are doable via key lookups
  • it favors reads over writes (for associations)
  • but it duplicates data

Note

We might offer alternative association data persistence options in the future based on feedback.

Again, there are specificities in how data is inherently stored in the specific NoSQL store. For example, in document oriented stores, the association information including the identifier to the associated entities can be stored in the entity owning the association. This is a more natural model for documents.

TODO: this sentence might be worth a diagram to show the difference with the key/value store.

Some identifiers require to store a seed in the datastore (like sequences for examples). The seed is stored in the value whose key is composed of:

  • the table name
  • the column name representing the segment
  • the column value representing the segment

Make sure to check the chapter dedicated to the NoSQL store you target to find the specificities.

Many NoSQL stores have no notion of schema. Likewise, the tuple stored by Hibernate OGM is not tied to a particular schema: the tuple is represented by a Map, not a typed Map specific to a given entity type. Nevertheless, JPA does describe a schema thanks to:

  • the class schema
  • the JPA physical annotations like @Table and @Column.

While tied to the application, it offers some robustness and explicit understanding when the schema is changed as the schema is right in front of the developers' eyes. This is an intermediary model between the strictly typed relational model and the totally schema-less approach pushed by some NoSQL families.

Since Hibernate OGM wants to offer all of JPA, it needs to support JP-QL queries. Hibernate OGM parses the JP-QL query string and extracts its meaning. From there, several options are available depending of the capabilities of the NoSQL store you target:

If the NoSQL datastore has some query capabilities and if the JP-QL query is simple enough to be executed by the datastore, then the JP-QL parser directly pushes the query generation to the NoSQL specific query translator. The query returns the list of matching identifiers snd uses Hibernate OGM to return managed objects.

Some of the JP-QL features are not supported by NoSQL solutions. Two typical examples are joins between entities - which you should limit anyways in a NoSQL environment - and aggregations like average, max, min etc. When the NoSQL store does not support the query, we use Teiid - a database federation engine - to build simpler queries executed to the datastore and perform the join or aggregation operations in Teiid itself.

Finally some NoSQL stores have poor query support, or none at all. In this case Hibernate OGM can use Hibernate Search as its indexing and query engine. Hibernate Search is able to index and query objects - entities - and run full-text queries. It uses the well known Apache Lucene to do that but adds a few interesting characteristics like clustering support and an object oriented abstraction including an object oriented query DSL. Let’s have a look at the architecture of Hibernate OGM when using Hibernate Search:


In this situation, Hibernate ORM Core pushes change events to Hibernate Search which will index entities accordingly and keep the index and the datastore in sync. The JP-QL query parser delegates the query translation to the Hibernate Search query translator and executes the query on top of the Lucene indexes. Indexes can be stored in various fashions:

  • on a file system (the default in Lucene)
  • in Infinispan via the Infinispan Lucene directory implementation: the index is then distributed across several servers transparently
  • in NoSQL stores like Voldemort that can natively store Lucene indexes
  • in NoSQL stores that can be used as overflow to Infinispan: in this case Infinispan is used as an intermediary layer to serve the index efficiently but persists the index in another NoSQL store.

Note that for complex queries involving joins or aggregation, Hibernate OGM can use Teiid as an intermediary query engine that will delegate to Hibernate Search.

Note that you can use Hibernate Search even if you do plan to use the NoSQL datastore query capabilities. Hibernate Search offers a few interesting options:

  • clusterability
  • full-text queries - ie Google for your entities
  • geospatial queries
  • query faceting (ie dynamic categorization of the query results by price, brand etc)

What’s the progress status on queries?

Well… now is a good time to remind you that Hibernate OGM is open source and that contributing to such cutting edge project is a lot of fun. Check out Chapter 1, How to get help and contribute on Hibernate OGM for more details.

But to answer your question, we have finished the skeleton of the architecture as well as the JP-QL parser implementation. The Hibernate Search query translator can execute simple queries already. However, we do not yet have a NoSQL specific query translator but the approach is quite clear to us. Teiid for complex queries is also not integrated but work is being done to facilitate that integration soon. Native Hibernate Search queries are fully supported.

Hibernate OGM favors ease of use and convention over configuration. This makes its configuration quite simple by default.

Hibernate OGM can be used via the Hibernate native APIs (Session) or via the JPA APIs (EntityManager). Depending of your choice, the bootstrapping strategy is slightly different.

The good news is that if you use JPA as your primary API, the configuration is extremely simple. Hibernate OGM is seen as a persistence provider which you need to configure in your persistence.xml. That’s it! The provider name is org.hibernate.ogm.jpa.HibernateOgmPersistence.


There are a couple of things to notice:

You also need to configure which NoSQL datastore you want to use and how to connect to it. We will detail how to do that later in Chapter 5, Datastores. In this case, we have used the defaults settings for Infinispan.

From there, simply bootstrap JPA the way you are used to with Hibernate ORM:

  • via Persistence.createEntityManagerFactory
  • by injecting the EntityManager / EntityManagerFactory in a Java EE container
  • by using your favorite injection framework (CDI - Weld, Spring, Guice)

If you want to bootstrap Hibernate OGM using the native Hibernate APIs, use the class org.hibernate.ogm.cfg.OgmConfiguration.


There are a couple of things to notice:

You also need to configure which NoSQL datastore you want to use and how to connect to it. We will detail how to do that later in Chapter 5, Datastores. In this case, we have used the defaults settings for Infinispan.

Hibernate OGM runs in various environments, pretty much what you are used to with Hibernate ORM. There are however environments where it works better and has been more thoroughly tested.

You don’t have to do much in this case. You need three specific settings:

If you use JPA, simply set the transaction-type to JTA and the transaction factory will be set for you.

If you use Hibernate ORM native APIs only, then set hibernate.transaction.factory_class to either:

Set the JTA platform to the right Java EE container. The property is hibernate.transaction.transaction.jta.platform and must contain the fully qualified class name of the lookup implementation. The list of available values are listed in Hibernate ORM’s configuration section. For example, in JBoss AS, use org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform.

In your persistence.xml, you also need to define an existing datasource. It is not needed by Hibernate OGM and won’t be used but the JPA specification mandates this setting.


java:DefaultDS will work for out of the box JBoss AS deployments.

There is a set of common misconceptions in the Java community about JTA:

None of that is true of course, let me show you how to use JBoss Transaction in a standalone environment with Hibernate OGM.

In Hibernate OGM, make sure to set the following properties:

On the JBoss Transaction side, add JBoss Transaction in your classpath. If you use maven, it should look like this:


The next step is you get access to the transaction manager. The easiest solution is to do as the following example:

TransactionManager transactionManager =

   com.arjuna.ats.jta.TransactionManager.transactionmanager();

Then use the standard JTA APIs to demarcate your transaction and you are done!


That was not too hard, was it? Note that application frameworks like Seam or Spring Framework should be able to initialize the transaction manager and call it to demarcate transactions for you. Check their respective documentation.

The most important options when configuring Hibernate OGM are related to the datastore. They are explained in Chapter 5, Datastores.

Otherwise, most options from Hibernate ORM and Hibernate Search are applicable when using Hibernate OGM. You can pass them as you are used to do either in your persistence.xml file, your hibernate.cfg.xml file or programmatically.

More interesting is a list of options that do not apply to Hibernate OGM and that should not be set:

  • hibernate.dialect
  • hibernate.connection.* and in particular hibernate.connection.provider_class
  • hibernate.show_sql and hibernate.format_sql
  • hibernate.default_schema and hibernate.default_catalog
  • hibernate.use_sql_comments
  • hibernate.jdbc.*
  • hibernate.hbm2ddl.auto and hibernate.hbm2ddl.import_file

Provided you’re deploying on JBoss AS 7.2 or JBoss EAP6, there is an additional way to add the OGM dependencies to your application.

In JBoss AS 7, class loading is based on modules that have to define explicit dependencies on other modules. Modules allow to share the same artifacts across multiple applications, getting you smaller and quicker deployments.

More details about modules are described in Class Loading in AS 7.2.

You can download the pre-packaged module from:

Unpack the archive into the modules folder of your JBoss AS 7.2 installation. The modules included are:

  • org.hibernate:ogm, containing the core OGM library and the infinispan datastore provider.
  • org.hibernate.ogm.ehcache:main, containing the ehcache datastore provider.
  • org.hibernate.ogm.mongodb:main, containing the mongodb datastore provider.
  • org.hibernate:main, containing the latest hibernate ORM libraries compatible with OGM.

Warning

The org.hibernate:main module changes the version of Hibernate ORM included in the default JBoss AS 7.2.

There are two ways to include the dependencies in your project:

Using the manifest
Add this entry to the MANIFEST.MF in your archive:
Dependencies: org.hibernate:ogm services
Using jboss-deployment-structure.xml
This is a proprietary JBoss AS descriptor. Add a WEB-INF/jboss-deployment-structure.xml in your archive with content:

<jboss-deployment-structure>
    <deployment>
        <dependencies>
            <module name="org.hibernate" slot="ogm" services="export" />
        </dependencies>
    </deployment>
</jboss-deployment-structure>

More information about the descriptor can be found in the JBoss AS 7.2 documentation.

Currently Hibernate OGM supports the following datastores:

  • Map: stores data in an in-memory Java map to store data. Use it only for unit tests.
  • Infinispan: stores data into Infinispan (data grid)
  • Ehcache: stores data into Ehcache (cache)
  • MongoDB: stores data into MongoDB (document store)

More are planned, if you are interested, come talk to us (see Chapter 1, How to get help and contribute on Hibernate OGM).

Hibernate OGM interacts with NoSQL datastores via two contracts:

  • a datastore provider which is responsible for starting and stopping the connection(s) with the datastore and prop up the datastore if needed
  • a grid dialect which is responsible for converting an Hibernate OGM operation into a datastore specific operation

The main thing you need to do is to configure which datastore provider you want to use. This is done via the hibernate.ogm.datastore.provider option. Possible values are the fully qualified class name of a DatastoreProvider implementation or one preferably of the following shortcuts:

  • map: stores data in an in-memory Java map to store data. Use it only for unit tests.
  • infinispan: stores data into Infinispan (data grid)
  • ehcache: stores data into Ehcache (cache)
  • mongodb: stores data into MongoDB (document store)
  • neo4j: stores data into Neo4j (graph)

You also need to add the relevant Hibernate OGM module in your classpath. In maven that would look like:


<dependency>
    <groupId>org.hibernate.ogm</groupId>
    <artifactId>hibernate-ogm-infinispan</artifactId>
    <version>4.0.0.Beta4</version>
</dependency>

We have respectively hibernate-ogm-infinispan, hibernate-ogm-ehcache, hibernate-ogm-mongodb and hibernate-ogm-neo4j. The map datastore is included in the Hibernate OGM engine module.

By default, a datastore provider chooses the best grid dialect transparently but you can manually override that setting with the hibernate.ogm.datastore.grid_dialect option. Use the fully qualified class name of the GridDialect implementation. Most users should ignore this setting entirely and live happy.

Infinispan is an open source in-memory data grid focusing on high performance. As a data grid, you can deploy it on multiple servers - referred to as nodes - and connect to it as if it were a single storage engine: it will cleverly distribute both the computation effort and the data storage.

It is trivial to setup on a single node, in your local JVM, so you can easily try Hibernate OGM. But Infinispan really shines in multiple node deployments: you will need to configure some networking details but nothing changes in terms of application behaviour, while performance and data size can scale linearly.

From all its features we’ll only describe those relevant to Hibernate OGM; for a complete description of all its capabilities and configuration options, refer to the Infinispan project documentation at infinispan.org.

Two steps basically:

Hibernate OGM will not use a single Cache but three and is going to use them for different purposes; so that you can configure the Caches meant for each role separately.

We’ll explain in the following paragraphs how you can take advantage of this and which aspects of Infinispan you’re likely to want to reconfigure from their defaults. All attributes and elements from Infinispan which we don’t mention are safe to ignore. Refer to the Infinispan User Guide for the guru level performance tuning and customizations.

An Infinispan configuration file is an XML file complying with the Infinispan schema; the basic structure is shown in the following example:


The global section contains elements which affect the whole instance; mainly of interest for Hibernate OGM users is the transport element in which we’ll set JGroups configuration overrides.

In the namedCache section (or in default if we want to affect all named caches) we’ll likely want to configure clustering modes, eviction policies and CacheStores.

In its default configuration Infinispan stores all data in the heap of the JVM; in this barebone mode it is conceptually not very different than using a HashMap: the size of the data should fit in the heap of your VM, and stopping/killing/crashing your application will get all data lost with no way to recover it.

To store data permanently (out of the JVM memory) a CacheStore should be enabled. The infinispan-core.jar includes a simple implementation able to store data in simple binary files, on any read/write mounted filesystem; this is an easy starting point, but the real stuff is to be found in the additional modules found in the Infinispan distribution. Here you can find many more implementations to store your data in anything from JDBC connected relational databases, other NoSQL engines, to cloud storage services or other Infinispan clusters. Finally, implementing a custom CacheStore is a trivial programming exercise.

To limit the memory consumption of the precious heap space, you can activate a passivation or an eviction policy; again there are several strategies to play with, for now let’s just consider you’ll likely need one to avoid running out of memory when storing too many entries in the bounded JVM memory space; of course you don’t need to choose one while experimenting with limited data sizes: enabling such a strategy doesn’t have any other impact in the functionality of your Hibernate OGM application (other than performance: entries stored in the Infinispan in-memory space is accessed much quicker than from any CacheStore).

A CacheStore can be configured as write-through, committing all changes to the CacheStore before returning (and in the same transaction) or as write-behind. A write-behind configuration is normally not encouraged in storage engines, as a failure of the node implies some data might be lost without receiving any notification about it, but this problem is mitigated in Infinispan because of its capability to combine CacheStore write-behind with a synchronous replication to other Infinispan nodes.


In this example we enabled both eviction and a CacheStore (the loader element). LIRS is one of the choices we have for eviction strategies. Here it is configured to keep (approximately) 2000 entries in live memory and evict the remaining as a memory usage control strategy.

The CacheStore is enabling passivation, which means that the entries which are evicted are stored on the filesystem.

Warning

You could configure an eviction strategy while not configuring a passivating CacheStore! That is a valid configuration for Infinispan but will have the evictor permanently remove entries. Hibernate OGM will break in such a configuration.

Tip

Currently with Infinispan 5.1, the FileCacheStore is neither very fast nor very efficient: we picked it for ease of setup. For a production system it’s worth looking at the large collection of high performance and cloud friendly cachestores provided by the Infinispan distribution.

The best thing about Infinispan is that all nodes are treated equally and it requires almost no beforehand capacity planning: to add more nodes to the cluster you just have to start new JVMs, on the same or different physical server, having your same Infinispan configuration and your same application.

Infinispan supports several clustering cache modes; each mode provides the same API and functionality but with different performance, scalability and availability options:

To use the replication or distribution cache modes Infinispan will use JGroups to discover and connect to the other nodes.

In the default configuration, JGroups will attempt to autodetect peer nodes using a multicast socket; this works out of the box in the most network environments but will require some extra configuration in cloud environments (which often block multicast packets) or in case of strict firewalls. See the JGroups reference documentation, specifically look for Discovery Protocols to customize the detection of peer nodes.

Nowadays, the JVM defaults to use IPv6 network stack; this will work fine with JGroups, but only if you configured IPv6 correctly. It is often useful to force the JVM to use IPv4.

It is also useful to let JGroups know which networking interface you want to use; especially if you have multiple interfaces it might not guess correctly.


Note

You don’t need to use IPv4: JGroups is compatible with IPv6 provided you have routing properly configured and valid addresses assigned.

The jgroups.bind_addr needs to match a placeholder name in your JGroups configuration in case you don’t use the default one.

The default configuration uses distribution as cache mode and uses the jgroups-tcp.xml configuration for JGroups, which is contained in the Infinispan jar as the default configuration for Infinispan users. Let’s see how to reconfigure this:


In the example above we specify a custom JGroups configuration file and set the cache mode for the default cache to distribution; this is going to be inherited by the ENTITIES and the ASSOCIATIONS caches. But for IDENTIFIERS we have chosen (for the sake of this example) to use replication.

Now that you have clustering configured, start the service on multiple nodes. Each node will need the same configuration and jars.

Tip

We have just shown how to override the clustering mode and the networking stack for the sake of completeness, but you don’t have to!

Start with the default configuration and see if that fits you. You can fine tune these setting when you are closer to going in production.

Hibernate Search, which can be used for advanced query capabilities (see Chapter 7, Query your entities), needs some place to store the indexes for its embedded Apache Lucene engine.

A common place to store these indexes is the filesystem which is the default for Hibernate Search; however if your goal is to scale your NoSQL engine on multiple nodes you need to share this index. Network sharing filesystems are a possibility but we don’t recommended that. Often the best option is to store the index in whatever NoSQL database you are using (or a different dedicated one).

Tip

You might find this section useful even if you don’t intend to store your data in Infinispan.

The Infinispan project provides an adaptor to plug into Apache Lucene, so that it writes the indexes in Infinispan and searches data in it. Since Infinispan can be used as an application cache to other NoSQL storage engines by using a CacheStore (see Section 5.1.2, “Manage data size”) you can use this adaptor to store the Lucene indexes in any NoSQL store supported by Infinispan:

  • Cassandra
  • Filesystem (but locked correctly at the Infinispan level)
  • MongoDB
  • HBase
  • JDBC databases
  • JDBM
  • BDBJE
  • A secondary (independent) Infinispan grid
  • Any Cloud storage service supported by JClouds

How to configure it? Here is a simple cheat sheet to get you started with this type of setup:

  • Add org.hibernate:hibernate-search-infinispan:4.4.0.Beta1 to your dependencies
  • set these configuration properties:

    • hibernate.search.default.directory_provider = infinispan
    • hibernate.search.default.exclusive_index_use = false
    • hibernate.search.infinispan.configuration_resourcename = [infinispan configuration filename]

The referenced Infinispan configuration should define a CacheStore to load/store the index in the NoSQL engine of choice. It should also define three cache names:


This configuration is not going to scale well on write operations: to do that you should read about the master/slave and sharding options in Hibernate Search. The complete explanation and configuration options can be found in the Hibernate Search Reference Guide

Some NoSQL support storage of Lucene indexes directly, in which case you might skip the Infinispan Lucene integration by implementing a custom DirectoryProvider for Hibernate Search. You’re very welcome to share the code and have it merged in Hibernate Search for others to use, inspect, improve and maintain.

When combined with Hibernate ORM, Ehcache is commonly used as a 2nd level cache, so caching data which is stored in a relational database. When used with Hibernate OGM it is not "just a cache" but is the main storage engine for your data.

This is not the reference manual for Ehcache itself: we’re going to list only how Hibernate OGM should be configured to use Ehcache; for all the tuning and advanced options please refer to the Ehcache Documentation.

MongoDB is a document oriented datastore written in C++ with strong emphasis on ease of use.

This implementation is based upon the MongoDB Java driver. The currently supported version is 2.10.1.

The following properties are available to configure MongoDB support:

MongoDB datastore configuration properties

hibernate.ogm.datastore.provider
To use MongoDB as a datastore provider, this property must be set to mongodb
hibernate.ogm.mongodb.host
The hostname of the mongodb instance. The default value is 127.0.0.1.
hibernate.ogm.mongodb.port
The port used by the mongodb instance. The default value is 27017
hibernate.ogm.mongodb.database
The database to connect to. This property has no default value.
hibernate.ogm.mongodb.username
The username used when connecting to the MongoDB server. This property has no default value.
hibernate.ogm.mongodb.password
The password used to connect to the MongoDB server. This property has no default value. This property is ignored if the username isn’t specified.
hibernate.ogm.mongodb.connection_timeout
Defines the timeout used by the driver when the connection to the MongoDB instance is initiated. This configuration is expressed in milliseconds. The default value is 5000.
hibernate.ogm.mongodb.associations.store
Defines the way OGM stores association information. There are three strategies to store association information. GLOBAL_COLLECTION stores the association information in a unique MongoDB collection for all associations. COLLECTION stores the association in a dedicated MongoDB collection per association. IN_ENTITY stores association information from within the entity. IN_ENTITY is the default.
hibernate.ogm.mongodb.writeconcern
Possible values are ERRORS_IGNORED, ACKNOWLEDGED, UNACKNOWLEDGED, FSYNCED, JOURNALED, NONE, NORMAL, SAFE, MAJORITY, FSYNC_SAFE, JOURNAL_SAFE, REPLICAS_SAFE. For more information, please refer to the official documentation. This option is case insensitive and the default value is ACKNOWLEDGED.

Hibernate OGM tries to make the mapping to the underlying datastore as natural as possible so that third party applications not using Hibernate OGM can still read and update the same datastore. We worked particularly hard on the MongoDB model to offer various classic mappings between your object model and the MongoDB documents.

Entities are stored as MongoDB documents and not as BLOBs which means each entity property will be translated into a document field. You can use the name property of @Table and @Column annotation to rename the collections and the document’s field if you need to.

Note that embedded objects are mapped as nested documents.


Hibernate OGM MongoDB proposes 3 strategies to store navigation information for associations. To switch between each of these strategies, use the hibernate.ogm.mongodb.associations.store configuration property. The three possible values are:

Neo4j is a robust (fully ACID) transactional property graph database. This kind of databases are suited for those type of problems that can be represented with a graph like social relationships or road maps for example.

At the moment only the support for the embeedded Neo4j is included in OGM.

This is our first version and a bit experimental. In particular we plan on using node navigation much more than index lookup in a future version.

TODO:

  • Talk about supported approaches (properties, embedded objects, inheritance)
  • Talk about associations
  • Talk about identifier recommendations

This section is a work in progress, if you find something that does not work as expected, let us know and we will update it (and fix the problem of course).

Pretty much all entity related constructs should work out of the box in Hibernate OGM. @Entity, @Table, @Column, @Enumarated, @Temporal, @Cacheable and the like will work as expected. If you want an example, check out Chapter 2, Getting started with Hibernate OGM or the documentation of Hibernate ORM. Let’s concentrate of the features that differ or are simply not supported by Hibernate OGM.

The various inheritance strategies are not supported by Hibernate OGM, only the table per concrete class strategy is used. f This is not so much a limitation but rather an acknowledgment of the dynamic nature of NoSQL schemas. If you feel the need to support other strategies, let us know (see Section 1.2, “How to contribute”). Simply do not use @Inheritance nor @DiscriminatorColumn.

Secondary tables are not supported by Hibernate OGM at the moment. If you have needs for this feature, let us know (see Section 1.2, “How to contribute”).

All SQL related constructs as well as HQL centered mapping are not supported in Hibernate OGM. Here is a list of feature that will not work:

  • Named queries
  • Native queries

All standard JPA id generators are supported: IDENTITY, SEQUENCE, TABLE and AUTO. If you need support for additional generators, let us know (see Section 1.2, “How to contribute”). We recommend you use a UUID based generator as this type of generator allows maximum scalability to the underlying data grid as no cluster-wide counter is necessary.


All association types are supported (@OneToOne, @OneToMany, @ManyToOne, @ManyToMany). Likewise, all collection types are supported (Set, Map, List). The way Hibernate OGM stores association information is however quite different than the traditional RDBMS representation. Check Section 3.2, “How is data persisted” for more information.

Keep in mind that collections with many entries won’t perform very well in Hibernate OGM (at least today) as all of the association navigation for a given entity is stored in a single key. If your collection is made of 1 million elements, Hibernate OGM stores 1 million tuples in the association key.

To query a NoSQL database is a complex feat, especially as not all NoSQL solutions support all forms of query. One of the goals of Hibernate OGM is to deal with this complexity so that users don’t have to. However, that’s not yet all implemented and depending on your use case there might be better approaches you can take advantage of.

If you skipped to this section without reading Chapter 3, Architecture, I’d suggest to read at least Section 3.3, “How is data queried” as it will greatly help you choosing a query approach.

We actually did use Hibernate Search already in the previous example; specifically the annotations @Indexed and @Field are Hibernate Search specific. In this example the query was defined using a JP-QL string and then defining parameters; that’s useful if all you have a is a JP-QL Query, but it is limiting.

Hibernate Search remaps the properties annotated with @Field in Lucene Documents, and manages the Lucene indexes so that you can then perform Lucene Queries.

To be extremely short, Apache Lucene is a full-text indexing and query engine with excellent query performance. Featurewise, full-text means you can do much more than a simple equality match as we did in the previous example.

Let’s show another example, now creating a Lucene Query instead:


Assuming our database contains an Hypothesis instance having description "tomorrow we release", the query above will not find the entity because we disabled text analysis in the previous mapping.

If we enable text analysis (which is the default):


Now the entity would match a query on "tomorrow" as we’re unlocking text similarity queries!

Text similarity can be very powerful as it can be configured for specific languages or domain specific terminology; it can deal with typos and synonyms, and above all it can return results by relevance.

Worth noting the Lucene index is a vectorial space of term occurrence statistics: so extracting tags from text, frequencies of strings and correlate this data makes it very easy to build efficient data analysis applications.

For a full explanation of all its capabilities and configuration options, see the Hibernate Search reference documentation.

While the potential of Lucene queries is very high, it’s not suited for all use cases Let’s see some of the limitations of Lucene Queries as our main query engine:

  • Lucene doesn’t support Joins. Any to-One relations can be mapped fine, and the Lucene community is making progress on other forms, but restrictions on OneToMany or ManyToMany can’t be implemented today.
  • Since we apply changes to the index at commit time, your updates won’t affect queries until you commit (we might improve on this).
  • While queries are extremely fast, write operations are not as fast (but we can make it scale).