Hibernate Search

You can download all dependencies from the Hibernate download site. You can also verify the dependency versions against the Hibernate Compatibility Matrix.

Instead of managing all dependencies yourself maven users have the possibility to use the JBoss maven repository. Just add the JBoss repository url to the repositories section of your pom.xml or settings.xml:

<repository>
  <id>repository.jboss.org</id>
  <name>JBoss Maven Repository</name>
  <url>http://repository.jboss.org/maven2</url>
  <layout>default</layout>
</repository>
      

Then add the following dependencies to your pom.xml:

<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-search</artifactId>
   <version>3.0.0.ga</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-annotations</artifactId>
   <version>3.3.0.ga</version>
</dependency>
<dependency>
   <groupId>org.hibernate</groupId>
   <artifactId>hibernate-entitymanager</artifactId>
   <version>3.3.1.ga</version>
</dependency>
      

Not all three dependencies are required. hibernate-search alone contains everything needed to use Hibernate Search. hibernate-annotations is only needed if you use non Hibernate Search annotations like we do in the examples of this tutorial. Last but not least, hibernate-entitymanager is only required if you use Hibernate Search in conjunction with JPA.

Once you have downloaded and added all required dependencies to your application you have to add a few properties to your hibernate configuration file. If you are using Hibernate directly this can be done in hibernate.properties or hibernate.cfg.xml. If you are using Hibernate via JPA you can also add the properties to persistence.xml. The good news is that for standard use most properties offer a sensible default.

Apache Lucene has a notion of Directory to store the index files. Hibernate Search handles the initialization and configuration of a Lucene Directory instance via a DirectoryProvider. In this tutorial we will use a subclass of DirectoryProvider called FSDirectoryProvider. This will give us the ability to physically inspect the Lucene indexes created by Hibernate Search (eg via Luke). Once you have a working configuration you can start experimenting with other directory providers (see Section 3.1, “Directory configuration”).

Lets assume that your application contains the Hibernate managed class example.Book and you now want to add free text search capabilities to your application in order to search body and summary of the books contained in your database.

package exmaple.Book
...
@Entity
public class Book {

  @Id
  private Integer id; 
  private String body;  
  private String summary; 
  @ManyToMany private Set<Author> authors = new HashSet<Author>();
  @ManyToOne private Author mainAuthor;
  private Date publicationDate;
  
  public Book() {
  } 
  
  // standard getters/setters follow here
... 
    

First you have to tell Hibernate Search which DirectoryProvider to use. This can be achieved by setting the hibernate.search.default.directory_provider property. You also have to specify the default root directory for all indexes via hibernate.search.default.indexBase.

...
# the default directory provider
hibernate.search.default.directory_provider = org.hibernate.search.store.FSDirectoryProvider

# the default base directory for the indecies
hibernate.search.default.indexBase = /var/lucene/indexes    
...
    

Next you have to add three annotations to the Book class. The first annotation @Indexed marks Book as indexable. By design Hibernate Search needs to store an untokenized id in the index to ensure index unicity for a given entity. @DocumentId marks the property to use for this purpose. Most if not all the time, the property is the database primary key. Last but not least you have to index the fields you want to make searchable. In our example these fields are body and summary. Both properties get annotated with @Field. The property index=Index.TOKENIZED will ensure that the text will be tokenized using the default Lucene analyzer whereas store=Store.NO ensures that the actual data will not be stored in the index. Usually, tokenizing means chunking a sentence into individual words (and potentially excluding common words like a, the etc).

These settings are sufficient for an initial test. For more details on entity mapping refer to Section 4.1, “Mapping an entity”. In case you want to store and retrieve the indexed data in order to avoid database roundtrips, refer to projections in Section 5.1.2.5, “Projection”

package exmaple.Book
...
@Entity
@Indexed
public class Book {

  @Id
  @DocumentId
  private Integer id;
  
  @Field(index=Index.TOKENIZED, store=Store.NO)
  private String body;
  
  @Field(index=Index.TOKENIZED, store=Store.NO)
  private String summary; 
  @ManyToMany private Set<Author> authors = new HashSet<Author>();
  @ManyToOne private Author mainAuthor;
  private Date publicationDate;
  
  public Book() {
  } 
  
  // standard getters/setters follow here
... 
  

Hibernate Search will index every entity persisted, updated or removed through Hibernate core transparently for the application. However, the data already present in your database needs to be indexed once to populate the Lucene index. Once you have added the above properties and annotations it is time to trigger an initial batch index of your books. You can achieve this by adding one of the following code examples to your code (see also Chapter 6, Manual indexing):

Example using Hibernate Session:

FullTextSession fullTextSession = Search.createFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
List books = session.createQuery("from Book as book").list();
for (Book book : books) {
    fullTextSession.index(book);
}
tx.commit(); //index are written at commit time       
    

Example using JPA:

EntityManager em = entityManagerFactory.createEntityManager();
FullTextEntityManager fullTextEntityManager = Search.createFullTextEntityManager(em);
List books = em.createQuery("select book from Book as book").getResultList();
for (Book book : books) {
    fullTextEntityManager.index(book);
} 
    

After executing the above code, you should be able to see a Lucene index under /var/lucene/indexes/example.Book. Go ahead an inspect this index. It will help you to understand how Hibernate Search works.

Now it is time to execute a first search. The following code will prepare a query against the fields summary and body, execute it and return a list of Books:

Example using Hibernate Session:

FullTextSession fullTextSession = Search.createFullTextSession(session);

Transaction tx = fullTextSession.beginTransaction();

MultiFieldQueryParser parser = new MultiFieldQueryParser( new String[]{"summary", "body"}, 
  new StandardAnalyzer());
Query query = parser.parse( "Java rocks!" );
org.hibernate.Query hibQuery = fullTextSession.createFullTextQuery( query, Book.class );
List result = hibQuery.list();
  
tx.commit();
session.close();  
    

Example using JPA:

EntityManager em = entityManagerFactory.createEntityManager();

FullTextEntityManager fullTextEntityManager = 
    org.hibernate.hibernate.search.jpa.Search.createFullTextEntityManager(em);
MultiFieldQueryParser parser = new MultiFieldQueryParser( new String[]{"summary", "body"}, 
  new StandardAnalyzer());
Query query = parser.parse( "Java rocks!" );
org.hibernate.Query hibQuery = fullTextEntityManager.createFullTextQuery( query, Book.class );
List result = hibQuery.list();
    

Assume that one of your indexed book entities contains the text "Java rocks" and you want to get hits for all of the following queries: "rock", "rocks", "rocked" and "rocking". In Lucene this can be achieved by choosing an analyzer class which applies word stemming during the indexing process. Hibernate Search offers several ways to configure the analyzer to use (see Section 4.1.5, “Analyzer”):

The following example uses the entity level annotation to apply a English language analyzer which would help you to achieve your goal. The class EnglishAnalyzer is a custom class using the Snowball English Stemmer from the Lucene Sandbox.

package example.Book
...
@Entity
@Indexed
@Analyzer(impl = example.EnglishAnalyzer.class)
public class Book {

  @Id
  @DocumentId
  private Integer id;
  
  @Field(index=Index.TOKENIZED, store=Store.NO)
  private String body;
  
  @Field(index=Index.TOKENIZED, store=Store.NO)
  private String summary; 
  @ManyToMany private Set<Author> authors = new HashSet<Author>();
  @ManyToOne private Author mainAuthor;
  private Date publicationDate;
  
  public Book() {
  } 
  
  // standard getters/setters follow here
... 
}

public class EnglishAnalyzer extends Analyzer {
    /**
     * {@inheritDoc}
     */
    @Override
    public TokenStream tokenStream(String fieldName, Reader reader) {
        TokenStream result = new StandardTokenizer(reader);
        result = new StandardFilter(result);
        result = new LowerCaseFilter(result);
        result = new SnowballFilter(result, name);
        return result;
    }
}
  

The above paragraphs hopefully helped you getting started with Hibernate Search. You should by now have a file system based index and be able to search and retrieve a list of managed objects via Hibernate Search. The next step is to get more familiar with the overall architecture ((Chapter 2, Architecture)) and explore the basic features in more detail.

Two topics which where only briefly touched in this tutorial were analyzer configuration (Section 4.1.5, “Analyzer”) and field bridges (Section 4.2, “Property/Field Bridge”), both important features required for more fine-grained indexing.

More advanced topics cover clustering (Section 3.4, “JMS Master/Slave configuration”) and large indexes handling (Section 3.2, “Index sharding”).

Hibernate Search consists of an indexing and an index search engine. Both are backed by Apache Lucene.

When an entity is inserted, updated or removed in/from the database, Hibernate Search keeps track of this event (through the Hibernate event system) and schedules an index update. All the index updates are handled for you without you having to use the Apache Lucene APIs (see Section 3.6, “Enabling Hibernate Search and automatic indexing”).

To interact with Apache Lucene indexes, Hibernate Search has the notion of DirectoryProviders. A directory provider will manage a given Lucene Directory type. You can configure directory providers to adjust the directory target (see Section 3.1, “Directory configuration”).

Hibernate Search can also use the Lucene index to search an entity and return a list of managed entities saving you the tedious object to Lucene document mapping. The same persistence context is shared between Hibernate and Hibernate Search; as a matter of fact, the Search Session is built on top of the Hibernate Session. The application code use the unified org.hibernate.Query or javax.persistence.Query APIs exactly the way a HQL, JPA-QL or native queries would do.

To be more efficient, Hibernate Search batches the write interactions with the Lucene index. There is currently two types of batching depending on the expected scope.

Outside a transaction, the index update operation is executed right after the actual database operation. This scope is really a no scoping setup and no batching is performed.

It is however recommended, for both your database and Hibernate Search, to execute your operation in a transaction be it JDBC or JTA. When in a transaction, the index update operation is scheduled for the transaction commit and discarded in case of transaction rollback. The batching scope is the transaction. There are two immediate benefits:

You can think of those two scopes (no scope vs transactional) as the equivalent of the (infamous) autocommit vs transactional behavior. From a performance perspective, the in transaction mode is recommended. The scoping choice is made transparently: Hibernate Search detects the presence of a transaction and adjust the scoping.

Hibernate Search offers the ability to let the scoped work being processed by different back ends. Two back ends are provided out of the box for two different scenarios.

2.2.1. Lucene

In this mode, all index update operations applied on a given node (JVM) will be executed to the Lucene directories (through the directory providers) by the same node. This mode is typically used in non clustered environment or in clustered environments where the directory store is shared.

This mode targets non clustered applications, or clustered applications where the Directory is taking care of the locking strategy.

The main advantage is simplicity and immediate visibility of the changes in Lucene queries (a requirement is some applications).

2.2.2. JMS

All index update operations applied on a given node are sent to a JMS queue. A unique reader will then process the queue and update the master Lucene index. The master index is then replicated on a regular basis to the slave copies. This is known as the master / slaves pattern. The master is the sole responsible for updating the Lucene index. The slaves can accept read as well as write operations. However, they only process the read operation on their local index copy and delegate the update operations to the master.

This mode targets clustered environments where throughput is critical, and index update delays are affordable. Reliability is ensured by the JMS provider and by having the slaves working on a local copy of the index.

The indexing work (done by the back end) can be executed synchronously with the transaction commit (or update operation if out of transaction), or asynchronously.

When executing a query, Hibernate Search interacts with the Apache Lucene indexes through a reader strategy. chosing a reader strategy will depend on the profile of the application (frequent updates, read mostly, asynchronous index update etc). See also Section 3.5, “Reader strategy configuration”

Apache Lucene has a notion of Directory to store the index files. The Directory implementation can be customized, but Lucene comes bundled with a file system (FSDirectoryProvider) and a in memory (RAMDirectoryProvider) implementation. Hibernate Search has the notion of DirectoryProvider that handles the configuration and the initialization of the Lucene Directory.

If the built-in directory providers does not fit your needs, you can write your own directory provider by implementing the org.hibernate.store.DirectoryProvider interface

Each indexed entity is associated to a Lucene index (an index can be shared by several entities but this is not usually the case). You can configure the index through properties prefixed by hibernate.search.indexname . Default properties inherited to all indexes can be defined using the prefix hibernate.search.default.

To define the directory provider of a given index, you use the hibernate.search.indexname.directory_provider

hibernate.search.default.directory_provider org.hibernate.search.store.FSDirectoryProvider
hibernate.search.default.indexBase=/usr/lucene/indexes

hibernate.search.Rules.directory_provider org.hibernate.search.store.RAMDirectoryProvider        

applied on

@Indexed(name="Status")
public class Status { ... }

@Indexed(name="Rules")
public class Rule { ... }

will create a file system directory in /usr/lucene/indexes/Status where the Status entities will be indexed, and use an in memory directory named Rules where Rule entities will be indexed.

You can easily define common rules like the directory provider and base directory, and overide those default later on on a per index basis.

Writing your own DirectoryProvider, you can utilize this configuration mechanism as well.

In some extreme cases involving huge indexes (in size), it is necessary to split (shard) the indexing data of a given entity type into several Lucene indexes. This solution is not recommended until you reach significant index sizes and index update time are slowing down. The main drawback of index sharding is that searches will end up being slower since more files have to be opend for a single search. In other words don't do it until you have problems :)

Despite this strong warning, Hibernate Search allows you to index a given entity type into several sub indexes. Data is sharded into the different sub indexes thanks to an IndexShardingStrategy. By default, no sharding strategy is enabled, unless the number of shards is configured. To configure the number of shards use the following property

hibernate.search.<indexName>.sharding_strategy.nbr_of_shards 5

This will use 5 different shards.

The default sharding strategy, when shards are set up, splits the data according to the hash value of the id string representation (generated by the Field Bridge). This ensures a fairly balanced sharding. You can replace the strategy by implementing IndexShardingStrategy and by setting the following property

hibernate.search.<indexName>.sharding_strategy my.shardingstrategy.Implementation

Each shard has an independent directory provider configuration as described in Section 3.1, “Directory configuration”. The DirectoryProvider default name for the previous example are <indexName>.0 to <indexName>.4. In other words, each shard has the name of it's owning index followed by . (dot) and its index number.

hibernate.search.default.indexBase /usr/lucene/indexes

hibernate.search.Animal.sharding_strategy.nbr_of_shards 5
hibernate.search.Animal.directory_provider org.hibernate.search.store.FSDirectoryProvider
hibernate.search.Animal.0.indexName Animal00
hibernate.search.Animal.3.indexBase /usr/lucene/sharded
hibernate.search.Animal.3.indexName Animal03

This configuration uses the default id string hashing strategy and shards the Animal index into 5 subindexes. All subindexes are FSDirectoryProvider instances and the directory where each subindex is stored is as followed:

It is possible to refine how Hibernate Search interacts with Lucene through the worker configuration. The work can be exected to the Lucene directory or sent to a JMS queue for later processing. When processed to the Lucene directory, the work can be processed synchronously or asynchronously to the transaction commit.

You can define the worker configuration using the following properties

This section describes in greater detail how to configure the Master / Slaves Hibernate Search architecture.

### slave configuration

## DirectoryProvider
# (remote) master location
hibernate.search.default.sourceBase = /mnt/mastervolume/lucenedirs/mastercopy

# local copy location
hibernate.search.default.indexBase = /Users/prod/lucenedirs

# refresh every half hour
hibernate.search.default.refresh = 1800

# appropriate directory provider
hibernate.search.default.directory_provider = org.hibernate.search.store.FSSlaveDirectoryProvider

## Backend configuration
hibernate.search.worker.backend = jms
hibernate.search.worker.jms.connection_factory = java:/ConnectionFactory
hibernate.search.worker.jms.queue = queue/hibernatesearch
#optional jndi configuration (check your JMS provider for more information)

## Optional asynchronous execution strategy
# org.hibernate.worker.execution = async
# org.hibernate.worker.thread_pool.size = 2
# org.hibernate.worker.buffer_queue.max = 50

### master configuration

## DirectoryProvider
# (remote) master location where information is copied to
hibernate.search.default.sourceBase = /mnt/mastervolume/lucenedirs/mastercopy

# local master location
hibernate.search.default.indexBase = /Users/prod/lucenedirs

# refresh every half hour
hibernate.search.default.refresh = 1800

# appropriate directory provider
hibernate.search.default.directory_provider = org.hibernate.search.store.FSMasterDirectoryProvider

## Backend configuration
#Backend is the default lucene one

@MessageDriven(activationConfig = {
      @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"),
      @ActivationConfigProperty(propertyName="destination", propertyValue="queue/hiebrnatesearch"),
      @ActivationConfigProperty(propertyName="DLQMaxResent", propertyValue="1")
   } )
public class MDBSearchController extends AbstractJMSHibernateSearchController implements MessageListener {
    @PersistenceContext EntityManager em;
    
    //method retrieving the appropriate session
    protected Session getSession() {
        return (Session) em.getDelegate();
    }

    //potentially close the session opened in #getSession(), not needed here
    protected void cleanSessionIfNeeded(Session session) 
    }
}

The different reader strategies are described in Reader strategy. The default reader strategy is shared. This can be adjusted:

hibernate.search.reader.strategy = not-shared

Adding this property switch to the non shared strategy.

Or if you have a custom reader strategy:

hibernate.search.reader.strategy = my.corp.myapp.CustomReaderProvider

where my.corp.myapp.CustomReaderProvider is the custom strategy implementation

Hibernate Search allows you to tune the Lucene indexing performance by specifying a set of parameters which are passed through to underlying Lucene IndexWriter such as mergeFactor, maxMergeDocs and maxBufferedDocs. You can specify these parameters either as default values applying for all indexes or on a per index basis.

There are two sets of parameters allowing for different performance settings depending on the use case. During indexing operations triggered by database modifications, the following ones are used:

When indexing occurs via FullTextSession.index() (see Chapter 6, Manual indexing), the following properties are used:

Unless the corresponding .batch property is explicitly set, the value will default to the .transaction property.

For more information about Lucene indexing performances, please refer to the Lucene documentation.

In Lucene all index fields have to be represented as Strings. For this reason all entity properties annotated with @Field have to be indexed in a String form. For most of your properties, Hibernate Search does the translation job for you thanks to a built-in set of bridges. In some cases, though you need a more fine grain control over the translation process.

/**
 * Padding Integer bridge.
 * All numbers will be padded with 0 to match 5 digits
 *
 * @author Emmanuel Bernard
 */
public class PaddedIntegerBridge implements StringBridge {

    private int PADDING = 5;

    public String objectToString(Object object) {
        String rawInteger = ( (Integer) object ).toString();
        if (rawInteger.length() > PADDING) 
            throw new IllegalArgumentException( "Try to pad on a number too big" );
        StringBuilder paddedInteger = new StringBuilder( );
        for ( int padIndex = rawInteger.length() ; padIndex < PADDING ; padIndex++ ) {
            paddedInteger.append('0');
        }
        return paddedInteger.append( rawInteger ).toString();
    }
}                

@FieldBridge(impl = PaddedIntegerBridge.class)
private Integer length;                

public class PaddedIntegerBridge implements StringBridge, ParameterizedBridge {

    public static String PADDING_PROPERTY = "padding";
    private int padding = 5; //default

    public void setParameterValues(Map parameters) {
        Object padding = parameters.get( PADDING_PROPERTY );
        if (padding != null) this.padding = (Integer) padding;
    }

    public String objectToString(Object object) {
        String rawInteger = ( (Integer) object ).toString();
        if (rawInteger.length() > padding) 
            throw new IllegalArgumentException( "Try to pad on a number too big" );
        StringBuilder paddedInteger = new StringBuilder( );
        for ( int padIndex = rawInteger.length() ; padIndex < padding ; padIndex++ ) {
            paddedInteger.append('0');
        }
        return paddedInteger.append( rawInteger ).toString();
    }
}


//property
@FieldBridge(impl = PaddedIntegerBridge.class,
             params = @Parameter(name="padding", value="10")
            )
private Integer length;                

public class PaddedIntegerBridge implements TwoWayStringBridge, ParameterizedBridge {

    public static String PADDING_PROPERTY = "padding";
    private int padding = 5; //default

    public void setParameterValues(Map parameters) {
        Object padding = parameters.get( PADDING_PROPERTY );
        if (padding != null) this.padding = (Integer) padding;
    }

    public String objectToString(Object object) {
        String rawInteger = ( (Integer) object ).toString();
        if (rawInteger.length() > padding) 
            throw new IllegalArgumentException( "Try to pad on a number too big" );
        StringBuilder paddedInteger = new StringBuilder( );
        for ( int padIndex = rawInteger.length() ; padIndex < padding ; padIndex++ ) {
            paddedInteger.append('0');
        }
        return paddedInteger.append( rawInteger ).toString();
    }

    public Object stringToObject(String stringValue) {
        return new Integer(stringValue);
    }
}


//id property
@DocumentId
@FieldBridge(impl = PaddedIntegerBridge.class,
             params = @Parameter(name="padding", value="10") 
private Integer id;
                

/**
 * Store the date in 3 different field year, month, day
 * to ease Range Query per year, month or day
 * (eg get all the elements of december for the last 5 years)
 *
 * @author Emmanuel Bernard
 */
public class DateSplitBridge implements FieldBridge {
    private final static TimeZone GMT = TimeZone.getTimeZone("GMT");

    public void set(String name, Object value, Document document, Field.Store
            store, Field.Index index, Float boost) {
                    
        Date date = (Date) value;
        Calendar cal = GregorianCalendar.getInstance( GMT );
        cal.setTime( date );
        int year = cal.get( Calendar.YEAR );
        int month = cal.get( Calendar.MONTH ) + 1;
        int day = cal.get( Calendar.DAY_OF_MONTH );
        //set year
        Field field = new Field( name + ".year", String.valueOf(year), store, index );
        if ( boost != null ) field.setBoost( boost );
        document.add( field );
        //set month and pad it if needed
        field = new Field( name + ".month", month < 10 ? "0" : "" + String.valueOf(month), store, index);
        if ( boost != null ) field.setBoost( boost );
        document.add( field );
        //set day and pad it if needed
        field = new Field( name + ".day", day < 10 ? "0" : "" + String.valueOf(day), store, index );
        if ( boost != null ) field.setBoost( boost );
        document.add( field );
    }
}


//property
@FieldBridge(impl = DateSplitBridge.class)
private Integer length;                

@Entity
@Indexed
@ClassBridge(name="branchnetwork",
             index=Index.TOKENIZED,
             store=Store.YES,
             impl = CatFieldsClassBridge.class,
             params = @Parameter( name="sepChar", value=" " ) )
public class Department {
    private int id;
    private String network;
    private String branchHead;
    private String branch;
    private Integer maxEmployees;
    ...
}

public class CatFieldsClassBridge implements FieldBridge, ParameterizedBridge {

    private String sepChar;

    public void setParameterValues(Map parameters) {
        this.sepChar = (String) parameters.get( "sepChar" );
    }

    public void set(String name, 
                Object value, //the department instance (entity) in this case
                Document document, //the Lucene document 
                Field.Store store, Field.Index index, Float boost) {
        // In this particular class the name of the new field was passed
        // from the name field of the ClassBridge Annotation. This is not
        // a requirement. It just works that way in this instance. The
        // actual name could be supplied by hard coding it below.
        Department dep = (Department) value;
        String fieldValue1 = dep.getBranch();
        if ( fieldValue1 == null ) {
            fieldValue1 = "";
        }
        String fieldValue2 = dep.getNetwork();
        if ( fieldValue2 == null ) {
            fieldValue2 = "";
        }
        String fieldValue = fieldValue1 + sepChar + fieldValue2;
        Field field = new Field( name, fieldValue, store, index );
        if ( boost != null ) field.setBoost( boost );
        document.add( field );
    }
}

Hibernate Search queries are built on top of Lucene queries. It gives you a total freedom on the kind of Lucene queries you are willing to execute. However, once built, Hibernate Search abstract the query processing from your application using org.hibernate.Query as your primary query manipulation API.

FullTextSession fullTextSession = Search.createFullTextSession( session );
org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery );

org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery, Customer.class );
//or
fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery, Item.class, Actor.class );

org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery, Customer.class );
fullTextQuery.setFirstResult(15); //start from the 15th element
fullTextQuery.setMaxResults(10); //return 10 elements

org.hibernate.search.FullTextQuery query = s.createFullTextQuery( query, Book.class );
org.apache.lucene.search.Sort sort = new Sort(new SortField("title"));
query.setSort(sort);
List results = query.list();

Criteria criteria = s.createCriteria( Book.class ).setFetchMode( "authors", FetchMode.JOIN );
s.createFullTextQuery( luceneQuery ).setCriteriaQuery( criteria );

org.hibernate.search.FullTextQuery query = s.createFullTextQuery( luceneQuery, Book.class );
query.setProjection( "id", "summary", "body", "mainAuthor.name" );
List results = query.list();
Object[] firstResult = (Object[]) results.get(0);
Integer id = firstResult[0];
String summary = firstResult[1];
String body = firstResult[2];
String authorName = firstResult[3];

org.hibernate.search.FullTextQuery query = s.createFullTextQuery( luceneQuery, Book.class );
query.setProjection( FullTextQuery.SCORE, FullTextQuery.BOOST, FullTextQuery.THIS, "mainAuthor.name" );
List results = query.list();
Object[] firstResult = (Object[]) results.get(0);
float score = firstResult[0];
float boost = firstResult[1];
Book book = firstResult[2];
String authorName = firstResult[3];

Once the Hibernate Search query is built, executing it is in no way different than executing a HQL or Criteria query. The same paradigm and object semantic apply. All the common operations are available: list(), uniqueResult(), iterate(), scroll().

org.hibernate.search.FullTextQuery query = s.createFullTextQuery( luceneQuery, Book.class );
assert 3245 == query.getResultSize(); //return the number of matching books without loading a single one

org.hibernate.search.FullTextQuery query = s.createFullTextQuery( luceneQuery, Book.class );
query.setMaxResult(10);
List results = query.list();
assert 3245 == query.getResultSize(); //return the total number of matching books regardless of pagination

org.hibernate.search.FullTextQuery query = s.createFullTextQuery( luceneQuery, Book.class );
query.setProjection( "title", "mainAuthor.name" );

query.setResultTransformer( 
    new StaticAliasToBeanResultTransformer( BookView.class, "title", "author" ) 
);
List<BookView> results = (List<BookView>) query.list();
for(BookView view : results) {
    log.info( "Book: " + view.getTitle() + ", " + view.getAuthor() );
}

Apache Lucene has a powerful feature that allows to filters results from a query according to a custom filtering process. This is a very powerful way to apply some data restrictions after a query, especially since filters can be cached and reused. Some interesting usecases are:

Hibernate Search pushes the concept further by introducing the notion of parameterizable named filters which are transparantly cached. For people familiar with the notion of Hibernate Core filters, the API is very similar.

fullTextQuery = s.createFullTextQuery( query, Driver.class );
fullTextQuery.enableFullTextFilter("bestDriver");
fullTextQuery.enableFullTextFilter("security").setParameter( "login", "andre" );
fullTextQuery.list(); //returns only best drivers where andre has credentials

In this example we enabled 2 filters on top of this query. You can enable (or disable) as many filters as you want.

Declaring filters is done through the @FullTextFilterDef annotation. This annotation can be on any @Indexed entity regardless of the filter operation.

@Entity
@Indexed
@FullTextFilterDefs( {
    @FullTextFilterDef(name = "bestDriver", impl = BestDriversFilter.class, cache=false), //actual Filter implementation
    @FullTextFilterDef(name = "security", impl = SecurityFilterFactory.class) //Filter factory with parameters
})
public class Driver { ... }

Each named filter points to an actual filter implementation.

public class BestDriversFilter extends org.apache.lucene.search.Filter {

    public BitSet bits(IndexReader reader) throws IOException {
        BitSet bitSet = new BitSet( reader.maxDoc() );
        TermDocs termDocs = reader.termDocs( new Term("score", "5") );
        while ( termDocs.next() ) {
            bitSet.set( termDocs.doc() );
        }
        return bitSet;
    }
}

BestDriversFilter is an example of a simple Lucene filter that will filter all results to only return drivers whose score is 5. The filters must have a no-arg constructor when referenced in a FulltextFilterDef.impl.

The cache flag, defaulted to true, tells Hibernate Search to search the filter in its internal cache and reuses it if found.

Note that, usually, filter using the IndexReader are wrapped in a Lucene CachingWrapperFilter to benefit from some caching speed improvement. If your Filter creation requires additional steps or if the filter you are willing to use does not have a no-arg constructor, you can use the factory pattern:

@Entity
@Indexed
@FullTextFilterDef(name = "bestDriver", impl = BestDriversFilterFactory.class) //Filter factory
public class Driver { ... }

public class BestDriversFilterFactory {

    @Factory
    public Filter getFilter() {
        //some additional steps to cache the filter results per IndexReader
        Filter bestDriversFilter = new BestDriversFilter();
        return new CachingWrapperFilter(bestDriversFilter);
    }
}

Hibernate Search will look for a @Factory annotated method and use it to build the filter instance. The factory must have a no-arg constructor. For people familiar with JBoss Seam, this is similar to the component factory pattern, but the annotation is different!

Named filters comes in handy where the filters have parameters. For example a security filter needs to know which credentials you are willing to filter by:

fullTextQuery = s.createFullTextQuery( query, Driver.class );
fullTextQuery.enableFullTextFilter("security").setParameter( "level", 5 );

Each parameter name should have an associated setter on either the filter or filter factory of the targeted named filter definition.

public class SecurityFilterFactory {
    private Integer level;

    /**
     * injected parameter
     */
    public void setLevel(Integer level) {
        this.level = level;
    }

    @Key
    public FilterKey getKey() {
        StandardFilterKey key = new StandardFilterKey();
        key.addParameter( level );
        return key;
    }

    @Factory
    public Filter getFilter() {
        Query query = new TermQuery( new Term("level", level.toString() ) );
        return new CachingWrapperFilter( new QueryWrapperFilter(query) );
    }
}

Note the method annotated @Key and returning a FilterKey object. The returned object has a special contract: the key object must implement equals / hashcode so that 2 keys are equals if and only if the given Filter types are the same and the set of parameters are the same. In other words, 2 filter keys are equal if and only if the filters from which the keys are generated can be interchanged. The key object is used as a key in the cache mechanism.

@Key methods are needed only if:

In most cases, using the StandardFilterKey implementation will be good enough. It delegates the equals/hashcode implementation to each of the parameters equals and hashcode methods.

Why should filters be cached? There are two area where filter caching shines:

Cache is enabled by default and use the notion of SoftReferences to dispose memory when needed. To adjust the size of the hard reference cache, use hibernate.search.filter.cache_strategy.size (defaults to 128). Don't forget to use a CachingWrapperFilter when the filter is cacheable and the Filter's bits methods makes use of IndexReader.

For advance use of filter caching, you can implement your own FilterCachingStrategy. The classname is defined by hibernate.search.filter.cache_strategy.

Query performance depends on several criteria:

If you wish to use some specific features of Lucene, you can always run Lucene specific queries. Check Chapter 8, Accessing Lucene natively for more informations.

It is sometimes useful to index an object even if this object is not inserted nor updated to the database. This is especially true when you want to build your index for the first time. You can achieve that goal using the FullTextSession.

FullTextSession fullTextSession = Search.createFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
for (Customer customer : customers) {
    fullTextSession.index(customer);
}
tx.commit(); //index are written at commit time    

For maximum efficiency, Hibernate Search batches index operations and executse them at commit time (Note: you don't need to use org.hibernate.Transaction in a JTA environment).

If you expect to index a lot of data, you need to be careful about memory consumption: since all documents are kept in a queue until the transaction commit, you can potentially face an OutOfMemoryException.

To avoid that, you can set up the hibernate.search.worker.batch_size property to a sensitive value: all index operations are queued until batch_size is reached. Every time batch_size is reached (or if the transaction is committed), the queue is processed (freeing memory) and emptied. Be aware that the changes cannot be rollbacked if the number of index elements goes beyond batch_size. Be also aware that the queue limits are also applied on regular transparent indexing (and not only when session.index() is used). That's why a sensitive batch_size value is expected.

Other parameters which also can affect indexing time and memory consumption are hibernate.search.[default|<indexname>].batch.merge_factor , hibernate.search.[default|<indexname>].batch.max_merge_docs and hibernate.search.[default|<indexname>].batch.max_buffered_docs . These parameters are Lucene specific and Hibernate Search is just passing these paramters through - see Section 3.7, “Tuning Lucene indexing performance” for more details.

Here is an especially efficient way to index a given class (useful for index (re)initialization):

fullTextSession.setFlushMode(FlushMode.MANUAL);
fullTextSession.setCacheMode(CacheMode.IGNORE);
transaction = fullTextSession.beginTransaction();
//Scrollable results will avoid loading too many objects in memory
ScrollableResults results = fullTextSession.createCriteria( Email.class ).scroll( ScrollMode.FORWARD_ONLY );
int index = 0;
while( results.next() ) {
    index++;
    fullTextSession.index( results.get(0) ); //index each element
    if (index % batchSize == 0) s.clear(); //clear every batchSize since the queue is processed
}
transaction.commit();

It is critical that batchSize in the previous example matches the batch_size value described previously.

It is equally possible to remove an entity or all entities of a given type from a Lucene index without the need to physically remove them from the database. This operation is named purging and is done through the FullTextSession.

FullTextSession fullTextSession = Search.createFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
for (Customer customer : customers) {
    fullTextSession.purge( Customer.class, customer.getId() );
}
tx.commit(); //index are written at commit time    

Purging will remove the entity with the given id from the Lucene index but will not touch the database.

If you need to remove all entities of a given type, you can use the purgeAll method.

FullTextSession fullTextSession = Search.createFullTextSession(session);
Transaction tx = fullTextSession.beginTransaction();
fullTextSession.purgeAll( Customer.class );
//optionally optimize the index
//fullTextSession.getSearchFactory().optimize( Customer.class );
tx.commit(); //index are written at commit time    

It is recommended to optimize the index after such an operation.

Hibernate Search can optimize automatically an index after:

The configuration can be global or defined at the index level:

hibernate.search.default.optimizer.operation_limit.max = 1000
hibernate.search.default.optimizer.transaction_limit.max = 100

hibernate.search.Animal.optimizer.transaction_limit.max = 50

An optimization will be triggered to the Animal index as soon as either:

If none of these parameters are defined, not optimization is processed automatically.

You can programmatically optimize (defragment) a Lucene index from Hibernate Search through the SearchFactory

searchFactory.optimize(Order.class);

searchFactory.optimize();

The first example reindex the Lucene index holding Orders, the second, optimize all indexes.

The SearchFactory can be accessed from a FullTextSession:

FullTextSession fullTextSession = Search.createFullTextSession(regularSession);
SearchFactory searchFactory = fullTextSession.getSearchFactory();

Note that searchFactory.optimize() has no effect on a JMS backend. You must apply the optimize operation on the Master node.

Apache Lucene has a few parameters to influence how optimization is performed. Hibernate Search expose those parameters.

Further index optimisation parameters include hibernate.search.[default|<indexname>].merge_factor, hibernate.search.[default|<indexname>].max_merge_docs and hibernate.search.[default|<indexname>].max_buffered_docs - see Section 3.7, “Tuning Lucene indexing performance” for more details.

The SearchFactory object keeps track of the underlying Lucene resources for Hibernate Search, it's also a convenient way to access Lucene natively. The SearchFactory can be accessed from a FullTextSession:

FullTextSession fullTextSession = Search.createFullTextSession(regularSession);
SearchFactory searchFactory = fullTextSession.getSearchFactory();

You can always access the Lucene directories through plain Lucene, the Directory structure is in no way different with or without Hibernate Search. However there are some more convenient ways to access a given Directory. The SearchFactory keeps track of the DirectoryProviders per indexed class. One directory provider can be shared amongst several indexed classes if the classes share the same underlying index directory. While usually not the case, a given entity can have several DirectoryProviders is the index is sharded (see Section 3.2, “Index sharding”).

DirectoryProvider[] provider = searchFactory.getDirectoryProviders(Order.class);
org.apache.lucene.store.Directory directory = provider[0].getDirectory();

In this example, directory points to the lucene index storing Orders information. Note that the obtained Lucene directory must not be closed (this is Hibernate Search responsibility).

Queries in Lucene are executed on an IndexReader. Hibernate Search caches such index readers to maximize performances. Your code can access such cached / shared resources. You will just have to follow some "good citizen" rules.

DirectoryProvider orderProvider = searchFactory.getDirectoryProviders(Order.class)[0];
DirectoryProvider clientProvider = searchFactory.getDirectoryProviders(Client.class)[0];

ReaderProvider readerProvider = searchFactory.getReaderProvider();
IndexReader reader = readerProvider.openReader(orderProvider, clientProvider);

try {
    //do read-only operations on the reader
}
finally {
    readerProvider.closeReader(reader);
}

The ReaderProvider (described in Reader strategy), will open an IndexReader on top of the index(es) referenced by the directory providers. This IndexReader being shared amongst several clients, you must adhere to the following rules:

Aside from those rules, you can use the IndexReader freely, especially to do native queries. Using the shared IndexReaders will make most queries more efficient.