Terminology

• DynaGroups - a mechanism enabling the automatic management of one or more group memberships based off of conjunctive ('and'), hierarchical search expressions; requires inventory manager permission

• Criteria Queries - a mechanism enabling search for entities matching simple filters; also supports paging / sorting, fetching related data, fuzzy matching, conjunctive ('and') as well as disjunctive ('or') condition sets, respects authorization

API Quick Reference

<resource-expr> [ <string-match-expr> ] = <value>
"groupby" <resource-expr>

empty <resource-expr> [ <string-match-expr> ]
not empty <resource-expr> [ <string-match-expr> ]

resource.id
resource.name
resource.version
resource.type.plugin
resource.type.name
resource.type.category
resource.availability
resource.pluginConfiguration[<property-name>]
resource.resourceConfiguration[<property-name>]
resource.trait[<property-name>]

resource.child
resource.parent
resource.grandParent

class Criteria {

    public void setCaseSensitive(boolean caseSensitive);
    public void setFiltersOptional(boolean filtersOptional); // disjunctive
    public void setStrict(boolean strict); // fuzzy-match

    public void setPageControl(PageControl pageControl);
    public void setPaging(int pageNumber, int pageSize);
}

class EntityCriteria extends Criteria {

    public void addFilterXXX(Type value);

    public void addSortXXX(PageOrdering columnOrdering);

    public void fetchXXX(boolean fetchRelationship);

}

Enhancements

DynaSearch will improve upon what DynaGroups started by expanding the syntax to including a wider array of searchable items as well as allow them to be coupled in various ways.

• Arbitrary boolean expressions including conjunctive and disjunctive operators supporting arbitrary levels of parenthetical nesting; 'AND' operator is implied between terms; 'OR' operator can use '|' or the literal word 'OR'; implicit 'AND' operator takes precedence over 'OR' operator if not explicitly grouped by parentheses

  • (a | b) (c or d), valid parenthetical expression

  • (a or b) (c | (d e)), valid two-level parenthetical expression

• Each term needs to be able to control case-sensitivity individually

  • expression = value, case-insensitive comparison

  • expression == value, case-sensitive comparison

  • expression != value, negated case-insensitive comparison

  • expression !== value, negated case-sensitive comparison

• Support for regex-like boundary matches, compatible with case sensitive or insensitive operators

  • expression = ^prefix, search for values beginning with the text 'prefix'

  • expression = suffix$, search for values ending in the text 'suffix'

  • expression = ^exact-match$, search for values which match the text 'exact-match' exactly

  • expression = inner, search for values which start with, end with, or contain the text 'inner'

• Null searches

  • expression = null, previously implemented as "empty <expression>"

  • expression != null, previously implemented as "not empty <expression>"

    • Note: if null is used with quotes as in "expression = 'null'", it is instead interpreted as a literal string instead

• Simplified syntax

  • "resource" token is implied, thus you say "name" instead of "resource.name"

  • All expressions reduced to a single word:

    • "type.plugin" becomes "plugin"

    • "type.name" becomes "type"

    • "type.category" becomes "category"

    • "pluginConfiguration" may become "connection"

    • "resourceConfiguration" may become "configuration"

• Improved hierarchical control (out of scope for RHQ

  • Support for "child[x]", where x > 0

  • Support for "parent[x]", where x > 0

  • Support for "platform" (shortcut for the platform ancestor managed by the agent which is also managing the resource being searched for)

Group Search

As it stands today, the resource browser is the primary display for both resources and groups. Accordingly, we might want to write a second (perhaps simplified) search solution for groups. Suggested tokens would be:

• group.name

• group.plugin (if compatible, the plugin defining the resource type for this group)

• group.type (if compatible, the resource type for this group)

• group.category ('platform', 'server', 'service')

• group.kind ('mixed' or 'compatible')

• group.availability (UP, DOWN, perhaps support a numeric number with units, i.e. 80%)

However, similar to the resource-specific search terms, the "group" prefix should be strictly optional. Thus, if you're on the group tab and you search for "category = Platform" it will search for compatible groups containing any platform.

Resource-Group Related Search

From a resource perspective, it might be nice to search for resources that are in a group with a particular name. From a group perspective, it might be nice to search for all groups that a particular resource is in. To accomplish this, both the resource search syntax and group search syntax could be supported in both views.

From the resource view: "group.availability = DOWN" would give me all of the resources I have visibility to that are members of groups where all of their members are down.

From the group view: "resource.connection['principal'] is null" would give me all of the groups I have visibility to that contain any resources that I haven't secured with principal/credentials yet.

Tag Search

The ability to search across tags should be exposed both to resources and groups. The full, explicit syntax will be as follows:

• resource.tag

• group.tag

Remember, if you're searching with the resource browser the 'resource' prefix will be implied. Likewise, if you're searching within the group browser the 'group' prefix will be implied.

Relationship Search

The RHQ system will support the ability to link resources in ways that can not be expressed via the standard hierarchical resource model. For instance, when an EAR is deployed to a JBossAS server, the SLSBs (stateless session beans) get discovered as siblings of the EAR under the server, even though they have a logical dependency on the deployed EAR. In this case, two unidirectional relationship definitions would be setup between the types:

• SLSB->"deployedUnder"->EAR

• EAR->"deploys"->SLSB

The relationship definition that links the SLSB resource type to the EAR resource type is called "deployedUnder". It is these relationship definition names that will be searchable. The syntax will look as follows:

resource.relationship = 'deployedUnder'

You can combine this search expression with others like:

(relationship = deployedUnder) and (name ~= myapp*) and (type = EAR)

So this will find all of the resources that are 'deployedUnder' enterprise application archives prefixed with 'myapp' (case-insensitively).

Simplified Search Syntax

Some may find the search syntax provided above a bit verbose. The auto-completion (see below) mechanism should mitigate that concern a bit, but it might be possible to support an alternate format which would be a condensed subset of the full search syntax.

Note: the type filter variant might help to make the transition from those familiar with RHEV syntax smoother.

The issue with supporting syntax variants is that they increase the cost of maintenance for the query parser as well as auto-completion mechanisms. The variant, by the fact that it seeks to make the syntax more concise also makes it denser. A denser syntax will lessen the impact of auto-completion, because at each completion point the number of possible matches explodes (relative to the more verbose but much more restrictive full syntax).

Pattern Grouping (for DynaGroups only)

This feature takes the concept of wildcard searches and applies them to pivoted (groupby) expressions. Often it is the case that resource names encode information that is specific to a particular company's topology, business tier, data center, etc. For instance, one organization might choose hostnames like 'DC001R001N001' to denote the datacenter, rack, and node information. Then, to create an expression which automatically creates one group for every datacenter, use the following syntax:

groupby substring(resource.name, 0, 5)

An alternate syntax might look like:

groupby substring(0,5) resource.name

Since simple substring functions are implemented in every major database (without requiring a vendor-specific package to be installed) we'll be able to implement the translated query in pure JPQL as follows:

  select substring(res.name, 0, 5) as datacenter
    from Resource res
group by datacenter

Though, due to a bug in Hibernate, the generated SQL will not properly alias the results of a projection of a string function. Thus, the following workaround JPQL will have to be used:

  select substring(res.name, 0, 5)
    from Resource res
group by substring(res.name, 0, 5)

Sorting Completion Results

• Frequency - number of times this search was retrieved / accessed by the user, can optionally show the absolute count or a mini pie chart for frequency

• Relevancy - saved searches that might be related to the pattern currently in the search window (not trivial!)

Syntax Highlighting

As the user types the search expression, coloring the boolean words will add clarity to the search terms by delimiting their boundaries clearly. Syntax highlighting can be seen on the accompanying UI mockup.

Value Auto-Completion

All search terms have the basic structure "<filter> <operator> <value>". Sytnax auto-completion is meant to fill in that basic structure including the known filters and operators. The values, however, are a different story. Some filters have a small set of known values, such as availability which can only be UP or DOWN. Other filters, such as the name of a resource, can be an arbitrary set of characters. Value auto-completion would be a mechanism to identify that a user is searching by one of the filtering with a known set of values, and would offer substring completion on it. For example, "EAR" would have an auto-completion result of "Enterprise Application (EAR)" if the user was filtering on "type".

JIRA

http://www.atlassian.com/software/jira/docs/v3.13/querysyntax.html JIRA uses Lucene for text indexing. Most is derived from the 2.2.0 version of http://jakarta.apache.org/lucene/2_2_0/queryparsersyntax.html Lucene Query Parser Syntax.

A query is broken up into terms and operators. There are two types of terms: Single Terms and Phrases. A Single Term is a single word such as "test" or "hello". A Phrase is a group of words surrounded by double quotes such as "hello dolly". Multiple terms can be combined together with Boolean operators to form a more complex query.

• Supports single ('?') and multiple ('*') character wildcard searches

• Note: wildcards cannot be the first character of a search

• Add the tilde ('~') at the end of a Single word Term to fuzzy search

• i.e., roam~ will find terms like foam and roams

• Add the tilde ('~') following by a number at the end of Phase to proximity search

• i.e., "atlassian jira"~10 will find documents where 'atlassian' and 'jira' are within 10 words of each other

• The caret ('^') symbol will boost Single word terms or Phrases to change the relevance level of document matches

• i.e., "atlassian jira"3 querying.2

• The defeault boost factor for all Single word terms and Phrases is 1

• Note: although the boost factor must be positive, it can be fractional (i.e., .2)

• Boolean operations must be all caps

• AND, "+"

• A required term

• OR, "||"

• This is the default operator is none exists between terms

• NOT, "!", "-"

• Note: the NOT operator cannot be used with just one term

• Parenthetical Grouping '(' and ')' can be used to add precedence to boolean expressions

• i.e., bugs AND (atlassian OR jira)

• Note: '(' can not begin a search expression

• Escaping special characters require the backslash ('\') before the character

• Special character set: {{+ - && || ! ( ) { } [ ] ^ " ~ * ? }}

Lucene

Sytnax Reference Guide is http://jakarta.apache.org/lucene/2_2_0/queryparsersyntax.html here.

A query is broken up into terms and operators. There are two types of terms: Single Terms and Phrases. A Single Term is a single word such as "test" or "hello". A Phrase is a group of words surrounded by double quotes such as "hello dolly". Multiple terms can be combined together with Boolean operators to form a more complex query.

• Everything from the JIRA syntax section above

• The colon (':') separates field names from the value search

• i.e., title:"The Right Way" AND go, since 'text' is the default text, the indicator was not required

• Range queries allow matching documents between inclusive or exclusive boundaries

• i.e., mod_date:[20020101 TO 20030101], inclusive search on mod_date field

• i.e., title:{Aide TO Carmen}, search on titles between Aide and Carmen, but not including Aida and Carmen

• Field grouping allows multiple clauses related to a single field

• i.e., title(+return +"pink panther") to search for titles containing the word "return" and the phrase "pink panther"

• Lucene has all of the special characters from JIRA as well as ':'

Solr

The standard http://wiki.apache.org/solr/SolrQuerySyntax Solr Query Parser syntax is a superset of the 2.4.0 version of http://lucene.apache.org/java/2_4_0/queryparsersyntax.html Lucene Query Parser Syntax. The differences are as follows:

• Range, prefix, and wildcard queries are constant-scoring; scoring factors tf, idf, index boost, and coord are not used (Lucene 2.1 also switched to ConstantScoreRangeQuery)

• There is no limit on the number of terms that match

• Range queries can use wildcards for either boundary

• i.e., field:[* TO 100] finds all field values less than or equal to 100

• Pure negative queries, where all clauses are prohibited, are allowed

• i.e., -inStock:false finds all field values where inStock is not false

• Extended grammar support for http://wiki.apache.org/solr/FunctionQuery FunctionQuery

• i.e., val:myCustomFunction(myfield)

• Support for nested queries via bracketed delimiters

• i.e., query:"{!dismax qf=myfield}how now brown cow"

• http://wiki.apache.org/solr/DisMaxRequestHandler DisMaxRequestHandler

• lightweight syntax for handling word search across several fields

• disjunction across all words in the phrase

• simplified subset of the Lucene QueryParser syntax, tailored for user-entered phrases without heavy syntax

Hibernate Search

This search technology is a thin integration layer on top of Lucene, where you pass the query to it and you can define the projection, result transformer/mapper, fetching rules, pagination and sorting. Its focus is integration with Lucene, not to define its own syntax variant. Below are some code examples:

// general integration
FullTextSession ftSession = Search.getFullTextSession(session);

// example 1
// all indexed entities will be searched, unless restricted
org.hibernate.search.FullTextQuery query =
   ftSession.createFullTextQuery(luceneQuery, Book.class, Magazine.class, Article.class);
query.setFirstResult(15); // pagination, start from 15th
query.setMaxResults(10); // pagination, return 10 elements
org.apache.lucene.search.Sort sort = new Sort(new SortField("title"));
query.setSort(sort); // sorting
List results = query.list();

// example 2
// if type restricted to a single class, you can customize the fetch for a specific use case
Criteria criteriaQuery = ftSession.createCriteria(Book.class);
criteriaQuery.setFetchMode("authors", FetchMode.JOIN); // preload related data
org.hibernate.search.FullTextQuery query = ftSession.createFullTextQuery(luceneQuery);
query.setCriteriaQuery(criteria);

// example 3
// returning direct data, instead of entities using Lucene search
org.hibernate.search.FullTextQuery query = ftSession.createFullTextQuery(luceneQuery, Book.class);
query.setProjection("id", "summary", "body", "mainAuthor.name");
List<Object[]> results = query.list();
Object[] firstResult = (Object[]) results.get(0);
Integer id = firstResult[0];
String summary = firstResult[1];
String body = firstResult[2];
String authorName = firstResult[3];

Confluence

Marketing Summary

• Everything in Confluence is searchable — pages, blogs, news items, comments, emails, attachments (Microsoft Word/Excel/PowerPoint documents; PDF files; HTML, XML, source and text files; ZIP files), and people

• Confluence's 'Did you mean' feature analyses your search term and suggests an alternative spelling to give you more relevant search results

• Narrow the search down to exactly what you are looking for — search by space, type of space (e.g. exclude all personal spaces), the type of content and/or the date it was last modified

• Saved searches - can embed live search directly into any page

Usage

Quick Search - find all content (except mail) in all global spaces, display the results in new screen
Advanced Search - location (within a particular space), group by (content type or space), type, date

Search Fields

Syntax

Supports Lucene-style queries for...

Red Hat Enterprise Virtualization (RHEV)

expression := resultType ':' criteria '[' sortby sortSpec ']'
resultType := 'VMS' | 'Host' | 'Template' | 'Event' | 'Cluster' | 'Datacenter' | 'Storage'
criteria := [objectType '.'] property operator value
operator := '=' | '!=' | '>' | '<' | '>=' | '<='
value := <wildcardString> | <integer> | <enumeration> | <date>
sortSpec := sortField 'asc' | sortField 'desc'
sortField := <string>