Release Notes for ModeShape 3.1.0.Final

The ModeShape 3.1.0.Final release is the second stable release of our new architecture. It's been a long journey with fourteen different alphas, betas, and candidate releases. But this is a huge improvement over the 2.x series. We hope you enjoy it!

What's new

3.1.0.Final provides a fast, distributed, hierarchical database that clients work with via the standard JCR 2.0 (JSR-283) API. ModeShape 3 is a major upgrade over 2.x and offers significant improvements in performance and scalability, while retaining all of ModeShape 2's JCR-related features. ModeShape 3 has complete integration with JBoss AS 7.1, allowing deployed components to simply lookup and use repositories managed by ModeShape's service.

This release brings these improvements:

• Federated content that exists in external systems and project it into the repository as regular content. Several connectors are provided out of the box: a file system connector (very similar to what was in 2.x) that accesses files and folders on the file system and projects them as 'nt:file' and 'nt:folder' nodes; and a Git connector that accesses a local Git repository (can be a clone of one or more remotes) and projects the branches, tags, commits and trees within the Git repository as a node structure in the repository.
• New support for the CMIS API. This is still a 'technology preview', and we're seeking users that can try it out and give us feedback.
• New JCA adapter to simplify deployment to application servers and containers other than JBoss AS and EAP. (Recall that ModeShape can already be deployed as a service within AS7/EAP6).
• Separately control the size of strings and binary values that are stored in the binary store. This repositories to always store binary values in binary storage, but only larger string values.
• Improved how repository configurations can also configure JGroups.
• Clustering bug fixes and improvements.
• Compatibility with Infinispan 5.1.2.FINAL through 5.1.8.Final. Expected to work with Infinispan 5.2.0.Final (when available).
• Fixes to how variables are handled in configuration files.
• Kit to install ModeShape into an existing JBoss AS7.1.1 installation now also installs CMIS API for all repositories. Configure connectors along with all the other components.
• Improved support for very large numbers of child nodes under a single parent, including when these nodes are projected from connectors.
• Over 50 issues (bugs, tasks, features, etc.) resolved in this release.

Features

ModeShape 3.1.0.Final has changed a lot since ModeShape 2.8.x:

• ModeShape uses Infinispan for all caching and storage, giving a powerful and flexible foundation for creating JCR repositories that are fast, scalable, and highly available. Infinispan offers a great deal of storage options (via cache loaders), but using Infinispan as a distributed, mulit-site, in-memory data grid provides incredible scalability and performance.
• Improved performance. ModeShape 3 is just plain seriously fast, and performance is all-around faster than 2.x - most operations are at least one if not several orders of magnitude faster! We'll publish performance and benchmarking results soon.
• Improved scalability. ModeShape 3 has been designed to store and access the content so that a node can have hundreds of thousands (or more!) of child nodes (even with same-name-siblings) yet still be incredibly fast. Additionally, repositories can scale to many millions of nodes and be deployed across many processes.
• Improved configuration. There is no more global configuration of the engine; instead, each repository is configured with a separate JSON file, which must conform to a ModeShape-specific JSON Schema and can be validated by ModeShape prior to use. Repository configurations can even be changed while the repository is running (some restrictions apply), making it possible to add/change/remove sequencers, authorization providers, and many other configuration options while the repository is in use.
• Each repository can be deployed, started, stopped, and undeployed while the engine and other repositories are still in use.
• Sessions immediately see all changes persisted/committed by other sessions, although transient changes made by the session always take precedence.
• Support for participation in JTA transactions, allowing (container-managed or bean-managed) EJBs and JCR clients that programmatically use transactions to commit the changes in the transactions.
• Monitoring API with over a dozen metrics.
• Sequencing SPI that uses the JCR API to get at the content being processed and create/update the derived content. Sequencers can also dynamically register namespaces and node types. Now it's easy to create custom sequencers.
• Connector SPI that defines how external systems are accessed and optionally updated to project the external information into the repository as regular nodes.
• Simplified API for implementing custom MIME type detectors. ModeShape still has a built-in Tika-based detector that determines MIME types using the filename extensions and binary content.
• New and simpler API for implementing custom text extractors, which extracts from binary values searchable text used in full-text searches and queries.
• Improved storage of binary values of all sizes, with a separate facility for storing these on the file system, in Infinispan caches, in relational DBMSes (via JDBC), and in MongoDB. Custom stores are also possible.
• Public API interfaces and methods that were deprecated in 2.7.0.Final (or later) have been removed. There weren't many of these; most of the ModeShape API remains the same as 2.x.
• Integration with JBoss AS 7. ModeShape runs as an integrated subsystem within AS7, and the AS7 tooling can be used to define and manage repositories independently of each other and while the server is running.
• Local and remote JDBC drivers for issuing JCR-SQL2 queries and getting database metadata via the JDBC API

All of the JCR features previously supported in 2.x are working and ready for use. If any issues are found, please log a bug report in our JIRA.

Accessing the Repository

• RepositoryFactory access
• JNDI registration of Repository
• JAAS Authentication
• Servlet Authentication
• Custom Authentication

Namespaces

• Session Remapping
• Permanent Addition/Deletion

Reading Repository Content

• Traversal Access
• Direct Access
• Same-Name Siblings
• Multi-Value Properties
• All Property Types Supported
• Property Type Conversion

Writing Repository Content

• Create/Update/Delete Nodes
• Create/Update/Delete Properties (Through Parent Nodes)
• Moving, Copying, Cloning
• Adding/Removing Mixins
• Referential integrity enforcement

Query / Search

• XPath
• JCR-SQL
• JCR-SQL2
• JCR-QOM
• Full-Text Search

Importing/Exporting Repository Content

• System View Import/Export
• Document View Import/Export

Node Types

• Inheritance Among Node Types
• Discovering available Node Types
• Discovering the Node Types of a Node
• Discovering the Definition of a Node Type
• Property Constraints
• Automatic Item Creation
• Predefined standard Node Types
• Custom Node Type Registration (CND-Based and and JCR 2.0 API Template-Based)

Repository Metadata under System Node

• Permanent Namespace Mappings
• Node Types (Built-In and User-Registered)
• Active Locks

Other JCR Optional Features

• Observation
• Locking
• Versioning
• Shareable nodes

Content Storage Options

• In-memory (local, replicated, and distributed)
• BerkleyDB
• Relational databases (via JDBC), including in-memory, file-based, or remote
• File system
• Cassandra
• Cloud storage (e.g., Amazon's S3, Rackspace's Cloudfiles, or any other provider supported by JClouds)
• Remote Infinispan

Binary Storage Options

• File system
• JDBC database
• Infinispan
• MongoDB

ModeShape Federation Connectors

• File system connector (read and write)
• Git repository connector (read-only)

ModeShape Sequencers

• Compact Node Definition (CND) Sequencer
• DDL Sequencer
• Image Sequencer
• Java Source Sequencer
• Java Class Sequencer
• MP3 Sequencer
• MS Office Sequencer
• Text Sequencers (Delimited and Fixed Width)
• XML Sequencer
• XML Schema Document (XSD) Sequencer
• Web Service Definition Lanaguage (WSDL) 1.1 Sequencer
• Zip File Sequencer (also WARs, JARs, and EARs)
• Teiid Relational Model Sequencer (coming soon)
• Teiid VDB Sequencer (coming soon)

ModeShape Deployment/Access Models

• JNDI-Based Deployment
• Deploy as a subsystem in JBoss AS7, with access to repositories via @Resource injection
• Deploy to other containers using ModeShape's JCA adapter
• Access through two RESTful Services (the 2.x-compatible API and a new improved API)
• Access through WebDAV Service
• Access through CMIS Service
• Local and remote JDBC drivers for accessing ModeShape content through JDBC API and JCR-SQL2 queries
• Embedded (in Server or JEE Archive) Deployment
• JTA support, allowing Sessions to participate in XA and container-managed transactions
• OSGi-Compatible Archives

Other ModeShape features

• Repository-wide backup and restoration
• Automatic MIME type detection of binary content
• Asynchronous sequencing operations, within completion notified via events

One feature not related to the JCR 2.0 API is enabling a repository to access the content in external systems (e.g., file system, SVN, JDBC, JCR, etc.). This feature was in 2.x but has been pushed to 3.1 (roughly 6 weeks after 3.0.0.Final) so that we could release 3.0 earlier with full support for JCR 2.0.

Bug Fixes, Features, and other Issues

The following are the bugs, features and other issues that have been fixed in the 3.1.0.Final release:

Bug

• MODE-1693 - Starting repository without transactions results in NPE
• MODE-1750 - FullTextSearch Left/Right outer join invalid behaviour on null join value
• MODE-1751 - Updating reference with already assigned node brings referential integrity exceptions
• MODE-1738 - SELECT ... JOIN ... ORDER BY array index out of bounds exception
• MODE-1739 - Data corruption when starting up another cluster instance
• MODE-1715 - Variable replacement not working in ModeShape
• MODE-1717 - SystemFailureException: Unable to initialize the '/jcr:system' content in repository 'sample'
• MODE-1733 - Problems with defining node types in clustered environment with eviction enabled
• MODE-1731 - Single-valued property is created when the property definition is actually multi-valued
• MODE-1737 - SELECT DISTINCT with JOIN results in class cast exception
• MODE-1734 - ModeShape loses nodes during concurrency
• MODE-1748 - Importing XML throws VersionException "node is checked in, preventing this action"
• MODE-1743 - Maven Scopes
• MODE-1742 - Git Connector - Certain commits cannot be parsed and therefore returned
• MODE-1747 - Default configuration for Infinispan used in workspace caches doesn't work well with Infinispan 5.2
• MODE-1746 - Deadlock when writing same nodes in different order using separate threads
• MODE-1745 - NullPointerException while running ModeShape embedded in spring with TransactionManager in clustered mode
• MODE-1744 - rename node with move() fails without SNS enabled
• MODE-1474 - Custom analyzer cannot be specified when running in AS7
• MODE-1721 - Session.move doesn't seem to work when same name siblings are not allowed
• MODE-1720 - Creating REFERENCE values by using Node.getIdentifier() strings results in unusable properties
• MODE-1722 - Unable to create property of type 'URI' using CND format
• MODE-1700 - Teiid Model Sequencer Throws Error When Model Import Has Annotation

Feature Request

• MODE-1487 - Add support in the Teiid sequencer for new VDB manifest (vdb.xml) features supported by Teiid 8.0
• MODE-1699 - Allow for separate string minimum size to store in binary store configuration
• MODE-1755 - Expose a "getPlan()" method on QueryResult
• MODE-1756 - Change copyright dates to 2013
• MODE-1445 - Store binary files in a MongoDB database
• MODE-295 - Content Management Interoperability Services API (a.k.a CMIS) implementation
• MODE-1344 - Support the shredding of the new Model Extension properties being added to the Teiid Models
• MODE-1513 - Add support for federating data from external systems
• MODE-1499 - Provide JCA Resource Adapter
• MODE-1703 - EOFException with clustered Modeshape3
• MODE-1724 - Provide Easier Way For The NamespaceRegistry To Indicate The Existence Of Namespace
• MODE-1292 - Support accessing file system content

Task

• MODE-1702 - Cleanup Corrupted Teiid Sequencer VDB Test Data
• MODE-1714 - Improve federation paging SPI
• MODE-1736 - Correct JCA module's build
• MODE-1741 - Only "tree/master" area of Git connector's content should be queryable
• MODE-1701 - Improve JGroups configuration mechanism

Sub-task

• MODE-1713 - Connector SPI's "update" method should inform connector of changes to documents/nodes
• MODE-1735 - Handle removing of projections and federated nodes
• MODE-1711 - Specify projections in the repository configuration
• MODE-1716 - The federated NodeKey - External Projection Documentid mappings should be persistent
• MODE-1732 - Binary values from a connector get lost unless the connector writes them to the repository's binary store
• MODE-1286 - Implement Git connector
• MODE-1534 - The ModeShape AS7 kit should not overwrite the Infinispan main module
• MODE-1729 - The JBoss AS7 kit should support federation configuration
• MODE-1728 - Connectors should be able to enroll in existing transactions
• MODE-1708 - External (federated) content should be indexed to allow it to be queried
• MODE-1705 - Create initial federation framework and connector SPI
• MODE-1706 - New file system connector