Release Notes for ModeShape 3.2.0.Final

This document outlines the changes that were made in ModeShape 3.2.0.Final. We hope you enjoy it!

What's new

3.2.0.Final provides a fast, elastic, distributed hierarchical database that clients work with via the standard JCR 2.0 (JSR-283) API. ModeShape 3 is a major upgrade over 2.x and offers significant improvements in performance and scalability, while retaining all of ModeShape 2's JCR-related features. ModeShape 3 has complete integration with JBoss AS 7.1, allowing deployed components to simply lookup and use repositories managed by ModeShape's service.

This release addesses a whopping 119 issues, most of which were bug fixes. Overall, clustering has been dramatically improved, both for embedded and EAP-based deployments. Other high-level changes include using JBoss EAP 6.1 (rather than JBoss AS7.1.1) and upgrades to Infinispan 5.2.5.Final, JGroups 3.2.7.Final, Lucene 3.6, and newer versions of several other third party libraries.

Note for EAP 6.1.0.Alpha1 users: there is a known issue (https://issues.jboss.org/browse/AS7-6639) where EAP intermittenly has problems upon startup concurrently starting Infinispan cache containers. If this occurs, simply kill the EAP process and restart. This has been fixed in EAP 6.1.0.Beta1.

Also, note that because we switched from AS7 to EAP, the artifact name for BOM for use in applications has been renamed from 'modeshape-bom-jbossas' to 'modeshape-bom-jbosseap'. This is to remain consistent and to help reinforce that ModeShape 3.2 works on EAP 6.1 now and not on JBoss AS7.1.1.

Features

ModeShape 3.2.0.Final has these features:

• ModeShape uses Infinispan for all caching and storage, giving a powerful and flexible foundation for creating JCR repositories that are fast, scalable, and highly available. Infinispan offers a great deal of storage options (via cache loaders), but using Infinispan as a distributed, mulit-site, in-memory data grid provides incredible scalability and performance.
• Strongly consistent. ModeShape is atomic, consistent, isolated and durable (ACID), so writing applications is very natural. Applications can even use JTA transactions.
• Fast. ModeShape 3 is just plain seriously fast, and performance is all-around faster than 2.x - most operations are at least one if not several orders of magnitude faster! We'll publish performance and benchmarking results soon.
• Larger content. ModeShape 3 has been designed to store and access the content so that a node can have hundreds of thousands (or more!) of child nodes (even with same-name-siblings) yet still be incredibly fast. Additionally, repositories can scale to many millions of nodes and be deployed across many processes.
• Simplified configuration. There is no more global configuration of the engine; instead, each repository is configured with a separate JSON file, which must conform to a ModeShape-specific JSON Schema and can be validated by ModeShape prior to use. Repository configurations can even be changed while the repository is running (some restrictions apply), making it possible to add/change/remove sequencers, authorization providers, and many other configuration options while the repository is in use.
• Elastic. Add processes to scale out, without having to have a single coordinator.
• Deploy, start, stop and undeploy repositories while the engine is running and while and other repositories are still in use.
• Sessions immediately see all changes persisted/committed by other sessions, although transient changes made by the session always take precedence.
• Monitoring API with over a dozen metrics.
• Sequencing SPI that uses the JCR API to get at the content being processed and create/update the derived content. Sequencers can also dynamically register namespaces and node types. Now it's easy to create custom sequencers.
• Connector SPI that defines how external systems are accessed and optionally updated to project the external information into the repository as regular nodes.
• Simple API for implementing custom MIME type detectors. ModeShape still has a built-in Tika-based detector that determines MIME types using the filename extensions and binary content.
• Simple API for implementing custom text extractors, which extracts from binary values searchable text used in full-text searches and queries.
• Ability to store binary values of any sizes, with a separate facility for storing these on the file system, in Infinispan caches, in relational DBMSes (via JDBC), and in MongoDB. Custom stores are also possible.
• Public API interfaces and methods that were deprecated in 2.7.0.Final (or later) have been removed. There weren't many of these; most of the ModeShape API remains the same as 2.x.
• Integration with JBoss AS 7. ModeShape runs as an integrated subsystem within AS7, and the AS7 tooling can be used to define and manage repositories independently of each other and while the server is running.
• Local and remote JDBC drivers for issuing JCR-SQL2 queries and getting database metadata via the JDBC API
• Use the RESTful API to talk to ModeShape repositories from non-Java and non-local applications
• Use the CMIS API to talk to ModeShape repositories
• Use WebDAV to mount ModeShape repositories as file servers

All of the JCR 2.0 features previously supported in 2.x are currently supported:

Accessing the Repository

• RepositoryFactory access
• JNDI registration of Repository
• JAAS Authentication
• Servlet Authentication
• Custom Authentication

Namespaces

• Session Remapping
• Permanent Addition/Deletion

Reading Repository Content

• Traversal Access
• Direct Access
• Same-Name Siblings
• Multi-Value Properties
• All Property Types Supported
• Property Type Conversion

Writing Repository Content

• Create/Update/Delete Nodes
• Create/Update/Delete Properties (Through Parent Nodes)
• Moving, Copying, Cloning
• Adding/Removing Mixins
• Referential integrity enforcement

Query / Search

• XPath
• JCR-SQL
• JCR-SQL2
• JCR-QOM
• Full-Text Search

Importing/Exporting Repository Content

• System View Import/Export
• Document View Import/Export

Node Types

• Inheritance Among Node Types
• Discovering available Node Types
• Discovering the Node Types of a Node
• Discovering the Definition of a Node Type
• Property Constraints
• Automatic Item Creation
• Predefined standard Node Types
• Custom Node Type Registration (CND-Based and and JCR 2.0 API Template-Based)

Repository Metadata under System Node

• Permanent Namespace Mappings
• Node Types (Built-In and User-Registered)
• Active Locks

Other JCR Optional Features

• Observation
• Locking
• Versioning
• Shareable nodes

Content Storage Options

• In-memory (local, replicated, and distributed)
• BerkleyDB
• Relational databases (via JDBC), including in-memory, file-based, or remote
• File system
• Cassandra
• Cloud storage (e.g., Amazon's S3, Rackspace's Cloudfiles, or any other provider supported by JClouds)
• Remote Infinispan

Binary Storage Options

• File system
• JDBC database
• Infinispan
• MongoDB

ModeShape also has features that go beyond the JCR API:

ModeShape Federation Connectors

• File system connector (read and write)
• Git repository connector (read-only)
• CMIS reposiotry connector (read and write, tech preview)

ModeShape Sequencers

• Compact Node Definition (CND) Sequencer
• DDL Sequencer
• Image Sequencer
• Java Source Sequencer
• Java Class Sequencer
• MP3 Sequencer
• MS Office Sequencer
• Text Sequencers (Delimited and Fixed Width)
• XML Sequencer
• XML Schema Document (XSD) Sequencer
• Web Service Definition Lanaguage (WSDL) 1.1 Sequencer
• Zip File Sequencer (also WARs, JARs, and EARs)
• Teiid Relational Model Sequencer
• Teiid VDB Sequencer

ModeShape Deployment/Access Models

• JNDI-Based Deployment
• Deploy as a subsystem in JBoss AS7, with access to repositories via @Resource injection
• Deploy to other containers using ModeShape's JCA adapter
• Access through two RESTful Services (the 2.x-compatible API and a new improved API)
• Access through WebDAV Service
• Access through CMIS Service
• Local and remote JDBC drivers for accessing ModeShape content through JDBC API and JCR-SQL2 queries
• Embedded (in Server or JEE Archive) Deployment
• JTA support, allowing Sessions to participate in XA and container-managed transactions
• OSGi-Compatible Archives

Other ModeShape features

• Repository-wide backup and restoration
• Automatic MIME type detection of binary content
• Asynchronous sequencing operations, within completion notified via events

Bug Fixes, Features, and other Issues

The following are the bugs, features and other issues that have been fixed in the 3.2.0.Final release:

Bug

• MODE-1830 - Modeshape not seeing changes made on other cluster nodes, unless doing write operation
• MODE-1833 - JQOM Wildcard Query fails with Column 'null' does not exist on the table
• MODE-1832 - ModeShape CMIS component has bug in JcrService
• MODE-1834 - Can't create new workspace at runtime in using JBoss Transactions
• MODE-1837 - Sometimes query returns duplicated records after commiting a transaction that contains VersionManager.checkin() call when using a real JTA transaction manager
• MODE-1836 - Modeshape clustering problem with ISPN eviction enabled
• MODE-1787 - Out of Memory error when performing bulk load
• MODE-1823 - Node Name with german Character :webdav and rest navigation to this node doesn't work?
• MODE-1785 - The query index rebuild configuration option doesn't work
• MODE-1821 - Nodes with the same name can be written to non-SNS node
• MODE-1783 - Property Constraint not checked if property already existed
• MODE-1827 - NullPointerException in RepositoryQueryManager
• MODE-1824 - AssertionError in Query Execution within QueryResultColumns
• MODE-1825 - NullPointerException in CanonicalPlanner when creating QOM query
• MODE-1828 - ConstraintViolationException when updating the definition of the same node in a session
• MODE-1829 - Full text search incorrect behaviour when wildcard is before the text
• MODE-1789 - "IllegalStateException: No match found" during parsing of config with federation
• MODE-1798 - Preconfigured projections should not be created if no alias is given or if "/" is used as an alias
• MODE-1799 - can't move node under root
• MODE-1790 - InfinispanBinaryStore does not clean up temporary files
• MODE-1791 - Correct "readonly" attribute for external sources (including in the AS7 kit)
• MODE-1793 - OutOfMemoryError when loading more than several hundred thousand nodes in AS7.1.1 using BDBJE
• MODE-1794 - Modeshape clustering trying to serialize InfinispanBinaryStore
• MODE-1795 - ModeShape does not allow importing a XML file that declares a default XML namespace
• MODE-1796 - Garbage collection doesn't work in RemoteCachestore
• MODE-1817 - NullPointerException when executing session.save()
• MODE-1816 - REST - Create node fails when jcr:mixinTypes specified after fields it declares
• MODE-1813 - Using CDI to inject Session results in ClassNotFoundException for org.xml.sax.ContentHandler in JBoss AS 7
• MODE-1811 - Infinispan Binary Stores don't work in the AS 7.1 kit
• MODE-1810 - ClassCastException when using tika text extractor
• MODE-1819 - Session.move() fails when using user transactions
• MODE-1898 - Teiid DDL Parser Problems Found When Integrating With Teiid Designer
• MODE-1895 - Hibernate Search's AS7 kit has incorrect dependencies for the Solr module
• MODE-1894 - When updating indexes, ModeShape stores both the old and the updated property values
• MODE-1893 - NullPointerException in background garbage collection
• MODE-1809 - SELECT UNION with JOIN results in class cast exception
• MODE-1802 - FileSystemConnector does not correctly handle projections to root
• MODE-1804 - modeshape json schema is wrong for jgroups-slave index backend channel
• MODE-1805 - Make it possible to disable querying in AS7 kit
• MODE-1806 - Projections storage is not consistent with internal node storage
• MODE-1807 - Custom CND file with residual child definitions cannot be imported
• MODE-1880 - FileSystemConnector uses incorrect ID when adding content to nt:file nodes
• MODE-1881 - ConstraintViolationException Creating Node With A Mixin That Extends Abstract Mixin
• MODE-1882 - Problems getting the binary value from a federated node
• MODE-1883 - Unable to restore a removed node
• MODE-1884 - Multiple possibilities for NullPointerException in PowerPointMetadataReader
• MODE-1760 - Modeshape JCA Dependencies
• MODE-1887 - Issue with Workspace merge
• MODE-1879 - Import/export data. Not all Import/Export tests are passed with persistent infinispan configuration
• MODE-1878 - Same name siblings are incorrectly prevented in some cases
• MODE-1874 - configuration in the AS7 places indexes in the wrong directory
• MODE-1876 - OOM due to index rebuilding (if_missing) when starting repository with 1M nodes
• MODE-1871 - Spelling mistake in exception message "prrimary"
• MODE-1870 - cacheConfiguration for Infinispan Index Storage doesn't find configuration file from classpath
• MODE-1873 - Querying using [mode:derivedFrom] path for sequenced nodes, yields no results
• MODE-1774 - The AS 7.1.1 kit reports some strange URL-related errors when starting up
• MODE-1775 - Getting system nodes by their identifier does not work when identifier is found from the "jcr:uuid" property
• MODE-1773 - Projection expression format is too constrained
• MODE-1770 - Sequencing should work with default Infinispan configuration
• MODE-1676 - Binary Metadata and Infinispan JDBC cache loader
• MODE-1765 - Startup Errors Installing 3.1.0.Final In JBAS 7.1.1.Final
• MODE-1867 - minimumStringSizeInBytes is not defined in the schema
• MODE-1601 - ClusteredRepositoryChangeBusTest fail with JGroups 3.0.10+
• MODE-1865 - Queries with LIMIT 1 return no results (when LIMIT >2 do return some)
• MODE-1862 - "java.lang.IllegalStateException: This is a tx cache!" when using federation
• MODE-1863 - rebuild-upon-startup = NEVER triggers a read of all indexed nodes on startup
• MODE-1861 - Database binary store schema is incompatible with MySQL 5.5
• MODE-1900 - DISTINCT possibly being incorrectly applied (in multi-value ref property join)
• MODE-1762 - Sequencers may not always close the binary's InputStream
• MODE-1853 - ClassCastException on Session.save()
• MODE-1852 - ClassNotFoundException in org.modeshape.jcr.bus.ClusteredRepositoryChangeBus
• MODE-1851 - Problem with three-way JOIN using ISCHILDNODE and reference property
• MODE-1850 - Missing dependency on org.jboss.as.clustering.jgroups module
• MODE-1856 - No-op mutations seems to trigger indexing updates
• MODE-1854 - Indexing took too much time to finish when using explicit JTA transaction compared to no transaction
• MODE-1859 - jboss cli creation and deletion of repositories fails
• MODE-1858 - IllegalStateException during query execution
• MODE-1754 - ModeShape 3.1 cannot use Infinispan 5.2.0.CR1 in a clustered environment when eviction is enabled.
• MODE-1844 - Repository restart fails if original start procedure was interrupted/failed
• MODE-1846 - Starting AS7.2 + ModeShape may result in Infinispan errors
• MODE-1840 - JCR-SQL2 Full text search does not support bind variables
• MODE-1841 - jdbc store fails if we use cache-index-storage and cache-binary-storage
• MODE-1842 - eviction fails in jdbc cache-store
• MODE-1848 - Image sequencer incorrectly advertises TIFF images
• MODE-1822 - VersionManager.checkin throws exception in the transaction when using JTA transaction manager
• MODE-1786 - Wrong param was read for binaryStorage with type database
• MODE-1826 - JDBC cache-store doesn't work with modeshape deployed in as7

Component Upgrade

• MODE-1763 - Upgrade to Infinispan 5.2.1.Final
• MODE-1778 - Upgrade to Tika 1.3 (or latest version)
• MODE-1304 - Upgrade Lucene to 3.6 version and Hibernate Search 4.2
• MODE-1907 - Upgrade to Infinispan 5.2.5.Final
• MODE-1849 - Replace AS7.2 with EAP 6.1.0.Alpha in our kits

Enhancement

• MODE-1587 - Support JBoss AS7.2 and/or EAP6.1
• MODE-1818 - Garbage collection process for binaries should be configurable
• MODE-1800 - Pass the primary node type for a new node to the connector when creating an identifier
• MODE-1761 - Enhance logging during content sequencing
• MODE-1857 - Allow a node type's property definition to override a supertype's, as long as constraints are more restrictive
• MODE-1767 - MimeType lookup require using JCR implementation APIs

Feature Request

• MODE-650 - Connector to CMIS
• MODE-1812 - DDL Parser and Sequencer for Teiid's dialect
• MODE-1891 - Support for Apache Chemistry OpenCMIS 0.9.0
• MODE-1888 - Capture warnings on queries, especially for common mistakes
• MODE-1768 - FileSystemConnector exposes child nodes that might not be accessible
• MODE-1906 - Add ability to load CND files specified in json config from file-system on startup
• MODE-1855 - Given a JCR Property object, retrieve local name and namespace URI
• MODE-1561 - TikaTextExtractor fails if extracted word count exceeds 100000 chars
• MODE-1680 - The xpath query is not well-formed: Multiple XPath expressions are not supported
• MODE-1784 - Change AS7 kit to deploy WARs on subsystem startup

Task

• MODE-1899 - Creation & initialization of a workspace should fail if it has a transactional cache configured
• MODE-1897 - Add/update clustered indexing test
• MODE-1896 - Deprecate transactionManagerLookup attribute
• MODE-1548 - AS7 support: enable the recorvering of exposed services in case the removal fails
• MODE-1872 - Re-indexing should be done asynchronously
• MODE-1777 - Add ability to skip long-running tests during developer builds
• MODE-1860 - Change kit for EAP to place modules under 'modules' directory rather than 'modules/system/layers/base'
• MODE-1758 - Remove the two TCK tests that ModeShape skips
• MODE-1753 - Merge two readme files into one

Component Upgrade

• MODE-1493 - Upgrade RESTEasy to 2.3.5.Final

Sub-task

• MODE-1771 - Clean up Schematic's CacheContext logic, which currently checks Infinispan versions
• MODE-1764 - Change AS7 kit to use Hibernate Search and Infinispan 5.2 modules