Release Notes for ModeShape 3.0.0.Beta3

The ModeShape 3.0.0.Beta3 release is the third beta release of our new architecture, and is suitable only for testing and previewing features. APIs and storage formats are still subject to change.

What's new

3.0.0.Beta3 provides a feature complete implementation of the JCR 2.0 (JSR-283) specification. This release adds support for full-text search, text extraction, and shareable nodes, plus it fixes quite a few bugs and issues. We've also improved the way binary values (and related information such as MIME types and extract text) are handled and stored within the BinaryStore. This release also provides kits for installing ModeShape as a service into JBoss AS7.1.1.Final and AS7.2 (requires building from code, since it hasn't yet been released).

Overall, ModeShape 3.0 has changed a lot since ModeShape 2.8.x:

• ModeShape now uses Infinispan for all caching and storage, giving a powerful and flexible foundation for creating JCR repositories that are fast, scalable, and highly available. Infinispan offers a great deal of storage options (via cache loaders), but can also be used as a distributed, mulit-site, in-memory data grid.
• Improved performance. ModeShape 3 is just plain seriously fast, and performance is all-around faster than 2.x - most operations are at least one if not several orders of magnitude faster! We'll publish performance and benchmarking results closer to the final release.
• Improved scalability. ModeShape 3 has been designed to store and access the content so that a node can have hundreds of thousands of nodes (even with same-name-siblings) yet still be incredibly fast. Additionally, repositories can scale to millions of nodes and be deployed across many processes. (Clustering is not yet supported in this release.)
• Improved configuration. There is no more global configuration of the engine; instead, each repository is configured with a separate JSON file, which must conform to a JSON Schema and can be validated by ModeShape prior to use. Repository configurations can even be changed while the repository is running (some restrictions apply), making it possible to add/change/remove sequencers, authorization providers, and many other configuration options while the repository is in use.
• Each repository can be deployed, started, stopped, and undeployed while the engine and other repositories are still in use.
• Sessions now immediately see all changes persisted/committed by other sessions, although transient changes of the session always take precedence.
• Support for participation in JTA and XA transactions, allowing (container-managed or bean-managed) EJBs and JCR clients that programmatically use XA transactions to commit the changes in the transactions.
• New monitoring API that allows accessing the history for over a dozen metrics.
• New sequencing API, so sequencers now use the JCR API to get at the content being processed and create/update the derived content. Sequencers can also dynamically register namespaces and node types. Now it's easy to create custom sequencers.
• Simplified API for implementing custom MIME type detectors. ModeShape still has built-in detectors that use the filename extensions and the binary content.
• New and simpler API for implementing custom text extractors.
• Improved storage of binary values of all sizes, with a separate facility for storing these on the file system. Storage of binary values in Infinispan and DBMSes will be added in upcoming releases.
• API interfaces and methods that were deprecated in 2.7.0.Final (or later) have been removed. There weren't many of these; most of the ModeShape API remains the same.
• Integration with JBoss AS 7. ModeShape runs as an integrated subsystem within AS7, and the AS7 tooling can be used to define and manage repositories independently of each other while the server is running.
• Local and remote JDBC drivers for issuing JCR-SQL2 queries and getting database metadata via the JDBC API
• Many bug fixes and minor improvements

Features

Most of the JCR features previously supported in 2.x are working and ready for testing. If any issues are found, please log a bug report in our JIRA.

Accessing the Repository

• RepositoryFactory access
• JNDI registration of Repository
• JAAS Authentication
• Servlet Authentication
• Custom Authentication

Namespaces

• Session Remapping
• Permanent Addition/Deletion

Reading Repository Content

• Traversal Access
• Direct Access
• Same-Name Siblings
• Multi-Value Properties
• All Property Types Supported
• Property Type Conversion

Writing Repository Content

• Create/Update/Delete Nodes
• Create/Update/Delete Properties (Through Parent Nodes)
• Moving, Copying, Cloning
• Adding/Removing Mixins
• Referential integrity enforcement

Query / Search

• XPath
• JCR-SQL
• JCR-SQL2
• JCR-QOM
• Full-Text Search

Importing/Exporting Repository Content

• System View Import/Export
• Document View Import/Export

Node Types

• Inheritance Among Node Types
• Discovering available Node Types
• Discovering the Node Types of a Node
• Discovering the Definition of a Node Type
• Property Constraints
• Automatic Item Creation
• Predefined standard Node Types
• Custom Node Type Registration (CND-Based and and JCR 2.0 API Template-Based)

Repository Metadata under System Node

• Permanent Namespace Mappings
• Node Types (Built-In and User-Registered)
• Active Locks

Other JCR Optional Features

• Observation
• Locking
• Versioning
• Shareable nodes

Content Storage Options

• In-memory (local, replicated, and distributed)
• BerkleyDB
• Relational databases (via JDBC), including in-memory, file-based, or remote
• File system
• Cassandra
• Cloud storage (e.g., Amazon's S3, Rackspace's Cloudfiles, or any other provider supported by JClouds)
• Remote Infinispan
• Separate large binary storage on file system (database and Infinispan coming soon)

ModeShape Sequencers

• Compact Node Definition (CND) Sequencer
• DDL Sequencer
• Image Sequencer
• Java Source Sequencer
• Java Class Sequencer
• MP3 Sequencer
• MS Office Sequencer
• Text Sequencers (Delimited and Fixed Width)
• XML Sequencer
• XML Schema Document (XSD) Sequencer
• Web Service Definition Lanaguage (WSDL) 1.1 Sequencer
• Zip File Sequencer (also WARs, JARs, and EARs)
• Teiid Relational Model Sequencer (coming soon)
• Teiid VDB Sequencer (coming soon)

ModeShape Deployment/Access Models

• JNDI-Based Deployment
• Deploy as a subsystem in JBoss AS7, with RHQ/JON monitoring, @Resource injection
• Access through RESTful Service
• Access through WebDAV Service
• Local and remote JDBC drivers for accessing ModeShape content through JDBC API and JCR-SQL2 queries
• Embedded (in Server or JEE Archive) Deployment
• JTA support, allowing Sessions to participate in XA and container-managed transactions
• OSGi-Compatible Archives

Other ModeShape features

• Automatic MIME type detection of binary content
• Asynchronous sequencing operations, within completion notified via events

A few features not related to the JCR 2.0 API have been planned and are still not yet implemented. The most important one is enabling a repository to access the content in external systems (e.g., file system, SVN, JDBC, JCR, etc.). This most likely will be pushed to 3.1 so that we can focus on releasing 3.0 with full support for JCR 2.0.

Bug Fixes, Features, and other Issues

The following are the bugs, features and other issues that have been fixed in this 3.0.0.Beta3:

Bug

• MODE-1541 - REST service fails with large BINARY values
• MODE-1607 - Unable to successfully build 'assembly' profile due to JavaDoc errors
• MODE-1605 - Provide an installation kit for AS7.2
• MODE-1602 - The primary item name is not read when NodeTypes are materialized after restart
• MODE-1597 - QueryResult.getNodes() cannot be called when query contains multiple selectors
• MODE-1592 - ShareableNodesTest failures
• MODE-1572 - Unparsable XML characters not handled fully when exporting/importing a system view
• MODE-1577 - DatabaseBinaryStore several issues

Component Upgrade

• MODE-1591 - Upgrade the TCK unit tests to 2.5.1

Enhancement

• MODE-1586 - BinaryStore interface - javadoc clarifications
• MODE-1583 - Improve JSON representation used in REST service

Feature Request

• MODE-887 - RESTful Service should include URLs of nodes in query results
• MODE-1291 - Support Infinispan-based BinaryStore

Task

• MODE-1554 - Improve ModeShape's AS7 subsystem to better adhere to conventions
• MODE-1461 - Use JDK 1.7 for building, but use 1.6 compatibility

Sub-task

• MODE-1581 - Add backup and restore functionality to repository

The following are the bugs, features and other issues that have been fixed in this 3.0.0.Beta2:

Bug

• MODE-1566 - ScheduledThreadPoolExecutor for modeshape-cron is not shutdowned
• MODE-1562 - modeshape-web-jcr-webdav: DefaultContentMapper.getResourceLength does not close InputStream
• MODE-1546 - Various & intermittent NPE, NodeNotFoundException and NodeParentNotFoundException
• MODE-1568 - Surefire ignores JGroups ipv4 and ipv6 settings
• MODE-1351 - Importing large binary values requires excessive memory
• MODE-1524 - Advanced externalizers not supported by AS7 subsystem
• MODE-1570 - Unit tests fail when using JGroups ipv6 addresses
• MODE-1360 - ConcurrentModificationException when session reads content being modified
• MODE-1573 - When importing binary values from a document view, the strings aren't decoded
• MODE-1575 - Schematic binary values aren't deserialized correctly, causing EOF exceptions

Enhancement

• MODE-1565 - Enable use of Infinispan 5.1.5.Final and JGroups 3.1.0.Final
• MODE-1275 - Consider changing the jboss deployment to use DatabaseInitializer class for initializing the database

Feature Request

• MODE-1448 - Store binary files in a relational database

Task

• MODE-1555 - Reuse TCK tests "known.issues" property to ignore certain tests

Bug Fixes, Features, and other Issues (Beta1)

The following are the bugs, features and other issues that have been fixed in 3.0.0.Beta1:

Bug

• MODE-1547 - Text extracted by the extractors, cannot be full-text-searched
• MODE-1549 - ModeShape should support the '<>' comparison operator in JCR-SQL2
• MODE-1161 - Shareable Nodes removeSharedSet() API throws InvalidItemStateException
• MODE-1469 - SQL query with path selectors does not find direct descendants of root node
• MODE-1416 - modeshape-rest query results have content-type: text/html instead of application/json
• MODE-1539 - CND Preference Page Is Uncategorized When Installing Just The CND Feature
• MODE-1550 - SQL/QOM query using ISCHILDNODE fail to find children of a node
• MODE-1556 - Queries with != or <> criteria do not work

Enhancement

• MODE-1545 - The FileBinaryStore should store the MIME type

Feature Request

• MODE-1544 - Create a Tika-based MIME type extractor
• MODE-1536 - [JCR Query] java.lang.IllegalArgumentException: The optimizationNanos argument value may not be negative
• MODE-1419 - Full-text search should be configurable
• MODE-1452 - Support multiple BinaryStores
• MODE-1458 - Implement shareable nodes support

Task

• MODE-1527 - Migrate Text Extractors
• MODE-1504 - Test container managed transactions
• MODE-1503 - Test JTA support with XA transactions
• MODE-1543 - Remove ModeShape Client JAR As A Dependency Of The JCR CND Editor Feature
• MODE-1481 - Support (or document) deploying JDBC DataSource for a repository
• MODE-1065 - Improve performance
• MODE-1329 - Convert the sequencers to the new 3.0 API
• MODE-1433 - Set up the JDBC data source for ModeShape in the JBoss AS7 kit
• MODE-1551 - Remove SLF4J dependencies in Publishing plugin
• MODE-1354 - Migrate more of the unit tests

Bug Fixes, Features, and other Issues (Alpha6)

The following are the bugs, features and other issues that have been fixed in 3.0.0.Alpha6:

Bug

• MODE-1525 - addNode(String) fails to find child node type
• MODE-1521 - Sequencers are not being configured correctly in AS7
• MODE-1520 - ClassNotFoundException: org.infinispan.schematic.internal.SchematicEntryLiteral
• MODE-1526 - Persisted data isn't found after server restart
• MODE-1518 - Valid sequencer path expression is not handled correctly
• MODE-1524 - Advanced externalizers not supported by AS7 subsystem
• MODE-1537 - Changes from transactions are not visible until restart
• MODE-1515 - Separate sequencers into separate modules in the AS7 kit
• MODE-1512 - Lucene lock file "write.lock" after shutdown repository in 3.0.0.Alpha4

Component Upgrade

• MODE-787 - Update JCR TCK unit tests
• MODE-1492 - Upgrade JCR TCK tests to 2.5

Enhancement

• MODE-1514 - Rename JcrEngine to ModeShapeEngine

Feature Request

• MODE-1538 - Allow queries to be cancelled
• MODE-1540 - Align ModeShape Tools Version To Match ModeShape
• MODE-872 - JDBC (non-local) driver needs to support queries
• MODE-1536 - [JCR Query] java.lang.IllegalArgumentException: The optimizationNanos argument value may not be negative

Task

• MODE-1522 - Sequencer configuration in JSON should be keyed by name, not array
• MODE-1519 - Sequencers configuration: rename the sequencer's "type" JSON attribute, to "classname"
• MODE-1395 - Port and update TCK tests to 3.x
• MODE-1516 - AS7 kit's sample repository should configure sequencers
• MODE-1507 - Enable WebDav support in 3.x

Bug Fixes, Features, and other Issues (Alpha5)

The following are the bugs, features and other issues that have been fixed in this release:

Bug

• MODE-1407 - Locking nodes does not add mandatory properties
• MODE-1501 - NullPointerException during Repository.login in EJB Startup
• MODE-1500 - Unable to get repository for JNDI name
• MODE-1489 - orderBefore causes infinite loop - hang
• MODE-1485 - QOM outer join tests return incorrect results
• MODE-1468 - JQOM queries are not converted to SQL2
• MODE-1409 - Node.orderBefore does not trigger (or triggers incomplete) cache events
• MODE-1510 - Configuration for storing indexes in Infinispan doesn't use the right default for 'cacheConfiguration'
• MODE-1456 - Update node.remove implementation to handle JCR validations
• MODE-1455 - Implement node.update functionality
• MODE-1453 - Implement reference handling support
• MODE-1497 - Exception Adding/Editing Property Definition's Value Constraint In CND Editor
• MODE-1496 - Query result column names don't follow the specification for wildcard selects
• MODE-1494 - xsd sequencer shows sramp:contentType="application/xsd"

Feature Request

• MODE-1465 - Sequencer Callbacks for Exception situations too
• MODE-1498 - Support JTA transactions
• MODE-1491 - Sequencers should check MIME type of Binary values

Task

• MODE-1509 - Infinispan Lucene Directory JARs should be added as modules in ModeShape's AS7 kit
• MODE-1495 - Remove The Tycho Compiler Plugin Entry From The Build Element In ModeShape Tools Pom Files
• MODE-1389 - Update Maven assemblies for 3.0
• MODE-1462 - Choose & update logging framework for 3.x
• MODE-1488 - Migrate ModeShape Tools Source Code To GitHub
• MODE-1477 - Port modeshape-jcr-tck module from 2.x
• MODE-1312 - Implement workspace copy & clone functionality
• MODE-1508 - Cleanup AS subsystem dependencies

Sub-task

• MODE-1318 - Create Repository Node

Bug Fixes, Features, and other Issues (Alpha4)

The following are the bugs, features and other issues that have been fixed in 3.0.0.Alpha4:

Bug

• [MODE-1361] - Sequencers path expressions missmatch
• [MODE-1408] - Namespace changes do not trigger repository events
• [MODE-1410] - Workspace.move does not trigger cache events
• [MODE-1423] - org.infinispan.marshall.NotSerializableException: org.infinispan.schematic.internal.SchematicEntryLiteral
• [MODE-1431] - Unable to locate the Log bundle
• [MODE-1442] - Admin user does not have admin role in JBoss AS kit
• [MODE-1464] - XSD Sequencer issue with Unsigned Longs
• [MODE-1466] - The current user should be set on the sequencer thread
• [MODE-1473] - Second property is ignored in org.modeshape.jcr.query.qom.JcrQueryObjectModelFactory.equiJoinCondition(String, String, String, String)
• [MODE-1479] - Applying changes to repository sequencer configs causes NPE
• [MODE-1480] - Clustering cannot be configured in ModeShape subsystem for AS7

Component Upgrade

• [MODE-1478] - Upgrade to Hibernate Search 4.1.1.Final

Feature Request

• [MODE-1220] - Add support for deploying Modeshape to JBoss AS 7.0

Task

• [MODE-1435] - Improved and working ModeShape AS7 subsystem
• [MODE-1437] - AS7 subsystem should provide and specify ChannelProvider and CacheManagerLookup classes
• [MODE-1440] - Improve AS7 subsystem configuration
• [MODE-1449] - Rename internal Binary interface to BinaryValue
• [MODE-1472] - Remove SLF4J from the modeshape-client jar

Bug Fixes, Features, and other Issues (Alpha3)

The following are the bugs, features and other issues that have been fixed in 3.0.0.Alpha3:

Bug

• [MODE-1405] - Importing content with protected properties results in exception
• [MODE-1436] - JGroups test configuration

Component Upgrade

• [MODE-1379] - Upgrade to Infinispan 5.1.2.Final
• [MODE-1425] - Upgrade Joda Time to 1.6.2
• [MODE-1426] - Upgrade PicketBox to 4.0.7.Final
• [MODE-1427] - Upgrade JGroups to 3.0.6.Final
• [MODE-1428] - Upgrade RESTEasy to 2.3.2.Final

Enhancement

• [MODE-1392] - Extend jcr event model to support sequencing
• [MODE-1412] - Upgrade Subsystem to JBoss AS 7.1.1 Final
• [MODE-1434] - JDBC drivers should define the service provider information in the JAR
• [MODE-1438] - modeshape-web-jcr repository resolution mechasim should be simplified
• [MODE-1439] - ModeShape API should contain JNDI-based RepositoryFactory

Task

• [MODE-1429] - Maven warnings (possibly errors) when building the AS 7.1 kit
• [MODE-1430] - Enable RESTful service in 3.0 builds
• [MODE-1441] - Update AS 7 Kit to Copy Subsystem Schema to docs/schema

Sub-task

• [MODE-1383] - Need to Create a JNDI Binding for the JcrEngine Service

Bug Fixes, Features, and other Issues (Alpha2)

The following are the bugs, features and other issues that have been fixed in 3.0.0.Alpha2:

Bug

• [MODE-1302] - PathNotFoundException when trying to retrieve version history for a node.
• [MODE-1391] - The disk location for the binary store cannot be specified in the repository configuration
• [MODE-1399] - Creating New RepositoryConfiguration Causes OoM Exception
• [MODE-1400] - CND syntax errors not reported by registerNodeTypes()
• [MODE-1401] - Checking in versionable node should make subgraph read-only
• [MODE-1404] - NPE in org.modeshape.common.util.MimeTypeUtil.getDefaultMappings
• [MODE-1411] - Dispatching events for the system workspace from a separate thread, corrupts the node caches
• [MODE-1417] - REST client utility is not making 'nt:file' nodes versionable
• [MODE-1418] - JCR-SQL2 CONTAINS() and Modeshape Search returning unexpected results
• [MODE-1420] - Extra log warning messages when re-indexing content

Feature Request

• [MODE-760] - ModeShape does not include "/jcr:system" content in query results

Task

• [MODE-1365] - Migrate query support to ModeShape 3.0
• [MODE-1378] - Incorporate improvements to our Maven usage
• [MODE-1397] - Implement JCR Observation in 3.0
• [MODE-1402] - Simplify the configuration for each sequencer in the repository configuration

Sub-task

• [MODE-1319] - Create Sequencer Node
• [MODE-1340] - Convert WSDL sequencer to the new 3.0 sequencing API
• [MODE-1341] - Convert XML sequencer to the new 3.0 sequencing API
• [MODE-1343] - Convert ZIP sequencer to the new 3.0 sequencing API
• [MODE-1382] - Create a Service for Each Repository Instance Defined by the Subsystem
• [MODE-1390] - Update org.modeshape Module Dependency to Include Infinispan

Bug Fixes, Features, and other Issues (Alpha1)

The following are the bugs, features and other issues that have been fixed in 3.0.0.Alpha1:

Bug

• [MODE-1041] - Modeshape unit test ZipSequencerTest fails when executed on Windows 2003
• [MODE-1118] - Events are fired incorrectly for UUID and non public properties
• [MODE-1296] - When sequencing of Teiid Virtual model, the transformation (transform_selectSql) is not being found in the [transform:withSql] table when using jdbc interface
• [MODE-1306] - don't start jetty when -DskipTests
• [MODE-1309] - Unstructured queries with inner joins do not work
• [MODE-1313] - Calling Session.getNodeByIdentifier(String) with a bogus identifier results in a non-JCR exception
• [MODE-1314] - JPA connector cannot properly configure 2nd level cache in Hibernate 3.3 and later
• [MODE-1315] - NPE in Observation when listener restricted by node type and property deleted
• [MODE-1326] - Oracle DDL Parser throws exception parsing CREATE UNIQUE INDEX
• [MODE-1327] - JPA source does not properly adjust the capabilities
• [MODE-1347] - Modeshape stuck - race condition in RepositoryConnectionPool
• [MODE-1348] - Exporting workspaces with large Binary values runs out of memory
• [MODE-1349] - JPA connection and 2nd level cache: LargeValueEntity should not be cacheable
• [MODE-1350] - Indexing workspace with large binary values results in OutOfMemoryError
• [MODE-1357] - Binary Store garbage collection thread blocks
• [MODE-1358] - Copying a file (in FileBinaryInputStream) should not take a long time
• [MODE-1359] - Join criteria that involves a null or nonexistent property on the left-hand side will result in NPE
• [MODE-1364] - JCR-SQL2 query with reference property in join condition does not work
• [MODE-1366] - Failure parsing JCR-SQL2 queries that use square brackets to surround paths with SNS indexes
• [MODE-1374] - ensureRequiredPrimaryTypesLoaded throwing NullPointerException on first access under Jetty
• [MODE-1377] - modeshape-web-jcr-webdav throws 500 Internal Server Error accessing repository with trailing slash

Component Upgrade

• [MODE-1300] - Upgrade to Tika 1.0

Enhancement

• [MODE-1187] - Support non-UUID identifiers
• [MODE-1308] - Enhance support for javax.jcr.Binary interface
• [MODE-1323] - Performance benchmark should graph results from all modules
• [MODE-1328] - Add to the public API methods to register node types in CND files
• [MODE-1345] - Allow JPA source to be configured without background garbage collection

Feature Request

• [MODE-951] - Changes made by one session via save() should be immediately visible to all other sessions (across the cluster)
• [MODE-1289] - JCR layer should use Infinispan for caching and storage
• [MODE-1293] - New sequencer API should expose JCR Session
• [MODE-1294] - JCR interfaces should be thread safe whenever possible
• [MODE-1295] - Performance benchmarks that use the JCR API
• [MODE-1310] - Add new Sequencer interfaces and service
• [MODE-1346] - Support centralized storage of Binary values, keeping content off of the heap
• [MODE-1375] - ModeShape 3 JSON configuration files should support using system properties

Task

• [MODE-1288] - Remove JCR 2.0 redundant interfaces & methods
• [MODE-1290] - Add compilation support for JDK 1.6
• [MODE-1303] - Update Maven plugin versions
• [MODE-1305] - Remove unused I18N classes method
• [MODE-1311] - Migrate query infrastructure
• [MODE-1367] - Change copyright dates to 2012
• [MODE-1368] - Remove unnecessary modules from 3.x branch
• [MODE-1371] - Add more metrics
• [MODE-1380] - Improve NodeTypeManager.registerNodeType methods
• [MODE-1386] - Update Maven build for 3.0

Sub-task

• [MODE-1148] - GraphSession code loads too many nodes
• [MODE-1330] - Convert image sequencer to the new 3.0 sequencing API
• [MODE-1331] - Convert classfile sequencer to the new 3.0 sequencing API
• [MODE-1332] - Convert CND sequencer to the new 3.0 sequencing API
• [MODE-1333] - Convert DDL sequencer to the new 3.0 sequencing API
• [MODE-1334] - Convert Java source file sequencer to the new 3.0 sequencing API
• [MODE-1336] - Convert MP3 sequencer to the new 3.0 sequencing API
• [MODE-1337] - Convert MS Office sequencer to the new 3.0 sequencing API
• [MODE-1339] - Convert text sequencer to the new 3.0 sequencing API
• [MODE-1342] - Convert XSD sequencer to the new 3.0 sequencing API
• [MODE-1363] - Add Attributes to Repository Node
• [MODE-1381] - Add JcrEngine Service in Subsystem
• [MODE-1384] - Replace Hard Coded ModeShape Versions in module.xml
• [MODE-1387] - Clean Up Unneeded Dependencies in Modules