Monitoring - ModeShape 3

ModeShape has a lot of moving parts, and with 3.8.1 we've made it much easier for your application to monitor the repository to understand how it's being used and what work is going on in the background.

Public API

ModeShape now includes as part of its public API a set of interfaces that your application can use to monitor the activities and health of your ModeShape repository. We did this because the standard JCR API doesn't cover monitoring at all, and we thought it's useful enough to make it available.

Metrics

ModeShape can capture a number of different measurements, called metrics, and these are broken into two categories: duration-based metrics (how long something takes) and simple value metrics.

Duration metrics are represented by the org.modeshape.jcr.api.monitor.DurationMetric enumeration, and include:

Metric	Description
Query execution time	The amount of time required to execute a query.
Session duration	The length of time that a session is used before being closed.
Sequencer duration	The length of time required to sequence a node, produce the output, and save the changes to the workspace.

Value metrics are represented by the org.modeshape.jcr.api.monitor.ValueMetric enumeration, and include:

Metric	Style	Description
Active sessions	continuous	The number of active sessions.
Active queries	continuous	The number of active queries.
Workspace count	continuous	The number of workspaces.
Session-scoped locks	continuous	The number of session-scoped locks held by clients.
Open-scoped locks	continuous	The number of open-scoped locks held by clients.
Listener count	continuous	The number of listeners registered with active sessions.
Event queue size	continuous	The number of events that are enqueued for processing and sending to listeners.
Event count	incremental	The number of events that have been sent to at least one listener.
Changed nodes	incremental	The number of nodes that were created, updated, or deleted.
Session saves	incremental	The number of `Session.save()` calls.
Sequencer queue size	continuous	The number of sequencing operations that are enqueued.
Sequenced nodes	incremental	The number of nodes sequenced.

Values for each of these metrics is captured every 5 seconds, where the continuous metrics are simply recorded as is (the values continue from one measurement to the next), while the incremental metrics represent distinct perturbations (or increments) from 0.

Windows and statistics

As mentioned above, ModeShape measures the values for each metric every 5 seconds. But it would take vast amounts of space to keep all these measurements around for long periods of time. Instead, ModeShape calculates the statistics for various intervals, and then rolls up the statistics into different time windows.

The statistics are straightforward:

Statistic	Data type	Description
Count	int	The number of samples.
Maximum	long	The maximum value from the samples.
Minimum	long	The minimum value from the samples.
Mean	double	The mean (or average) value from the samples.
Variance	double	The average of the squared differences from the mean.
Standard Deviation	double	A measure of how spread out the samples are and is the square root of the variance

and are represented in the API by the org.modeshape.jcr.api.monitor.Statistics interface (with getter methods for each statistic). The statistics were chosen because multiple Statistics objects can easily be rolled-up into a single Statistic object.

The rollup process is pretty simple. For each metric:

The value is captured every 5 seconds, and recorded a Statistics instance with a single sample. This is repeated 9 more times.
After 60 seconds, the 10 Statistics objects recorded in the previous step are rolled-up into a single Statistics object for this minute. This is repeated 59 more times.
After 60 minutes, the 60 Statistics objects recorded in the previous step are rolled-up into a single Statistics object for this hour. This is repeated 23 more times.
After 24 hours, the 24 Statistics objects recorded in the previous step are rolled-up into a single Statistics object for the day. This is repeated 7 more times.
After 7 days, the 7 Statistics objects recorded in the previous step are rolled-up into a single Statistics object for the week. This is repeated 52 more times.

Each of these periods represents a window in time during with the Statistics are captured:

Window timeframe	Description
60 seconds	A `Statistics` for each of the ten 5-second intervals during the last minute.
60 minutes	A `Statistics` for each minute during the last hour.
24 hours	A `Statistics` for each hour during the last day.
7 days	A `Statistics` for each day during the last week.
52 weeks	A `Statistics` for each week during the last year.

The org.modeshape.jcr.api.monitor.Window enumeration is used to represent each of these windows in time.

Histories

The set of Statistics objects for a particular metric during a Window is called the history of the metric, which is represented by the org.modeshape.jcr.api.monitor.History interface:

public interface History {

    /**
     * Get the kind of window.
     *
     * @return the window type; never null
     */
    public Window getWindow();

    /**
     * Get the total duration of this history window.
     *
     * @param unit the desired time unit; if null, then {@link TimeUnit#SECONDS} is used
     * @return the duration
     */
    public long getTotalDuration( TimeUnit unit );

    /**
     * Get the timestamp (including time zone information) at which this history window starts.
     *
     * @return the time at which this window starts
     */
    public DateTime getStartTime();

    /**
     * Get the timestamp (including time zone information) at which this history window ends.
     *
     * @return the time at which this window ends
     */
    public DateTime getEndTime();

    /**
     * Get the statistics for that make up the history.
     *
     * @return the statistics; never null, but the array may contain null if the window is
     *         longer than the lifetime of the repository
     */
    Statistics[] getStats();
}

The org.modeshape.jcr.api.value.DateTime interface is an immutable representation of an instant in time. It includes timezone information and methods for converting or obtaining the various representations and/or parts of the instant. It is based upon initial work by the JSR-310 effort, and is far superior to the mutable and difficult-to-use java.util.Calendar class.

RepositoryMonitor

The org.modeshape.jcr.api.monitor.RepositoryMonitor interface can then be used to get the available metrics and windows, as well as obtaining the history for a given metric and window:

public interface RepositoryMonitor {

    /**
     * Get the ValueMetric enumerations that are available for use by the caller
     * with {{getHistory(ValueMetric, Window)}}.
     *
     * @return the immutable set of ValueMetric instances; never null but possibly
     *         empty if the caller has no permissions to see any value metrics
     */
    Set<ValueMetric> getAvailableValueMetrics();

    /**
     * Get the DurationMetric enumerations that are available for use by the caller
     * with {{getHistory(DurationMetric, Window)}}.
     *
     * @return the immutable set of DurationMetric instances; never null but possibly
     *         empty if the caller has no permissions to see any value metrics
     */
    Set<DurationMetric> getAvailableDurationMetrics();

    /**
     * Get the Window enumerations that are available for use by the caller with
     * {{getHistory(DurationMetric, Window)}} and {{getHistory(ValueMetric, Window)}}.
     *
     * @return the immutable set of DurationMetric instances; never null but possibly
     *         empty if the caller has no permissions to see any value metrics
     */
    Set<Window> getAvailableWindows();

    /**
     * Get the statics for the specified value metric during the given window in time.
     * The oldest statistics will be first, while the newest statistics will be last.
     *
     * @param metric the value metric; may not be null
     * @param windowInTime the window specifying which statistics are to be returned;
     *        may not be null
     * @return the history of the metrics; never null but possibly empty if there are
     *         no statistics being captures for this repository
     * @throws AccessDeniedException if the session does not have privileges to monitor the repository
     * @throws RepositoryException if there is an error obtaining the history
     */
    public History getHistory( ValueMetric metric,
                               Window windowInTime )
                   throws AccessDeniedException, RepositoryException;

    /**
     * Get the statics for the specified duration metric during the given window in time.
     * The oldest statistics will be first, while the newest statistics will be last.
     *
     * @param metric the duration metric; may not be null
     * @param windowInTime the window specifying which statistics are to be returned;
     *        may not be null
     * @return the history of the metrics; never null but possibly empty if there are
     *         no statistics being captures for this repository
     * @throws AccessDeniedException if the session does not have privileges to monitor the repository
     * @throws RepositoryException if there is an error obtaining the history
     */
    public History getHistory( DurationMetric metric,
                               Window windowInTime )
                   throws AccessDeniedException, RepositoryException;

    /**
     * Get the longest-running activities recorded for the specified metric.
     * The results contain the duration records in order of increasing duration,
     * with the activity with the longest duration appearing last in the array.
     *
     * @param metric the duration metric; may not be null
     * @return the activities with the longest durations; never null but possibly
     *         empty if no such activities were performed
     * @throws AccessDeniedException if the session does not have privileges to monitor the repository
     * @throws RepositoryException if there is an error obtaining the history
     */
    public DurationActivity[] getLongestRunning( DurationMetric metric )
                   throws AccessDeniedException, RepositoryException;

And finally, your application can get the RepositoryMonitor instance from the Session's workspace,
using ModeShape's org.modeshape.jcr.api.Workspace interface that extends the standard
javax.jcr.Workspace interface:

Session session = ...
org.modeshape.jcr.api.Workspace workspace = (org.modeshape.jcr.api.Workspace)session.getWorkspace();
RepositoryMonitor monitor = workspace.getRepositoryManager().getRepositoryMonitor();

Examples

The following examples show a few ways of accessing the history of different metrics using different windows.

Active sessions during the last hour

This example shows how to get the history containing the number of active sessions during each minute of the last hour:

RepositoryMonitor monitor = workspace.getRepositoryManager().getRepositoryMonitor();
History history = monitor.getHistory(ValueMetric.SESSION_COUNT,Window.PREVIOUS_60_MINUTES);

// Use the history information to build a graph and determine the axes labels ...
int duration = history.getTotalDuration(TimeUnit.MINUTES);  // will be '60'
DateTime started = history.getStartTime();
DateTime ended = history.getEndTime();
Statistics[] stats = history.getStats();  // will contain 60 elements

Here, each Statistics object represents the number of active sessions that existed during each minute.
If, for example, all the sessions were closed in the second-to-last minute, then the second-to-last Statistics
object will reflect some of them closing, while the first Statistics object will have average, maximum, and minimum values of 0.

Query durations during the last day

This example shows how to obtain the statistics for the durations of queries executed during the last 24 hours:

RepositoryMonitor monitor = workspace.getRepositoryManager().getRepositoryMonitor();
History history = monitor.getHistory(DurationMetric.QUERY_EXECUTION_TIME,Window.PREVIOUS_24_HOURS);

// Use the history information to build a graph and determine the axes labels ...
int duration = history.getTotalDuration(TimeUnit.MINUTES);  // will be '1440' (or 24 x 60 )
DateTime started = history.getStartTime();
DateTime ended = history.getEndTime();
Statistics[] stats = history.getStats();  // will contain 24 elements

Each Statistics object will represent the number, average, maximum, minimum, variance, and standard deviation for the queries that were executed during an hour of the last 24 hours.

Worst performing queries during the last day

Just as we can obtain the statistics for the queries that were submitted during the last 24 hours, we can also get information about the longest-running queries:

RepositoryMonitor monitor = workspace.getRepositoryManager().getRepositoryMonitor();
// Get the 'DurationActivity' object for each long-running query, where the longest is last ...
DurationActivity[] longestQueries = monitor.getLongestRunning(DurationMetric.QUERY_EXECUTION_TIME);

for ( DurationActivity queryActivity : longestQueries ) {
    long duration = queryActivity.getDuration(TimeUnit.MILLISECONDS);
    Map<String,String> payload = queryActivity.getPayload();
    String query = payload.get("query");
}

Event queue backlog during the last hour

This example shows how to get the history containing the number of events in the event queue during each minute of the last hour:

RepositoryMonitor monitor = workspace.getRepositoryManager().getRepositoryMonitor();
History history = monitor.getHistory(ValueMetric.EVENT_QUEUE_SIZE,Window.PREVIOUS_60_MINUTES);

// Use the history information to build a graph and determine the axes labels ...
int duration = history.getTotalDuration(TimeUnit.MINUTES);  // will be '60'
DateTime started = history.getStartTime();
DateTime ended = history.getEndTime();
Statistics[] stats = history.getStats();  // will contain 60 elements

Here, each Statistics object represents the number of events that are in the queue during each minute.
If, for example, the number of events is increasing during each minute, then ModeShape is falling behind
in notifying the listeners. This likely will happen when sessions are making frequent changes, while
registered listeners are taking too long to process the event.

Listeners should not take too long to process the event, since one thread is being used to notify all listeners. So if your listeners might, consider having your listener enqueuing work into a separate java.util.concurrent.Executor, where the actual work is performed on separate threads.

Also, be careful if the listener needs to look up content using a session. Generally speaking, it's not good practice for a listener to reuse the same session on which it's registered, since all listeners will share the same session. ModeShape 3.x is thread-safe, but any changes made by one listener will be visible to other listeners.