Monitoring and Managing JBoss Web

Introduction

Monitoring is a very important question today. Looking inside the running server, grab some statistic data or reconfigure some aspects are daliy adminstration tasks.

Enabling JMX Remote

The Sun website includes the list of options and how to configure JMX Remote on Java 5: http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html.

For quick installation you find here a short installation guide:

Add the following parameters to your JBoss Web startup script:

    set CATALINA_OPTS="-Dcom.sun.management.jmxremote \
    -Dcom.sun.management.jmxremote.port=%my.jmx.port% \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Dcom.sun.management.jmxremote.authenticate=false"

When you think authorisation is a good, add and change this :

    -Dcom.sun.management.jmxremote.authenticate=true \
    -Dcom.sun.management.jmxremote.password.file=../conf/jmxremote.password \
    -Dcom.sun.management.jmxremote.access.file=../conf/jmxremote.access \

edit the access allow file $CATALINA_BASE/conf/jmxremote.access :
```
monitorRole readonly
controlRole readwrite
    
```
edit the password file $CATALINA_BASE/conf/jmxremote.password :
```
monitorRole tomcat
controlRole tomcat
    
```
Tipp: Password File must be readonly and not accessable from every other user! Remove all other users under windows to access this file.

Note:The JSR 160 JMX-Adaptor opens a second data protocol port. That is a problem when you have installed a local firewall.

Manage JBoss Web with JMX remote Ant Tasks

For simple JBoss Web ant task usage with ant 1.6.x we have integrate import and antlib support. Copy your catalina-ant.jar from $CATALINA_HOME/server/lib to $ANT_HOME/lib. The following example show the JMX Accessor usage:

<project name="Catalina Ant JMX" 
        xmlns:jmx="antlib:org.apache.catalina.ant.jmx" 
        default="state"
        basedir=".">
    <property name="jmx.server.name" value="localhost" />
    <property name="jmx.server.port" value="9012" />
    <property name="cluster.server.address" value="192.168.1.75" />
    <property name="cluster.server.port" value="9025" />
 
    <target name="state" description="Show JMX Cluster state">
        <jmx:open
            host="${jmx.server.name}"
            port="${jmx.server.port}"
            username="controlRole"
            password="tomcat"/>
        <jmx:get
            name="Catalina:type=IDataSender,host=localhost,senderAddress=${cluster.server.address},senderPort=${cluster.server.port}" 
            attribute="connected"
            resultproperty="IDataSender.backup.connected"
            echo="false"
        />
       <jmx:get
            name="Catalina:type=ClusterSender,host=localhost" 
            attribute="senderObjectNames"
            resultproperty="senderObjectNames"
            echo="false"
        />
        <!-- get current maxActiveSession from ClusterTest application
             echo it to ant output and store at 
             property <em>clustertest.maxActiveSessions.orginal</em>
        -->
       <jmx:get
            name="Catalina:type=Manager,path=/ClusterTest,host=localhost" 
            attribute="maxActiveSessions"
            resultproperty="clustertest.maxActiveSessions.orginal"
            echo="true"
        />
        <!-- set maxActiveSession to 100
        -->
        <jmx:set
            name="Catalina:type=Manager,path=/ClusterTest,host=localhost" 
            attribute="maxActiveSessions"
            value="100"
            type="int"
        />
        <!-- get all sessions and split result as delimiter <em>SPACE</em> for easy
             access all session ids directly with ant property sessions.[0..n].
        -->
        <jmx:invoke
            name="Catalina:type=Manager,path=/ClusterTest,host=localhost" 
            operation="listSessionIds"
            resultproperty="sessions"
            echo="false"
            delimiter=" "
        />
        <!-- Access session attribute <em>Hello</em> from first session.
        -->
        <jmx:invoke
            name="Catalina:type=Manager,path=/ClusterTest,host=localhost" 
            operation="getSessionAttribute"
            resultproperty="Hello"
            echo="false"
        >
          <arg value="${sessions.0}"/>
          <arg value="Hello"/>
        </jmx:invoke> 
        <!-- Query for all application manager.of the server from all hosts
             and bind all attributes from all found manager mbeans.
        -->
        <jmx:query
            name="Catalina:type=Manager,*" 
            resultproperty="manager"
            echo="true"
            attributebinding="true"
        />
        <!-- echo the create properties -->
        <echo>
           senderObjectNames: ${senderObjectNames.0}
           IDataSender.backup.connected: ${IDataSender.backup.connected}
           session: ${sessions.0}
           manager.length: ${manager.length}
           manager.0.name: ${manager.0.name}
           manager.1.name: ${manager.1.name}
           hello: ${Hello}
           manager.ClusterTest.0.name: ${manager.ClusterTest.0.name}
           manager.ClusterTest.0.activeSessions: ${manager.ClusterTest.0.activeSessions}
           manager.ClusterTest.0.counterSend_EVT_SESSION_EXPIRED: ${manager.ClusterTest.0.counterSend_EVT_SESSION_EXPIRED}
           manager.ClusterTest.0.counterSend_EVT_GET_ALL_SESSIONS: ${manager.ClusterTest.0.counterSend_EVT_GET_ALL_SESSIONS}
        </echo>   

    </target>
 
</project>

import: Import the Catalina Ant tasks with <import file="${CATALINA.HOME}/bin/catalina-tasks.xml" /> and reference the tasks with jmxOpen, jmxSet, jmxGet, jmxQuery, jmxInvoke,jmxEquals and jmxCondition.

JMXAccessorOpenTask - jmx open connection task

List of Attributes

Attribute	Description
`url`	Set jmx connection url - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi
`host`	Set the host, shortcut the very long url syntax. [localhost]
`port`	Set the remote connection port. [8050]
`username`	Remote jmx connection user name.
`password`	Remote jmx connection password.
`ref`	Name of the internal connection reference. With this attribute you can configure more the one connection inside the same ant project. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]
`if`	Only execute if a property of the given name exists in the current project.
`unless`	Only execute if a property of the given name not exists in the current project.

Example to open a new jmx connection

    <jmx:open
            host="${jmx.server.name}"
            port="${jmx.server.port}"
    />

Example to open a jmx connection from url, with authorisation and store at other reference

    <jmx:open
            url="service:jmx:rmi:///jndi/rmi://localhost:9024/jmxrmi"
            ref="jmx.server.9024"
            username="controlRole"
            password="tomcat"    
    />

Example to open a jmx connection from url, with authorisation and store at other reference, but only when property jmx.if exists and jmx.unless not exists

    <jmx:open
            url="service:jmx:rmi:///jndi/rmi://localhost:9024/jmxrmi"
            ref="jmx.server.9024"
            username="controlRole"
            password="tomcat"    
            if="jmx.if"    
            unless="jmx.unless"    
    />

Note: All properties from jmxOpen task also exists at all other tasks and conditions.

JMXAccessorGetTask: get attribute value ant task

Attribute	Description
`name`	Full qualified JMX ObjectName -- Catalina:type=Server
`attribute`	Existing Mbean attribute (see JBoss Web mbean description above).
`ref`	JMX Connection reference. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]
`resultproperty`	Save result at this project property.
`delimiter`	Split result with delimiter (java.util.StringTokenizier) and use resultproperty as prefix to store tokens.
`separatearrayresults`	When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.lenght). [true]

Example to get remote mbean attribute from default jmx connection

    <jmx:get
        name="Catalina:type=Manager,path=/servlets-examples,host=localhost" 
        attribute="maxActiveSessions"
        resultproperty="servlets-examples.maxActiveSessions"
    />

Example to get and result array and split it at separate properties

    <jmx:get
        name="Catalina:type=ClusterSender,host=localhost" 
        attribute="senderObjectNames"
        resultproperty="senderObjectNames"
    />

Access the senderObjectNames properties with:

    ${senderObjectNames.lenght} give the number of returned sender list.
    ${senderObjectNames.[0..N]} found all sender object names

Example to get IDataSender attribute connected only when cluster is configured.

<jmx:query
    failonerror="false"
    name="Catalina:type=Cluster,host=${tomcat.application.host}"
    resultproperty="cluster"
/>
<jmx:get
    name="Catalina:type=IDataSender,host=${tomcat.application.host},senderAddress=${cluster.backup.address},senderPort=${cluster.backup.port}" 
    attribute="connected"
    resultproperty="datasender.connected"
    if="cluster.0.name" />

JMXAccessorSetTask: set attribute value ant task

Attribute	Description
`name`	Full qualified JMX ObjectName -- Catalina:type=Server
`attribute`	Existing Mbean attribute (see JBoss Web mbean description above).
`value`	Value to set to attribute.
`type`	Type of the attribute. [java.lang.String]
`ref`	JMX Connection reference. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]

Example to set remote mbean attribute value

    <jmx:set
        name="Catalina:type=Manager,path=/servlets-examples,host=localhost" 
        attribute="maxActiveSessions"
        value="500"
        type="int"
    />

JMXAccessorInvokeTask: invoke Mbean operation ant task

Attribute	Description
`name`	Full qualified JMX ObjectName -- Catalina:type=Server
`operation`	Existing Mbean operation.
`ref`	JMX Connection reference. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]
`resultproperty`	Save result at this project property.
`delimiter`	Split result with delimiter (java.util.StringTokenizier) and use resultproperty as prefix to store tokens.
`separatearrayresults`	When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.lenght). [true]

stop an application

    <jmx:invoke
        name="Catalina:type=Manager,path=/servlets-examples,host=localhost" 
        operation="stop"/>

Now you can find the sessionid at ${sessions.[0..N} properties and access the count with ${sessions.lenght} property.

Example to get all sessionids

    <jmx:invoke
        name="Catalina:type=Manager,path=/servlets-examples,host=localhost" 
        operation="listSessionIds"
        resultproperty="sessions"
        delimiter=" "        
    />

Now you can find the sessionid at ${sessions.[0..N} properties and access the count with ${sessions.lenght} property.

Example to get remote mbean session attribute from session ${sessionid.0}

    <jmx:invoke
        name="Catalina:type=Manager,path=/ClusterTest,host=localhost" 
        operation="getSessionAttribute"
        resultproperty="hello">
         <arg value="${sessionid.0}"/>
         <arg value="Hello" />
 </jmx:invoke>

Example to create a new access logger valve at vhost localhost

 <jmx:invoke
         name="Catalina:type=MBeanFactory" 
         operation="createAcccesLoggerValve"
         resultproperty="acccesLoggerObjectName"
 >
     <arg value="Catalina:type=Host,host=localhost"/>
 </jmx:invoke>

Now you can find new Mbean with name stored at ${acccesLoggerObjectName} proeprty.

JMXAccessorQueryTask: query Mbean ant task

Attribute	Description
`name`	Full qualified JMX ObjectName -- Catalina:type=Server
`ref`	JMX Connection reference. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]
`resultproperty`	Save result at this project property.
`attributebinding`	Bind ALL MBean attributes in addition to name. [false]
`delimiter`	Split result with delimiter (java.util.StringTokenizier) and use resultproperty as prefix to store tokens.
`separatearrayresults`	When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.lenght). [true]

Get all Manager ObjectNames from all services and Hosts

  <jmx:query
           name="Catalina:type=Manager,* 
           resultproperty="manager" />

Now you can find the Session Manager at ${manager.[0..N].name} properties and access the result object counter with ${manager.length} property.

Example to get the Manager from servlet-examples application an bind all mbean properties

  <jmx:query
           name="Catalina:type=Manager,path=/servlet-examples,host=localhost*" 
           attributebinding="true"
           resultproperty="manager.servletExamples" />

Now you can find the manager at ${manager.servletExamples.0.name} property and can access all properties from this manager with ${manager.servletExamples.0.[manager attribute names]}. The result object counter from MBeans is stored ad ${manager.length} property.

Example to get all MBeans from a server and store inside an external xml property file

<project name="jmx.query"         
            xmlns:jmx="antlib:org.apache.catalina.ant.jmx"
            default="query-all" basedir=".">
<property name="jmx.host" value="localhost"/>
<property name="jmx.port" value="8050"/>
<property name="jmx.username" value="controlRole"/>
<property name="jmx.password" value="tomcat"/>

<target name="query-all" description="Query all MBeans of a server">
<!-- Configure connection -->
<jmx:open 
    host="${jmx.host}"
    port="${jmx.port}"
    ref="jmx.server"
    username="${jmx.username}"
    password="${jmx.password}"/>
<!-- Query MBean list -->
<jmx:query 
    name="*:*"
    resultproperty="mbeans"
    attributebinding="false"/>
    
<echoproperties
    destfile="mbeans.properties"
    prefix="mbeans."
    format="xml"/>
    
<!-- Print results -->
<echo
    message="Number of MBeans in server ${jmx.host}:${jmx.port} is ${mbeans.length}"/>
</target>
</project>

Now you can find all MBeans inside the file mbeans.properties.

JMXAccessorCreateTask: remote create mbean ant task

Attribute	Description
`name`	Full qualified JMX ObjectName -- Catalina:type=Server
`className`	Existing MBean full qualified classname (see JBoss Web mbean description above).
`classLoader`	ObjectName of server or web application classloader ( Catalina:type=ServerClassLoader,name=[server,common,shared] or Catalina:type=WebappClassLoader,path=/myapps,host=localhost).
`ref`	JMX Connection reference. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]

Example to create remote mbean

    <jmx:create
             ref="${jmx.reference}"
             name="Catalina:type=MBeanFactory"
             className="org.apache.commons.modeler.BaseModelMBean"
             classLoader="Catalina:type=ServerClassLoader,name=server">             
             <Arg value="org.apache.catalina.mbeans.MBeanFactory" />
    </jmx:create>

Warning: A lot of JBoss Web mbeans can't be really create and connect with
the parent. The valve, cluster or realm Mbeans are not autconnect with there parent.
Use MBeanFacrory create operation instead.

JMXAccessorUnregisterTask: remote unregister mbean ant task

Attribute	Description
`name`	Full qualified JMX ObjectName -- Catalina:type=Server
`ref`	JMX Connection reference. [jmx.server]
`echo`	Echo the command usage (to analyze access or debugging). [false]

Example to unregister remote mbean

    <jmx:unregister
        name="Catalina:type=MBeanFactory" 
    />

Warning: A lot of JBoss Web mbeans can't be really unregister.
The Mbeans are not deregister from parent. Use MBeanFacrory
remove operation instead.

JMXAccessorCondition: express condition

Attribute	Description
`url`	Set jmx connection url - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi
`host`	Set the host, shortcut the very long url syntax. [localhost]
`port`	Set the remote connection port. [8050]
`username`	Remote jmx connection user name.
`password`	Remote jmx connection password.
`ref`	Name of the internal connection reference. With this attribute you can configure more the one connection inside the same ant project. [jmx.server]
`name`	Full qualified JMX ObjectName -- Catalina:type=Server.
`echo`	Echo the command usage (to analyze access or debugging). [false]
`if`	Only execute if a property of the given name exists in the current project.
`unless`	Only execute if a property of the given name not exists in the current project.
`value`	Second arg for operation.
`type`	Value type to express operation (supports long and double). [long]
`operation`	Express one == equals != not equals > greater than (>) >= greater than or equals (>=) < lesser than (<) <= lesser than or equals (<=) [`==`]

Wait for server connection and that cluster backup node is accessable

      <target name="wait">
         <waitfor maxwait="${maxwait}" maxwaitunit="second" timeoutproperty="server.timeout" >
            <and>
                <socket server="${server.name}" port="${server.port}"/>
                <http url="${url}"/>
                <jmx:condition
                    operation="==" 
                    host="localhost" 
                    port="9014"
                    username="controlRole"
                    password="tomcat"
                    name="Catalina:type=IDataSender,host=localhost,senderAddress=192.168.111.1,senderPort=9025"
                    attribute="connected"
                    value="true"
                />
            </and>
        </waitfor>
        <fail if="server.timeout" message="Server ${url} don't answer inside ${maxwait} sec" />
        <echo message="Server ${url} alive" />
    </target>

JMXAccessorEqualsCondition: equals Mbean ant condition

Attribute	Description
`url`	Set jmx connection url - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi
`host`	Set the host, shortcut the very long url syntax. [localhost]
`port`	Set the remote connection port. [8050]
`username`	Remote jmx connection user name.
`password`	Remote jmx connection password.
`ref`	Name of the internal connection reference. With this attribute you can configure more the one connection inside the same ant project. [jmx.server]
`name`	Full qualified JMX ObjectName -- Catalina:type=Server.
`echo`	Echo the command usage (to analyze access or debugging). [false]

Wait for server connection and that cluster backup node is accessable

      <target name="wait">
         <waitfor maxwait="${maxwait}" maxwaitunit="second" timeoutproperty="server.timeout" >
            <and>
                <socket server="${server.name}" port="${server.port}"/>
                <http url="${url}"/>
                <jmx:equals 
                    host="localhost" 
                    port="9014"
                    username="controlRole"
                    password="tomcat"
                    name="Catalina:type=IDataSender,host=localhost,senderAddress=192.168.111.1,senderPort=9025"
                    attribute="connected"
                    value="true"
                />
            </and>
        </waitfor>
        <fail if="server.timeout" message="Server ${url} don't answer inside ${maxwait} sec" />
        <echo message="Server ${url} alive" />
    </target>