JBoss Community Archive (Read Only)

RHQ 4.9

Shutdown process on AS7

Shutdown expectations

  • Shutdown code should stop some RHQ services gracefully.

  • Currently running Quartz jobs should be able to finish.
    This is something important as job code may not be resilient to interrupts

  • The server should be switched to maintenance mode.

The start script that RHQ server ships sends a TERM signal every two seconds as long as the server remains up.

The old days (RHQ running on top of AS4.2.3)

The shutdown code was executed in a JMX notification listener. On startup, we registered the ShutdownListener:

From the StartupServlet
ObjectName jbossServerName = new ObjectName("jboss.system:type=Server");
MBeanServer jbossServer = MBeanServerLocator.locateJBoss();
jbossServer.addNotificationListener(jbossServerName, new ShutdownListener(), null, null);

Then the ShutdownListener would react on the STOP_NOTIFICATION event:

From the ShutdownListener
if (org.jboss.system.server.Server.STOP_NOTIFICATION_TYPE.equals(notification.getType()))
    // Do shutdown

No component was undeployed before the ShutdownListener was fully executed.

Current situation (RHQ 4.7 running on EAP6.1)

The ShutdownListener has been turned into a @Singleton @Startup bean. The shutdown code is inside its @PreDestroy. This implementation has some issues.

Unpredictable bean shutdown order

When the shutdown code is executed, there is no guarantee that all our beans will still be deployed. The Wildfly team suggested to use @DependsOn annotation to declare dependencies of the ShutdownListener. This is doable for the few dependencies needed to switch the server to maintenance mode. But the Quartz jobs still running during shutdown could virtually depend on any RHQ bean. Obviously we cannot manually add all RHQ beans as dependencies of the ShutdownListener.

Dependencies on MXBeans

Some of RHQ's EJBs require RHQ's MXBeans (like CoreServer) to be deployed. MXBeans are declared in the services SAR. There is no guarantee that MXBeans will still be deployed while the shutdown code is executed.

Besides, the SchedulerService MXBean shutdowns the Quartz scheduler when it is stopped, without waiting for running jobs to complete. So the Quartz scheduler could even be stopped before the ShutdownListener code is executed.

Dependency on the NoTxRHQDS

The Quartz scheduler uses the NoTxRHQDS to persist its state to the database. But there is no guarantee that the NoTxRHQDS will still be deployed while the scheduler is shutdown.

Proposed resolution

Inject the NoTxRHQDS as a @Resource in the ShutdownListener

This will prevent the NoTxRHQDS from being undeployed before the Quartz scheduler finishes to gracefully shutdown.

Convert SAR services into @Singleton beans

The MXBeans classes are already part of the the server EJBs JAR. @Singleton beans will get all features of session beans (injection, dependencies... etc). They will still be exposed via JMX for backward compatibility (registration to the MBeanServer in the @PostConstruct method).

The services SAR module will be deleted.

Tweak RHQ EAR deployment with a set of DeploymentUnitProcessor classes

As a result, the @PostConstruct method of the @RhqLifecycleManager annotated bean would be called only when every other bean is deployed, and the @PreDestroy method before any other bean is undeployed.

These deployment processors would be part of a new RHQ AS7 extension which the RHQ installer would setup.

JBoss.org Content Archive (Read Only), exported from JBoss Community Documentation Editor at 2020-03-13 08:19:36 UTC, last content change 2013-09-18 19:42:21 UTC.