JBoss Community Archive (Read Only)

RHQ 4.9

Agent Plugin Administration

This page discussed agent plugins. If you are interested in updating server plugins, please see Server Plugin Administration.

Adding or Updating Agent Plugins

There are times when you want to install a new agent plugin or update an existing agent plugin to your RHQ environment. This page explains the steps needed to perform this task.

  1. Log into the RHQ GUI as a user with the proper permissions to edit the server settings

  2. Go to the Administration>Configuration>AgentPlugins

    images/author/download/attachments/73139937/admin-agentplugins.png

  3. Upload the new or updated plugin jar file using the "Upload Plugin" option in the GUI (select the plugin jar file then press the "Upload" button).

    • If you do not want to use the GUI to upload the jar file, you can explicitly copy your plugin .jar file to the plugins dropbox directory located at <server-install-dir>/plugins. Note that for RHQ 2.3 and below, the dropbox location is not supported, the place to copy the agent plugin jars for those versions of RHQ is <server-install-dir>/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins.

    • If you see problems hot deploying plugins into the server, you can shut down the server and copy the plugin jars to the plugins directory, then restart the server.

    • If you have more than one RHQ Server in your RHQ Server Cloud, you only need to deploy the plugins to one of the servers; the other servers will automatically detect and copy the plugins.

  4. At this point, you can wait for the server to auto-detect your new plugin jar file, or you can force it to detect it immediately by pressing the "Scan for Updates" button. The server will log messages in its log file when the plugin has been detected and deployed properly.

  5. Once the RHQ Server has detected and deployed the plugin jars, you can then push the plugins out to the agents. You can do this in one of several ways:

    • If you have the agent prompt available, you can type in and execute the command "plugins update" at the agent prompt.

    • If you have the agent imported into your RHQ inventory, you can execute the operation "Update All Plugins" which tells the agent to pull down any new or updated plugins that the agent does not yet have

    • If you have multiple agents imported into inventory, you can execute a group operation of "Update All Plugins" so all of your agents pull down the plugins.

Plugin Versioning

Note that a plugin will get updated if the new plugin being installed has a higher version string than the one already deployed. A plugin defines its version one of two ways:

  1. In the plugin jar's META-INF/MANIFEST.MF file, in the value for "Implementation-Version:"

  2. In the plugin jar's rhq-plugin.xml descriptor in the root "plugin" XML element node's "version" attribute (e.g. "<plugin version="1.0" ... />")
    If the rhq-plugin.xml defines a version, any Implementation-Version specified in MANIFEST.MF is ignored.

If you try to update a plugin and the new plugin has the same version as the old plugin, the new plugin will only take effect if the timestamp of the new plugin jar file is newer than when the old plugin jar was installed.

Plugin Scanner Period

The RHQ Server will periodically scan for new, enabled, disabled or deleted plugins. It does so, by default, every 5 minutes. However, you can configure this time by editing rhq-server.properties and setting the value rhq.server.plugin-scan-period-ms to the number of milliseconds you want your RHQ Server to periodically check (if that setting does not exist in your properties file yet, just add it to the bottom).

For example, if you are developing a new plugin, you can have your RHQ Server check for new or updated plugins every 30 seconds by setting this value to 30000 in rhq-server.properties like this: 

rhq.server.plugin-scan-period-ms=30000

Disabling Agent Plugins

In RHQ 3.x and after, you can disable plugins. To do this, shutdown your agents, select the plugins to disable, and click the Disable button. You can then restart the agents.

There is one plugin that you must never disable - it is required for your agents to run properly. The "platform" plugin (which you will see has the name "Operating System Platforms") should always be enabled.

Read https://bugzilla.redhat.com/show_bug.cgi?id=535894#c10 - this feature was implemented such that disabling plugins seems to work better than I thought. It may not require the agents to be restarted - need someone to test this and document if that is really not needed.

Deleting Agent Plugins

Deleting agents plugins is new to RHQ 4. Deleting a plugin causes several things to happen.

  • All resource types defined by the plugin are purged from the system.

  • All meta data associated with the aforementioned resource types (e.g., metric definitions, operation definitions, configuration definitions, etc.) are purged from the system.

    • To be precise, this meta data includes anything that can be declared inside any of <platform>, <server>, or <service>.

  • All resources of the aforementioned types are purged from the system.

  • The plugin is marked deleted in the database (This will be further discussed later).

  • The plugin JAR file is deleted from each RHQ Server.

  • The plugin JAR file is deleted from the plugins directory on each agent.

  • All resource types defined by the plugin are purged from agents' local inventories.

  • All meta data associated with the aforementioned resource types is purged from agents' local inventories.

  • All resources of the aforementioned types are purged from agents' local inventories.

Plugin Deletion Cannot Be Undone

Disabling a plugin is a reversible operation in that the plugin can later be re-enabled. Deleting a plugin however cannot be undone. Its results are permanent. Specifically, all of the resources and type definitions that are deleted cannot be restored. Proceed with caution when deleting a plugin.

Two Step Deletion

Plugin deletion is a two-step process. The first part happens synchronously when the user submits the deletion request from the UI. This part is intended to run quickly so that control is returned back to the user as quickly as possible. In the first step the plugin, all resource types from the plugin, and all resources of those types are marked deleted. Once a plugin, resource type, or resource is marked deleted, it is effectively deleted from the user's perspective. A resource type or resource that has been marked deleted is not accessible from the UI or from the CLI.

The second step is done asynchronously as part of a scheduled job because it can potentially be a time-consuming operation. The scheduled job performs the resource type deletion which includes all of the associated meta data definitions such as metric definitions, operation definitions, and configuration definitions.

Plugin Dependencies

A plugin cannot be deleted without also deleting all of the other plugins that depend on it. Suppose you have plugins X, Y, and Z installed. X depends on Y, and Y depends on Z. If you want to delete Z, then you will also have to delete Y and X. If you try to delete a plugin without also deleting all of the other plugins that depend on it, then you will get an error message indicating all of the plugins that need to be deleted.

Updating Agents

Agents are automatically updated; however the update may not happen immediately. An agent periodically sends inventory reports up to the server. If the inventory report contains a resource type that has been deleted, the server will throw an exception that gets propagated back down to the agent. This lets the agent know it has one or more stale type definitions. At this point the agent will restart itself. Part of the agent start up process includes checking for updated plugins. It is during this update process that the plugin will get deleted. The agent's local inventory will be purged of the deleted type definitions and resources a bit further along in the start up process. Since plugin deletion on the agent side is carried out as part of the start up process, you can force an agent to update itself immediately by restarting the agent.

With the automatic restart there are a couple different possible failure points that need to be discussed. First, it is possible that the agent will fail to shut down. If this happens, the agent will log the following message, 

The agent could not be shut down and rebooting the plugin container failed. Please check the logs for errors and manually restart the agent as soon as possible.

If the agent fails to shut down, plugin deletion on the agent will not take place. On the one hand failure to shut down is not necessarily a severe issue because the agent can operate normally with a plugin that has been deleted on the server. It essentially is just doing more work than it needs to. On the other hand, failure to shut down probably indicates another problem that should be investigated. The second failure point involves the agent failing to start back up. If this happens, the agent will log the following message, 

Restarting the agent has failed. Please check the logs for errors and manually restart the agent as soon as possible.

This is a severe problem since the agent is basically offline, and as the log message says, a manual restart will be required.

Purging Plugins

Several things get removed from the system as part of plugin deletion. One thing though that does not get deleted, is the row in the RHQ_PLUGINS table. There is a column named, STATUS, which has a value of either INSTALLED or DELETED. When a plugin is deleted, the STATUS column is set to DELETED. Since the row is actually deleted, we make a distinction between deleting and purging a plugin. The reason that this distinction is necessary is because if in an HA deployment, one RHQ server deletes the row in RHQ_PLUGINS, another RHQ could reinstall the plugin. This would happen because the second server would find the plugin JAR file on its file system but not in the database; so, it could then only assume it has detected a new plugin. If the RHQ servers are running on a shared file system, then this would not be an issue; however, no assumptions are made about the file systems on which RHQ servers are deployed.

Let's consider the scenarios for when purging a plugin is necessary. Fortunately, there are not many scenarios when purging is necessary. If you delete a plugin, and try to reinstall the same or a different version of it, the installation will fail, and you should find something like the following in your server log, 

ERROR [org.rhq.enterprise.server.core.plugin.ProductPluginDeployer] Failed to register RHQ plugin file
[file:/rhq/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins/rhq-twitter-plugin-4.0.0-SNAPSHOT.jar]
javax.ejb.EJBException: javax.persistence.EntityExistsException: org.hibernate.exception.ConstraintViolationException:
Could not execute JDBC batch update

Reinstalling the plugin, whether it is the same or a different version, is the one and only scenario where you absolutely have to purge a plugin. Let's look at how we go about purging a plugin.

First, you need to press the "Show Deleted" button which shows all plugins, even those previously deleted, in the list of plugins.

images/author/download/attachments/73139937/admin-agentdeletedplugin.png

Here you see the plugins that have been previously deleted. To purge this plugin completely from the system, select the deleted plugins you want purged and click the "Purge" button.

Unexpected Results Possible When Purging in Clustered Deployment

There is still a timing issue when you purge a plugin. If you have two servers, A and B, and purge the plugin from A before B has removed the plugin from its file system, then B would reinstall the plugin. It is recommended that you wait a minimum of five minutes after deleting a plugin before purging it. Five minutes is the default interval of the agent plugin scanner (see above for more information on Plugin Scanner Period setting). The only way to be absolutely certain that the plugin will not get reinstalled is to first verify that it has been removed from each file system of every RHQ server in the cluster.

JBoss.org Content Archive (Read Only), exported from JBoss Community Documentation Editor at 2020-03-13 08:47:53 UTC, last content change 2013-09-18 19:43:31 UTC.