JBoss Community Archive (Read Only)

RHQ

Agent Plugin Administration

This page discussed agent plugins. If you are interested in updating server plugins, please see Server Plugin Administration.

Adding or Updating Agent Plugins

There are times when you want to install a new agent plugin or update an existing agent plugin to your RHQ environment. This page explains the steps needed to perform this task.

  1. Log into the RHQ GUI as a user with the proper permissions to edit the server settings

  2. Go to the Administration>Configuration>AgentPlugins

    images/author/download/attachments/30900879/admin-agentplugins.png

  3. Upload the new or updated plugin jar file using the "Upload Plugin" option in the GUI (select the plugin jar file then press the "Upload" button).

    • If you do not want to use the GUI to upload the jar file, you can explicitly copy your plugin .jar file to the plugins dropbox directory located at <server-install-dir>/plugins. Note that for RHQ 2.3 and below, the dropbox location is not supported, the place to copy the agent plugin jars for those versions of RHQ is <server-install-dir>/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins.

    • If you see problems hot deploying plugins into the server, you can shut down the server and copy the plugin jars to the plugins directory, then restart the server.

    • If you have more than one RHQ Server in your RHQ Server Cloud, you only need to deploy the plugins to one of the servers; the other servers will automatically detect and copy the plugins.

  4. At this point, you can wait for the server to auto-detect your new plugin jar file, or you can force it to detect it immediately by pressing the "Scan for Updates" button. The server will log messages in its log file when the plugin has been detected and deployed properly.

  5. Once the RHQ Server has detected and deployed the plugin jars, you can then push the plugins out to the agents. You can do this in one of several ways:

    • If you have the agent prompt available, you can type in and execute the command "plugins update" at the agent prompt.

    • If you have the agent imported into your RHQ inventory, you can execute the operation "Update All Plugins" which tells the agent to pull down any new or updated plugins that the agent does not yet have

    • If you have multiple agents imported into inventory, you can execute a group operation of "Update All Plugins" so all of your agents pull down the plugins.

Plugin Versioning

Note that a plugin will get updated if the new plugin being installed has a higher version string than the one already deployed. A plugin defines its version one of two ways:

  1. In the plugin jar's META-INF/MANIFEST.MF file, in the value for "Implementation-Version:"

  2. In the plugin jar's rhq-plugin.xml descriptor in the root "plugin" XML element node's "version" attribute (e.g. "<plugin version="1.0" ... />")
    If the rhq-plugin.xml defines a version, any Implementation-Version specified in MANIFEST.MF is ignored.

If you try to update a plugin and the new plugin has the same version as the old plugin, the new plugin will only take effect if the timestamp of the new plugin jar file is newer than when the old plugin jar was installed.

Plugin Scanner Period

The RHQ Server will periodically scan for new, enabled, disabled or deleted plugins. It does so, by default, every 5 minutes. However, you can configure this time by editing rhq-server.properties and setting the value rhq.server.plugin-scan-period-ms to the number of milliseconds you want your RHQ Server to periodically check (if that setting does not exist in your properties file yet, just add it to the bottom).

For example, if you are developing a new plugin, you can have your RHQ Server check for new or updated plugins every 30 seconds by setting this value to 30000 in rhq-server.properties like this: 

rhq.server.plugin-scan-period-ms=30000

Disabling Agent Plugins

In RHQ 3.x and after, you can disable plugins. To do this, shutdown your agents, select the plugins to disable, and click the Disable button. You can then restart the agents.

There is one plugin that you must never disable - it is required for your agents to run properly. The "platform" plugin (which you will see has the name "Operating System Platforms") should always be enabled.

Read https://bugzilla.redhat.com/show_bug.cgi?id=535894#c10 - this feature was implemented such that disabling plugins seems to work better than I thought. It may not require the agents to be restarted - need someone to test this and document if that is really not needed.

Deleting Agent Plugins

Deleting agents plugins is new to RHQ 4. Deleting a plugin causes several things to happen.

  • All resource types defined by the plugin are purged from the system.

  • All meta data associated with the aforementioned resource types (e.g., metric definitions, operation definitions, configuration definitions, etc.) are purged from the system.

    • To be precise, this meta data includes anything that can be declared inside any of <platform>, <server>, or <service>.

  • All resources of the aforementioned types are purged from the system.

  • The plugin is marked deleted in the database (This will be further discussed later).

  • The plugin JAR file is deleted from each RHQ Server.

  • The plugin JAR file is deleted from the plugins directory on each agent.

  • All resource types defined by the plugin are purged from agents' local inventories.

  • All meta data associated with the aforementioned resource types is purged from agents' local inventories.

  • All resources of the aforementioned types are purged from agents' local inventories.

Plugin Deletion Cannot Be Undone

Disabling a plugin is a reversible operation in that the plugin can later be re-enabled. Deleting a plugin however cannot be undone. Its results are permanent. Specifically, all of the resources and type definitions that are deleted cannot be restored. Proceed with caution when deleting a plugin.

Three Step Deletion

Plugin deletion is a three-step process. The first part happens synchronously when the user submits the deletion request from the UI. This part is intended to run quickly so that control is returned back to the user as quickly as possible. In the first step the plugin, all resource types from the plugin, and all resources of those types are marked deleted. Once a plugin, resource type, or resource is marked deleted, it is effectively deleted from the user's perspective. A resource type or resource that has been marked deleted is not accessible from the UI or from the CLI.

The second step is done asynchronously as part of a scheduled job because it can potentially be a time-consuming operation. The scheduled job performs the resource type deletion which includes all of the associated meta data definitions such as metric definitions, operation definitions, and configuration definitions.

The third step is the actual removal of the plugin from the database. This happens only once all the resource types of the plugin are removed and all the RHQ servers in the HA cluster have acknowledged they know about plugin deletion. The acknowledgement is needed to prevent a "race condition" where one server would delete the plugin from the database and the other server, seeing the plugin still on its file system and not in the database would think it was new and installed it (again).

The latter two steps are automatic and happen without user intervention in the background. From the user's point of view, the plugin is effectively deleted after the first (quick) step.

As mentioned above, the plugin deletion is not immediate and requires asynchronous background processing after the control returns back to the user. Therefore it may happen that if you want to redeploy a plugin you only recently deleted (some minutes ago), the operation may fail. To prevent that from happening, wait at least 5 minutes before trying to redeploy a deleted plugin. Notably, the plugin will NOT be deleted before ALL servers in the HA cluster have acknowledged its deletion.

Plugin Dependencies

A plugin cannot be deleted without also deleting all of the other plugins that depend on it. Suppose you have plugins X, Y, and Z installed. X depends on Y, and Y depends on Z. If you want to delete Z, then you will also have to delete Y and X. If you try to delete a plugin without also deleting all of the other plugins that depend on it, then you will get an error message indicating all of the plugins that need to be deleted.

Updating Agents

Agents are automatically updated; however the update may not happen immediately. An agent periodically sends inventory reports up to the server. If the inventory report contains a resource type that has been deleted, the server will throw an exception that gets propagated back down to the agent. This lets the agent know it has one or more stale type definitions. At this point the agent will restart itself. Part of the agent start up process includes checking for updated plugins. It is during this update process that the plugin will get deleted. The agent's local inventory will be purged of the deleted type definitions and resources a bit further along in the start up process. Since plugin deletion on the agent side is carried out as part of the start up process, you can force an agent to update itself immediately by restarting the agent.

With the automatic restart there are a couple different possible failure points that need to be discussed. First, it is possible that the agent will fail to shut down. If this happens, the agent will log the following message, 

The agent could not be shut down and rebooting the plugin container failed. Please check the logs for errors and manually restart the agent as soon as possible.

If the agent fails to shut down, plugin deletion on the agent will not take place. On the one hand failure to shut down is not necessarily a severe issue because the agent can operate normally with a plugin that has been deleted on the server. It essentially is just doing more work than it needs to. On the other hand, failure to shut down probably indicates another problem that should be investigated. The second failure point involves the agent failing to start back up. If this happens, the agent will log the following message, 

Restarting the agent has failed. Please check the logs for errors and manually restart the agent as soon as possible.

This is a severe problem since the agent is basically offline, and as the log message says, a manual restart will be required.

JBoss.org Content Archive (Read Only), exported from JBoss Community Documentation Editor at 2020-03-11 13:29:50 UTC, last content change 2014-06-13 15:33:43 UTC.