<server name="JBossAS" ...> ... <destination-base-dir>...</destination-base-dir> ... </server>
Minutes: http://people.redhat.com/~manderse/irc.freenode.org/meetings/rhq/2011/rhq.2011-05-26-17.00.html
Log: http://people.redhat.com/~manderse/irc.freenode.org/meetings/rhq/2011/rhq.2011-05-26-17.00.log.html
Today, bundles can only be provisioned to groups of platform resources.
However, deploying bundles to other non-platform resource types could be possible (at least conceptually). For example, if we had a group of JBossAS Server resources, it could mean that we deploy bundles at the "jboss.server.home.dir" location of the JBossAS Servers. This way we could use bundles to deploy patches to core JBossAS libraries or deploy applications to the deploy/ directory.
This obviously would require some changes to the bundle implementation code and we would have to workaround or change some assumptions we made when we originally designed and built the bundle subsystem.
We first must decide if this is even a good idea. While it is technically feasible to deploy to non-platform resources, do we want to enable this?
The bundle subsystem can be confusing to a person new to RHQ. The UI can be complex to fully understand right away. Does adding the ability to install things to other resources other than platforms create the potential for more confusion? Does it add enough value to make any added complexity worth it?
These questions need to be answered before we move on to the design/implementation phase.
We would need to introduce a new metadata tag to the agent plugin descriptor schema - let's call it the "<destination-base-dir>" tag since it would refer to a root directory location that can be used as a bundle destination directory.
<server name="JBossAS" ...> ... <destination-base-dir>...</destination-base-dir> ... </server>
To support further enhancement in the future, we should probably enclose this in an outer <bundle-target> tag, thus providing several extension points (e.g. we can add attributes to the outer bundle tag, we can support multiple destination directories a user can choose from and/or we can add additional child elements should we have more metadata that needs to be given for <bundle-target>)
<server name="JBossAS" ...>
...
<bundle-target>
<destination-base-dir>...</destination-base-dir>
</bundle-target>
...
</server>
Resource types are not required to specify this tag, it is optional (thus we still support legacy plugins and type definitions). If a type is defined without such a <bundle-target> tag, then resources of that type can not serve as targets for bundle provisioning.
We could use some special notation inside this new tag that would make sense to the core server. For example, the installation directory of all JBossAS Server instances are stored in the resource's plugin configuration. We can introduce something like what the dynagroup language has:
<destination-base-dir>pluginConfiguration[install.dir]</destination-base-dir>
This would require <plugin-configuration> to have defined inside of it a simple property called "install.dir". Another possible value can be one that signifies that the destination directory is obtained via resource configuration:
<destination-base-dir>resourceConfiguration[install.dir]</destination-base-dir>
This would require <resource-configuration> to have defined inside of it a simple property called "install.dir".
Yet another possible value we can support is the specification of a measurement trait:
<destination-base-dir>trait[install.dir]</destination-base-dir>
This would require a <metric> trait to be defined whose name is "install.dir". For example, So "trait[deploy.dir]" could evaluate to something like "/opt/jboss/server/default/deploy".
We could additionally support multiple destination base directories, in case the resource supports multiple places where bundles can be installed - notice to support multiple ones, we'd provide a "name" attribute to the destination-base-dir element (a default name can be assumed if none is specified, something like "Destination Base Directory":
<server name="JBossAS:JBossAS Server" ...>
...
<bundle-target>
<destination-base-dir name="Library Directory">pluginConfiguration[lib.dir]</destination-base-dir>
<destination-base-dir name="Deploy Directory">pluginConfiguration[deploy.dir]</destination-base-dir>
</bundle-target>
...
</server>
After thinking of this some more, rather than perform our own special parsing after we parse the XML, let's use the XML parsing do the job for us. So rather than use the special notation "pluginConfig[propName]", split those two pieces of information into their own XML elements:
<destination-base-dir name="Deploy Directory"> <value-context>pluginConfiguration</value-context> <value-name>deploy.dir</value-name> </destination-base-dir>
This provides the additional feature of supporting a fixed number of values for <value-context> (e.g. "pluginConfiguration", "resourceConfiguration", "measurementTrait") thus allowing the XML parser to perform its own syntax checking and validation of those values.
The UI will show help text next to the "name" of the destination base directory, so it is best to supply the optional "description" attribute too.
So, a full example would be this:
<server name="JBossAS:JBossAS Server" ...>
<bundle-target>
<destination-base-dir name="Library Directory" description="Where the jar libraries are">
<value-context>pluginConfiguration</value-context>
<value-name>lib.dir</value-name>
</destination-base-dir>
<destination-base-dir name="Deploy Directory" description="Where the deployments are">
<value-context>pluginConfiguration</value-context>
<value-name>deploy.dir</value-name>
</destination-base-dir>
</bundle-target>
</server>
We could add a "targetResourceTypes" attribute to the Ant recipe XML to help RHQ determine where this recipe is allowed to be deployed.
<rhq:deployment-unit targetResourceTypes="JBossAS:JBossAS Server,JBossAS5:JBossAS Server" .../>
If not specified, we just assume the bundle can be deployed anywhere.
We could also add a "targetDestinationBaseDir" that is a hint from the recipe author to indicate which <destination-base-dir> the bundle should be deployed to. This is how we can indicate that a bundle is a patch to the core libraries of a server, or if its an application that can be deployed in an app server. Here's an example:
<!-- In the agent plugin descriptor: -->
<bundle-target>
<destination-base-dir name="Install Directory">...</destination-base-dir>
<destination-base-dir name="Deploy Directory">...</destination-base-dir>
</bundle-target>
<!-- In the ANT recipe: -->
<rhq:bundle name="Server Patch Bundle">
<rhq:deployment-unit targetDestinationBaseDir="Install Directory" targetResourceTypes="JBossAS:JBossAS Server">
<!-- ...and the rest... -->
</rhq:deployment-unit>
</rhq:bundle>
Note that neither targetDestinationBaseDir nor targetResourceTypes are required or necessary for the bundle to be deployed successfully. If not specified, the bundle can be deployed to any destination base directory on any resource that supports bundles. However, when specified, they help provide hints to RHQ to make sure the bundle is being deployed to the properly type of resource for which it was built.
If targetDestinationBaseDir is specified, then the UI will not have to ask the user where he wants the bundle deployed - the destination base directory is already determined. However, if not specified, the UI needs to provide a drop down menu to let the user pick which base directory that want to install to. If there is only one <destination-base-dir> specified for the resource type (determined by the compatible group type), then again there is no need to ask the user since the base dir is already determined.
It would be most desirable to not have to store these attribute values anywhere in the database, to avoid db upgrade problems in the future. They should only be useful in the UI.
Because I'm not sure where these optional recipe data will be stored, and if not in the DB, I'm not sure how easy and efficient it will be for the GWT UI to obtain this data, I'm going to skip implementing changes to the recipe. When the actual feature is done, we can revisit this and add in this optional feature. By then, it will be more clear how all of this will work and how we can have the GWT UI support this
There are a couple of additions on the database schema to support this. First, there must now be a relationship from ResourceType to Configuration. This is the bundle-target's metadata, specifically the destination base directory metadata.
Next we need a new attribute on BundleDestination called something like "bundleDestinationBaseDirectoryName". This is the name of the <destination-base-dir> as selected by the user. This is where the bundle will be destined to be deployed. For backward compatibility, we'll need to update all existing bundle destinations so this value is "Root File System" which is what our platform plugin calls its one and only dest base dir. Since all bundle destinations up till now could only have been deployed on platforms, this is a valid default to force it to.
The server only needs to know that a resource type has a <destination-base-dir> defined. It technically doesn't need to know its value - it doesn't need to know the root directory is obtained from the plugin config, or resource config, or trait. If a resource type has any <bundle-target> metadata, the RHQ Server knows resources of that type can be targets for bundle deployment. With this basic knowledge, we can query the database for types that support bundle deployment.
Today, the GWT bundle wizard provides a drop down menu item of compatible groups whose members are only platforms. This is because, today, it doesn't make sense to attempt to deploy bundles to non-platforms - we have no idea where to put the bundle under, say, an Apache Web Server or JBossAS Server. However, with this added support of <bundle-target> metadata, the GWT wizard will need to provide a drop down of all groups whose members are types that support bundle deployments - which can be non-platforms.
Other than this new query that needs to be used, the wizards can probably remain as-is. The user can still specify a destination directory as always. Today, this is the full, absolute path of where the bundle is to be deployed on the platform's file system. However, if the user specifies a relative path, it can be relative to the <destination-base-dir>, thus allowing the user to deploy to a subdirectory under the resource.
We will need to also prevent/warn users from doing things which we would expect they would not want to do, e.g. deploying a bundle into the root of the server in such a way that the bundle will wipe out their jboss install. Maybe a warning on the wizard screen would be sufficient here. I'm not sure what types of things we should warn about, but there might be some things we can pre-emptively discourage from the GUI.
Now that we are introducing a resource-specific bundle distribution subsystem, we'd have to add a new "Bundle" tab, OR a new subtab under the Content tab.
The <bundle-target> metadata tag tells the RHQ Agent (more specifically, probably the Bundle Manager in the plugin container) what it should use to determine where a bundle should be deployed on a specific resource. Should it look in a plugin configuration property? A resource config property? A trait? What's the name of the property or trait? Once an agent knows this, it will use that directory as the "base" or root directory for the bundle destination. If the user specified a relative subdirectory as a destination directory, that relative subdirectory is under the root directory and is useful if the user wants to put the bundle somewhere underneath the root location of the destination directory. For example, if a plugin descriptor has this:
<destination-base-dir name="Install Directory"> <value-context>pluginConfiguration</value-context> <value-name>install.dir</value-name> </destination-base-dir>
and a resource's "install.dir" plugin configuration property value is "/opt/my-resource", and the user specified "my-subdir/blah" as the bundle destination directory in the UI, the agent would deploy the bundle as if the full, absolute path for the bundle deployment is "/opt/my-resource/my-subdir/blah".
So far I have no envisioned a requirement that plugins implement a BundleTargetFacet. All of the work to support this is done in the plugin container and the plugin developer only has to add a metadata tag in the descriptor to enable support. However, it may be a nice feature to incorporate an optional bundle target facet to facility the plugin to inject its own handling of the bundle deployment. For example, we could call the facet before and after a bundle deployment so a resource component can do their own custom work.
One problem with introducing a facet is that it would require a resource to be started in order for bundle deployments to work. If a resource component has not been started, the facet will not be able to be called. Unsure how this would negatively affect the bundle deployment capability (i.e. is it OK to require a component to have been started in order to be able to deploy bundles?)
This new capability is a change in our assumptions we made when we originally implemented bundle provisioning. The main change is that we no longer assume platforms are the only types of resources that can have bundles deployed to them. This means we need to ensure any assumptions we make in the code is revised. For example, we assume users can only select groups that only have platforms as members so we have code that verifies that groups have only platform members. Also, the UI shows each resource with the platform icon next to it when looking at the bundle deployment information - we need to make sure we do no hardcode things like icons to display and we need to make sure our UI displays show information that is appropriate for non-platforms as well as platforms (columns that we show, for example).
We also have to check the agent bundle plugin code to make sure there are no assumptions about the resources were we are deploying bundles. For example, do we assume the root platform resource is being used when the agent is deploying bundles?
The bundle subsystem data model needs to be examined to make sure it doesn't assume platforms are the only resources that bundles can be deployed to. The bundle tables and their relationships to other tables needs to be checked for this kind of assumption.
We'll have to update Provisioning to reflect the new functionality.
This design is mixing plugin type knowledge (from the agent plugin descriptor) with bundle recipes. This may open a can of worms with respect to DB upgrades. It also possibly may limit the ability to develop other bundle handlers (like the file template bundle handler or other custom handlers that might be built in the future). We need to really make sure we do not expose too much and make sure we do not cause more problems than we solve. Persisting plugin type metadata that we find in the recipe to the DB should be avoided if at all possible. But even so, there will be type metadata encoded in recipes. If a plugin is updated and this type metadata is changed or removed, it will invalidate those bundles.
Originally, bundles were designed to lay down complete servers (app server with apps already in place, etc). The latest proposal moves toward smaller, one-off deployments. We may need to work out the semantics around when one bundle's deployment directory is a subdirectory of another bundle's deployment directory. For example, suppose deployed bundle-1 is an app-server and deployed bundle-2 is a web-app inside bundle-1's deploy/ directory. What happens to bundle-2 (the web-app) when bundle-1 (the app server) gets reverted or upgraded?
Currently, the only solution we can offer is to document the need for bundle authors to ensure they add the "inner bundle destination directory" is flagged as an "ignored" directory in the outer-bundle recipe (e.g. bundle-1's recipe needs to be told to ignore its subdirectory where bundle-2 will be deployed.
If multiple bundles are allowed to be deployed in the same bundle destination base dir, where is the deployment meta-data kept? It can't be stored under the destination base dir because it would get mixed up/overwrite other bundles that get deployed there, too. For example, suppose we have a JBossAS Server and it can have webapps deployed to its deploy/ directory. Today if I were to deploy bundle-A to that JBossAS Server, the metadata is stored in deploy/.rhqdeployments. But if I then deploy bundle-B to that JBossAS Server, the metadata will also try to be stored in the same location. This will conflict with bundle-A's metadata.
This feature adds a new twist to the bundle subsystem, and as such will require some release notes.
In past versions, RHQ supported deploying bundles to any group, so long as that group had only platforms as its members. So long as the member resources were of the category PLATFORM, it did not care what resource types those members were (i.e. you could have had a mixed group of Linux, Windows and Solaris platforms, for example).
However, now RHQ supports bundle deployments to non-platforms and as such as changed the way it implements this feature. Because RHQ can now support deploying to any type of resource (but only if those resource types can support bundle deployments), RHQ can no longer support deploying to mixed groups, since mixed groups do not have a single, associated resource type (they can have members of many kinds of resource types - and each kind of platform, Linux or Windows or Solaris for example, are considered different resource types).
Therefore, any previous Bundle Destinations that have been created with a mixed group can no longer be used. You must create a new set of compatible groups and assign them to their own new bundle destination. If you need to deploy to heterogeneous group of platforms, split your mixed group into separate compatible groups. For example, if you had a bundle destination that was assigned a mixed group consisting of 3 Linux platforms and 2 windows platforms, you must now create two compatible groups (one containing the 3 Linux platforms and the other containing the 2 windows platforms) and then create two new destinations for your bundle - one destination assigned the Linux compatible group and one destination assigned the Windows compatible group. Now you can deploy your bundle to the two sets of machines, albeit in two deployments, rather the one.
Due to the new bundle feature allowing RHQ users to deploy bundles to non-platforms, changes needed to be made to the bundle API. If you wrote remote CLI scripts to deploy bundles, the following changes may affect you.
org.rhq.enterprise.server.bundle.BundleManagerRemote.createBundleDestination
This method adds a new "destBaseDirName" parameter. It is the name of the base directory location where the bundle will be deployed. This isn't the directory itself, it refers to the named location as defined in the agent plugin's descriptor for the resource's type.
org.rhq.enterprise.server.bundle.BundleManagerRemote.getResourceTypeBundleConfiguration
Used mainly to support user interfaces (like the RHQ GWT GUI), this provides the bundle configuration for a given compatible group (this is going to be the compatible group's resource type's bundle configuration). Use this to find out the valid values that can be passed as the above "createBundleDestination" method's "destBaseDirName" parameter.