rhq.cassandra.seeds=127.0.0.1|9160|9042,127.0.0.2|9160|9042,127.0.0.3|9160|9042
The purpose of this document is to provide details about when the metrics database will need to be installed, how it might be installed, and the options for configuring the installation.
The installer will lay down a Cassandra distribution with modifications and customizations tailored for RHQ. While we do not intend source code modifications to Cassandra, we are not simply using a stock distribution. For example, the the Apache distribution, which we are using, does not include the JNA libraries which Cassandra uses for compression. We are packaging JNA. As of Cassandra 1.2.0, the Apache distribution does not ship with any authentication/authorization libraries. We are including our own, which are based off the examples in the Cassandra source tree.
During new RHQ installation
Adding nodes to cluster for existing, running RHQ server
On machine that also needs to have RHQ agent installed
On machine that already has RHQ agent installed
During (as part of) RHQ upgrade
Just like the RDBMS, Cassandra is a dependency for RHQ; therefore, it needs to be installed prior to installing RHQ. More precisely though, part of the work that the RHQ installer does involves verifying that it can connect to both the RDBMS as well as Cassandra. This means that once the installer reaches the point where it verifies the connection, Cassandra needs to be installed.
As the demands on Cassandra grow due to a growing inventory and increase in the number of metrics being collected or a increase in the frequency of metrics collections, it may be necessary to expand the cluster by adding more nodes. RHQ will support expanding the cluster without incurring any down time.
A user may want to add nodes to the cluster during an upgrade of RHQ. More importantly the upgrade may involve upgrading Cassandra itself. Upgrading Cassandra will be treated as its own topic and discussed separately.
Stand alone installer (similar to agent installer)
RHQ server installer
RHQ agent installer
RHQ provisioning
A stand alone installer would come in the form of an executable JAR file like the agent installer. It would primarily be used prior to running the RHQ installer during a fresh install of RHQ.
The RHQ installer could be modified such that it has the ability to install Cassandra instead of or in addition to installing the RHQ server. This approach would primarily be used when performing a fresh install of RHQ. Note that this would require putting the full RHQ distribution on each machine where you want to install Cassandra.
The agent installer could be modified such that it has the ability to install Cassandra in addition to the agent. This approach could potentially be used during a fresh install of RHQ as well as for adding nodes to an existing cluster for a running RHQ server.
Using RHQ provisioning requires a running agent on the machine where you want to install Cassandra. Once the agent is up and running it is just a matter of going through the bundle deployment UI or using the remote APIs to execute the installation and deployment of Cassandra. Note that the Cassandra bundle is pre-installed. This approach would be used to add nodes to an existing cluster for a running RHQ server. It is also the approach we will use to upgrade Cassandra.
This section will provide some examples of the different options discussed in the previous two sections.
User downloads Cassandra installer to each machine on which he wants to run Cassandra.
User runs Cassandra installer on each machine.
On the machine where the RHQ server will run, set the rhq.cassandra.seeds property in rhq-server.properties.
See Installing the Server for details about editing rhq-server.properties during RHQ server installation.
Run the RHQ server installer.
After the server installation completes, install the agent on each machine on which Cassandra was installed.
Note that installing the agent is not required for the RHQ server to be operational. It is required so that RHQ can manage Cassandra.
About the rhq.cassandra.seeds property
The rhq.cassandra.seeds property specifies the information RHQ needs to connect to the cluster. It is a comma-delimited list that looks like,
rhq.cassandra.seeds=127.0.0.1|9160|9042,127.0.0.2|9160|9042,127.0.0.3|9160|9042
Each item in the list consists of three, pipe-delimited fields. The first field is the hostname or IP address. The second field is the Thrift or RPC port, and the third field is the native transport or CQL port.
User downloads RHQ installer to each machine on which he wants to run Cassandra.
User runs installer on each machine, specifying the necessary options to install Cassandra instead of RHQ.
On the machine where the RHQ server will run, set the rhq.cassandra.seeds property in rhq-server.properties.
See Installing the Server for details about editing rhq-server.properties during RHQ server installation.
Run the installer to install RHQ server.
After the server installation completes, install the agent on each machine on which Cassandra was installed.
Note that installing the agent is not required for the RHQ server to be operational. It is required so that RHQ can manage Cassandra.
User installs agent on machine.
User starts agent, registering it with RHQ server.
User goes through bundle deployment UI to submit deployment request.
Cassandra node is imported into inventory.
We might want to consider auto-importing the node.
User receives notification that the node is up and operational.
User installs agent on machine, specify necessary options to also install Cassandra.
The agent installer installs Cassandra.
The agent installer installs the agent.
User starts agent, registering it with RHQ server.
Cassandra node is imported into inventory.
We might want to consider auto-importing the node.
User receives notification that the node is up and operational.
this would require the agent update binary (what you are calling the "agent installer") to contain additional "bloat" but for a limited number of agent nodes that actually will need it. The agent update binary is used for auto-updating (it can be downloaded via http://localhost:7080/agentupdate/download - this is how an old agent will download the bits for an upgrade automatically). So I would hesitate to say put it in the standard agent update binary since we want to limit the size of this binary file (it will need to be streamed down to all agents that need to auto-update). We could, however, ship TWO versions of this - say, localhost:7080/agentupdate/download and localhost:7080/agentupdate/download-with-metricsdb. We could then configure the agents to know which to download during an auto-upgrade (there is already a configuration preference in the agent that has this). Manually, an admin would just go to the URL for the agent he is interested in installing.