Creating the ModelControllerClient

To create a management client that can connect to your target process's native management socket, simply:

ModelControllerClient client = ModelControllerClient.Factory.create(InetAddress.getByName("localhost"), 9999);

The address and port are what is configured in the target process' <management><management-interfaces><native-interface.../> element.

Typically, however, the native management interface will be secured, requiring clients to authenticate. On the client side, the custom client will need to provide the user's authentication credentials, obtained in whatever manner is appropriate for the client (e.g. from a dialog box in a GUI-based client.) Access to these credentials is provided by passing in an implementation of the javax.security.auth.callback.CallbackHandler interface. For example:

static ModelControllerClient createClient(final InetAddress host, final int port, 
                  final String username, final char[] password, final String securityRealmName) {

    final CallbackHandler callbackHandler = new CallbackHandler() {

        public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
            for (Callback current : callbacks) {
                if (current instanceof NameCallback) {
                    NameCallback ncb = (NameCallback) current;
                    ncb.setName(username);
                } else if (current instanceof PasswordCallback) {
                    PasswordCallback pcb = (PasswordCallback) current;
                    pcb.setPassword(password.toCharArray());
                } else if (current instanceof RealmCallback) {
                    RealmCallback rcb = (RealmCallback) current;
                    rcb.setText(rcb.getDefaultText());
                } else {
                    throw new UnsupportedCallbackException(current);
                }
            }
        }
    };
        
    return ModelControllerClient.Factory.create(host, port, callbackHandler);
}

Creating an operation request object

Management requests are formulated using the org.jboss.dmr.ModelNode class from the jboss-dmr library. The jboss-dmr library allows the complete JBoss AS management model to be expressed using a very small number of Java types. See Detyped management and the jboss-dmr library for full details on using this library.

Let's show an example of creating an operation request object that can be used to read the resource description for the web subsystem's HTTP connector:

ModelNode op = new ModelNode();
op.get("operation").set("read-resource-description");

ModelNode address = op.get("address");
address.add("subsystem", "web");
address.add("connector", "http");

op.get("recursive").set(true);
op.get("operations").set(true);

What we've done here is created a ModelNode of type ModelType.OBJECT with the following fields:

• operation – the name of the operation to invoke. All operation requests must include this field and its value must be a String.

• address – the address of the resource to invoke the operation against. This field's must be of ModelType.LIST with each element in the list being a ModelType.PROPERTY. If this field is omitted the operation will target the root resource. The operation can be targeted at any address in the management model; here we are targeting it at the resource for the web subsystem's http connector.

In this case, the request includes two optional parameters:

• recursive – true means you want the description of child resources under this resource. Default is false

• operations – true means you want the description of operations exposed by the resource to be included. Default is false.

Different operations take different parameters, and some take no parameters at all.

See Format of a Detyped Operation Request for full details on the structure of a ModelNode that will represent an operation request.

The example above produces an operation request ModelNode equivalent to what the CLI produces internally when it parses and executes the following low-level CLI command:

[localhost:9999 /] /subsystem=web/connector=http:read-resource-description(recursive=true,operations=true)

Execute the operation and manipulate the result:

The execute method sends the operation request ModelNode to the process being managed and returns a ModelNode the contains the process' response:

ModelNode returnVal = client.execute(op);
System.out.println(returnVal.get("result").toString());

See Format of a Detyped Operation Response for general details on the structure of the "returnVal" ModelNode.

The execute operation shown above will block the calling thread until the response is received from the process being managed. ModelControllerClient also exposes and API allowing asynchronous invocation:

Future<ModelNode> future = client.executeAsync(op);
. . .  // do other stuff
ModelNode returnVal = future.get();
System.out.println(returnVal.get("result").toString());

Close the ModelControllerClient

A ModelControllerClient can be reused for multiple requests. Creating a new ModelControllerClient for each request is an anti-pattern. However, when the ModelControllerClient is no longer needed, it should always be explicitly closed, allowing it to close down any connections to the process it was managing and release other resources:

client.close();

Simple Operations

A text representation of simple operation would look like this:

{
    "operation" => "write-attribute",
    "address" => [
        ("profile" => "production"),
        ("subsystem" => "threads"),
        ("bounded-queue-thread-pool" => "pool1")
    ],
    "name" => "count",
    "value" => 20
}

Java code to produce that output would be:

ModelNode op = new ModelNode();
op.get("operation").set("write-attribute");
ModelNode addr = op.get("address");
addr.add("profile", "production");
addr.add("subsystem", "threads");
addr.add("bounded-queue-thread-pool", "pool1");
op.get("name").set("count");
op.get("value").set(20);
System.out.println(op);

The order in which the outermost elements appear in the request is not relevant. The required elements are:

• operation – String – The name of the operation being invoked.

• address – the address of the managed resource against which the request should be executed. If not set, the address is the root resource. The address is an ordered list of key-value pairs describing where the resource resides in the overall management resource tree. Management resources are organized in a tree, so the order in which elements in the address occur is important.

The other key/value pairs are parameter names and their values. The names and values should match what is specified in the operation's description.

Parameters may have any name, except for the reserved words operation, address and operation-headers.

Operation Headers

Besides the special operation and address values discussed above, operation requests can also include special "header" values that help control how the operation executes. These headers are created under the special reserved word operation-headers:

ModelNode op = new ModelNode();
op.get("operation").set("write-attribute");
ModelNode addr = op.get("address");
addr.add("base", "domain");
addr.add("profile", "production");
addr.add("subsystem", "threads");
addr.add("bounded-queue-thread-pool", "pool1");
op.get("name").set("count");
op.get("value").set(20);
op.get("operation-headers", "rollback-on-runtime-failure").set(false);
System.out.println(op);

This produces:

{
    "operation" => "write-attribute",
    "address" => [
        ("profile" => "production"),
        ("subsystem" => "threads"),
        ("bounded-queue-thread-pool" => "pool1")
    ],
    "name" => "count",
    "value" => 20,
    "operation-headers" => {
        "rollback-on-runtime-failure => false
    }
}

The following operation headers are supported:

• rollback-on-runtime-failure – boolean, optional, defaults to true. Whether an operation that successfully updates the persistent configuration model should be reverted if it fails to apply to the runtime. Operations that affect the persistent configuration are applied in two stages – first to the configuration model and then to the actual running services. If there is an error applying to the configuration model the operation will be aborted with no configuration change and no change to running services will be attempted. However, operations are allowed to change the configuration model even if there is a failure to apply the change to the running services – if and only if this rollback-on-runtime-failure header is set to false. So, this header only deals with what happens if there is a problem applying an operation to the running state of a server (e.g. actually increasing the size of a runtime thread pool.)

• rollout-plan – only relevant to requests made to a Domain Controller or Host Controller. See "Operations with a Rollout Plan" for details.

• allow-resource-service-restart – boolean, optional, defaults to false. Whether an operation that requires restarting some runtime services in order to take effect should do so. See discussion of resource-services in the "Applying Updates to Runtime Services" section of the Description of the Management Model section for further details.

Composite Operations

The root resource for a Domain or Host Controller or an individual server will expose an operation named "composite". This operation executes a list of other operations as an atomic unit (although the atomicity requirement can be relaxed. The structure of the request for the "composite" operation has the same fundamental structure as a simple operation (i.e. operation name, address, params as key value pairs).

{
    "operation" => "composite",
    "address" => [],
    "steps" => [
         {
              "operation" => "write-attribute",
              "address" => [
                   ("profile" => "production"),
                   ("subsystem" => "threads"),
                   ("bounded-queue-thread-pool" => "pool1")
              ],
              "count" => "count",
              "value" => 20
         },
         {
              "operation" => "write-attribute",
              "address" => [
                   ("profile" => "production"),
                   ("subsystem" => "threads"),
                   ("bounded-queue-thread-pool" => "pool2")
              ],
              "name" => "count",
              "value" => 10
         }
    ],
    "operation-headers" => {
        "rollback-on-runtime-failure => false
    }
}

The "composite" operation takes a single parameter:

• steps – a list, where each item in the list has the same structure as a simple operation request. In the example above each of the two steps is modifying the thread pool configuration for a different pool. There need not be any particular relationship between the steps. Note that the rollback-on-runtime-failure and rollout-plan operation headers are not supported for the individual steps in a composite operation.

The rollback-on-runtime-failure operation header discussed above has a particular meaning when applied to a composite operation, controlling whether steps that successfully execute should be reverted if other steps fail at runtime. Note that if any steps modify the persistent configuration, and any of those steps fail, all steps will be reverted. Partial/incomplete changes to the persistent configuration are not allowed.

Operations with a Rollout Plan

Operations targeted at domain or host level resources can potentially impact multiple servers. Such operations can include a "rollout plan" detailing the sequence in which the operation should be applied to servers as well as policies for detailing whether the operation should be reverted if it fails to execute successfully on some servers.

If the operation includes a rollout plan, the structure is as follows:

{
    "operation" => "write-attribute",
    "address" => [
        ("profile" => "production"),
        ("subsystem" => "threads"),
        ("bounded-queue-thread-pool" => "pool1")
    ],
    "name" => "count",
    "value" => 20,
    "operation-headers" => {
        "rollout-plan" => {
            "in-series" => [
                {
                    "concurrent-groups" => {
                        "groupA" => {
                            "rolling-to-servers" => true,
                            "max-failure-percentage" => 20
                        },
                        "groupB" => undefined
                    }
                },
                {
                   "server-group" => {
                        "groupC" => {
                            "rolling-to-servers" => false,
                            "max-failed-servers" => 1
                        }
                    }
                },
                {
                    "concurrent-groups" => {
                        "groupD" => {
                            "rolling-to-servers" => true,
                            "max-failure-percentage" => 20
                        },
                        "groupE" => undefined
                    }
                }
            ],
            "rollback-across-groups" => true
        }
    }
}

As you can see, the rollout plan is another structure in the operation-headers section. The root node of the structure allows two children:

• in-series – a list – A list of activities that are to be performed in series, with each activity reaching completion before the next step is executed. Each activity involves the application of the operation to the servers in one or more server groups. See below for details on each element in the list.

• rollback-across-groups – boolean – indicates whether the need to rollback the operation on all the servers in one server group should trigger a rollback across all the server groups. This is an optional setting, and defaults to false.

Each element in the list under the in-series node must have one or the other of the following structures:

• concurrent-groups – a map of server group names to policies controlling how the operation should be applied to that server group. For each server group in the map, the operation may be applied concurrently. See below for details on the per-server-group policy configuration.

• server-group – a single key/value mapping of a server group name to a policy controlling how the operation should be applied to that server group. See below for details on the policy configuration. (Note: there is no difference in plan execution between this and a "concurrent-groups" map with a single entry.)

The policy controlling how the operation is applied to the servers within a server group has the following elements, each of which is optional:

• rolling-to-servers – boolean – If true, the operation will be applied to each server in the group in series. If false or not specified, the operation will be applied to the servers in the group concurrently.

• max-failed-servers – int – Maximum number of servers in the group that can fail to apply the operation before it should be reverted on all servers in the group. The default value if not specified is zero; i.e. failure on any server triggers rollback across the group.

• max-failure-percentage – int between 0 and 100 – Maximum percentage of the total number of servers in the group that can fail to apply the operation before it should be reverted on all servers in the group. The default value if not specified is zero; i.e. failure on any server triggers rollback across the group.

If both max-failed-servers and max-failure-percentage are set, max-failure-percentage takes precedence.

Looking at the (contrived) example above, application of the operation to the servers in the domain would be done in 3 phases. If the policy for any server group triggers a rollback of the operation across the server group, all other server groups will be rolled back as well. The 3 phases are:

1. Server groups groupA and groupB will have the operation applied concurrently. The operation will be applied to the servers in groupA in series, while all servers in groupB will handle the operation concurrently. If more than 20% of the servers in groupA fail to apply the operation, it will be rolled back across that group. If any servers in groupB fail to apply the operation it will be rolled back across that group.

2. Once all servers in groupA and groupB are complete, the operation will be applied to the servers in groupC. Those servers will handle the operation concurrently. If more than one server in groupC fails to apply the operation it will be rolled back across that group.

3. Once all servers in groupC are complete, server groups groupD and groupE will have the operation applied concurrently. The operation will be applied to the servers in groupD in series, while all servers in groupE will handle the operation concurrently. If more than 20% of the servers in groupD fail to apply the operation, it will be rolled back across that group. If any servers in groupE fail to apply the operation it will be rolled back across that group.

Simple Responses

Simple responses are provided by the following types of operations:

• Non-composite operations that target a single server. (See below for more on composite operations).

• Non-composite operations that target a Domain Controller or slave Host Controller and don't require the responder to apply the operation on multiple servers and aggregate their results (e.g. a simple read of a domain configuration property.)

The response will always include a simple boolean outcome field, with one of three possible values:

• success – the operation executed successfully

• failed – the operation failed

• cancelled – the execution of the operation was cancelled. (This would be an unusual outcome for a simple operation which would generally very rapidly reach a point in its execution where it couldn't be cancelled.)

The other fields in the response will depend on whether the operation was sucessful.

The response for a failed operation:

{
    "outcome" => "failed",
    "failure-description" => "[JBAS012345] Some failure message"
    "rolled-back" => true
}

A response for a successful operation will include an additional field:

• result – the return value, or undefined for void operations or those that return null

A non-void result:

{
    "outcome" => "success",
    "result" => {
        "name" => "Brian",
        "age" => 22
    }
}

A void result:

{
    "outcome" => "success",
    "result" => undefined
}

The response for a cancelled operation has no other fields:

{
    "outcome" => "cancelled"
}

Response Headers

Besides the standard outcome, result and failure-description fields described above, the response may also include various headers that provide more information about the affect of the operation or about the overall state of the server. The headers will be child element under a field named response-headers. For example:

{
    "outcome" => "success",
    "result" => undefined,
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}

A response header is typically related to whether an operation could be applied to the targeted runtime without requiring a restart of some or all services, or even of the target process itself. Please see the "Applying Updates to Runtime Services" section of the Description of the Management Model section for a discussion of the basic concepts related to what happens if an operation requires a service restart to be applied.

The current possible response headers are:

• operation-requires-reload – boolean – indicates that the specific operation that has generated this response requires a restart of all services in the process in order to take effect in the runtime. This would typically only have a value of 'true'; the absence of the header is the same as a value of 'false.'

• operation-requires-restart – boolean – indicates that the specific operation that has generated this response requires a full process restart in order to take effect in the runtime. This would typically only have a value of 'true'; the absence of the header is the same as a value of 'false.'

• process-state – enumeration – Provides information about the overall state of the target process. One of the following values:

  • starting – the process is starting

  • running – the process is in a normal running state. The process-state header would typically not be seen with this value; the absence of the header is the same as a value of 'running'.

  • reload-required – some operation (not necessarily this one) has executed that requires a restart of all services in order for a configuration change to take effect in the runtime.

  • restart-required – some operation (not necessarily this one) has executed that requires a full process restart in order for a configuration change to take effect in the runtime.

  • stopping – the process is stopping

Basic Composite Operation Responses

A composite operation is one that incorporates more than one simple operation in a list and executes them atomically. See the "Composite Operations" section for more information.

Basic composite responses are provided by the following types of operations:

• Composite operations that target a single server.

• Composite operations that target a Domain Controller or a slave Host Controller and don't require the responder to apply the operation on multiple servers and aggregate their results (e.g. a list of simple reads of domain configuration properties.)

The high level format of a basic composite operation response is largely the same as that of a simple operation response, although there is an important semantic difference. For a composite operation, the meaning of the outcome flag is controlled by the value of the operation request's rollback-on-runtime-failure header field. If that field was false (default is true), the outcome flag will be success if all steps were successfully applied to the persistent configuration even if none of the composite operation's steps was successfully applied to the runtime.

What's distinctive about a composite operation response is the result field. First, even if the operation was not successful, the result field will usually be present. (It won't be present if there was some sort of immediate failure that prevented the responder from even attempting to execute the individual operations.) Second, the content of the result field will be a map. Each entry in the map will record the result of an element in the steps parameter of the composite operation request. The key for each item in the map will be the string "step-X" where "X" is the 1-based index of the step's position in the request's steps list. So each individual operation in the composite operation will have its result recorded.

The individual operation results will have the same basic format as the simple operation results described above. However, there are some differences from the simple operation case when the individual operation's outcome flag is failed. These relate to the fact that in a composite operation, individual operations can be rolled back or not even attempted.

If an individual operation was not even attempted (because the overall operation was cancelled or, more likely, a prior operation failed):

{
    "outcome" => "cancelled"
}

An individual operation that failed and was rolled back:

{
    "outcome" => "failed",
    "failure-description" => "[JBAS-12345] Some failure message",
    "rolled-back" => true
}

An individual operation that itself succeeded but was rolled back due to failure of another operation:

{
    "outcome" => "failed",
    "result" => {
        "name" => "Brian",
        "age" => 22
    },
    "rolled-back" => true
}

An operation that failed and was rolled back:

{
    "outcome" => "failed",
    "failure-description" => "[JBAS-12345] Some failure message",
    "rolled-back" => true
}

Here's an example of the response for a successful 2 step composite operation:

{
    "outcome" => "success",
    "result" => [
        {
            "outcome" => "success",
            "result" => {
                "name" => "Brian",
                "age" => 22
            }
        },
        {
            "outcome" => "success",
            "result" => undefined
        }
    ]
}

And for a failed 3 step composite operation, where the first step succeeded and the second failed, triggering cancellation of the 3rd and rollback of the others:

{
    "outcome" => "failed",
    "failure-description" => "[JBAS-99999] Composite operation failed; see individual operation results for details",
    "result" => [
        {
            "outcome" => "failed",
            "result" => {
                "name" => "Brian",
                "age" => 22
            },
            "rolled-back" => true
        },
        {
            "outcome" => "failed",
            "failure-description" => "[JBAS-12345] Some failure message",
            "rolled-back" => true
        },
        {
            "outcome" => "cancelled"
        }
    ]
}

Multi-Server Responses

Multi-server responses are provided by operations that target a Domain Controller or slave Host Controller and require the responder to apply the operation on multiple servers and aggregate their results (e.g. nearly all domain or host configuration updates.)

Multi-server operations are executed in several stages.

First, the operation may need to be applied against the authoritative configuration model maintained by the Domain Controller (for domain.xml confgurations) or a Host Controller (for a host.xml configuration). If there is a failure at this stage, the operation is automatically rolled back, with a response like this:

{
    "outcome" => "failed",
    "domain-failure-description" => "[JBAS-33333] Failed to apply X to the domain model"
}

If the operation was addressed to the domain model, in the next stage the Domain Controller will ask each slave Host Controller to apply it to its local copy of the domain model. If any Host Controller fails to do so, the Domain Controller will tell all Host Controllers to revert the change, and it will revert the change locally as well. The response to the client will look like this:

{
    "outcome" => "failed",
    "host-failure-descriptions" => {
        "hostA" => "[DOM-3333] Failed to apply to the domain model",
        "hostB" => "[DOM-3333] Failed to apply to the domain model"
    }
}

If the preceding stages succeed, the operation will be pushed to all affected servers. If the operation is successful on all servers, the response will look like this (this example operation has a void response, hence the result for each server is undefined):

{
    "outcome" => "success",
    "result" => undefined,
    "server-groups" => {
        "groupA" => {
            "serverA-1" => {
                "host" => "host1",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            },
            "serverA-2" => {
                "host" => "host2",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            }
        },
        "groupB" => {
            "serverB-1" => {
                "host" => "host1",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            },
            "serverB-2" => {
                "host" => "host2",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            }
        }
    }
}

The operation need not succeed on all servers in order to get an "outcome" => "success" result. All that is required is that it succeed on at least one server without the rollback policies in the rollout plan triggering a rollback on that server. An example response in such a situation would look like this:

{
    "outcome" => "success",
    "result" => undefined,
    "server-groups" => {
        "groupA" => {
            "serverA-1" => {
                "host" => "host1",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            },
            "serverA-2" => {
                "host" => "host2",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            }
        },
        "groupB" => {
            "serverB-1" => {
                "host" => "host1",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined,
                    "rolled-back" => true
                }
            },
            "serverB-2" => {
                "host" => "host2",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined,
                    "rolled-back" => true
                }
            },
            "serverB-3" => {
                "host" => "host3",
                "response" => {
                    "outcome" => "failed",
                    "failure-description" => "[DOM-4556] Something didn't work right",
                    "rolled-back" => true
                }
            }
        }
    }
}

Finally, if the operation fails or is rolled back on all servers, an example response would look like this:

{
    "outcome" => "failed",
    "server-groups" => {
        "groupA" => {
            "serverA-1" => {
                "host" => "host1",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            },
            "serverA-2" => {
                "host" => "host2",
                "response" => {
                    "outcome" => "success",
                    "result" => undefined
                }
            }
        },
        "groupB" => {
            "serverB-1" => {
                "host" => "host1",
                "response" => {
                    "outcome" => "failed",
                    "result" => undefined,
                    "rolled-back" => true
                }
            },
            "serverB-2" => {
                "host" => "host2",
                "response" => {
                    "outcome" => "failed",
                    "result" => undefined,
                    "rolled-back" => true
                }
            },
            "serverB-3" => {
                "host" => "host3",
                "response" => {
                    "outcome" => "failed",
                    "failure-description" => "[DOM-4556] Something didn't work right",
                    "rolled-back" => true
                }
            }
        }
    }
}