JBoss.orgCommunity Documentation
There are two different ways to consume messages from a topic or queue. You can wait and have the messaging server push them to you, or you can continuously poll the server yourself to see if messages are available. This chapter discusses the latter. Consuming messages via a pull works almost identically for queues and topics with some minor, but important caveats. To start consuming you must create a consumer resource on the server that is dedicated to your client. Now, this pretty much breaks the stateless principle of REST, but after much prototyping, this is the best way to work most effectively with HornetQ through a REST interface.
You create consumer resources by doing a simple POST to the URL
published by the msg-pull-consumers response header if
you're interacting with a queue, the
msg-pull-subscribers response header if you're
interacting with a topic. These headers are provided by the main queue or
topic resource discussed in Chapter 3. Doing an empty POST to one of these
URLs will create a consumer resource that follows an auto-acknowledge
protocol and, if you're interacting with a topic, creates a temporty
subscription to the topic. If you want to use the acknowledgement protocol
and/or create a durable subscription (topics only), then you must use the
form parameters (application/x-www-form-urlencoded)
described below.
A value of true or false
can be given. This defaults to true if you do not
pass this parameter.
A value of true or false
can be given. This defaults to false if you do
not pass this parameter. Only available on topics. This specifies
whether you want a durable subscription or not. A durable
subscription persists through server restart.
This is the name of the durable subscription. If you do not provide this parameter, the name will be automatically generated by the server. Only usable on topics.
This is an optional JMS selector string. The HornetQ REST interface adds HTTP headers to the JMS message for REST produced messages. HTTP headers are prefixed with "http_" and every '-' charactor is converted to a '$'.
For a topic subscription, idle time in milliseconds in which the consumer connections will be closed if idle.
Boolean value, If true, a topic subscription will be deleted (even if it is durable) when an the idle timeout is reached.
This section focuses on the auto-acknowledge protocol for consuming messages via a pull. Here's a list of the response headers and URLs you'll be interested in.
The URL of a factory resource for creating queue consumer resources. You will pull from these created resources.
The URL of a factory resource for creating topic subscription resources. You will pull from the created resources.
The URL you will pull the next message from. This is returned with every response.
This is a URL pointing back to the consumer or subscription resource created for the client.
Here is an example of creating an auto-acknowledged queue pull consumer.
Find the pull-consumers URL by doing a HEAD or GET request to the base queue resource.
HEAD /queues/jms.queue.bar HTTP/1.1 Host: example.com --- Response --- HTTP/1.1 200 Ok msg-create: http://example.com/queues/jms.queue.bar/create msg-pull-consumers: http://example.com/queues/jms.queue.bar/pull-consumers msg-push-consumers: http://example.com/queues/jms.queue.bar/push-consumers
Next do an empty POST to the URL returned in the
msg-pull-consumers header.
POST /queues/jms.queue.bar/pull-consumers HTTP/1.1 Host: example.com --- response --- HTTP/1.1 201 Created Location: http://example.com/queues/jms.queue.bar/pull-consumers/auto-ack/333 msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/auto-ack/333/consume-next-1
The Location header points to the JMS
consumer resource that was created on the server. It is good to
remember this URL, although, as you'll see later, it is
transmitted with each response just to remind you.
Creating an auto-acknowledged consumer for a topic is pretty much the same. Here's an example of creating a durable auto-acknowledged topic pull subscription.
Find the pull-subscriptions URL by doing
a HEAD or GET request to the base topic resource
HEAD /topics/jms.topic.bar HTTP/1.1 Host: example.com --- Response --- HTTP/1.1 200 Ok msg-create: http://example.com/topics/jms.topic.foo/create msg-pull-subscriptions: http://example.com/topics/jms.topic.foo/pull-subscriptions msg-push-subscriptions: http://example.com/topics/jms.topic.foo/push-subscriptions
Next do a POST to the URL returned in the
msg-pull-subscriptions header passing in a
true value for the durable
form parameter.
POST /topics/jms.topic.foo/pull-subscriptions HTTP/1.1 Host: example.com Content-Type: application/x-www-form-urlencoded durable=true --- Response --- HTTP/1.1 201 Created Location: http://example.com/topics/jms.topic.foo/pull-subscriptions/auto-ack/222 msg-consume-next: http://example.com/topics/jms.topic.foo/pull-subscriptions/auto-ack/222/consume-next-1
The Location header points to the JMS
subscription resource that was created on the server. It is good
to remember this URL, although, as you'll see later, it is
transmitted with each response just to remind you.
After you have created a consumer resource, you are ready to
start pulling messages from the server. Notice that when you created
the consumer for either the queue or topic, the response contained a
msg-consume-next response header. POST to the URL
contained within this header to consume the next message in the queue
or topic subscription. A successful POST causes the server to extract
a message from the queue or topic subscription, acknowledge it, and
return it to the consuming client. If there are no messages in the
queue or topic subscription, a 503 (Service Unavailable) HTTP code is
returned.
For both successful and unsuccessful posts to the msg-consume-next URL, the response will contain a new msg-consume-next header. You must ALWAYS use this new URL returned within the new msg-consume-next header to consume new messages.
Here's an example of pulling multiple messages from the consumer resource.
Do a POST on the msg-consume-next URL that was returned with the consumer or subscription resource discussed earlier.
POST /queues/jms.queue.bar/pull-consumers/consume-next-1 Host: example.com --- Response --- HTTP/1.1 200 Ok Content-Type: application/xml msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-2 msg-consumer: http://example.com/queues/jms.queue.bar/pull-consumers/333 <order>...</order>
The POST returns the message consumed from the queue. It also returns a new msg-consume-next link. Use this new link to get the next message. Notice also a msg-consumer response header is returned. This is a URL that points back to the consumer or subscription resource. You will need that to clean up your connection after you are finished using the queue or topic.
The POST returns the message consumed from the queue. It also returns a new msg-consume-next link. Use this new link to get the next message.
POST /queues/jms.queue.bar/pull-consumers/consume-next-2 Host: example.com --- Response --- Http/1.1 503 Service Unavailable Retry-After: 5 msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-2
In this case, there are no messages in the queue, so we get a 503 response back. As per the HTTP 1.1 spec, a 503 response may return a Retry-After head specifying the time in seconds that you should retry a post. Also notice, that another new msg-consume-next URL is present. Although it probabley is the same URL you used last post, get in the habit of using URLs returned in response headers as future versions of HornetQ REST might be redirecting you or adding additional data to the URL after timeouts like this.
POST to the URL within the last
msg-consume-next to get the next
message.
POST /queues/jms.queue.bar/pull-consumers/consume-next-2 Host: example.com --- Response --- HTTP/1.1 200 Ok Content-Type: application/xml msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-3 <order>...</order>
If you experience a network failure and do not know if your post to a msg-consume-next URL was successful or not, just re-do your POST. A POST to a msg-consume-next URL is idempotent, meaning that it will return the same result if you execute on any one msg-consume-next URL more than once. Behind the scenes, the consumer resource caches the last consumed message so that if there is a message failure and you do a re-post, the cached last message will be returned (along with a new msg-consume-next URL). This is the reason why the protocol always requires you to use the next new msg-consume-next URL returned with each response. Information about what state the client is in is embedded within the actual URL.
If the server crashes and you do a POST to the msg-consume-next URL, the server will return a 412 (Preconditions Failed) response code. This is telling you that the URL you are using is out of sync with the server. The response will contain a new msg-consume-next header to invoke on.
If the client crashes there are multiple ways you can recover. If you have remembered the last msg-consume-next link, you can just re-POST to it. If you have remembered the consumer resource URL, you can do a GET or HEAD request to obtain a new msg-consume-next URL. If you have created a topic subscription using the name parameter discussed earlier, you can re-create the consumer. Re-creation will return a msg-consume-next URL you can use. If you cannot do any of these things, you will have to create a new consumer.
The problem with the auto-acknowledge protocol is that if the client or server crashes, it is possible for you to skip messages. The scenario would happen if the server crashes after auto-acknowledging a message and before the client receives the message. If you want more reliable messaging, then you must use the acknowledgement protocol.
The manual acknowledgement protocol is similar to the auto-ack protocol except there is an additional round trip to the server to tell it that you have received the message and that the server can internally ack the message. Here is a list of the respone headers you will be interested in.
The URL of a factory resource for creating queue consumer resources. You will pull from these created resources
The URL of a factory resource for creating topic subscription resources. You will pull from the created resources.
URL used to obtain the next message in the queue or topic subscription. It does not acknowledge the message though.
URL used to acknowledge a message.
This is a URL pointing back to the consumer or subscription resource created for the client.
Here is an example of creating an auto-acknowledged queue pull consumer.
Find the pull-consumers URL by doing a HEAD or GET request to the base queue resource.
HEAD /queues/jms.queue.bar HTTP/1.1 Host: example.com --- Response --- HTTP/1.1 200 Ok msg-create: http://example.com/queues/jms.queue.bar/create msg-pull-consumers: http://example.com/queues/jms.queue.bar/pull-consumers msg-push-consumers: http://example.com/queues/jms.queue.bar/push-consumers
Next do a POST to the URL returned in the
msg-pull-consumers header passing in a
false value to the autoAck
form parameter .
POST /queues/jms.queue.bar/pull-consumers HTTP/1.1 Host: example.com Content-Type: application/x-www-form-urlencoded autoAck=false --- response --- HTTP/1.1 201 Created Location: http://example.com/queues/jms.queue.bar/pull-consumers/acknowledged/333 msg-acknowledge-next: http://example.com/queues/jms.queue.bar/pull-consumers/acknowledged/333/acknowledge-next-1
The Location header points to the JMS
consumer resource that was created on the server. It is good to
remember this URL, although, as you'll see later, it is
transmitted with each response just to remind you.
Creating an manually-acknowledged consumer for a topic is pretty much the same. Here's an example of creating a durable manually-acknowledged topic pull subscription.
Find the pull-subscriptions URL by doing
a HEAD or GET request to the base topic resource
HEAD /topics/jms.topic.bar HTTP/1.1 Host: example.com --- Response --- HTTP/1.1 200 Ok msg-create: http://example.com/topics/jms.topic.foo/create msg-pull-subscriptions: http://example.com/topics/jms.topic.foo/pull-subscriptions msg-push-subscriptions: http://example.com/topics/jms.topic.foo/push-subscriptions
Next do a POST to the URL returned in the
msg-pull-subscriptions header passing in a
true value for the durable
form parameter and a false value to the
autoAck form parameter.
POST /topics/jms.topic.foo/pull-subscriptions HTTP/1.1 Host: example.com Content-Type: application/x-www-form-urlencoded durable=true&autoAck=false --- Response --- HTTP/1.1 201 Created Location: http://example.com/topics/jms.topic.foo/pull-subscriptions/acknowledged/222 msg-acknowledge-next: http://example.com/topics/jms.topic.foo/pull-subscriptions/acknowledged/222/consume-next-1
The Location header points to the JMS
subscription resource that was created on the server. It is good
to remember this URL, although, as you'll see later, it is
transmitted with each response just to remind you.
After you have created a consumer resource, you are ready to
start pulling messages from the server. Notice that when you created
the consumer for either the queue or topic, the response contained a
msg-acknowledge-next response header. POST to the
URL contained within this header to consume the next message in the
queue or topic subscription. If there are no messages in the queue or
topic subscription, a 503 (Service Unavailable) HTTP code is returned.
A successful POST causes the server to extract a message from the
queue or topic subscription and return it to the consuming client. It
does not acknowledge the message though. The response will contain the
acknowledgement header which you will use to
acknowledge the message.
Here's an example of pulling multiple messages from the consumer resource.
Do a POST on the msg-acknowledge-next URL that was returned with the consumer or subscription resource discussed earlier.
POST /queues/jms.queue.bar/pull-consumers/consume-next-1 Host: example.com --- Response --- HTTP/1.1 200 Ok Content-Type: application/xml msg-acknowledgement: http://example.com/queues/jms.queue.bar/pull-consumers/333/acknowledgement/2 msg-consumer: http://example.com/queues/jms.queue.bar/pull-consumers/333 <order>...</order>
The POST returns the message consumed from the queue. It
also returns a msg-acknowledgement link. You
will use this new link to acknowledge the message. Notice also a
msg-consumer response header is returned. This
is a URL that points back to the consumer or subscription
resource. You will need that to clean up your connection after you
are finished using the queue or topic.
Acknowledge or unacknowledge the message by doing a POST to
the URL contained in the msg-acknowledgement
header. You must pass an acknowledge form
parameter set to true or
false depending on whether you want to
acknowledge or unacknowledge the message on the server.
POST /queues/jms.queue.bar/pull-consumers/acknowledgement/2 Host: example.com Content-Type: application/x-www-form-urlencoded acknowledge=true --- Response --- Http/1.1 200 Ok msg-acknowledge-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/acknowledge-next-2
Whether you acknowledge or unacknowledge the message, the response will contain a new msg-acknowledge-next header that you must use to obtain the next message.
If you experience a network failure and do not know if your post
to a msg-acknowledge-next or
msg-acknowledgement URL was successful or not, just
re-do your POST. A POST to one of these URLs is idempotent, meaning
that it will return the same result if you re-post. Behind the scenes,
the consumer resource keeps track of its current state. If the last
action was a call to msg-acknowledge-next, it will
have the last message cached, so that if a re-post is done, it will
return the message again. Same goes with re-posting to
msg-acknowledgement. The server remembers its last
state and will return the same results. If you look at the URLs you'll
see that they contain information about the expected current state of
the server. This is how the server knows what the client is
expecting.
If the server crashes and while you are doing a POST to the
msg-acknowledge-next URL, just re-post. Everything
should reconnect all right. On the other hand, if the server crashes
while you are doing a POST to msg-acknowledgement,
the server will return a 412 (Preconditions Failed) response code.
This is telling you that the URL you are using is out of sync with the
server and the message you are acknowledging was probably re-enqueued.
The response will contain a new
msg-acknowledge-next header to invoke on.
As long as you have "bookmarked" the consumer resource URL
(returned from Location header on a create, or the
msg-consumer header), you can recover from client
crashes by doing a GET or HEAD request on the consumer resource to
obtain what state you are in. If the consumer resource is expecting
you to acknowledge a message, it will return a
msg-acknowledgement header in the response. If the
consumer resource is expecting you to pull for the next message, the
msg-acknowledge-next header will be in the
response. With manual acknowledgement you are pretty much guaranteed
to avoid skipped messages. For topic subscriptions that were created
with a name parameter, you do not have to "bookmark" the returned URL.
Instead, you can re-create the consumer resource with the same exact
name. The response will contain the same information as if you did a
GET or HEAD request on the consumer resource.
Unless your queue or topic has a high rate of message flowing
though it, if you use the pull protocol, you're going to be receiving a
lot of 503 responses as you continuously pull the server for new
messages. To alleviate this problem, the HornetQ REST interface provides
the Accept-Wait header. This is a generic HTTP
request header that is a hint to the server for how long the client is
willing to wait for a response from the server. The value of this header
is the time in seconds the client is willing to block for. You would
send this request header with your pull requests. Here's an
example:
POST /queues/jms.queue.bar/pull-consumers/consume-next-2 Host: example.com Accept-Wait: 30 --- Response --- HTTP/1.1 200 Ok Content-Type: application/xml msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-3 <order>...</order>
In this example, we're posting to a msg-consume-next URL and telling the server that we would be willing to block for 30 seconds.
When the client is done with its consumer or topic subscription it should do an HTTP DELETE call on the consumer URL passed back from the Location header or the msg-consumer response header. The server will time out a consumer with the value configured from Chapter 2.3, so you don't have to clean up if you dont' want to, but if you are a good kid, you will clean up your messes. A consumer timeout for durable subscriptions will not delete the underlying durable JMS subscription though, only the server-side consumer resource (and underlying JMS session).