MSMQ Heartbeat checks with QueueMonitor

Here’s how wikipedia describes heartbeats in computing:

In computer science, a heartbeat is a periodic signal generated by hardware or software to indicate normal operation or to synchronize other parts of a computer system.

How can we apply heartbeats to MSMQ?

We can start with simple heartbeat checks – QueueMonitor sends message to a queue, and tries to receive it from that same queue. If message arrives within specified timeout, everything is OK, MSMQ is fully working and messages can be both sent and received. Otherwise, we raise alarm and someone should check what’s wrong.

Why is this better than just checking Message Queueing service?

Sometimes service is running, but actual MSMQ queues are not – we used all free space on disk, remote machine can’t be reached, queues are filled up, there’s some internal MSMQ problem, permissions are mixed up, whatever. However if we can successfully send and receive message, we know that basic MSMQ functionality 100% works.

That’s all fine, but why not include user’s application in this check too?

Let’s assume that application already receives messages from some queue. If QueueMonitor sends heartbeat messages to that queue, and application resends heartbeats to another queue, we can receive them from that second queue.

If heartbeat arrives within specified timeout, we’ll know that BOTH user’s application and MSMQ work OK. Small drawback is that we have to modify application as well to recognize heartbeat messages and sends them to some queue. This type of check is called “advanced” heartbeat check in QueueMonitor.

In order to minimize changes to outside application, QueueMonitor has configurable message label and body – they could contain anything, as long they also contain heartbeat token somewhere. This token is just a small piece of text containing timestamp:

Heartbeat:’1760848984445′

In order to put this in message, you should use {heartbeat_token} macro, which QueueMonitor replaces with current timestamp when sending heartbeat.

Just create new Heartbeat task from QueueMonitor 1.6+ and your MSMQ monitoring will be greatly improved.

Installing QueueMonitor on a cluster

QueueMonitor since version 1.5.9 can be installed on a Windows Server cluster as clustered application. Main addition to v1.5.9 release was ability to put QueueMonitor’s data folder on shared Cluster storage. That way there will be single shared task database used by both nodes. Or to be more precise – used by whichever cluster node is currently active.

For this tutorial we’ll install QueueMonitor Agent on Windows Server 2012 two node cluster, which already has MSMQ installed as clustered resource. We’ll add QueueMonitor to existing cluster MSMQ role, because that way QM has local access to MSMQ. Local access is always preferred since it offers better performance and it’s easier to set up security wise. Beside that, if we access MSMQ in remote instead of local mode there are some limitations meaning we’re limited in features QM can use.

Step by step instructions

  • Install QueueMonitor on both nodes. We only need Agents here, Admin part is optional and can be installed here or on some other machine.
  • Once installation is finished, stop QueueMonitor services on both nodes. While you’re at services panel, check out which account service uses to log on. By default it’s “Local System account”. That account will be used for all QM’s operations – access to queues, files, etc. So change it to e.g. some domain account if it better suits your setup.
  • Following steps can performed on active node only, because clustering will take care of migrating changes to passive node
  • Run regedit and locate HKLM\SOFTWARE\Cogin\QueueMonitor


    Creating registry key

    Create new String value named “Folder” and enter path to a folder on shared cluster storage. In our example it’s “e:\QueueMonitorData\”


    7

  • Go to “Failover Cluster Manager”. Locate MSMQ role. While it’s selected, find “Add Resource” in right panel and choose “Generic Service”
  • Select QueueMonitor service and finish wizard.


    2

  • After QueueMonitor is added, and while it’s still offline, right click on it and go to Registry Replication. Enter “SOFTWARE\Cogin\QueueMonitor” there. This is registry key we configured before. This way, it will be replicated to another node automatically.


    4

  • Now you can click “Bring Online” on QueueMonitor.
  • Start QueueMonitor Admin service and connect to agent. You should use cluster’s address/name, and not a name of individual nodes! That way agent will always connect to clustered resource, regardless of which mahcine is currently active node for cluster.

QueueExplorer 4 Beta – next gen queue manager

QueueExplorer 4 is finally available for Beta testing. It’s been a long wait but not without a reason. It was overhauled to support multiple queueing systems, and for start we added support for ActiveMQ.

There are lot of improvements for MSMQ users as well. You no longer have to wait big queues to load, messages now show up progressively as soon as they are received. Neither you have to press F5 to refresh queues and messages as they are refreshed in the background, with new/updated stuff flashing for few seconds to easily see changes.

 

For Professional edition you can now extract business data from your messages using XPath, JSON or Regex. That business data is then displayed as a new column, and you can see, filter, or sort it.

Also, you can create your own folders to organize queues. Just drag and drop queues there or set up wildcard filtering. For example you can use “test*” and have all test queues grouped there. In that case folder will dynamically update itself to include newly created queues which fit wildcard search.

This is just a quick glance on new features, you can check full list and download Beta here:

Full list of improvements in QueueExplorer 4

Automatically resend expired MSMQ dead letter messages using QueueMonitor

There were few inquires recently on what to do with messages which were moved by MSMQ to dead letter queue because they reached “Time to be received” timeout.

When is Time to be received property useful?

Sometimes messages are of temporary value and are not important unless they are processed immediately. For instance if we use queue to send current status of some process, next message will anyway be come after few seconds. Even if we stop processing we don’t want to process all pending messages if we’re only interested in latest one. For these scenarios Time to be received makes sense, since MSMQ will automatically remove these messages from queue once timeout is reached. If messages are set to use Dead letter queue, they will be moved to one of two System Dead-letter queues – one for transactional messages and one for non-transactional.

However sometimes reality gets in a way. Sender has some expectations and sets Time to be received to few minutes, but circumstances change and we find out that these messages are still needed and important even if they were moved to dead letter queue.

How to automatically resend messages to their original queue using QueueMonitor

For this to work you’ll need QueueMonitor Professional, since it offers more complex processing.

 

Step 1 – New task

Click on “New Task…” and pick “Process messages – scheduled” task.

QueueMonitor - Select a task

Why scheduled processing? Because dead letter queues can contain messages which came there for various reasons – for instance purged messages, etc. We don’t want to return all of these messages back to their queues, we’re interested only in expired ones. There’s similar task “Process messages – constantly” but it constantly watches a queue and should be used when all arriving messages should be processed as soon as they arrive. If we don’t want to remove all messages after they’re processed, constant processing will be suboptimal because it will keep iterating through queue all the time which could be a problem for system resources.

 

Step 2 – Pick a queue

Choose one of System queues – “Dead letter messages” or “Transactional dead-letter messages”. In you want to monitor both DL queues, you should create two separate tasks because of limitations in current version of QueueMonitor we’ll discuss later.

Select a queue

Step 3 – Schedule

Pick how often you want this task to occur:

Schedule

Step 4 – Message filter

If we want to only resend messages which were expired, we should make appropriate filter:

Message filter for ReceiveTimeout

Turn on Advanced query option, select Acknowledgment property and set it to equals ReceiveTimeout

You can make more complex query here, if you want to process only certain messages.

 

Step 5 – Actions

Change Time to be received

Add Change message properties action and configure it to change “Time to be received” property.

Send to queue action

Add Send to queue action. For destination name type {msg_destination} (you can also insert macros by clicking on M button). This will make sure that message goes back to its original destination.

And here’s a reason why you had to create separate tasks for transactional and non-transactional DLQs. Sending in MSMQ requires that you either wrap it in Transaction or not, depending on type of queue. For local queues QueueMonitor can determine whether queue is transactional, but if it’s remote queue or if macros are used (like in this case), it can’t and you have to set it manually. So if this task is for transactional DLQ check Transactional checkbox, otherwise leave it unchecked.

Step 6 – Done

You can save new task now. In order to read system queues, QueueMonitor service will have to work under account with administrative privileges. If it’s not already set, go to Computer Management, find QueueMonitor service and change it’s Log On as setting.

You can use “Run once now” option to test new task whether it works correctly. If everything is ok, you can enable this task and it will continue to work in background, happily resending all expired messages back to their original queues.

QueueExplorer for ActiveMQ

Yes, that’s right. We’re extending support to other queueing systems beyond MSMQ. First pick was ActiveMQ. More to come in the future.

You’ll be able to use more or less all features available to MSMQ users – copy/paste messages, save/load to files, create test messages, bulk sending for performance tests, etc. Standard message views like JSON, XML, Text, Hex will continue to work as well. We’ll also cover ActiveMQ specific features. Message properties and headers will be available for filtering and editing, allowing you to quickly find relevant message and change it if needed.

QueueExplorer for ActiveMQ

ActiveMQ Topics

One of biggest differences between ActiveMQ and MSMQ are Topics. Topics are a way to achieve publish/subscribe messaging pattern. Unlike regular queues, topic can have multiple subscribers and each of them receives all messages sent to topic. On the other hand, these messages are not stored like messages sent to a regular queue. They are just passed to currently active subscribers *. So when you subscribe to a topic you only get messages which are published from that moment on.

* What I wrote here is not actually true, but a simplification to help distinguish between regular queue and topic. You can configure topic to keep messages for some time so that if you drop subscription and resubscribe you get messages which came during that period.

Anyway, apart from functionality we already had working for MSMQ, QueueExplorer for ActiveMQ will also allow you to subscribe to topic and see messages as they come in real time.

Sing up to try ActiveMQ support

ActiveMQ support is not yet ready for public yet, but if you want to be the first to know when it’s available sign up here.

QueueMonitor 1.0 released

Thanks for your feedback and support during beta testing of QueueMonitor! That helped us a lot to reach v1.0.

What’s new for v1.0?

Apart from smaller fixes, there are two significant improvements. First one is import/export of tasks. It makes it easy to configure tasks on one server and transfer them to another one. Or to create backup before you start modifying tasks.

Second improvement is documentation. Whenever you’re creating or editing task you can hit F1 and get help for that specific task.
(more…)

QueueMonitor Beta – monitoring and automation for MSMQ

QueueMonitor Beta is here

Thanks for your support during alpha testing of QueueMonitor! That helped us a lot to reach this point. We are now ready for Beta stage. What does Beta mean in our case? It means QueueMonitor passes our internal tests, doesn’t have any known issues and doesn’t consume much memory or CPU when left running for prolonged periods of time. In other words, it behaves as fully functional software.

Of course, it’s rare that any software survives first contact with real world usage. That’s why we need your help with Beta testing.

(more…)

QueueExplorer 3.3 – Queue views (PRO)

QueueExplorer 3.3 bring queue views for Professional edition.

Queue views allow user to configure how messages are filtered, displayed, etc. on queue by queue basis. Following settings are stored:

  • Which columns are displayed
  • Filtering and sorting
  • Limiting number of messages (i.e. “Top x messages”)
  • .Net viewer assembly path
  • Encodings for body and extension when displayed as a column

(more…)