RabbitQM with Springboot
In this article, I will demonstrate using RabbitMQ with Springboot technologies.
- RabbitMQ: https://www.rabbitmq.com/
- Springboot: https://spring.io
Today, most applications are designed as decoupled into smaller microservices, independent building blocks that are easier to develop, deploy, and maintain. The Message queues can provide communication and coordination for these distributed applications.
The Message queues can significantly simplify the coding of decoupled applications while improving performance, reliability, and scalability. You can also combine message queues with Pub/Sub messaging in a fanout design pattern. In this discussion, I would like to pick up RabbitMQ Message type.
Prerequisites
I assume the RabbitMQ is installed and running on localhost using the standard port (5672). In case you use a different host, port or credentials, connections settings would require adjusting.
Install RabbitMQ with docker
https://www.rabbitmq.com/download.html
docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-managementUsage
The sample uses Maven build. To build it run
mvn spring-boot:runRabbitMQ
RabbitMQ is the most widely deployed open-source message broker. It is written in the Erlang programming language and is built on the Open Telecom Platform framework for clustering and failover.
RabbitMQ is lightweight and easy to deploy on-premises and in the cloud. It supports multiple messaging protocols. RabbitMQ can be deployed in distributed and federated configurations to meet high-scale, high-availability requirements.
There are many client libraries supported to interface with the broker that are available for all major programming languages such as Java, Python, PHP, C#, JavaScript, Ruby, Erlang, Go, Object-C, Swift, and more.
When using RabbitMQ the publisher never directly sends a message to a queue. Instead, the publisher sends messages to an exchange. The Exchange is responsible for sending the message to an appropriate queue based on routing keys, bindings, and header attributes. The Exchanges are the message routing agents that we can define and bindings are what connect the exchanges to the queues.
In this sample project, we will use the Queue and Exchange and then bind them together.
There are 4 types of Exchanges in RabbitMQ-
- Direct Exchange
- Fanout Exchange
- Topic Exchange
- Header Exchange
Direct Exchange
Based on the routing key, a message is sent to the queue having the same routing key specified in the binding rule. The routing key of exchange and the binding queue have to be an exact match. A message is sent to exactly one queue.
curl “http://localhost:8080/api/rabbitmq/producer?code=001&description=MyDescription"
- Sender: see class RabbitMQSender.java
- Consumer: see class RabbitMQReciver.java
- RabbitMQDirectconfigurationType.java
Fanout Exchange
The message is routed to all the available bounded queues. The routing key if provided it is completely ignored. So the fanout Exchange is a kind of publish-subscribe design pattern which broadcasts the message to all consumers.
curl -i “http://localhost:8080/api/rabbitmq/fanout?exchangeName=fanout-exchange&messageData=message"
Topic Exchange
The Topic Exchange looks similar to Direct Message type, but the difference is that the routing key of the exchange and the bound queues should not necessarily be an exact match. Using regular expressions like wildcard we can send the exchange to multiple bound queues.
@Bean
Binding allBinding(Queue allQueueTopic, TopicExchange topicExchange) {
return BindingBuilder.bind(allQueueTopic).to(topicExchange).with(“queue.*”);
}
curl “http://localhost:8080/api/rabbitmq/topic?exchangeName=topic-exchange&routingKey=queue.admin&messageData=MessageData"
Header Exchange
In this type of exchange, the routing queue is selected based on the criteria specified in the headers instead of the routing key. This is similar to topic exchange type, but here we can specify complex criteria for selecting routing queues
curl “http://localhost:8080/api/rabbitmq/header?exchangeName=header-exchange&department=marketing&messageData=MessageValue"
12 reasons why considering to use message queues
1. Redundancy via Persistence
Redundancy is one of the most obvious advantages to message queues. Application crashes, timeouts, errors in your code, and other problems are just part of the norm. This is especially true in applications that process millions or billions of transactions per month.
Queues help with redundancy by making the process that reads the message confirming that it completed the transaction and it is safe to remove it. If anything fails, worst-case scenario, the message is persisted to storage somewhere and won’t be lost. It can be reprocessed later.
2. Traffic Spikes
You don’t always know exactly how much traffic your application is going to have. For example, we receive billions of messages a month. We have no way to know what our clients are going to send us. By queuing the data we can be assured the data will be persisted and then be processed eventually, even if that means it takes a little longer than usual due to a high traffic spike.
3. Improve Web Application Page Load Times
Queues can be useful in web applications to do complex logic in a background thread so the request can finish for the user quickly. If someone places an order on your website, that could involve a lot of different things that have to happen. You can do the minimum and return success to your user and kick off the rest of them in a background thread to finish up, without using a full message queuing system and background apps. Most programming languages have ways to do this now.
4. Batching for Efficiency
Batching is a great reason to use message queues. It is much more efficient to insert 100 records into a database at a time instead of 1 at a time, 100 times. We insert a lot of data into elasticsearch and SQL Server. Batching helps us optimize their performance by tuning the size of the transactions.
5. Asynchronous Messaging
Queues can be great in scenarios where your application needs something done but doesn’t need it done now, or doesn’t even care about the result. Instead of calling a web service and waiting for it to complete, you can write the message to a queue and let the same business logic happen later. Queues are an excellent way to implement an asynchronous programming pattern.
6. Decouple by Using Data Contracts
By using a queue between different parts of your software you can decouple the hard dependencies. The format of the message in the queue becomes your data contract and anything that knows how to read that message format can be used to process the transaction. This could be useful for parts of your code that are even written in different programming languages.
7. Transaction Ordering and Concurrency Challenges
If 1000 people are placing an order on your website at one time, that could create some problems with concurrency and ensuring that the first order-in finishes first. By queuing them up, you could then guarantee their order and control how many are even processed concurrently.
8. Improve Scalability
Message queues enable you to decouple different parts of your application and then scale them independently. Using Azure, AWS, or other hosting solutions you could even dynamically scale that background service based on CPU usage or other metrics. Message queues can help a lot with scalability and elasticity.
9. Create Resiliency
By breaking your app up and separating different components by queues, you inherently create more resiliency. Your website can still be operational even if part of the back end processing of the order is slightly delayed. we design our incoming APIs to do as little as possible beyond queuing the data to ensure that nothing else can bring down the ability to receive data. Even if the database server is down, we want to be able to accept data!
10. Guarantee Transaction Occurs Once
Message queues are designed to be transactional. When you read a message off of a queue, you have to tell it where you completed processing the message before it is completely removed from the queue. This helps to ensure that a transaction only occurs once.
11. Break Larger Tasks into Many Smaller Ones
Another good use for queues is breaking up a larger task into lots of little pieces and then queuing them all up. We have a good example of this at Stackify. If you edit a monitoring template for a group of servers, we then need to update the monitors on every server that uses that template. We can queue up a message for every server and potentially do them concurrently as smaller operations.
12. Monitoring
Message queuing systems enable you to monitor how many items are in a queue, the rate of processing messages, and other stats. This can be very helpful from an application monitoring standpoint to keep an eye on how data is flowing through your system and if it is getting backed up.
Ref: https://stackify.com/message-queues-12-reasons/
Final Thought
Using MQ messages it can make your applications better and stable and scalable.
I hope this article can help you and enlarge your knowledge.
The full source code can be found: https://github.com/sophea/rabbitmq-springboot