A kind of Microservice Architecture

Developer: What problems are we trying to solve?

Enterprise Architect: There are no problems. I’ve got multiple solutions and I’m trying to understand which one is the best one.

-- Conversation with a former manager

Maybe you have heard of Microservice Architecture. In this this entry I'll simply call it MSA. As you'll probably know MSA is an approach to organising systems through the use of independently deployable services communicating through dumb asynchronous communication infrastructures. That's basically the gist of it. In case you want a more complete understanding of what it is all about Martin Fowler more or less sums it up in the article Microservices.

There are several questions that can be raised when considering an MSA approach. An important but seldom asked question is if a monolithic design approach is in opposition to an MSA approach. I don't believe it is as I will explain in this posting.

The idea

The approach I'll go through here is not novel by any standards and does not shed light on anything fundamentally new. However, since I hardly see it being used in software projects I think it's well worth presenting it.

The idea is simple: drum up a messaging infrastructure having enough polymorphism baked in so that it transparently can be used for communication between objects inside the same address space as well as between objects deployed in processes. Not to forget, the infrastructures polymorphic properties must support adaptation so that it is optimised for the environment in which it is being used.

Next: construct a number of services where a service is a piece of software communicating with other services using the infrastructure. A service could be an instance of a class executing together with other services inside the same executable or a full-fledged executable as long as it receives and sends messages using the infrastructure..

What I call service here is a piece of functionality receiving a message, processing the message and sending the message to some other place. It does not support a request/reply machinery. It only supports a fire and forget mechanisms seen from a client viewpoint.

The restriction of invoking a service using fire and forget may seem like a limiting factor. However, there are a surprsing amount of services which may be implemented this way.

The infrastructure I'll use here is a queue based one implemented as an extension to boost::asio. In fact, it is the same one I've described in a few of my previous blog entries (Using queues with boost::asio - Part I, boost::asio Queues Revamped - Part II and boost::asio Queues Rediscovered - Part III).

Well, almost the same one. In fact, the implementation I use in this entry is a slightly modified version of the one described previously. I've modified the queue classes to accept a base class as a template parameter in order that queue_senders and queue_listeners can use queues in a polymorphic way.

I will not describe the boost::asio queue extension in detail here. Instead I refer to the github repository https://github.com/hansewetz/blog.

Have the cake and eat it too

So, what's the big deal in being able to decide late in the process how to package services? The big deal is that we can have the cake and eat it too! What I mean here is that we don't have to choose if two services should execute within a single process or be deployed in two separate processes. Again, the big deal here is that the deployment of a service is often viewed as something that we must decide on very early - and if we change our mind later, the code making up the service must be hacked to make it fit the new deployment scheme.

For example, say I have one service which chops up text into sentences and another which translates the sentences into one or more target languages. For various technical reasons I want the services to receive input asynchrounosly, do some processing and send results asynchrounosly. Now, when starting the design and coding I'm not really sure if the services will be packaged into a single executable or possibly in multiple executables. Maybe, for a small system I just want everything to run as a single executable having say 20 or so translation services being deployed - again within a single executable. In a larger system I might want to spread the services across multiple executables and possibly across multiple machines. In either case, I want the services to receive input and send output in exactly the same way. I do not want to modify the interfaces by going in and hacking the code. Also, I want to be able to take advantage of services running in the same address space so I don't have to serialise messages when communicating with some other service. That's about sums up what I want.

A simple way to do exactly what I want is to write my services so that they act on messages they receive asynchrounosly. A standard way, well - at least in the C++ community - is to use boost::asio as a baseline. Unfortunatly boost::asio doesn't support general messaging - boost::asio deals mostly with sending buffers of bytes. Fortunatly, it's not difficult to add extensions to boost::asio. One extension is the queue extension I have presented in three of the past blog entries. That's the one I'll use (with some slight modifications).

A simple echo server

Because the implementation of a translation service as mentioned earlier is quite involved, I'll stick to a simple echo service here. The echo service receives a message and simply echo (or forward it) to a different location.

I'll start off by coding the service as one header file and one source file. The header for the service is simple:

#include "QueueDefs.h"
#include <memory>
// service class
class Service{
public:
Service(boost::asio::io_service&ios,std::shared_ptr<QueueDeqBase>qserviceReceive,std::shared_ptr<QueueEnqBase>qserviceReply);
private:
  std::shared_ptr<QueueListener>serviceReceiver_;
  std::shared_ptr<QueueSender>serviceSender_;

 // handler for received messages
  void recvHandler(boost::system::error_code const&ec,QueueValueType const&msg);
};

The service is started in the constructor. Alternativly we could have supplied a start and stop method. The implementation is short and simple:

#include "Service.h"
#include <functional>
using namespace std;
using namespace std::placeholders;
namespace asio=boost::asio;

// ctor
Service::Service(asio::io_service&ios,shared_ptr<QueueDeqBase>qserviceReceive,shared_ptr<QueueEnqBase>qserviceReply):
serviceReceiver_(make_shared<QueueListener>(ios,qserviceReceive.get())),
serviceSender_(make_shared<QueueSender>(ios,qserviceReply.get())){
  // setup asio callback
  serviceReceiver_->async_deq(std::bind(&Service::recvHandler,this,_1,_2));
}
// handler for receiving message
void Service::recvHandler(boost::system::error_code const&ec,QueueValueType const&msg){
  // echo back message to output queue
  boost::system::error_code ec1;
  serviceSender_->sync_enq(msg,ec1);

  // trigger for receiving messages again
  serviceReceiver_->async_deq(std::bind(&Service::recvHandler,this,_1,_2));
}

There are a number of types such as QueueListener which have not yet been defined. I'll define them in the file QueueDefs.h:

#include <boost/asio_queue.hpp>
#include <string>
#include <iosfwd>
// queue value type
using QueueValueType=std::string;

// message separator
constexpr char sep='|';

// serialiser/de-serialiser
auto msgDeserializer=[](std::istream&is)->QueueValueType{std::string line;getline(is,line,sep);return line;};
auto msgSerializer=[](std::ostream&os,QueueValueType msg)->void{os<<msg;};

// typedefs for base classes for msg queues
using QueueBase=boost::asio::detail::base::queue_interface_base<QueueValueType>;
using QueueDeqBase=boost::asio::detail::base::queue_interface_deq<QueueValueType>;
using QueueEnqBase=boost::asio::detail::base::queue_interface_enq<QueueValueType>;

// typedefs for concrete queues
using MemQueue=boost::asio::simple_queue<QueueValueType,QueueBase>;
using SockServQueue=boost::asio::sockserv_queue<QueueValueType,decltype(msgDeserializer),decltype(msgSerializer),QueueBase>;
using SockClientQueue=boost::asio::sockclient_queue<QueueValueType,decltype(msgDeserializer),decltype(msgSerializer),QueueBase>;

// typedef for qsender/qlistener based on queue base classes
using QueueSender=boost::asio::queue_sender<QueueEnqBase>;
using QueueListener=boost::asio::queue_listener<QueueDeqBase>;

As you can see, I've defined a number of queues all using different media. The idea is to create concrete queue objects and use them through their base classes. The queue creation is factored out and implemented in the file: QueueCreation.h:

#include "QueueDefs.h"
#include <memory>
#include <string>
// struct holding queues to be used
struct queue_struct{
  std::shared_ptr<QueueDeqBase>qserviceReceive; // service receives msg
  std::shared_ptr<QueueEnqBase>qserviceReply;   // service responds on message
  std::shared_ptr<QueueEnqBase>qclientRequest;  // client sends request
  std::shared_ptr<QueueDeqBase>qclientReceive;  // client receives msg
};
// create memory queues
queue_struct createMemQueues(){
queue_struct ret;
  // we only need to create two queues
  size_t maxq{10};
  std::shared_ptr<QueueBase>q1=std::make_shared<MemQueue>(maxq);
  std::shared_ptr<QueueBase>q2=std::make_shared<MemQueue>(maxq);
  ret.qserviceReceive=q1;
  ret.qclientRequest=q1;
  ret.qserviceReply=q2;
  ret.qclientReceive=q2;
  return ret;
}
// create ip queues with Service having one ipservice queue and one ip-client queue
queue_struct createAsymetricIpQueues(){
  queue_struct ret;
  // ip data
  constexpr int serverListenPort=7787;
  constexpr int clientListenPort=7788;
  std::string const server{"localhost"};

  // create 4 queues
  ret.qserviceReceive=std::make_shared<SockServQueue>(serverListenPort,msgDeserializer,msgSerializer,sep);
  ret.qclientRequest=std::make_shared<SockClientQueue>(server,serverListenPort,msgDeserializer,msgSerializer,sep);
  ret.qserviceReply=std::make_shared<SockClientQueue>(server,clientListenPort,msgDeserializer,msgSerializer,sep);
  ret.qclientReceive=std::make_shared<SockServQueue>(clientListenPort,msgDeserializer,msgSerializer,sep);
  return ret;
}
// create ip queues with Service having two ip-servoce queues
queue_struct createSymetricIpQueues(){
  queue_struct ret;
  // ip data
  constexpr int serverListenPortReceive=7787;
  constexpr int serverListenPortSend=7788;
  std::string const server{"localhost"};
  // create 4 queues
  ret.qserviceReceive=std::make_shared<SockServQueue>(serverListenPortReceive,msgDeserializer,msgSerializer,sep);
  ret.qserviceReply=std::make_shared<SockServQueue>(serverListenPortSend,msgDeserializer,msgSerializer,sep);
  ret.qclientRequest=std::make_shared<SockClientQueue>(server,serverListenPortReceive,msgDeserializer,msgSerializer,sep);
  ret.qclientReceive=std::make_shared<SockClientQueue>(server,serverListenPortSend,msgDeserializer,msgSerializer,sep);
  return ret;
}

I've only implemented three queue creation functions - one for creating in-memory queues, one for creating ip-queues where Service has a service queue and a client queue and finally one where Service has two ip-service queues. I can now implement a mini test program acting as a client choosing any one of the three queue creation functions.

The test program wires up queues between itself and the service. It then revs up a timer which sends a message to the service each time it ticks. At the same time the test program listens on messages from the service - when one is received it prints it.

#include "QueueDefs.h"
#include "QueueCreation.h"
#include "Service.h"
#include <boost/asio_queue.hpp>
#include <string>
#include <iostream>
#include <memory>
using namespace std;
using namespace std::placeholders;
namespace asio=boost::asio;

// asio io service
asio::io_service ios;

// client queue sender/receiver
shared_ptr<QueueSender>clientSender;
shared_ptr<QueueListener>clientReceiver;

// handle messages from service
void clientMsgHandler(boost::system::error_code const&ec,QueueValueType const&msg){
  cout<<"Client received message: \""<<msg<<"\""<<endl;
  clientReceiver->async_deq(clientMsgHandler);
}
// timer handler - sends a message each time timer pops
void timerHandler(boost::system::error_code const&ec,boost::asio::deadline_timer*timer){
  // send a message
  boost::system::error_code ec1;
  clientSender->sync_enq("Hello",ec1);

  // set timer again
  timer->expires_from_now(boost::posix_time::milliseconds(1000));
  timer->async_wait(std::bind(timerHandler,_1,timer));
}
// echo test program
int main(){
  // --- create queues (choose between type of queues)
  // queue_struct queues=createMemQueues();
  // queue_struct queues=createAsymetricIpQueues();
  queue_struct queues=createSymetricIpQueues();


  // --- create service
  Service serv(::ios,queues.qserviceReceive,queues.qserviceReply);

  // --- create client sender/receiver
  clientSender=make_shared<QueueSender>(::ios,queues.qclientRequest.get());
  clientReceiver=make_shared<QueueListener>(::ios,queues.qclientReceive.get());

  // --- setup client event handler
  clientReceiver->async_deq(clientMsgHandler);

  // --- setup timer for sending messages
  boost::asio::deadline_timer timer(::ios,boost::posix_time::milliseconds(1000));
  timer.async_wait(std::bind(timerHandler,_1,&timer));

  // --- start asio event loop
  ::ios.run();
}

Now what is interesting in this design is that it is possible to test the service inside a single executable and still use ip-queues. I could factor out the service into it's own executable - i.e., wrap the Service in an executable and it would still work exactly the same way.

As you probably noticed the message I used was simple - a straight forward std::string. It is not difficult to send more complicated messages. All that is needed is to be able to serialise and de-serialise them - typically using boost::serialization.

Seems like a lot of work for an echo service ...?

Yes, there is quite a lot of work for simple echo service. However, if I factor in the polymorphism in the design which allows me packaging options which may span across address spaces as well as iseemless integration with the boost::asio framwework, I don't think the code overhead is that large. Also, if you look carefully at the code you'll see that most of the code is simply a steep ramp which you ony have to climb ones no matter how many services you implement and how complicated they are.

A note of caution

Clearly the type of design I've shown here is not always a good one. For instance, conceptually it might be simpler to communicate via a single message broker instead of having a queue between any two services needing to communicate. Error handling and recovery might also be simpler using a message broker. At the end, what design approach you use depends on the requirements, the environment and also on your preferred way of designing.

The services I've described here are not request-response based services. They are fire and forget services. If you need a request-response type service the design described here may not be the best one.

Yet another problem wth the design I presented here is that services are tied to the communictaion structure. As a consequence the process is not decoupled from the services. In general it is not a good idea to design in such a way. However, for special purpose designs where we have a stable structure for the message flow, it is possible to build very high performance message based systems using the approach presented here.

A real system

I have recently built a system translating text from one language to one or more languages using the ideas above. The main benefit I believe I had from the approach was that I could quickly re-factor the architecture and deploy (at compile time) services in way that suited my needs and requirements.

One system was designed as a single executable where services were all using in-memory queues - that was what the customer needed. A set of other systems were partitioned in various ways - still using the same services - but having services deployed into separate executables.

There were also several disadvantages when designing using this type of approach. One obvious one was that services and processes were tightly coupled - in the translation system it did not really matter so much though. But in other systems it may make a big difference.

All in all, designing using this methodology is simply another experience which, as always, leads to better designs in the future.