Event Driven Architecture: Publishing Events using an IOC container
In my last post I talked about advanced IOC usage and how it is possible to use an IOC container to resolve an open generic from a closed implementation. This is technically cool, but it does not explain why this is important. In this post I want to show some additional code that will demonstrate a basic concept of Event Driven Architecture: Publishing an Event.
Why is Event Driven Architecture good?
Event driven architecture is extremely extensible. In my mind, it the perfect solution to address the Open Closed Principal and enable Single Responsibility in code.
Consider a system that processes orders. There is a method called SubmitOrder(order) on an OrderController. It just validates and saves the new order to the database today.
In a following story, the product owner says ?I need the system to send an email to the customer when the order is submitted?. No problem, inject the MailService into the OrderController, send the mail in the SubmitOrder(order) method. great.
The next story the product owner says ?I need you to deduct the ordered quantities from the OnHand inventory?. Check, so we inject in the inventory service into our OrderController, and we call it from the SubmitOrder(order) method.
More stories?. the product owner says ?We need to send a message to the warehouse fulfillment system to let them know a new order has been submitted?
Ugh, another service, another method. The tests for SubmitOrder become out of control. Even when extracting methods and encapsulating more logic into services, there is still violation of SRP & OCP. Not to mention there could be performance degradation from performing all of these tasks synchronously while the user waits.
Its not the job of SubmitOrder to do all of this, there has to be a better way.
Publishing an Event Example
What if I ?tell? the system that an event had occurred, and anyone interested could take action? (like an old school observer right?) In the following code sample I am going to inject in an ?EventPublisher? into my OrderController, then use that object to notify the rest of the system by publishing an event using this line:
_eventPublisher.Publish(new OrderSubmittedEvent{OrderId = order.Id});
The OrderController
public class OrderController : Controller
{
private readonly IRepository<Order> _orderRepository;
private readonly IEventPublisher _eventPublisher;
public OrderController(IRepository<Order> orderRepository,
IEventPublisher eventPublisher)
{
_orderRepository = orderRepository;
_eventPublisher = eventPublisher;
}
public ActionResult SubmitOrder(OrderViewModel viewModel)
{
try
{
if (ModelState.IsValid)
{
var order = MapOrder(viewModel);
_orderRepository.Add(order);
_orderRepository.SaveChanges();
_eventPublisher.Publish(new OrderSubmittedEvent{OrderId = order.Id});
//Display success message
//ViewInfo.AddSuccessMessage(Language.SubmitOrderSuccess);
}
}
catch
{
ModelState.AddModelError("__Form", Language.SubmitOrderError);
}
return View(viewModel);
}
//other
}
Now, using this simple Consumer (Handler) interface:
public interface IConsumer<T>
{
void Handle(T eventMessage);
}
I can independently implement each of the system requirements, as needed:
public class EmailOrderConfirmation : IConsumer<OrderSubmittedEvent>
{
public void Handle(OrderSubmittedEvent eventMessage)
{
//send email
}
}
public class NotifyWarehouse : IConsumer<OrderSubmittedEvent>
{
public void Handle(OrderSubmittedEvent eventMessage)
{
//notify warehouse
}
}
public class DeductOnHandInventory : IConsumer<OrderSubmittedEvent>
{
public void Handle(OrderSubmittedEvent eventMessage)
{
//deduct inventory
}
}
One of my favorite parts about this code: the container is building up all of these event consumers, which makes it is easy to satisfy all of the dependencies using constructor injection. This makes testing each of these handlers a breeze, and everything follows the same patterns and conventions keeping the code base understandable and clean. (Services, Controllers, Consumers all work the same)
Just plug in the dependencies in the EmailOrderConfirmation consumer ctor* like so:
public class EmailOrderConfirmation : IConsumer<OrderSubmittedEvent>
{
private readonly IRepository<Order> _orderRepository;
private readonly ISmtpService _smtpService;
private readonly ILogger _logger;
public EmailOrderConfirmation(IRepository<Order> orderRepository,
ISmtpService smtpService,
ILogger logger)
{
_orderRepository = orderRepository;
_smtpService = smtpService;
_logger = logger;
}
public void Handle(OrderSubmittedEvent eventMessage)
{
var order = _orderRepository.Single(x => x.Id == eventMessage.OrderId);
var message = new SmtpMessage();
//get customer info from order & populate message
_smtpService.SendMessage(message);
}
}
* Note that you could also inject the IEventPublisher into a Consumer and publish more events.
How Does it Work?
Under the hood the IoC container is doing most of the work. It keeps track of the event subscriptions and also provides the consumer instantiation.
Event Subscriptions
Here is how the event subscriptions are added to the container. In this example I use an interface so that I can enable dependency injection into components, and then mock the GetSubscriptions<T> dependency for easy testing. I also use a static method for adding subscriptions since the registration process operation usually happens at bootstrapping time, which is generally all static. The IoC.Container here is thread safe.
public interface ISubscriptionService
{
IEnumerable<IConsumer<T>> GetSubscriptions<T>();
}
public class EventSubscriptions : ISubscriptionService
{
public static void Add<T>()
{
var consumerType = typeof(T);
consumerType.GetInterfaces()
.Where(x => x.IsGenericType)
.Where(x => x.GetGenericTypeDefinition() == typeof(IConsumer<>))
.ToList()
.ForEach(x => IoC.Container.RegisterType(x,
consumerType,
consumerType.FullName));
}
public IEnumerable<IConsumer<T>> GetSubscriptions<T>()
{
var consumers = IoC.Container.ResolveAll(typeof(IConsumer<T>));
return consumers.Cast<IConsumer<T>>();
}
}
Event Publishing
Now that the subscriptions are setup, the publisher can read those subscriptions, get the consumers from the container, and pass the event message instance to each of them. Again I am using an Interface so I can enable dependency injection (and mocking capabilities) for the components which need to do publishing (controllers, services)
public interface IEventPublisher
{
void Publish<T>(T eventMessage);
}
public class EventPublisher : IEventPublisher
{
private readonly ISubscriptionService _subscriptionService;
public EventPublisher(ISubscriptionService subscriptionService)
{
_subscriptionService = subscriptionService;
}
public void Publish<T>(T eventMessage)
{
var subscriptions = _subscriptionService.GetSubscriptions<T>();
subscriptions.ToList().ForEach(x => PublishToConsumer(x, eventMessage));
}
private static void PublishToConsumer<T>(IConsumer<T> x, T eventMessage)
{
try
{
x.Handle(eventMessage);
}
catch(Exception e)
{
//log and handle internally
}
finally
{
var instance = x as IDisposable;
if (instance != null)
{
instance.Dispose();
}
}
}
}
– Keep in mind in this example is all running on the same thread, therefore even though semantically we are handling events as if they are asynchronous, it is a blocking operation
Additional Benefits of this Architecture
– Events are in messages, which can be serialized and processed on different threads, or even different nodes in a cluster. In a web environment, this might mean offloading the workload out of your IIS processes freeing web-server threads for performance & scalability (see Service Bus section)
– Get multicast delegate observer like functionality without having to deal with static events, event registration & deregistration (the whole += ?= biz)
– Each event handler has its own class, each having 1 responsibility
– Very extensible, very maintainable (wait, I said that already?)
This is not a Service Bus
This code above could work great in a simple MVC.NET application for implementing clean separation and structuring your code base into events. Though it gets a lot of influence from the OSS services busses, it is greatly simplified and merely demonstrates a simple concept of publishing & consuming domain events using an IoC container.
A service bus does quite a bit more than just publishing local messages on single thread. If you find yourself enjoying this posts and want to take it to the next level, check out MassTransit (Chris Patterson & Dru Sellers) & nServiceBus (Udi Dahan). I would also recommend the EAI patterns books by Gregor Hohpe and recognize the reference patterns. (one of these days Ill make it through it instead of using it for reference!)
Some additional concepts and common patterns you might find:
- Publish-Subscribe
- Request-Response
- Sagas (Workflow)
- Correlation
- Competing Consumers
- Distributed Load (processing across many nodes in cluster)
- Transport (Http, MSMQ, ActiveMq, WCF TCP)
- Serialization (Binary, XML, JSON)
- Threadpooling
The Specification
Normally I don’t make a habit of testing the container, but in this case its so close to the container I feel its appropriate vs mocking out the subscriptions. Plus since this is a blog post demonstrating advanced IOC usage, I want explicitly show all of the dependencies required at bootstrap time.
Please note the LoggerSpy for assertions. In this spec we are really only concerned whether or not they get called. Because of the nature of eventing, using a Test Spy is really the only good way I have found to assert whether or not your handlers get invoked. (If your processing on multiple threads you will have to make sure they are all done before you check assertions, perhaps a future blog post)
[TestFixture]
public class When_An_Order_Is_Submitted : Specification_Context
{
private LoggerSpy _loggerSpy;
private Mock<IRepository<Order>> _orderRepositoryMock;
readonly IUnityContainer _container = IoC.Container;
protected override void Context()
{
//setup EF repository mock
_orderRepositoryMock = new Mock<IRepository<Order>>();
//logger spy to capture handler messages
_loggerSpy = new LoggerSpy();
//register with container
_container.RegisterInstance<ILogger>(_loggerSpy);
_container.RegisterInstance(_orderRepositoryMock.Object);
_container.RegisterType<IEventPublisher, EventPublisher>();
_container.RegisterType<ISubscriptionService, EventSubscriptions>();
_container.RegisterType<OrderController>();
//add event subscriptions
EventSubscriptions.Add<EmailOrderConfirmation>();
EventSubscriptions.Add<NotifyWarehouse>();
EventSubscriptions.Add<DeductOnHandInventory>();
}
protected override void Because()
{
_container.Resolve<OrderController>().SubmitOrder(new OrderViewModel());
}
[Test]
public void Order_Should_Be_Persisted()
{
_orderRepositoryMock.Verify(x => x.Add(It.IsAny<Order>()));
_orderRepositoryMock.Verify(x => x.SaveChanges());
}
[Test]
public void Email_Notification_Should_Be_Sent_To_Customer()
{
_loggerSpy.LoggedMessages.ShouldContain(Language.OrderConfirmationEmailSuccess);
}
[Test]
public void On_Hand_Inventory_Should_Be_Reduced()
{
_loggerSpy.LoggedMessages.ShouldContain(Language.DeductOnHandInventorySuccess);
}
[Test]
public void Warehouse_Should_Be_Notified()
{
_loggerSpy.LoggedMessages.ShouldContain(Language.NotifyWarehouseSuccess);
}
}
This is a great concept. More devs should employ patterns like this. MvcContrib has an application bus in it as well. IOC container support as well. Mvccontrib.org
Nice article,
Any source code to go with it? Other than the one listed.
I especially like the fact that I now know a way of dealing with super-fat controllers. Often enough I find myself using to many services in one constructor.
I’ve played with the type of system you have written here, and I must say I agree, it’s the ultimate way of applying SRP and OCP.
I do however see a declared dependency on the type of message you publish, which is ok if I think about it, it makes the code more explicit.
I totally agree with Palermo, more devs should post this type of articles.
A very interesting article.
Downloadable source code would be great, so I can play and experiment with it.
Thanks.
Udi Dahan explained the same technic months ago http://www.uidahan.com/2008/02/29/how-to-create-fully-encapsulated-domain-models/ and implementation http://www.udidahan.com/2008/08/25/domain-events-take-2/
@Jeffrey – Thanks. It has been a while since I have looked at MvcContrib, perhaps I will pull down the latest and check it out.
@Mihai – Thank you. Yes, you would have dependency on the event message. I usually put all these in a separate App.Messages dll. In larger impl this is handy to include in any part of the system, client/server or server cluster.
@Mihai/Deiter – I will package up the code I used here I publish a download location. Stay tuned.
Jarod – seriously…thanks a stack for this.
This solution in its Elegance has been etched onto my brain – somewhat like poetry in the way it reads.
Keep up the great advice!
All the best,
Rob G
@Mike – Thanks for posting those links. Udi is very influential in this space. I am certainly not trying to claim original thought here, just trying promote awareness of better ways to build software.(I still don’t know many that build software like this)
@Rob G – That’s quite a compliment! I appreciate it, thank you.
I have created a VS2010 B2 project which contains the code from this post. Let me know if you have any questions or issues. Enjoy!
http://guild3.com/files/samples/ECDomainEvents.zip
To go one step further you could Publish(submitOrderMessage) and have a separate handler to persist, that then performs the Publish(orderSubmittedEvent). Regardless, I think this type of code can help to eliminate controller bloat. Thanks for the great read!
Hello,
although I can understand that IEventPublisher helps in terms of testing (because it’s mockable) and allows easy invertion of control as it’s injected in the constructor, I’d move the raising of events to a generic method to the base Controller. The method signature would look like the:
protected void Publish(T eventMessage);
The gain is hiding use of interface for all derivers, and providing one way to raise events in all containers, which can be altered in some way (for instance logging only for controllers or getting the publisher of the specified string key). What do you think about it?
Ever seen a chunk of code that makes you slap yourself in the forehead and say “Why didn’t I think of that?” This post definitely gave me that experience. Very simple and elegant. Looks like I need to redo the event broker quickstart for Unity!
@Szymon – I appreciate your feedback. IMO, I think that is a classic “Composition vs Inheritance” argument. You may gain some ease of use initially, but I believe the price is too steep.
Here would be some of my arguments against the inheritance approach:
-How would you test your controller logic that calls base.Publish?
-What if there are new implementations of IEventPublisher? For example, AzureCouldEventPublisher? (With constructor dependency injection you can just change the registration w/o compiling old code and it works)
-What if you need to make a change to Publish which affects only new code?(Versioning) does the new version go in the base for all controllers? At some point would you also have to version or derive a new controller?
In my experience inheritance can lead to brittle and inflexible design, so I tend to shy away unless I have no alternative. IoC *auto-wiring* enables the luxury of leaning heavily towards composition & testability, w/o too much PIA.
@Chris – Why yes, that happens to me all the time (and some of it may have been yours!) :P. Thank you for the kind words.
This is really beautiful, and I’ve always liked the pub/sub model, but in an ASP.NET MVC world how can you use it and effect the view? For example, if I wanted to change what was rendered to the user based on the result of the NotifyWarehouse task, is there an easy solution to get that back to the Controller? I realize the controller could also subscribe to events, but it may not know what events it needs to even watch for. Seems like there’s a catch-22 here.
If anyone is interested in doing this with Structure Map, I posted a question on SO that was answered.
http://stackoverflow.com/questions/2033794/unity-to-structure-map
@Jarod Ferguson
Jarod, I would add one more observation to your list of arguments: the controller does not pass the instance (or maybe instances, if it requires additional IEventPublisher, for example AzureCouldEventPublisher ;-)) to its base class, so all dependencies are resolved only for the specified type of controller, not the base one. This makes it more explicit or simply: better. Hence, I must agree, that using inheritance for this purpose should be rather called: abusing:]
Thank you for your anwer!
@Ryan – The View would only be immediately affected by the SubmitOrder Action (request/response). In a pub/sub model you do not get a response from the subscribers (NotifyWareHouse).
What you could do here is generate a token (Guid) which represents the event, and pass back this token in the ViewData for the SubmitOrder Action. Now you can poll from the client (JqUery) to an Action like AcknowledgeWarehouseReceipt using the token to correlate the initial published event. (Warehouse, do you have an order for this event?)
Check out this post, specifically the “Http Solutions” section: http://www.udidahan.com/2008/07/30/scaling-long-running-web-services/
@beckelmw – For StructureMap, you might want to take a look at ‘cfg.ConnectImplementationsToTypesClosing’. Check out this blog post: http://www.lostechies.com/blogs/jimmy_bogard/archive/2009/12/17/advanced-structuremap-connecting-implementations-to-open-generic-types.aspx
@Szymon – You bet!
@Ryan – you could do it the same way you do with normal events. Have a field in the payload that’s for the “return value”. Since all this stuff is synchronous you could easily have one or more subscribers set this field when they get it. Then grab the information out after the publish call finishes and do whatever you want with it.
Having said that, I’d suggest instead using the “Post/Process/Redirect” pattern (or whatever the name is). Basically, you never return a view from an action that processes HTTP POST requests – you only redirect afterwards to a GET request. The action that processes the GET can then query the state of the system as normal, and since everything should have been processed by then (or at least status updated to “processing”) everything just flows.
Yes, what @Chris suggested will work perfectly for most applications.
Keep in mind though if you use that approach, you will not be able to process events asynchronously (the next evolution here, the queue). As I mentioned in this post, this can be one of that main advantages to EDA.
How does this (in terms of design) compare to http://blog.robustsoftware.co.uk/2009/08/better-domain-event-raiser.html?
@gsogol – Conceptually they are the same (which is the important part, thinking/coding in terms of Events). The Impl is a little different. I would check out both examples and see which one works best for you.
A small suggestion. I prefer to have the Domain itself Publish the event rather than clients of the Domain. It would be easy for one client to ‘forget’ to Publish the event. This makes the Controller action even cleaner, it just translates the ViewModel, creates an Order and asks for it to be persisted.
You know, I was thinking “how am I going to do the email on created account thing without a mess” and voila. Found you through the morning brew
Since this runs on the current thread has anybody looked to make it run on multiple threads? I am thinking about trying it with the ManagedThreadPool that subtext uses (http://www.koders.com/csharp/fidD26A22C3944C4B6D66D2DA45BA3D502389B154CB.aspx?s=file:semaphore.cs)
I was thinking to queue up two threads that could be used to process the published messages then you could have a pub/sub model without having to tie in something like MSMQ.
@Joe Young
Yes. I have used this with a custom threadpool. The Koders class you linked should work, or you could also look at ThreadPool.QueueUserWorkItem. (though you will find much controversy using ThreadPool.QueueUserWorkItem in ASP.NET, I have used it with success)
See this article as well as all the related links: http://stackoverflow.com/questions/1325718/using-threadpool-queueuserworkitem-in-asp-net-in-a-high-traffic-scenario
how about adding a cross-cutting aspect to said “SubmitOrder” method instead ctor injections?
@mahmud khaled
I meant leveraging Aspect-Oriented Programming techniques
Great article. I am going to have to try this out, but I have a question. How would you handle a multi-tenant environment where maybe some clients want the email, but others do not and instead want to be notified with SMS?
Hi Jarrod,
Thanks for this article and this example. I appreciate how clearly your examples demonstrate a very bare bones, clean, implementation of architectural strategies. It helps that I also use EF POCO, and reviewing yours and Szymon’s code has convinced me to shift to using Unity.
In any case, I wonder if you could do the same for a CQRS architecture in which the client makes no use of the actual domain objects and read/write responsibilities are fully segregated. I find your style of example code to be very easy to follow which makes it much easier to see all of the very smart design decisions being made. Sometimes it’s helpful to see a fully functional application like the one Mark offers in his excellent CQRS post, but I find that when I’m studying an architectural implementation, it’s most helpful for the Domain to be as simple as possible (at least in the initial exploration). I appreciate for instance, that your Order object in this example consists of only an ID and that submitting an Order simply logs the action to a fake logger. 🙂
When I first read the comments for this post, I thought some of the responses were a little over the top in their praise. But having spent some time exploring the code, I get it. Thanks for putting this stuff out there. The contributions to the community from the Elegant Code bloggers are much appreciated.
Jason
@mahmud khaled
I might use AOP for cross-cutting concerns like logging, binding etc, but cant say I would use it for domain events. I want these events very visible, and IMO AOP decreases testability.
@Jarrod
I would handle multi-tenant or session based rules outside of the event publisher in the example. That said, I think you could come up with a dynamic subscription mechanism that could read a ‘ClientSubscriptions Map’ from a data store on session start. Then when EventPublisher.Publish is called, you could have it check the ClientSubscriptions, if the event subscription is present then publish the event. I would just make sure to keep a clear distinction between a system event and a client/customer event.
@Jason
Thank you for the positive feedback, I am very glad that you find my posts helpful. I do have plans for additional posts that build on the EDA posts and also EF POCO. I will put some CQRS posts on the radar and well see what comes out. I have a lot on my plate at the moment, so my blog has been on the back-burner.
@Jarod Ferguson
Sure thing… I look forward to it.
Excellent article – thanks for posting this!
The SubmitOrder action above feels like it is violating SRP by inheriting the responsibility of publishing domain events.
I suppose from a certain point of view the task of submitting an order could consist of creating the order as well as notifying any listeners that the order was created.
Your approach also puts the onus on the developer to remember to raise an event anytime that an action performed against the domain is executed that should logically result in the creation of an event. I have a personal aversion to multi-step processes. I guess it feels like the event being raised should be encapsulated with the method invocation related to it.
Thoughts?
p.s. Do you have a link to “Mark”‘s CQRS post? I don’t know which Mark you are speaking of.
The SubmitOrder action above feels like it is violating SRP by inheriting the responsibility of publishing domain events.
I suppose from a certain point of view the task of submitting an order could consist of creating the order as well as notifying any listeners that the order was created.
Your approach also puts the onus on the developer to remember to raise an event anytime that an action performed against the domain is executed that should logically result in the creation of an event. I have a personal aversion to multi-step processes. I guess it feels like the event being raised should be encapsulated with the method invocation related to it.
Thoughts?
p.s. Do you have a link to “Mark”‘s CQRS post? I don’t know which Mark you are speaking of.
agree with you Robert that the logic in the controller could/should be pushed into the domain. I wrote this a few years ago- I think I was referring to this as ‘Marks post’ 😕https://elegantcode.com/2009/11/11/cqrs-la-greg-young/
Office cleaners can provide quality
service and perform a variety of duties. It does not matter how big or small
your office or working environment is. A little office cleaning can make an
excellent impression on clients and employees.