Specifications
I received a couple questions about the Specification Framework that I use in the CQRS example and thought lets talk about that for a bit. The first thing that should be underlined is that this is _not_ a framework, they are a few classes and extension methods that rely on NUnit for the actual assertions and and Moq for mocking of the dependencies. I got the initial bits from Greg Young at his DDD course which I extended a little bit for my specific needs.
I have the following base test fixture classes:
- BaseTestFixture
- BaseTestFixture<TSubjectUnderTest>
- AggregateRootTestFixture<TAggregateRoot>
- CommandTestFixture<TCommand, TCommandHandler, TAggregateRoot>
- EventTestFixture<TEvent, TEventHandler>
- PresenterTestFixture<TPresenter>
These different classes are all very specific towards a specific need, which is a direct opposite from what a framework usually provides.
Black box
I try my best to make my tests treat the subject under test (SUT) as a black box, meaning that in my tests I don’t directly interact with the actual class that I am testing. Instead I want to trigger the behavior by executing behavior that lies further outside. The behavior that triggers the behavior on the SUT may be an actual implementation, or it could be a fake.
The same applies to the result of the behavior that gets tested, instead of verifying some state in the SUT I want to verify what happens outside of the SUT. So what matters is that I test the behavior not the state of the subject under test.
I also try to get further away from the SUT then its immediate usage. Doing so makes the tests less brittle for change. This in itself is not always an easy task, but I recommend you try it anyway.
The BaseTestFixture
This is the simplest test fixture class that I have and I use this for and I actually don’t use this anywhere in the example code, but it serves a really good basic overview of the semantics that are shared among the other test fixture classes.
1 namespace Test.Fohjin.DDD
2 {
3 [Specification]
4 public abstract class BaseTestFixture
5 {
6 protected Exception CaughtException;
7 protected virtual void Given() { }
8 protected abstract void When();
9 protected virtual void Finally() { }
10
11 [Given]
12 public void Setup()
13 {
14 CaughtException = new ThereWasNoExceptionButOneWasExpectedException();
15 Given();
16
17 try
18 {
19 When();
20 }
21 catch (Exception exception)
22 {
23 CaughtException = exception;
24 }
25 finally
26 {
27 Finally();
28 }
29 }
30 }
31
32 public class GivenAttribute : SetUpAttribute { }
33
34 public class ThenAttribute : TestAttribute { }
35
36 public class SpecificationAttribute : TestFixtureAttribute { }
37 }
It is following the Given When Then (GWT) approach, and as you can see it is really simple. Also note that I introduced some other named attributes by simply inheriting from the default NUnit attributes, this was purely done to stay with the GWT approach.
Below here is an incredible KISS example of how you would use this BaseTestFixture, which I believe doesn’t need further explanation. (I know I am misusing the term KISS here, but I thought if was fitting anyway).
1 namespace Test.Fohjin.DDD
2 {
3 public class Very_adding_a_number_to_the_subject_under_test : BaseTestFixture
4 {
5 private int SubjectUnderTest;
6
7 protected override void Given()
8 {
9 SubjectUnderTest = 2;
10 }
11
12 protected override void When()
13 {
14 SubjectUnderTest += 2;
15 }
16
17 [Then]
18 public void Then_the_new_value_of_the_subject_under_test_will_be_4()
19 {
20 SubjectUnderTest.WillBe(4);
21 }
22 }
23 }
The BaseTestFixture<TSubjectUnderTest>
Now we are getting into a more interesting case because now my subject under test is actually provided by the generic parameter of the base test fixture class. And to be honest this class is only used with 12 specifications out of the 122 specification classes. This is mostly because it is still a very generic solution, but again a nice way to ease into it.
1 namespace Test.Fohjin.DDD
2 {
3 [Specification]
4 public abstract class BaseTestFixture<TSubjectUnderTest>
5 {
6 private Dictionary<Type, object> mocks;
7
8 protected Dictionary<Type, object> DoNotMock;
9 protected TSubjectUnderTest SubjectUnderTest;
10 protected Exception CaughtException;
11 protected virtual void SetupDependencies() { }
12 protected virtual void Given() { }
13 protected abstract void When();
14 protected virtual void Finally() { }
15
16 [Given]
17 public void Setup()
18 {
19 mocks = new Dictionary<Type, object>();
20 DoNotMock = new Dictionary<Type, object>();
21 CaughtException = new ThereWasNoExceptionButOneWasExpectedException();
22
23 BuildMocks();
24 SetupDependencies();
25 SubjectUnderTest = BuildSubjectUnderTest();
26
27 Given();
28
29 try
30 {
31 When();
32 }
33 catch (Exception exception)
34 {
35 CaughtException = exception;
36 }
37 finally
38 {
39 Finally();
40 }
41 }
42
43 public Mock<TType> OnDependency<TType>() where TType : class
44 {
45 return (Mock<TType>)mocks[typeof(TType)];
46 }
47
48 private TSubjectUnderTest BuildSubjectUnderTest()
49 {
50 var constructorInfo = typeof(TSubjectUnderTest).GetConstructors().First();
51
52 var parameters = new List<object>();
53 foreach (var mock in mocks)
54 {
55 object theObject;
56 if (!DoNotMock.TryGetValue(mock.Key, out theObject))
57 theObject = ((Mock) mock.Value).Object;
58
59 parameters.Add(theObject);
60 }
61
62 return (TSubjectUnderTest)constructorInfo.Invoke(parameters.ToArray());
63 }
64
65 private void BuildMocks()
66 {
67 var constructorInfo = typeof(TSubjectUnderTest).GetConstructors().First();
68
69 foreach (var parameter in constructorInfo.GetParameters())
70 {
71 mocks.Add(parameter.ParameterType, CreateMock(parameter.ParameterType));
72 }
73 }
74
75 private static object CreateMock(Type type)
76 {
77 var constructorInfo = typeof(Mock<>).MakeGenericType(type).GetConstructors().First();
78 return constructorInfo.Invoke(new object[] { });
79 }
80 }
81 }
Wow there is _a lot_ more going on here! You are right, because here I make the base test fixture responsible for instantiating the subject under test, including providing mocks for any dependencies that it may have. So it is an auto mocker as well, but the interesting part here is that it puts a reference of the injected mocks in a collection that you can access inside your tests by using the OnDependency<TType> method that returns a Moq object.
Take a look at a specification using this base test fixture class:
1 namespace Test.Fohjin.DDD.Scenarios.Receiving_money_transfer
2 {
3 public class When_receiving_a_money_transfer : BaseTestFixture<MoneyReceiveService>
4 {
5 protected override void SetupDependencies()
6 {
7 OnDependency<IReportingRepository>()
8 .Setup(x => x.GetByExample<AccountReport>(It.IsAny<object>()))
9 .Returns(new List<AccountReport> { new AccountReport(Guid.NewGuid(), Guid.NewGuid(), "AccountName", "target account number") });
10 }
11
12 protected override void When()
13 {
14 SubjectUnderTest.Receive(new MoneyTransfer("source account number", "target account number", 123.45M));
15 }
16
17 [Then]
18 public void Then_the_newly_created_account_will_be_saved()
19 {
20 OnDependency<IBus>().Verify(x => x.Publish(It.IsAny<ReceiveMoneyTransferCommand>()));
21 }
22 }
23 }
So the first thing you see is the method SetupDependencies that is requesting the mock object for injected type IReporintgRepository and it is using the Moq way of setting up the Mock object. This is only needed when in your specification you need the mocks to be setup in a specific way. I intentionally separated the SetupDependencies from the Given as they may be two different things. And in the actual test you see the usage of the OnDependency method again where its being used to verify that something was indeed triggered on the injected class.
Now indeed this is not really treating the subject under test as a black box, for example in the When I make a direct call to a method on the subject under test. So my test knows about this now, meaning when it changes I need to change this test as well. So lets take a look where I go a bit further into the black box mentality.
The PresenterTestFixture<TPresenter>
Here I am not going to show you the code of the PresenterTestFixture implementation as it is almost identical the the previous base test fixture. So lets go straight to an actual specification:
1 namespace Test.Fohjin.DDD.Scenarios.Adding_a_new_client
2 {
3 public class When_in_the_GUI_the_phone_number_of_the_new_client_is_saved : PresenterTestFixture<ClientDetailsPresenter>
4 {
5 private object CreateClientCommand;
6
7 protected override void SetupDependencies()
8 {
9 OnDependency<IPopupPresenter>()
10 .Setup(x => x.CatchPossibleException(It.IsAny<Action>()))
11 .Callback<Action>(x => x());
12
13 OnDependency<IBus>()
14 .Setup(x => x.Publish(It.IsAny<object>()))
15 .Callback<object>(x => CreateClientCommand = x);
16 }
17
18 protected override void Given()
19 {
20 Presenter.SetClient(null);
21 Presenter.Display();
22 On<IClientDetailsView>().ValueFor(x => x.ClientName).IsSetTo("New Client Name");
23 On<IClientDetailsView>().FireEvent(x => x.OnFormElementGotChanged += null);
24 On<IClientDetailsView>().FireEvent(x => x.OnSaveNewClientName += null);
25
26 On<IClientDetailsView>().ValueFor(x => x.Street).IsSetTo("Street");
27 On<IClientDetailsView>().ValueFor(x => x.StreetNumber).IsSetTo("123");
28 On<IClientDetailsView>().ValueFor(x => x.PostalCode).IsSetTo("5000");
29 On<IClientDetailsView>().ValueFor(x => x.City).IsSetTo("Bergen");
30 On<IClientDetailsView>().FireEvent(x => x.OnFormElementGotChanged += null);
31 On<IClientDetailsView>().FireEvent(x => x.OnSaveNewAddress += null);
32
33 On<IClientDetailsView>().ValueFor(x => x.PhoneNumber).IsSetTo("1234567890");
34 On<IClientDetailsView>().FireEvent(x => x.OnFormElementGotChanged += null);
35 }
36
37 protected override void When()
38 {
39 On<IClientDetailsView>().FireEvent(x => x.OnSaveNewPhoneNumber += null);
40 }
41
42 [Then]
43 public void Then_the_save_button_will_be_disabled()
44 {
45 On<IClientDetailsView>().VerifyThat.Method(x => x.DisableSaveButton()).WasCalled();
46 }
47
48 [Then]
49 public void Then_a_create_client_command_with_all_collected_information_will_be_published()
50 {
51 On<IBus>().VerifyThat.Method(x => x.Publish(It.IsAny<CreateClientCommand>())).WasCalled();
52
53 CreateClientCommand.As<CreateClientCommand>().ClientName.WillBe("New Client Name");
54 CreateClientCommand.As<CreateClientCommand>().Street.WillBe("Street");
55 CreateClientCommand.As<CreateClientCommand>().StreetNumber.WillBe("123");
56 CreateClientCommand.As<CreateClientCommand>().PostalCode.WillBe("5000");
57 CreateClientCommand.As<CreateClientCommand>().City.WillBe("Bergen");
58 CreateClientCommand.As<CreateClientCommand>().PhoneNumber.WillBe("1234567890");
59 }
60
61 [Then]
62 public void Then_overview_panel_will_be_shown()
63 {
64 On<IClientDetailsView>().VerifyThat.Method(x => x.Close()).WasCalled();
65 }
66 }
67 }
Again there is the setting up of a dependency in the beginning and then there is the Given method, Presenter in this case is the subject under test, so you can see that the specification still knows about the SUT in the Given, but you will notice that this is not used in either the When or the Then.
Hey what is that On thing in there? Well that is a very small DSL wrapping the Moq API. I did this to make it slightly more readable, and in this case it is very adapt towards working with a view and presenter. In the Given I am setting up my IClientDetailsView with the correct data, but I am also simulating that an event was triggered. This is not logic that the test is concerned about, all we do here is bring the view and the presenter in the correct state for this particular specification. So instead of setting these things directly on the presenter this will all be directed from the view.
Then in the When we again fire an event from the view, but in this case it is the going to trigger the behavior on the presenter that we want to verify. And in the Then methods we verify on the view again that the presenter actually did the correct things, but we also verify on other dependencies that the correct methods where called, in this case the IBus.
I am not completely happy with the mini DSL yet, but I think it is cleaner then the default Moq API. Just for those that are curious, here is the mini DSL which gets returned by the On method:
1 namespace Test.Fohjin.DDD
2 {
3 public class MockDsl<TType> where TType : class
4 {
5 private readonly IDictionary<Type, object> _mocks;
6
7 public MockDsl(IDictionary<Type, object> mocks)
8 {
9 _mocks = mocks;
10 }
11
12 public ValueSetter<TType, TProperty> ValueFor<TProperty>(Expression<Func<TType, TProperty>> selector)
13 {
14 return new ValueSetter<TType, TProperty>(_mocks, selector);
15 }
16
17 public void FireEvent(Action<TType> fieldSelector)
18 {
19 if (!_mocks.ContainsKey(typeof(TType)))
20 throw new Exception(string.Format("The requested dependency '{0}' is not specified in the constructor", typeof(TType).FullName));
21
22 var mock = (Mock<TType>)_mocks[typeof(TType)];
23 mock.Raise(fieldSelector);
24 }
25
26 public Verifier<TType> VerifyThat { get { return new Verifier<TType>(_mocks); } }
27 }
28
29 public class Verifier<TType> where TType : class
30 {
31 private readonly IDictionary<Type, object> _mocks;
32
33 public Verifier(IDictionary<Type, object> mocks)
34 {
35 _mocks = mocks;
36 }
37
38 public void ValueIsSetFor(Action<TType> selector)
39 {
40 if (!_mocks.ContainsKey(typeof(TType)))
41 throw new Exception(string.Format("The requested dependency '{0}' is not specified in the constructor", typeof(TType).FullName));
42
43 var mock = (Mock<TType>)_mocks[typeof(TType)];
44 mock.VerifySet(selector);
45 }
46
47 public MethodVerifier<TType> Method(Expression<Action<TType>> selector)
48 {
49 return new MethodVerifier<TType>(_mocks, selector);
50 }
51 }
52
53 public class MethodVerifier<TType> where TType : class
54 {
55 private readonly IDictionary<Type, object> _mocks;
56 private readonly Expression<Action<TType>> _fieldSelector;
57
58 public MethodVerifier(IDictionary<Type, object> mocks, Expression<Action<TType>> fieldSelector)
59 {
60 _mocks = mocks;
61 _fieldSelector = fieldSelector;
62 }
63
64 public void WasCalled()
65 {
66 if (!_mocks.ContainsKey(typeof(TType)))
67 throw new Exception(string.Format("The requested dependency '{0}' is not specified in the constructor", typeof(TType).FullName));
68
69 var mock = (Mock<TType>)_mocks[typeof(TType)];
70 mock.Verify(_fieldSelector);
71 }
72 }
73
74 public class ValueSetter<TType, TProperty> where TType : class
75 {
76 private readonly IDictionary<Type, object> _mocks;
77 private readonly Expression<Func<TType, TProperty>> _fieldSelector;
78
79 public ValueSetter(IDictionary<Type, object> mocks, Expression<Func<TType, TProperty>> fieldSelector)
80 {
81 _mocks = mocks;
82 _fieldSelector = fieldSelector;
83 }
84
85 public void IsSetTo(TProperty value)
86 {
87 if (!_mocks.ContainsKey(typeof(TType)))
88 throw new Exception(string.Format("The requested dependency '{0}' is not specified in the constructor", typeof(TType).FullName));
89
90 var mock = (Mock<TType>)_mocks[typeof(TType)];
91 mock.SetupGet(_fieldSelector).Returns(value);
92 }
93 }
94 }
CQRS and Event Sourcing
By combining CQRS and Event Sourcing we get an architecture that is very suitable for black box testing its behavior, which was a real eye opener when Greg demonstrated this to me. He says that the way to bring your aggregate root back into the desired state is to playback the events that are needed to do so. Then you can execute the behavior on the aggregate root, and finally to actually verify your behavior you retrieve the published events and verify that they are as expected.
Now the beauty of this is that the setup and the verification this will work on any aggregate root because we are using the IEventProvider interface there that they all implement. The only actual knowledge about the aggregate root that remains is the specific behavior that you trigger.
But I went a little bit further then what was shown during the course, I am saying that instead of executing the behavior on the aggregate root we could also just provide the command that should trigger this behavior to be executed. Because the command would be handled by a specific command handler which in turn would execute the domain behavior.
Now below here is the command test fixture that allows me to do just that, I need to provide the actual command, command handler and aggregate root types that are to be used in this specification.
1 namespace Test.Fohjin.DDD
2 {
3 [Specification]
4 public abstract class CommandTestFixture<TCommand, TCommandHandler, TAggregateRoot>
5 where TCommand : class, ICommand
6 where TCommandHandler : class, ICommandHandler<TCommand>
7 where TAggregateRoot : class, IOrginator, IEventProvider<IDomainEvent>, new()
8 {
9 private IDictionary<Type, object> mocks;
10
11 protected TAggregateRoot AggregateRoot;
12 protected ICommandHandler<TCommand> CommandHandler;
13 protected Exception CaughtException;
14 protected IEnumerable<IDomainEvent> PublishedEvents;
15 protected virtual void SetupDependencies() { }
16 protected virtual IEnumerable<IDomainEvent> Given()
17 {
18 return new List<IDomainEvent>();
19 }
20 protected virtual void Finally() { }
21 protected abstract TCommand When();
22
23 [Given]
24 public void Setup()
25 {
26 mocks = new Dictionary<Type, object>();
27 CaughtException = new ThereWasNoExceptionButOneWasExpectedException();
28 AggregateRoot = new TAggregateRoot();
29 AggregateRoot.LoadFromHistory(Given());
30
31 CommandHandler = BuildCommandHandler();
32
33 SetupDependencies();
34 try
35 {
36 CommandHandler.Execute(When());
37 PublishedEvents = AggregateRoot.GetChanges();
38 }
39 catch (Exception exception)
40 {
41 CaughtException = exception;
42 }
43 finally
44 {
45 Finally();
46 }
47 }
48
49 public Mock<TType> OnDependency<TType>() where TType : class
50 {
51 return (Mock<TType>)mocks[typeof(TType)];
52 }
53
54 private ICommandHandler<TCommand> BuildCommandHandler()
55 {
56 var constructorInfo = typeof(TCommandHandler).GetConstructors().First();
57
58 foreach (var parameter in constructorInfo.GetParameters())
59 {
60 if (parameter.ParameterType == typeof(IDomainRepository<IDomainEvent>))
61 {
62 var repositoryMock = new Mock<IDomainRepository<IDomainEvent>>();
63 repositoryMock.Setup(x => x.GetById<TAggregateRoot>(It.IsAny<Guid>())).Returns(AggregateRoot);
64 repositoryMock.Setup(x => x.Add(It.IsAny<TAggregateRoot>())).Callback<TAggregateRoot>(x => AggregateRoot = x);
65 mocks.Add(parameter.ParameterType, repositoryMock);
66 continue;
67 }
68
69 mocks.Add(parameter.ParameterType, CreateMock(parameter.ParameterType));
70 }
71
72 return (ICommandHandler<TCommand>)constructorInfo.Invoke(mocks.Values.Select(x => ((Mock) x).Object).ToArray());
73 }
74
75 private static object CreateMock(Type type)
76 {
77 var constructorInfo = typeof (Mock<>).MakeGenericType(type).GetConstructors().First();
78 return constructorInfo.Invoke(new object[]{});
79 }
80 }
81
82 public class ThereWasNoExceptionButOneWasExpectedException : Exception {}
83 }
Please note that the Given method now returns an IEnumerable<IDomainEvent> this is to be used to provide the events that are needed to bring the aggregate root into the correct state for this specification. This is using the exact same mechanism as the actual code uses to make state changes in the aggregate root, so there cannot be a case that you are testing your aggregate root using a state that it cannot get into.
The When method returns the expected command, so all you do there is create the command with the correct information and return it to the specification.
Then in the try catch block you may have noticed that a command handler is executing the provided command and that after that the events are being retrieved from the aggregate root. These events are what you would verify to make sure your domain behavior is correct.
But this may all sound very abstract, lets look at a simple specification and see how clean and readable is really is:
1 namespace Test.Fohjin.DDD.Scenarios.Withdrawing_cash
2 {
3 public class When_withdrawing_cash : CommandTestFixture<WithdrawlCashCommand, WithdrawlCashCommandHandler, ActiveAccount>
4 {
5 protected override IEnumerable<IDomainEvent> Given()
6 {
7 yield return new AccountOpenedEvent(Guid.NewGuid(), Guid.NewGuid(), "AccountName", "1234567890");
8 yield return new CashDepositedEvent(20, 20);
9 }
10
11 protected override WithdrawlCashCommand When()
12 {
13 return new WithdrawlCashCommand(Guid.NewGuid(), 5);
14 }
15
16 [Then]
17 public void Then_a_cash_withdrawn_event_will_be_published()
18 {
19 PublishedEvents.Last().WillBeOfType<CashWithdrawnEvent>();
20 }
21
22 [Then]
23 public void Then_the_published_event_will_contain_the_amount_and_new_account_balance()
24 {
25 PublishedEvents.Last<CashWithdrawnEvent>().Balance.WillBe(15);
26 PublishedEvents.Last<CashWithdrawnEvent>().Amount.WillBe(5);
27 }
28 }
29 }
So you provide historical events to bring the aggregate root into the expected state, you fire of the command, then you verify the published events to ensure your domain behavior is correct. If you choose the correct naming for your events and commands, then a business person would be able to understand the specification. Especially if you parse the text and do a little bit of formatting.
Where is Should?
Hey what are all those WillBe and WillBeOfType things that I see in your specifications, should they not be ShouldBe and ShouldBeOfType? Well I used to think so as well, until I attended a presentation by Kevlin Henney at NDC where he explained that Should is not specific enough. Should indicates that it might not. I like to use the example; “I should really do the dishes, but I won’t”. By using Will Be and Must you are much more dictating what will or must happen, its not a question anymore.
Finally
I am completely taken by this approach and as you see you don’t need a big BDD framework for this. I think using something like this gives a good learning experience before going towards an actual BDD framework like MSpec. Also Uncle Bob just wrote a good post about to not abuse the Given When Then approach and also take a look at the Mocks aren’t Stubs article by Martin Fowler.
By now you must <grin> understand that I like to throw with code examples so yes the post is very long, but I hope it provides enough value instead of just confusion.
Thanks Mark. This post was loaded with value for me. Thank you for the detailed information.
Thanks for sharing and explaining. I really like your approach. Pulling the Given and When up into the setup makes perfect sense, it really cleans up the specs – why didn’t I think of that?! Will copy shamelessly
@Jason Jarrett
Glad to hear it was useful!
@Torbjørn Marø
this is actually something I got from Greg’s example, but there are other examples on the net as well doing the same thing.
That is what makes blogging worth it, when others start stealing from it
Thanks for sharing this. I like what you’ve done with the test fixture base classes. I cloned your git repo for Fohjin projects and will study it further.
Hate to ask for something when you’ve already given something of value freely; any chance to have a stylesheet that will wrap the code when printing rather than cutting it off (even in landscape)?
@Al Gonzalez
Sorry, no printer friendly version, at least not that I know of.