Comments on: Testing a Membership Provider

By: Bryan Watts

Bryan Watts — Sun, 20 Apr 2008 16:29:22 +0000

I didn’t mean that you should register a provider per test – I meant that you should register all implementations that need to be tested. In your case, there is only 1 implementation, VirtualMembershipProvider, and that provider is the sole target of your test suite.

The same set of tests could be applied to another provider by registering it under a new name. You can target an entirely new implementation with the same tests using this technique. You end up with a line in the provider block per implementation, not per test.

A provider can be considered a pluggable implementation of an API. It is not meant to be stateful, i.e. requiring a “fresh” instance for each test. The only state in a provider is its configuration, which is scoped to all operations. If you need to test a different configuration, you now have a new implementation to register.and refer to.

What you are seeking to test is the particular implementation of an API represented by your provider. As such, you should be using the API to test, and ensure that API is hooked up to the correct implementation, not bypass it completely.

Consuming the provider and testing it are synonymous in a unit test.

By: trasa

trasa — Sun, 20 Apr 2008 14:56:28 +0000

This doesn’t allow me to meet the testing requirements of writing code like (apologies for this being all munged up):
[Test] public void CreateValidUser() {
VirtualMembershipProvider provider = new VirtualMembershipProvider();
MembeshipUser testResult = provider.CreateUser(….);
}

Microsoft has had a hostile approach towards TDD and alternative approaches in general; for more evidence of that just google for anything “ALT.Net”.

By: trasa

trasa — Sun, 20 Apr 2008 14:05:14 +0000

Adding a new line to the providers block for each test in the test app.config doesn’t strike me as scalable at all – and to test individual parts of this provider, i need the ability to unload & reload “fresh” copies of it. so the immutable dependency is exactly the problem.

I think you’re confusing consuming the provider with testing the provider itself. I’m not talking about writing unit tests for a web application that also uses membership provider, I’m talking about writing unit tests for developing the provider itself.

By: Bryan Watts

Bryan Watts — Sun, 20 Apr 2008 04:35:27 +0000

Also, to be clear: your unit testing framework should support a way to load App.config which is not a manual build step, i.e. done automatically like in ASP.NET and Forms projects.

For example, a Google search of “nunit app.config” has this as the first result:

http://www.cornetdesign.com/2005/08/nunit-appconfig-files-its-all-about.html

You can tell your NUnit project the name of the configuration file; it will be loaded when the app is run, and reloaded with rebuilt DLLs.

_ReadOnly, in this case, is akin to a thin metal ruler:

http://blogs.msdn.com/ericlippert/archive/2003/11/03/53333.aspx

By: Bryan Watts

Bryan Watts — Sun, 20 Apr 2008 04:26:49 +0000

App.config is the authority on the state of the framework running underneath your application. An app domain running a unit test is no different from a web app, Forms app, console app, or Windows service, from the perspective of the configuration system.

That is why there is a direct, immutable dependency on that system. It is *the* way to do what you seek to do. Any other approach is proprietary, introduces an unnecessary dichotomy, and most likely won’t scale.

Your unit testing framework should have a way to load an App.config. If you need to test multiple providers of a system, you may register several and refer to each by name.

By: Reflective Perspective - Chris Alcock » The Morning Brew #76

Reflective Perspective - Chris Alcock » The Morning Brew #76 — Fri, 18 Apr 2008 07:17:01 +0000

[...] Testing a Membership Provider – Tony Rasa looks at creating an in memory Membership provider for test use. [...]

By: trasa

trasa — Thu, 17 Apr 2008 19:25:30 +0000

Also – just to be clear – the initialization procedures here are to set up unit tests *developing the provider itself* – and not meant for consumers of the provider. They’d use app.config and the standard initialization pipeline as normal.

That’s why I can override the _ReadOnly field and not worry about what else that is going to break. You wouldn’t do that “for real.”

By: trasa

trasa — Thu, 17 Apr 2008 19:23:55 +0000

Ah, but I don’t want to configure it from App.config, because I’m driving this particular initialization from unit tests.

I agree that a typical usage of a provider would be through the standard configuration mechanism. But the provider doesn’t give us a mechanism for following an alternative other than typical.

If there’s a standard way for initializing membership providers in a unit test scenario, aside from some sort of hack involving multiple app.configs overloaded at run-time, I’d love to hear it.

By: Bryan Watts

Bryan Watts — Thu, 17 Apr 2008 18:18:56 +0000

Membership.Providers is a run-time collection representing the configured providers for that system. It is made read-only so it can be initialized and then let loose in the wild.

The solution here is to configure your provider in App.config so it falls neatly into the initialization pipeline.

Conclusions:

* Don’t assume roadblocks are arbitrary. It is most likely there to prevent abuse in the first place.

* Be sure to investigate the standard ways of accomplishing your goal before “just getting it done” – be sensitive to intent vs. mechanism.