Will Typemock make you stop designing for testability?

[royo]

This thread is created based on my blog post over here:
http://weblogs.asp.net/rosherove/archive/2008/01/17/is-typemock-too-powerful-will-it-kill-design-for-testability.aspx

so that we can open up a conversation around this topic. Feel free to write what you think, and what us, at Typemock, can do to help address this belief.

Roy Osherove

[tillig]

I blogged about this a couple of days back:

http://www.paraesthesia.com/archive/2008/01/14/designing-for-testability-with-typemock.aspx

I figure that if you keep yourself to Natural mocks, you're pretty much in the same boat as if you were using a mocking framework that forces you to design for testability... but you're not losing that Typemock power for when you really do need it.

Granted, there are some edge-case exceptions, like the fact that Typemock can mock a public static call using Natural mocks and that wouldn't happen in other frameworks, so you'll still have to be careful. It boils down to the fact that you can't rely on your mocking framework to enforce good design - that's up to you, the developer.
_________________
Travis Illig
Sr. Software Developer, Electronic Banking Services
Fiserv
http://www.fiserv.com

[Jacob]

Here's the thing: what does "Design for Testability" mean. More specifically, what does it mean if TypeMock is a part of your testing tool chest?

You see, with TypeMock, pretty much any design is testable. This has two effects that I don't think people are ready to face.

First, it means re-evaluating current architecture dogma. A lot of "best practices" have a lot of their power sapped if you no longer have to inject a dependency in order to test your public methods. Why exactly do you need Inversion of Control in general and Dependency Injection specifically if you have no specific need for that degree of separation of concerns? (this is a rhetorical question on my part. I answered it for myself, in public, recently.)

Please note that I'm not saying IoC and DI aren't powerful tools. I'm just saying that their importance diminishes quite sharply in the absence of requiring them for testing. You can go from needing it for every object to only needing it for a very small group of objects (and sometimes not needing it at all).

Unfortunately, enough people have needed DI for testing purposes that its presence has become synonymous with "good design". This brings me to the second important effect of making every design testable: you make a lot of investments (emotional, training, mindset, and code base) obsolete. There are a lot of people who have spent immense effort and pain urging their companies and teams to fundamentally change the way they code. That spent capital is never coming back and nothing TypeMock can do will change that.

What do you do if you've spent all this time convincing the powers that be that DI is absolutely essential and now . . . it isn't? That's going to hurt. Much better to just dismiss TypeMock as "too powerful" and move on.

Jacob

[royo]

Jacob, thanks for the informative answer. I really like the way you describe things, which makes me sometime doubt my own inner motives for believing in things.
For making me think twice about what design for testability really means to me, I thank you.

I don't think people will dismiss typemock just because fear of wasted time preaching the other way, at least not most people. But it does raise a concern.
The .net world as a whole has been doing great movement towards a world of better design, better practices and more. DI is a relatively new concept in the .NET world for most people (who didn't' come from Java).

In that light, it almost scares me to try and dismiss DI or design for testability simply because of all the other good things this thinking represents.

Perhaps people aren't ready to say "yes, we know DI exists, but we know where we don't necessarily need it", because they don't know enough in this world.
perhaps bringing DI and testability concepts is good, if for no other reason than to teach people what decoupling really means, with "testing" being just the excuse to make it happen.

We have put so much effort in making testing and testability drive lots of good design decisions, that, if "unit testing" didn't catch on in the .net world, we'd have a hard time introducing them.

Perhaps (being devils advocate) it is too early to dismiss this thinking, or to drive holes in it, for fear of throwing the baby out with the bath water.

What do you think?

Roy

[paulo.morgado]

I agree with Travis when he says that the API is a deliverable.
I also have a problem with any pattern or technique that advocates testing as the ultimate testing as the ultimate goal in software development.
For me, testing is just like compiling. It’s a step in the development process. As compiled code gives you more confidence than interpreted code (C# vs. Jscript), tested code gives you more confidence than non tested code.
I use TypeMock in my development in three ways:

1. Unit testing: I test every path of a method and mock everything outside of it (even if it belongs to the same class or object). This gives me confidence on each unit of code. The goal here is to achieve 100% of code coverage.

2. Class testing: I test the public interface of a class mocking everything outside of it. Here is a bit more tricky (and sometimes practically impossible) to cover all possible combinations.

3. Framework testing: The same as before but targeting a framework composed by a set of classes.

We must not lose track of what we are trying to achieve with testing, which is Correctness provability (see what Frans Bouma has to say about this).
_________________
Paulo Morgado
http://PauloMorgado.NET/

[Jacob]

[boo]

I think using type-mock has 0 to do with designing for testability and DI has absolutely 0 impact on whether the end product will be more testable or flexable. As an example I worked with a company that was using pure DI, 'to be flexable' (so they said) and possibly more testable, but they were doing it horribly and the program was a mess and extremely complicated to figure out. They actually were less agile because of it.

I think 'design for testability' is like salt, use as much as you personally need. I think many people have lost the pragmatism of programming and have become so focused on following a check-list of to-do's from some book and maintaining some status quo that they've forgotten that the ultimate goal is to deliver a functional product to the customer on time and on budget. The second goal is to be able to support and extend that product without re-writing the program. If you can do this using purely GoF/Fowler patterns, GREAT! If you can do this using DI and RM GREAT! If you can do this in VSTS with Microsoft Test and Typemock GREAT! To the person paying your paycheck (unless your self employed) it really doesn't matter what you use as long as you meet those two goals.

We use Typemock and we design for flexibility, but not through tests, but through our design of where we know we need to be flexible. In some cases we have used DI, in many case we don't as it would just be falling under Fowler's speculative generalities if we did and was the cause of the aforementioned company's reason for having a less agile product and why they couldn't make a date or extend there system in a timely manner to save their lives.

By using Typemock we let the design of the system stand on it's own and testing doesn't have to suffer based upon the patterns we use, and our design doesn't have to suffer because of the testing framework we use. We get the best of both worlds, or rather I get my cake and I get to eat it too.

[doron]

Hear hear! Some excellent points, boo. It's almost two years now since this discussion got started, and its been reiterated countless times on several forums. Why do you think this happens?