| View previous topic :: View next topic |
| Author |
Message |
Mikael
Newbie
Joined: 16 Jun 2009
Posts: 9
|
 Posted: Fri Jun 26, 2009 10:11 am Post subject: Tight coupling between unit test and implementation code |
 |
|
By using mocking in unit tests you are also doing white-box testing, because you are forced to know what your implementation code is doing to be able to mock correctly. This results in a tight coupling between the unit test and the implementation code.
A simple refactoring of your implementation code can result in broken tests - even if you do not change the functionality in the implementation code.
I think that this tight coupling smells very bad 
Does any one have any opinions on that?
Is this coupling acceptable?
Can we do something about it? |
|
| Back to top |
|
 |
Elisha
Moderator
Joined: 24 Jun 2009
Posts: 168
|
| Posted: Fri Jun 26, 2009 11:52 am Post subject: |
|
|
Hi Mikael,
The issue you've pointed out is very important in the unit testing world.
Coupling a test to the implementaion of the code it's testing can cause tests become fragile and hard to maintain. There are many ways to do unit testing and to use mock objects, doing it right can help avoiding the coupling.
The main advantage of using mock objects is that it helps isolating the logic being tested. Unit test focuses on a logic of a specific class, but it's also affected by the class dependencies. Using mock objects can assist controlling the dependencies and therefore make sure the only logic being tested belongs to the class under test.
Mock objects don't necessarily mean white-box testing. The mock object stands instead of real object, but it has the same contract as the real one. The fact the mock object fakes a behavior only means it knows the contract, but it doesn't have to know class under test implemetation (and not even the real class it's faking).
Let's take a look at a simple example of a class which creates mail message using a username. The class needs a dependency to find the user email.
| Code: | public interface IUsersRepository
{
string GetEmail(string userName);
}
public class EmailFactory
{
private readonly IUsersRepository usersRepository;
public EmailFactory(IUsersRepository usersRepository)
{
this.usersRepository = usersRepository;
}
public string GenerateMailMessage(string userName)
{
string email = usersRepository.GetEmail(userName);
string message =
string.Format("To: {0}; Subject: Hello: {1}; Body: Welcome {1}", email, userName);
return message;
}
}
[TestClass]
public class EmailFactoryTests
{
[TestMethod]
public void GenerateMailMessage_UserNameExistsInRepository_MailAddressPlacedInMessage()
{
var fakedUsersRepository = Isolate.Fake.Instance<IUsersRepository>();
const string mailFromRepository = "user@email.com";
Isolate.WhenCalled(()=> fakedUsersRepository.GetEmail(null)).WillReturn(mailFromRepository);
var emailFactory = new EmailFactory(fakedUsersRepository);
const string userName = "userName";
string actualMessage = emailFactory.GenerateMailMessage(userName);
StringAssert.StartsWith(actualMessage, "To: " + mailFromRepository);
}
}
|
As can be seen, the test or the mock knows nothing about the factory implementation. All the test does is to supply an object which provides basic functionality for the repository contract. The factory implementation can change (use inner cache for example) and the test won't have to change.
It is important to notice the main advantage - the logic here is isolated. We tested only if the message composes correctly. We don't have to worry here if the email is resolved incorrectly. If the test fails we know it's the composing logic that broke it. It's simple to know what is the exact place that needs to be fixed.
Another advantage is that we don't have to setup a dependency object which may have complicated setup. The real UsersRepository could be a database and to make the test run we it must be able to connect to it. This way the test is free to run without being concerned about its availability.
Of course there are many ways to abuse mock objects and tests and make them coupled with the implemetation. One of the simplest ways to abuse mocks is over-specification. Over-specification means the test verifies too much that calls where made on fake objects instead of verifying the state which resulted from the action.
An example for over-specification could be in a test similar to the one in the example, where the test will generate a message twice to the same user. Over-specifying will be verifying on GetEmail that it was called twice. This is over-specifying since it couples the test to the implementation. If the factory will use cache, it will use the GetMail once and work correctly, but the test will fail due to the unnecessary expectation.
I hope it helps using mock objects with less coupling between them and the implementation.
Best regards,
Elisha
Typemock Support Team |
|
| Back to top |
|
|
Mikael
Newbie
Joined: 16 Jun 2009
Posts: 9
|
| Posted: Tue Jun 30, 2009 6:09 am Post subject: |
|
|
Hi Elisha
Thank you for writing such a detailed reply.
I understand what you are saying. Mocking calls to methods that is outside the scope of your test is good and nice and result in a lot less (parhaps even no) coupling between your implementation code and the dependencies it may have. And I also agree that it very important not to over specify the test.
But I still believe that using mocking will always result in a tight coupling between the test and implementation code. When you do mocking you MUST be able to know the inside of the implementation code – or else your have no way of knowing what to mock. Your test code is by nature coupled to the implementation code. Of course not all changes in your implementation code will break your test, but changes to areas that you mock will potentially break your test.
Let me explain that by showing you an example using your code:
Let’s say that I choose to refactor your implementation code like this:
| Code: | public string GenerateMailMessage(string userName)
{
string email = usersRepository.GetEmail2(userName);
string message =
string.Format("To: {0}; Subject: Hello: {1}; Body: Welcome {1}", email, userName);
return message;
} |
I just changed the call GetEmail to GetEmail2. Where GetEmail2 is a new and faster way of retrieving the email, and I want to use that in my code. The functionality of GenerateMailMessage is NOT changed. I actually just did a small refactoring to improve performance. This simple change will break the test.
Here is the way I see it (when using mocking):
1. Accept that your test code is coupled to your implementation code. See it as one piece of work (if you do TDD this will properly not be a hard thing to do).
2. Enjoy the fact that you are able to decoupled all dependencies your implementation code have.
Or in other words: Trading coupling between implementation code and it's dependencies for coupling between implementation and test code is not a bad trade.
Any comments on my view of things are very welcome? (Am I wrong?) |
|
| Back to top |
|
|
Elisha
Moderator
Joined: 24 Jun 2009
Posts: 168
|
| Posted: Wed Jul 01, 2009 4:28 pm Post subject: |
|
|
Hi Mikael,
It is true that in some cases the test depends on the implementation (e.g. only one method of a service was faked where few methods serves similar functionality). The better the test is written the little it depends on the implementation.
In many cases the contract of the dependency is simple enough to be covered with few behaviors. In these cases there is no dependency between the code the test. If a dependency serves the same functionallity through several methods it can be covered by faking all to avoid the coupling.
Best Regards,
Elisha
Typemock Support Team |
|
| Back to top |
|
|
Mikael
Newbie
Joined: 16 Jun 2009
Posts: 9
|
| Posted: Wed Jul 01, 2009 9:15 pm Post subject: |
|
|
Hi Elisha
You wrote:
| Quote: | | In many cases the contract of the dependency is simple enough to be covered with few behaviors. In these cases there is no dependency between the code the test. |
I am sorry, but I do not follow you. Can you provide a unit test sample that uses mocking that does not depend on the implementation code? I would really like to see that.
Even if I try hard, I am not able to come up with a unit test (that uses mocking) that is so decoupled from implementation that I can do whatever refactoring (only refactoring - no change in functionality) without breaking the test.
Also. You wrote this:
| Quote: | | If a dependency serves the same functionallity through several methods it can be covered by faking all to avoid the coupling. |
Are you recommending that all methods variants to be mocked, just to make sure that when/if the implementation code change, then the unit test does not need to be changed?
If we go back to the sample code in my previous post, you recommend that all methods that return email addresses are mocked (e.g. GetEmail and GetEmail2)?
Would that not just couple your unit test even further? Imagine that some of the methods that are being mocked are changed (e.g. deleted, renamed or some change in parameters/return type). Then you most likely also will need to change all unit tests that is mocking the methods (even if the unit test does not rally use them – they are there “just in case”)
Plus. What if someone later on adds a new GetEmail3 and change the implementation code to use that method?
I am not trying to be a grumbler. I am just interested in learning the best approach for using mocks in unit tests.  |
|
| Back to top |
|
|
tillig
TMExpert
Joined: 28 Sep 2006
Posts: 86
|
| Posted: Wed Jul 01, 2009 10:34 pm Post subject: |
|
|
I think you've found that line between "design for testability" and "test what's designed."
If you've got a testable design that uses patterns like inversion of control, then mocking/stubbing is helpful in testing so you don't have to have a bunch of tiny objects floating around that just, say, implement a bunch of interfaces just for testing purposes.
On the other hand, if you're adding unit tests to legacy code, or if you've got an API that you've intentionally designed a certain way that encapsulates a lot of logic, then you'll run more into this tight coupling of test logic to implementation logic. You might be able to mitigate it some by creating a single "setup" method that will set standard expectations and mocks for a given scenario - that way you only have one place in your tests that is coupled to the implementation instead of every test - but you're right, there will still be some sort of white box testing going on. You obviously have to know what's getting called internally so you know what dependencies you have and, thus, know what to mock.
I think this is just one of those necessary/accepted evils. If you find your test logic is too tightly coupled to implementation (and if you have the luxury to do so) you may need to look at your design. On the other hand, if you have an intentional design that maybe is less conducive to testing, or if you have some code you're trying to add tests to that you can't change... the best you can really do is try to reduce the number of places where the tight coupling occurs.
_________________
Travis Illig
Sr. Software Developer, Electronic Banking Services
Fiserv
http://www.fiserv.com |
|
| Back to top |
|
|
Mikael
Newbie
Joined: 16 Jun 2009
Posts: 9
|
| Posted: Thu Jul 02, 2009 5:17 am Post subject: |
|
|
Hi Travis
Thanks for replying - I agree with you. Especially this:
| Quote: | ...there will still be some sort of white box testing going on. You obviously have to know what's getting called internally so you know what dependencies you have and, thus, know what to mock.
I think this is just one of those necessary/accepted evils.
|
Even if you design your code for testability your implementation code will be coupled to your test code as soon you start using mocks. |
|
| Back to top |
|
|
|
FAQ
Search
Register
Rss
Profile 
Log in 
