CS SYD - Why mocking is a bad idea

4 Likes

My impression is that most engineers don’t use mocking for integration tests, but to be able to unit test certain parts of the system without requiring the entirety of the integration test environment.

I’d say it’s pretty obvious that an integration test is not an integration test if it relies on mocking.

9 Likes

The purpose of mocking is not to test the behavior of code in a vacuum, but to validate that the code behaves as expected with respect to some other object that obeys certain laws. Of course if either your mocked object or the real object violates the laws you’re coding against, the mocked test is pointless. Very frequently you can get mocked and real objects that obey some useful laws.

A super useful form of mocking is to have mocked resources which are in a sense worse-behaved than the real thing. For example, finding concurrency bugs with dejafu by mocking out concurrent operations is seriously impressive and useful. Dejafu allows you to test your program with every possible concurrency ordering to find deadlocks and other concurrency related bugs. This is actually better for testing than the “real” thing.

I don’t think “only ever test against real resources” is actionable or helpful. Testing the behavior of your program with respect to a spec is extremely useful and testing it in a way that doesn’t require you to actually spin up a database or use the filesystem or hit a website means you’re going to write and execute these tests more frequently.

There are also non-testing benefits to coding in a way that’s conducive to mocking (MTL, capabilities, whatever) such as that you’re specifically describing the effects required by procedures in your program rather than describing some specific implementation that happens to satisfy the properties you actually need. E.g. If I were looking for a library that did cryptographic operations, I would prefer one that used cryptonite’s MonadRandom over one that used IO.

10 Likes