Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I consider the distinction between unit and integration test useless. The important part is tests give me confidence that my system is working, or if not working I can quickly find the problem. I don't care if the bug is in my code or in some third party code, I care that there is a bug. More than once the bug was because the third party code works different from what I expected, but mocks only tell me that if the code works as I expect then my code works - which isn't very good.

Fast tests are important. However I find that most integration tests run very fast if I select the right data. I can write unit tests that run very slot if I select the wrong data. I find that the local filesystem is very fast and so I don't worry about testing it (I place my data in a temporary directory - the problems with temporary directories are about security which I don't worry about in unit tests). Likewise I find that testing with a real database is fast enough (my DB is sqlite which supports in-memory databases - not everyone can work with a real database in that way). Which is to say you should challenge your assumptions about what makes a test slow and see if you can work around them.



You call both tests (which they both are) but if one specifically uses the word "unit test" then they should try using it appropriately, since there is a reason why the extra word is there.


The problem is that “unit” isn’t well defined. Anything from a method to a service can be considered the unit under test.


That's not a problem, that's a hangup you have.

We don't need a global definition of what a unit is, we just need to know what unit is being tested by a given unit test. This is important because it's how we understand coverage:

1. We need to understand what's within the unit (covered by the test) so we don't write redundant tests, and...

2. ...we need to understand what's not within the unit (not covered by the test) so we can write separate tests to cover that unit.

The unit can be anything, as long as we know what the unit we're testing when we read/write/run the test.

An integration test also needs to know what it's testing, but now you're looking at it from an outside perspective. You no longer care which specific areas of code (units) are within the integration, you just care what features are within the integration. Sometimes a refactor will mean that feature A no longer causes class B to call class C: that probably means changes to B's unit tests, because the code unit changed, but it doesn't mean changes to the integration test because the feature didn't change.

The important thing is to understand what's being tested by a given test.

For example, a project I'm working on has integration tests which are written with Selenium and run in a headless browser. Most of these tests involve users, but most of these tests aren't testing user creation, so I don't care how the users get created. Instead of going through the signup interface using Selenium, I'm just injecting the users into the database directly because that's faster and easier. The one exception is I'm not injecting the users into the database in my signup/login integration tests, because whether the user gets created is part of the feature under test.

Of course there are significant overlaps between these two categories, that is to say, there are some unit tests that are also integration tests.


It certainly doesn’t bother me, but considering the length of your rant, which I won’t bother to read after your first sentence, you certainly seem to have quite a few hangups.


Not wanting to respond to things you haven't read is a hangup you should have, but don't, apparently.

Nothing says "doesn't bother me" like having to tell everyone you aren't bothered.


Someone woke up on the wrong side today, or need to go get something to eat.


Go eat, then. :)


I'll also recommend you to read HN guidelines.


At Google they draw the distinction based on i/o. Anything that does i/o wouldn't be considered a unit test by them. Unit tests are usually for algorithms, math functions, data structure classes, and stuff like that. Most developers have never needed to write any of those things because they're abstracted by modern languages, which is why I suspect the terminology confuses some people.


Unit testing in the realm of software originated from the Smalltalk world, where a unit described the test suite for a class. That, with some liberty around the use of class in languages that are not class-centric, reflects the vast, vast, vast majority of tests I see out there in the wild.

The reality is that unit testing swept the world and has become synonymous with testing. The days of yore when things like capture and replay testing were it are behind us. Today, it is uncommon for anyone to write anything other than unit tests (as originally defined).

And that's why "unit testing" has become confusing. Its original meaning no longer serves a purpose. Today, we just call it "testing", leaving those who wish to continue to use "unit" for the sake of fashion to invent pet definitions; definitions not shared by anyone else.


That's a useful distinction for creating a shared language within Google, and it might be a useful distinction for your team too.

But it's definitely not the only way to divide things up. The important thing is that the terminology your term uses is useful for your team.

One project I'm working on has two kinds of tests: unit tests run within the code and test classes, state transitions, etc. by calling the code directly, while integration tests are run in Selenium in a headless browser from the perspective of the user. Some of the unit tests on models do hit the DB (i.e. they do I/O).


I think it's actually quite simple to come to a common language for unit and integration tests if you start from "integration" instead: if there is any sort of integration (two things interacting: your test setup can tell you that's happening), it's an integration test.

If there are no integrations, it's a unit test.

As in, it's all in the name. How does one fit a whole "service" (if we are talking something like a networking service like web API server there and not a DDD-style service) there really beats me, but I am happy to hear you out.

I'd note that things like setting up a client and then configuring it to access a service already fails that test, even if you hide that complexity into fixtures or "external" test setup.

Edit: for avoidance of any doubt, these are terms without clear, widely accepted definitions. I am mostly sharing how I am making what I think is a useful distinction to drive me to write properly scoped tests and structured code.


> if there is any sort of integration (two things interacting: your test setup can tell you that's happening), it's an integration test.

Two things of what? There are people that claim that as soon as there are more than one method involved, it's an integration test, and more common, that as soon as there are more than one class involved, it is. This lead to absurd mocking of utility classes to make sure that only one thing is tested at the time. And then, of course, the tests become very fragile.

And while there are situations when it is useful to only test one specific thing, what we really care about is the behaviour of whole system of many parts. We mock (fake or stub) to make the tests faster, repeatable and reliable. That is, what we don't want in our tests are slow calls, over slow networks or lengthy calculations. We want the calls to be repeatable, so we don't want to call, say, a database where the state changes between test runs. And we don't want to make calls to unreliable services; unreliable because they are under development or because they are over an unreliable network.


I would be in the camp that says two methods calling each other are an integration, but I avoid mocking as much as possible.

But if you need to use mocking there heavily, your code is not structured well: look for functional style and acyclic directed graph structure of calls.


The whole point of unit tests is that to write them you have to break the system up into units and that decision is an important one that depends on the problem being solved. If they are not well defined then unit tests force you to do define them.


> there is a reason why the extra word is there.

Fashion, but there is no good reason for it to be there. It does not communicate anything.


I appreciate your historical account but words have acquired or changed meaning frequently in the past ("hacker" or "cloud" just to name a couple of obvious ones).

There is a large number of people using terms like unit, integration, system or end-to-end to differentiate between different types of automated tests ran in an xUnit-style framework: why insist on not allowing the term to gain a new meaning?


The term is free to find new meaning, but so far nobody has found any scenario that is in need of differentiation.


As this discussion made obvious, a lot of people have: it's ok that you are in disagreement, but I think it's disingenuous to claim otherwise.


It's funny you bring up speed — people have different ideas of fast: if you need to run 50k tests, filesystem or database through ORM are usually not fast (you won't have those tests complete in sub-9s if that's the not-lose-focus time).

Having 50k tests complete in that time when backed by memory on a modern CPU is trivial (easily achievable with proper unit and integration tests).

I've been on one too many projects where tests were exactly slow because developers didn't care about testing just-enough when unit- or integration-testing — not to mention becoming parallyzed by any tiny change breaking a ton of unrelated tests.

What you are talking about are tests that cover even more than just integration points, and are usually described as end-to-end or system tests (depending on the scope and context of the code under test). I suggest having as few of those, but in general people structure their code badly so sometimes it's hard to avoid it (most web frameworks don't help, for instance).


I have tools like make (not actually make, but I expect you to know what make is unlike my tools) that ensure while I in theory have more than 100k tests and so running them all would take 10s of minutes, in practice I'm never running more than a few hundred at a time and so the total time is a couple seconds and I don't have to worry.

There is more than one way to solve the problem you suggest one alternative. I have found other answers that let me not need to worry so much.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: