Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate early warnings for callback API mock misses #210

Open
Schoonology opened this issue Mar 17, 2017 · 1 comment
Open

Investigate early warnings for callback API mock misses #210

Schoonology opened this issue Mar 17, 2017 · 1 comment

Comments

@Schoonology
Copy link
Contributor

Schoonology commented Mar 17, 2017

I'm currently working with testdouble.js with a callback-based API, and if I miss a stubbing, I only know that when the test times out (or when I see it hang). Looking through the API again, I don't see a way to throw an early warning (or error) for missed mocks, which would be really helpful with these callback-based APIs (i.e. request).

Commence thinking out loud:

Stubbings are currently unordered, right? One solution would be to order them, so that stubbing A has to occur before stubbing B for any given double. If that were true, then we could throw on the first miss—your invocations were out of order. Not elegant, but it could be a valid solution.

Another would be to throw on any miss. Why don't we do this currently?

I'm currently running v2.0.2, which is otherwise working really well. I hooked it into a dependable container with zero issue to replace request proper, and it's perfect. 😍

@searls
Copy link
Member

searls commented Mar 18, 2017

Stubbings are currently unordered, right?

There are two things you could mean here so I'll comment on both.

First, does the order of the stubbing configurations matter?

If you're talking about setting up multiple stubbing configurations on a single double, they are ordered: it's last-in-that-satisfies-the-stubbing-wins, and yes you can use these to specify fallback conditions before the more specific "correct" invocation that you want.

Here's an example: https://runkit.com/searls/58cd0ef764cab80014bb474f

var td = require('testdouble')

var get = td.func()
td.when(get(), {ignoreExtraArgs: true}).thenReturn('A') // <-- fallback case
td.when(get(td.matchers.isA(Number))).thenReturn('B') // <-- any number at all
td.when(get(5)).thenReturn('C') // <-- exactly 5
td.when(get(5), {times: 1}).thenReturn('D') // <-- fire at most one time.

console.log(get(4)) // B
console.log(get(5)) // D
console.log(get(5)) // C
console.log(get(3)) // B
console.log(get('trollface')) // A

Second, can you specify that two invocations happen in the order you expect?

Which is "in order for the second stubbing on a double to fire, the first must be satisfied". This is not supported explicitly via the API, because it was my experience in Java/Ruby that this led to really knotty test code that was hard to sort out after the fact (e.g. 'why isn't my stubbing working? oh b/c there's an obscure config being used at the top that means i have to call it with X first')

However, almost every time I use multiple doubles or invoke one multiple times in a test, I implicitly design the test to ensure that they are called in the right order, like so:

var is = td.func()

td.when(is('1st arg')).thenReturn('1st result')
td.when(is('1st result')).thenReturn('2nd result')

// Then in my production source I essentially have constrained my subject to call `is` a second time with the args in the order i want
var thing = is('1st arg') // 1st result
var otherThing = is(thing) // 2nd result

You also asked:

Another would be to throw on any miss. Why don't we do this currently?

This was a rather big contentious fracas in the GOOS community / Java-land in the late aughts as spies started to become popular as less-noisy/hassley alternatives to mocks. Traditional mock objects blow up whenever they're invoked in any unexpected way or not invoked in _every single expected way.

This does accomplish the aim of immediately identifying the source of the problem and might alleviate the pain you're experiencing while writing the test, but IME it was a super duper bad time. Every time I tried to refactor anything, the stub/verify configurations for mock objects was inevitably going to break, need to be revisited and reworked, and much of the time it was really painful to adjust that configuration to match the slightly-changed reality.

This is one of the ways mocking earned its reputation for being a pain in the ass and "too much coupling between the test and the implementation" because the slightest change to how things got called would blow up at the mock level (i.e. a framework constraint) instead of the test level (i.e. an expectation miss). That's why I've since been promoting people use spies and try to craft tests that imply the expectations you have about the subject whereever possible, and not worry too much if you leave a little breathing room for the subject to call the spies in ways you didn't anticipate, especially when you're not sure it would be wrong.


Ok, all that said, can I have an example of your code to try my hand at writing a test of?

@jasonkarns jasonkarns added this to Backlog in Test Double Trouble Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

2 participants