Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail fast if invocation matches never expectation #679

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

floehopper
Copy link
Member

@floehopper floehopper commented Nov 7, 2024

TODO

  • Come up with a version of this that emits a deprecation warning rather than actually changing the behaviour.
  • Ensure all the relevant documentation is updated, e.g. method dispatch in README, class docs for Mock, and docs for any relevant methods.
  • Update commit note(s) to reference being consistent with the behaviour described in the following test which was added in d358377

def test_should_fail_fast_if_method_is_never_expected_but_is_called_once_even_if_everything_is_stubbed
test_result = run_as_test do
stub = stub_everything('stub')
stub.expects(:method).never
1.times { stub.method }
end
assert_failed(test_result)
assert_equal [
'unexpected invocation: #<Mock:stub>.method()',
'unsatisfied expectations:',
'- expected never, invoked once: #<Mock:stub>.method(any_parameters)'
], test_result.failure_message_lines
end

Previously when an invocation matched an expectation which did not allow invocations (i.e. Expectation#never had been called on it), but the invocation also matched another expectation which did allow invocations, then the test would not fail with an unexpected invocation error.

This was happening because neither the if condition was true, because the "never" expectation was not returned by ExpectationList#match_allowing_invocation, but the other expectation allowing expectations was returned. Thus Expectation#invoke was called on the latter and Mock#raise_unexpected_invocation_error was not called.

This behaviour was confusing and had led to a number of issues being raised over the years: #44, #131, #490 & most recently #678. Previously I had thought this was a symptom of the JMock v1 dispatch behaviour (which might still be true) and thus would be best addressed by adopting the JMock v2 dispatch behaviour (#173). However, having considered this specific scenario involving a "never" expectation more carefully, I've decided to try to fix it with the changes in this PR.

Now a test like this will fail with an unexpected invocation error:

mock = mock('mock')
mock.stubs(:method)
mock.expects(:method).never
mock.method

unexpected invocation: #<Mock:mock>.method()
unsatisfied expectations:
- expected never, invoked once: #<Mock:mock>.method(any_parameters)
satisfied expectations:
- allowed any number of times, invoked never: #<Mock:mock>.method(any_parameters)

Closes #678. Also addresses #490, #131 & #44.

@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from 7caf33f to 5a214c0 Compare November 7, 2024 15:39
@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from 5a214c0 to fc5898a Compare November 9, 2024 11:10
@floehopper floehopper changed the title WIP Fail fast if invocation matches never expectation Nov 9, 2024
floehopper added a commit that referenced this pull request Nov 9, 2024
Currently when an invocation matches an expectation which does not allow
invocations (i.e. `Expectation#never` has been called on it), but the
invocation also matches another expectation which does allow
invocations, then the test does not fail with an unexpected invocation
error.

In #679 I'm planning to change this behaviour so the test fails fast
with an unexpected invocation error. This commit displays a deprecation
warning instead to give users a chance to update their code before the
actual change is made.
@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from fc5898a to ca5bf20 Compare November 9, 2024 14:32
floehopper added a commit that referenced this pull request Nov 13, 2024
Currently when an invocation matches an expectation which does not allow
invocations (i.e. `Expectation#never` has been called on it), but the
invocation also matches another expectation which does allow
invocations, then the test does not fail with an unexpected invocation
error.

In #679 I'm planning to change this behaviour so the test fails fast
with an unexpected invocation error. This commit displays a deprecation
warning instead to give users a chance to update their code before the
actual change is made.
@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from ca5bf20 to 73c4ea6 Compare November 13, 2024 14:05
floehopper added a commit that referenced this pull request Nov 24, 2024
Currently when an invocation matches an expectation which does not allow
invocations (i.e. `Expectation#never` has been called on it), but the
invocation also matches another expectation which does allow
invocations, then the test does not fail with an unexpected invocation
error.

In #679 I'm planning to change this behaviour so the test fails fast
with an unexpected invocation error. This commit displays a deprecation
warning instead to give users a chance to update their code before the
actual change is made.

The new behaviour will bring the behaviour into line with what is
already the case for mocks where `Mock#stub_everything` has been called
as per this acceptance test [1] that was introduced in this commit [2].

[1]: https://github.com/freerange/mocha/blob/f2fa99197f35e2d2ce9554db63f6964057e29ce0/test/acceptance/expected_invocation_count_test.rb#L202-L214
[2]: d358377
@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch 2 times, most recently from f832c55 to b98a6dc Compare November 24, 2024 10:44
Previously when an invocation matched an expectation which did not allow
invocations (i.e. `Expectation#never` had been called on it), but the
invocation also matched another expectation which did allow invocations,
then the test would not fail with an unexpected invocation error.

After #681 a deprecation warning was displayed in this scenario, but now
an unexpected invocation error is reported.

This behaviour was confusing and had led to a number of issues being
raised over the years: #44, #131, #490 & most recently #678. Previously
I had thought this was a symptom of the JMock v1 dispatch behaviour
(which _might_ still be true) and thus would be best addressed by
adopting the JMock v2 dispatch behaviour (#173). However, having
considered this specific scenario involving a "never" expectation, I've
decided to try to fix it with the changes in this commit.

Also the new behaviour will bring the behaviour into line with what is
already the case for mocks where `Mock#stub_everything` has been called
as per this acceptance test [1] that was introduced in this commit [2].

Now a test like this will fail with an unexpected invocation error:

    mock = mock('mock')
    mock.stubs(:method)
    mock.expects(:method).never
    mock.method

    unexpected invocation: #<Mock:mock>.method()
    unsatisfied expectations:
    - expected never, invoked once: #<Mock:mock>.method(any_parameters)
    satisfied expectations:
    - allowed any number of times, invoked never: #<Mock:mock>.method(any_parameters)

Closes #678. Also addresses #490, #131 & #44.

[1]: https://github.com/freerange/mocha/blob/f2fa99197f35e2d2ce9554db63f6964057e29ce0/test/acceptance/expected_invocation_count_test.rb#L202-L214
[2]: d358377
@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from b98a6dc to 15985bc Compare November 24, 2024 11:17
@floehopper floehopper marked this pull request as ready for review November 24, 2024 11:18
@davidstosik
Copy link

davidstosik commented Nov 27, 2024

@floehopper Hi there! 👋🏻
I started seeing the warning introduced by #681 and would like to check what's expected to happen on a specific scenario.
Here it is:

require "test_helper"

class NeverTest < ActiveSupport::TestCase
  class Foo
    def self.create_if(condition)
      new if condition
    end
  end

  test "it creates only if condition is true" do
    Foo.expects(:new).never
    Foo.create_if(false)

    Foo.expects(:new).once
    Foo.create_if(true)
  end
end

This test currently passes, though it triggers the deprecation warning.

Are we expecting this to still work in the future?
If not, how can I write a test that follows the same logic as above?

@floehopper
Copy link
Member Author

@davidstosik Thanks for reporting this. It does look as if something's not quite right with the deprecation warning logic. I'll try to look into it as soon as I can.

@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from d74c4c6 to 5daa0cc Compare November 27, 2024 12:24
floehopper added a commit that referenced this pull request Nov 27, 2024
In #681 I added logic to display a deprecation warning about a change I
planned to make in #679. However, it turns out the logic in both was
faulty as demonstrated in this example [1] where the deprecation
warning was incorrectly displayed for the 2nd call to `Foo.create_if`:

    class Foo
      def self.create_if(condition)
        new if condition
      end
    end

    test "it creates only if condition is true" do
      Foo.expects(:new).never
      Foo.create_if(false)

      Foo.expects(:new).once
      Foo.create_if(true)
    end

This commit adds a new acceptance test to cover an example where the
logic was incorrect and updates the logic in `Mock#handle_method_call`
to fix the logic so this new test passes and none of the existing tests
fail. Unfortunately the implementation is now rather complicated and
hard to follow, but I think it will do for now. I have a couple of ideas
for ways to simplify it, but I don't have time to work on that now.

I plan to update the equivalent logic in #679 to match the new logic
implemented here.

[1]: #679 (comment)
@floehopper
Copy link
Member Author

floehopper commented Nov 27, 2024

@davidstosik I've attempted a fix for the deprecation warning logic in #686. If you have a chance, could you try running your tests against the fix-deprecation-warning-logic-for-expectation-with-never-cardinality branch of mocha to see whether (a) the deprecation warning goes away in the case you identified; and (b) there are no other unexpected regressions...? Oh, and just to be clear, I don't expect the test you described to fail when #679 is merged.

c.f. 01874f1

In #681 I added logic to display a deprecation warning about a change I
planned to make in this PR. However, it turns out the logic in both was
faulty as demonstrated in this example [1] where the deprecation warning
was incorrectly displayed for the 2nd call to `Foo.create_if`:

    class Foo
      def self.create_if(condition)
        new if condition
      end
    end

    test "it creates only if condition is true" do
      Foo.expects(:new).never
      Foo.create_if(false)

      Foo.expects(:new).once
      Foo.create_if(true)
    end

This commit adds a new acceptance test to cover an example where the
logic was incorrect and updates the logic in `Mock#handle_method_call`
to fix the logic so this new test passes and none of the existing tests
fail. Unfortunately the implementation is now rather complicated and
hard to follow, but I think it will do for now. I have a couple of ideas
for ways to simplify it, but I don't have time to work on that now.

[1]: #679 (comment)
@floehopper floehopper force-pushed the expectation-with-never-cardinality-should-fail-fast-if-invoked branch from 5daa0cc to 29a8e08 Compare November 28, 2024 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

expects.never doesn't handle stubs methods correctly
2 participants