Question about the tests

[karfau]

I'm currently trying to understand the tests to port them for the ActionScript implementation (see CC-Archived/promise-as3#11).

When i look at the first section suite I'm recognizing that it is only asserted that the promise doesn't get rejected but i don't see an assertion that it was fulfilled.

Which i would expect like this

setTimeout(function (){
  assert.strictEqual(onFulfilledCalled, true);
  done();
}, 100);

is this checked somewhere else or did i miss something in how the tests are executed?

[domenic]

Indeed, that is only testing

2.1.2.1: When fulfilled, a promise: must not transition to any other state.

It's not testing that if a promise is fulfilled, then it must be fulfilled. It's testing that if a promise is fulfilled, it must not also be rejected.

You are probably looking for the tests for

2.2.2.1: If onFulfilled is a function, it must be called after promise is fulfilled, with promise’s value as its first argument.

[karfau]

Understood: the timeout is actually there to give the test time to fail when rejected async.

But the assertion in onRejected only fails if onFulfilled has not been called even. But the section only talks about fulfilled ones.
And this test would not fail if the rejected handler would be called before the fulfilled handler right?

Just trying to understand how the spec section maps to the assertion,
thx for fast response

[karfau]

After thinking about it a bit more what i am able to read out of this assertion and the logic in the helpers is:

make sure fulfillment doesn't (also) cause a rejection afterwards (testThreeCases -> no matter when fullfillment happens)

is this what the call to testFulfilled is for in this section?

[domenic]

Yes, that's it exactly :)

[karfau]

I thought a lot about those tests now, while implementing them in ActionScript.
And i found that commenting out the equivalent of e.g. line 36 or 37 (or both, same for lines 53 and 54) or even switching them around would not fail any of those tests.

(If this is not the case when you execute the JS-tests please tell me how they fail.)

I think all this is caused by the fact that in the case where the test works no assertion is done at all.
For making more sense to me I just added an assertion for the onFulfilledCalled before i "stop waiting for something to happen" (the call to done() in JS).

Does this make any sense for you or do I still don't get the scope of those tests?

By the way: Is there a section 1.X anywhere?

[karfau]

Oh, maybe there is one "invisible" assertion in your JS test framework:
will not calling done() let the test fail?

[domenic]

will not calling done() let the test fail?

Yes, indeed! A timeout error will fail the test if 200 ms go by without done being called.

[karfau]

Now a lot of things make more sense :-)

Could you please also respond to my questions from my post before the last one?
Because most of them are still valid for me.
(maybe even just a link to a good article or talk about your approach to those "spec-tests"?)

Thx

[domenic]

Does this make any sense for you or do I still don't get the scope of those tests?

I wouldn't add assertions that aren't there. The scope of the 2.1.2.1 tests is to test 2.1.2.1, not to test 2.2.2.1.

By the way: Is there a section 1.X anywhere?

Section 1 is at http://promisesaplus.com/#terminology; it does not have tests.

[karfau]

Ok, now I refactored how my tests wok so it matches the 200ms timeout per default and ails for that.
Really looks and feels a lot better now.

I think what irritated me at the first point (beside the implicit fail) was that I normally write tests in a more TDD like fashion, so i first see them break and them make them work. Which was not possible the way i implemented those tests before today.
And I also really like explorative testing and looking for edge cases, but for those spec tests I think it really important, that only those tests fail, where the implementation doesn't fit to the spec.

Together with the fact that you can not really "prove" that something will not happen (e.g. by waiting 150 ms), this irritated me a lot.

And so starting with those strange promise-state tests not knowing "the big picture" of the test suite, brought up all those question marks.

My AS-Port of the test suite currently has 21 tests with finishing section

2.2.2.2: it must not be called before promise is fulfilled.

I will leave this issue open for the time I'm working on this, so we can discuss further questions in one "thread".

[domenic]

Yeah, the tests are not very good for TDDing; see #48.

[karfau]

I have read that thread before but didn't get what it is all about.
Now I understand more of it :)

[karfau]

would the testsuite also fail when done() is called twice, or would it silently ignore the second call?

[domenic]

It would fail

[karfau]

I expected somthing like this :)
the testing framework I use is not good about this.
I am currently able to call the equivalent of done after a time bigger then timeout,
but then the scope of the actual test from which this was caused is lost and it can cause another test to fail or (the flash player) brings up a modal error popup to the user that stops the execution like when a breakpoint gets hit when debugging, or it will not get called because the swf-instance is not open anymore.
This would be a killer for e.g. automated execution.
At the moment it looks like when I want to "fix" that for the scope of my tests I would need to work around this with some extra logic.
But I think this can be avoided depending on how the tests are written, so I won't address this right now.

[karfau]

I actually refactored some stuff so that those things doesn't happen anymore when using the helper methods.
And I finally did pushed my current state, so everybody interested can have a look:
karfau/promises-tests-as3

[karfau]

When I do understand it correctly, the last two specs in tests/2.2.2.js will not fail when only the last onFulfilled will get executed, as the done method is called there?
Is this by intention again, is there another place that verifies that all calls to then will get their fulfillment handlers called?
Or is the intention of the test to verify in the end that all three handlers should get executed?

[domenic]

And I finally did pushed my current state, so everybody interested can have a look:
karfau/promises-tests-as3

Really exciting! I'll be sure to link to this in the readme.

[domenic]

(Ooops, pressed the submit button earlier than intended.)

When I do understand it correctly, the last two specs in tests/2.2.2.js will not fail when only the last onFulfilled will get executed, as the done method is called there?

Yeah, that's true. Because they're not testing that onFulfilled is called at all---they're testing that it isn't called more than once. So the test would actually be OK if the asserts didn't run at all. But we had to put the done() somewhere. I suppose it might have been clearer to put it outside, in e.g. a setTimeout(..., 200).

Is this by intention again, is there another place that verifies that all calls to then will get their fulfillment handlers called?

Right, exactly. The 2.2.6 tests.

[karfau]

Is there actually a reason for that everything regarding rejection is called (on)rejected but for resolving there is onFulfilled vs. resolve?

[domenic]

Resolve and fulfill are very different operations. See "States and Fates".

[karfau]

👍

[karfau]

is there a special reason why you are calling resolved() with no parameters in 2.2.4 line 69 ?
(same in line 82, 83, 141, 142, 155)

[domenic]

It generates a fulfilled promise with fulfillment value undefined. I could have used dummy I suppose for clarity...

[karfau]

or leave dummy out in all the other cases where it is not relevant...