assert package API change + random.Unique helper

AnyOf

Align the AnyOf assertion helper with other existing assertion helper functions in the assert package.
The new approach should be idiomatic with the rest of the assert package and more convenient.

assert.AnyOf(tb, func(a *assert.A) {
	a.Case(func(it assert.It) {
		it.Must.True(outcome)
	})

	a.Case(func(it assert.It) {
		it.Must.False(outcome)
	})
})

random.Unique

Introduce a random.Unique utility function to simplify the generation of unique random values in tests,
reducing the risk of flaky test results where an assertion expects district values from two randomly made input

rnd := random.New(random.CryptoSeed{})
v1 := rnd.Int()
v2 := random.Unique(rnd.Int, v1)
v3 := random.Unique(rnd.Int, v1, v2)

Breaking changes

assert.AnyOf -> assert.A

To fix your codebase, run the following shell script (GNU coreutils)

find . -type f -name "*.go" -exec sed -i'' -e 's/assert\.AnyOf)/assert.A)/g' {} \; 

eventually assertion refact + new import path introduction

Update: New Import Path go.llib.dev/testcase

Transition to our updated import path go.llib.dev/testcase.
Previously, it was through github.com, but now it's streamlined to go.llib.dev/testcase. This change allows for more adaptability regarding where we store our source code, be it on my personal GitHub account or within a GitHub organization.

Modification Alert: Changes to assert.Eventually

The assertion retry helper has undergone a name change:

assert.Eventually -> assert.Retry
assert.EventuallyWithin -> assert.MakeRetry

The function assert.Eventually has now become a top-tier test assisting function, enabling easier creation of 'eventually' assertions similar to our other helper functions.

Critical Update to testcase/assert Library: Improved Assertion Function Handling!

Hello, testing enthusiasts! 🚀

We've got some important news to share based on the invaluable feedback from our community. There's been a common misstep in how some of our users have been utilizing assertion functions with testcase, and we've taken steps to address it.

The Issue:
Several of you have been inadvertently using assertion functions meant for a single parameter with two parameters. A typical example we've seen:

assert.Error(t, expectedErr, actualErr, "expectation message")

This isn't ideal because the third argument for these functions isn't for comparison but is a variadic assertion message argument. In such cases, expectedErr gets checked as an error, and actualErr is interpreted as a failure assertion message.

Our Solution:
We've updated the function's signature to mitigate this and ensure clarity in function usage. It now mandates a string-based type for assertion failure messages. This change ensures that the compiler will flag any type mismatches, making it much more challenging to misuse the function.

We're confident that this update will streamline your testing experience and reduce inadvertent errors. We recommend updating your testcase/assert library to the latest version to leverage this enhancement.

Your feedback drives our improvements, and we're grateful for your continued support and insights. Please don't hesitate to reach out with any questions, concerns, or further feedback.

Happy coding, and thank you for being an integral part of our community! 🚀

v0.139.2

• Random Interface Values and Subtypes: The 'random' package was updated to allow empty interface values and subtypes.
• Assertion Improvements: The behaviour of 'assert.NotEmpty' was updated to behave the same as 'assert.NotNil'. The 'assert.NotNil' function was also updated only to check if the value is not nil, avoiding potential race conditions. A new visitor pattern was introduced to support method-based comparison using the 'assert' package, allowing custom equality checkers to be registered.
• Clock Time Format: The clock was updated to return time in local format.
• Spec Based Suites: The 'testcase.AsSuite' spec option was introduced, allowing you to provide a name to enhance convenience while using a Spec as a testing suite. Support was also added for the OpenSuite interface with Spec to enable non-testcase specific suite generation.
• Pretty Print and Diff: The 'pp.Diff' function was updated to behave like 'pp.PP' for convenience.
• Random Contract Generation: A new feature was introduced to generate coherent contact details randomly.
• Assertion Helper: The 'assert.OneOf' assertion helper was added.
• Test Ordering: Test ordering was temporarily removed to comply with changes in go1.20.
• Fault Injection: Support was added for manual fault injection.
• Random Value Generation: Improvements were made to generating random values, including the ability to generate deterministic random UUIDs in tests.
• Variable Binding: Improvements were made to 'Var.Bind', including preventing it from doing anything when the variable is already bound to the context.
• Error Feedback: Improved error feedback was provided for a potentially reused 'testcase#Var.ID'.
• Time Manipulation: The ability to unfreeze time with 'timecop' was added.

clock, timecop, table driven tests

• add the ability to manipulate observed time through the clock and timecop package
• add experimental spec helpers to the httpspec
• add support to simulate random io reads with random.Random
• add the ability to pretty print any object, diff them in GNU side-by-side Diff style
• support accessing a testcase.Var's Super value when you define it in a sub context
  • this allows you to wrap entities with Spies and Stubs to do various things, such as fault injection.
• testcase.LetValue accepts more value as long they are safe to use due to pass-by-value.
• support table tests with table test helper.

Happy testing!

v0.94.0

upgrade testcase to the latest

• a new fault injection framework
  • allows injecting errors into a specific package/receiver/function's context.Err call
  • includes a fault injection middleware/round-tripper that allows propagating faults in distributed systems.
  • Added the ability to propagate injected faults in the round-tripper
  • allows the ability to inject errors through the HTTP API in a controlled manner
  • has a Global Enable Disable toggle to exclude all possibilities where faults injection is are not expected on an environment.
    • default is off

defer faultinject.Enable()()

ctx := context.Background()

// all fault field is optional.
// in case left as zero value,
// it will match every caller context,
// and returns on the first .Err() / .Value(faultinject.Fault{})
ctx = faultinject.Inject(ctx, faultinject.CallerFault{
	Package:  "targetpkg",
	Receiver: "*myreceiver",
	Function: "myfunction",
}, errors.New("boom"))

// from and after call stack reached: targetpkg.(*myreceiver).myfunction
if err := ctx.Err(); err != nil {
	fmt.Println(err) // in the position defined by the Fault, it will yield an error
}

• added support for a deterministic random generation as io.Reader with random.Random

rnd := random.New(rand.NewSource(time.Now().Unix()))

p := make([]byte, 42)
n, err := rnd.Read(p)

_, _ = n, err

• added support for deterministic random error generation

rnd := random.New(rand.NewSource(time.Now().Unix()))
err := rnd.Error()
_ = err

• nil is an acceptable constant value for the testcase.LetValue
• more examples in the pkg documentation
  • https://pkg.go.dev/github.com/adamluzsi/testcase#pkg-examples
  • https://pkg.go.dev/github.com/adamluzsi/testcase/random#pkg-examples
  • https://pkg.go.dev/github.com/adamluzsi/testcase/faultinject#pkg-examples
• changed signature to accept receiver in the Before hook of a testcase.Var,
  • This allows the use of a global testcase.Var with a dynamic Before hook arrangement without the need to use the init() function.

var v = testcase.Var[int]{
	ID:   "myvar",
	Init: func(t *testcase.T) int { return 42 },
	Before: func(t *testcase.T, v testcase.Var[int]) {
		t.Logf(`I'm from the Var.Before block, and the value: %#v`, v.Get(t))
	},
}

• [EXPERIMENT] deprecate Around, AroundAll, After, AfterAll to trim down on the testcase's API
  • They can be expressed using a Before/BeforeAll with T.Defer or T.Cleanup
• added support for sandboxing function blocks
  • This allows testing test helpers easily without letting your test exit with runtime.Goexit on testing.TB.FailNow

var tb testing.TB = &testcase.StubTB{}
outcome := testcase.Sandbox(func() {
	// some test helper function calls fatal, which cause runtime.Goexit after marking the test failed.
	tb.FailNow()
})

fmt.Println("The sandbox run has finished without an issue", outcome.OK)
fmt.Println("runtime.Goexit was called:", outcome.Goexit)
fmt.Println("panic value:", outcome.PanicValue)

• added support for assert.NoError

var tb testing.TB
assert.NoError(tb, nil)                // passes
assert.NoError(tb, errors.New("boom")) // fails

• improved the usability of StubTB to make it easier to assert the log content

stub := &testcase.StubTB{}
stub.Log("hello", "world")
fmt.Println(stub.Logs.String())

• added commonly used charsets to make random string generation more convenient for a given charset

rnd := random.New(rand.NewSource(time.Now().Unix()))
rnd.StringNWithCharset(42, random.Charset())
rnd.StringNWithCharset(42, random.CharsetASCII())
rnd.StringNWithCharset(42, random.CharsetAlpha())
rnd.StringNWithCharset(42, "ABC")

• added alias for random.StringWithCharset to make it easier to use

rnd := random.New(rand.NewSource(time.Now().Unix()))
rnd.StringNC(42, random.Charset())

• extend assert.ErrorIs to check equality even with wrapped values
• move Eventually async test helper into the assert package
  • backward compatibility is ensured for awhile
• add more safety guards to prevent variables without ID
• added http.Handler middleware contract to httpspec
  • if you write an http.Handler-based middleware ensures that the minimum expectations are there.

myHandlerWithFaultInjectionMiddleware := fihttp.Handler{
	Next:        myHandler,
	ServiceName: "our-service-name",
	FaultsMapping: fihttp.FaultsMapping{
		"mapped-fault-name": func(ctx context.Context) context.Context {
			return faultinject.Inject(ctx, FaultTag{}, errors.New("boom"))
		},
	},
}

• many small improvements

assert Package level assertion functions

Add package level assertion functions to testcase/assert package
This will make the assert package align with other popular assertion packages in terms of conventions.

Examples:

• assert.Contain
• assert.ContainExactly
• assert.Empty
• assert.Equal
• assert.ErrorIs
• assert.False
• assert.True
• assert.Nil
• assert.NotContain
• assert.NotEmpty
• assert.NotEqual
• assert.NotNil
• assert.NotPanic
• assert.NotPanic
• assert.Panic

add testcase.T.SkipUntil

Add testcase.T.SkipUntil that is equivalent to Log followed by SkipNow if the test is executing prior to the given deadline time.
SkipUntil is useful when you need to skip something temporarily, but you don't trust your memory enough to return to it on your own.

add support for IsEqual function

assert.Asserter.Equal and other assertions that test equality will respect the .IsEqual method on the value.
When it does a comparison and .IsEqual is defined, it will use that instead of deep equality checking.

If a value has unexported values or a stateful field that requires a delicate way to compare, through this method, it becomes possible.

example code for .IsEqual

http.RoundTripper middleware contract

httpspec is extended with a RoundTripperMiddleware contract.

You can verify common scenarios with your round-tripper middleware using the shared spec.
This shared contract should allow you to focus on the round-trippers business logic rather than the generic middleware behaviour.