Skip to content

Commit

Permalink
Pull request #53: Get fixture fields with reflection
Browse files Browse the repository at this point in the history
Merge in AW/go-starter from aj/fixture-fields-with-reflection to master

* commit '7ef261e577f3c7b60125525e4494795a9c04ea06':
  Update changelog
  fix CVE-2022-41723 vulnerability
  Remove not needed test snapshot
  Fix linter
  Use GetFieldsImplementing with Upsertable
  Remove fixture dependency in struct tests
  Use generics
  fix trivy CVE-2022-41723 vulnerability
  get fixture insertable fields with reflection
  • Loading branch information
anjankow committed Apr 20, 2023
2 parents efa9f04 + 7ef261e commit d9979f3
Show file tree
Hide file tree
Showing 7 changed files with 329 additions and 34 deletions.
18 changes: 10 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@
- Major: Upgrade distroless app image from base-debian10 to base-debian11
- Major: Dockerfile is now build to support amd64 and arm64 architecture
- Improve speed of `make swagger` when dealing with many files in `/api` by generating to a docker volume instead of the host filesystem, rsyncing only to changes into `/internal/types`. Furthermore split our swagger type generation and validation into two separate make targets, that can run concurrently (requires `./docker-helper.sh --rebuild`).
- Note that `/app/api/tmp`, `/app/tmp` and `/app/bin` are now baked by proper docker volumes when using our `docker-compose.yml`/`./docker-helper.sh --up`. You **cannot** remove these directories directly inside the container (but its contents) and you can also no longer see its files on your host machine directly!
- Note that `/app/api/tmp`, `/app/tmp` and `/app/bin` are now baked by proper docker volumes when using our `docker-compose.yml`/`./docker-helper.sh --up`. You **cannot** remove these directories directly inside the container (but its contents) and you can also no longer see its files on your host machine directly!
- Fix `make check-gen-dirs` false positives hidden files.
- Allow to trace/benchmark `Makefile` targets execution by using a custom shell wrapper for make execution. See `SHELL` and `.SHELLFLAGS` within `Makefile` and the custom `rksh` script in the root working directory. Usage: `MAKE_TRACE_TIME=true make <target>`
- Minor: add `GetFieldsImplementing` to utils and use it to easier add new fixture fields.
- `go.mod` changes:
- Minor: [Bump github.com/BurntSushi/toml from 1.1.0 to 1.2.1](https://github.com/BurntSushi/toml/releases/tag/v1.2.1)
- Minor: [Bump github.com/gabriel-vasile/mimetype from 1.4.0 to 1.4.1](https://github.com/gabriel-vasile/mimetype/releases/tag/v1.4.1)
Expand All @@ -46,6 +47,7 @@
- Minor: [Bump golang.org/x/crypto from v0.0.0-20220411220226-7b82a4e95df4 to 0.3.0](https://cs.opensource.google/go/x/crypto)
- Minor: [Bump golang.org/x/sys from v0.0.0-20220412211240-33da011f77ad to 0.2.0](https://cs.opensource.google/go/x/sys)
- Minor: [Bump golang.org/x/text from 0.3.7 to 0.4.0](https://cs.opensource.google/go/x/text) (Fixing CVE-2022-32149)
- Minor: [Bump golang.org/x/net from 0.2.0 to 0.7.0](https://cs.opensource.google/go/x/net) (Fixing CVE-2022-41723)
- Minor: [Bump google.golang.org/api from 0.74.0 to 0.103.0](https://github.com/googleapis/google-api-go-client/compare/v0.80.0...v0.103.0)

## 2022-09-13
Expand Down Expand Up @@ -76,7 +78,7 @@
- `go.mod` changes:
- Major: [Bump `github.com/rubenv/sql-migrate` from v0.0.0-20210614095031-55d5740dbbcc to v1.1.1](https://github.com/rubenv/sql-migrate/compare/55d5740dbbccbaa4934009263b37ba52d837241f...v1.1.1) (though this should not lead to any major changes)
- Minor: [Bump github.com/volatiletech/sqlboiler/v4 from 4.6.0 to v4.9.2](https://github.com/volatiletech/sqlboiler/blob/v4.9.2/CHANGELOG.md#v492---2022-04-11) (your generated model might slightly change, minor changes).
- Note that v5 will prefer wrapping errors (e.g. `sql.ErrNoRows`) to retain the stack trace, thus it's about time for us to start to enforce proper `errors.Is` checks in our codebase (see above).
- Note that v5 will prefer wrapping errors (e.g. `sql.ErrNoRows`) to retain the stack trace, thus it's about time for us to start to enforce proper `errors.Is` checks in our codebase (see above).
- Minor: [#178: Bump github.com/labstack/echo/v4 from 4.6.1 to 4.7.2](https://github.com/allaboutapps/go-starter/pull/178) (support for HEAD method query params binding, minor changes).
- Minor: [#160: Bump github.com/rs/zerolog from 1.25.0 to 1.26.1](https://github.com/allaboutapps/go-starter/pull/160) (minor changes).
- Minor: [#179: Bump github.com/nicksnyder/go-i18n/v2 from 2.1.2 to 2.2.0](https://github.com/allaboutapps/go-starter/pull/179) (minor changes).
Expand All @@ -96,7 +98,7 @@
- This does not require a development container restart.
- We override the env within the app process through `config.DefaultServiceConfigFromEnv()`, so this does not mess with the actual container ENV.
- See `.env.local.sample` for further instructions to use this.
- Note that `.env.local` is **NEVER automatically** applied during **test runs**. If you really need that, use the specialized `test.DotEnvLoadLocalOrSkipTest` helper before loading up your server within that very test! This ensures that this test is automatically skipped if the `.env.local` file is no longer available.
- Note that `.env.local` is **NEVER automatically** applied during **test runs**. If you really need that, use the specialized `test.DotEnvLoadLocalOrSkipTest` helper before loading up your server within that very test! This ensures that this test is automatically skipped if the `.env.local` file is no longer available.
- VSCode windows closes now explicitly stop Docker containers via [`shutdownAction: "stopCompose"`](https://code.visualstudio.com/docs/remote/devcontainerjson-reference) within `.devcontainer.json`.
- Use `./docker-helper --halt` or other `docker` or `docker-compose` management commands to do this explicitly instead.
- Drone CI specific (minor): Fix multiline ENV variables were messing up our `.hostenv` for `docker run` command testing of the final image.
Expand All @@ -121,9 +123,9 @@

- **BREAKING** Username format change in auth handlers
- Added the `util.ToUsernameFormat` helper function, which will **lowercase** and **trim whitespaces**. We use it to format usernames in the login, register, and forgot-password handlers.
- This prevents user duplication (e.g. two accounts registered with the same email address with different casing) and
- cases where users would inadvertently register with specific casing or a trailing whitespace after their username, and subsequently struggle to log into their account.
- **This effectively locks existing users whose username contains uppercase characters and/or whitespaces out of their accounts.**
- This prevents user duplication (e.g. two accounts registered with the same email address with different casing) and
- cases where users would inadvertently register with specific casing or a trailing whitespace after their username, and subsequently struggle to log into their account.
- **This effectively locks existing users whose username contains uppercase characters and/or whitespaces out of their accounts.**
- Before rolling out this change, check whether any existing users are affected and migrate their usernames to a format that is compatible with this change.
- Be aware that this may cause conflicts in regard to the uniqueness constraint of usernames and therefore need to be resolved manually, which is why we are not including a database migration to automatically migrate existing usernames to the new format.
- For more information and a possible manual database migration flow please see this special WIKI page: https://github.com/allaboutapps/go-starter/wiki/2022-02-28
Expand All @@ -133,7 +135,7 @@
### Changed

- Changed order of make targets in the `make swagger` pipeline. `make swagger-lint-ref-siblings` will now run after `make swagger-concat`, always linting the current version of our swagger file. This helps avoid errors regarding an invalid `swagger.yml` when resolving merge conflicts as those are often resolved by running `make swagger` and generating a fresh `swagger.yml`.

## 2022-02-02

### Changed
Expand Down Expand Up @@ -225,7 +227,7 @@

### Changed

- **Hotfix**: We will pin the `Dockerfile` development and builder stage to `golang:1.16.7-buster` (+ `-buster`) for now, as currently the [new debian bullseye release within the go official docker images](https://github.com/docker-library/golang/commit/48a7371ed6055a97a10adb0b75756192ad5f1c97) breaks some tooling. The upgrade to debian bullseye and Go 1.17 will happen ~simultaneously~ **separately** within go-starter in the following weeks.
- **Hotfix**: We will pin the `Dockerfile` development and builder stage to `golang:1.16.7-buster` (+ `-buster`) for now, as currently the [new debian bullseye release within the go official docker images](https://github.com/docker-library/golang/commit/48a7371ed6055a97a10adb0b75756192ad5f1c97) breaks some tooling. The upgrade to debian bullseye and Go 1.17 will happen ~simultaneously~ **separately** within go-starter in the following weeks.

## 2021-08-16

Expand Down
6 changes: 3 additions & 3 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ require (
github.com/volatiletech/sqlboiler/v4 v4.13.0
github.com/volatiletech/strmangle v0.0.4
golang.org/x/crypto v0.3.0
golang.org/x/sys v0.2.0
golang.org/x/text v0.4.0
golang.org/x/sys v0.5.0
golang.org/x/text v0.7.0
google.golang.org/api v0.103.0
)

Expand Down Expand Up @@ -107,7 +107,7 @@ require (
github.com/volatiletech/inflect v0.0.1 // indirect
go.mongodb.org/mongo-driver v1.11.0 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/net v0.2.0 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/oauth2 v0.2.0 // indirect
golang.org/x/time v0.2.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
Expand Down
11 changes: 7 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -856,8 +856,9 @@ golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0 h1:sZfSu1wtKLGlWI4ZZayP0ck9Y73K1ynO6gqzTdBVdPU=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
Expand Down Expand Up @@ -971,12 +972,13 @@ golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0 h1:ljd4t30dBnAvMZaQCevtY0xLLD0A+bRZXbgLMLU1F/A=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0 h1:z85xZCsEl7bi/KwbNADeBYoOP0++7W1ipu+aGnpwzRM=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
Expand All @@ -986,8 +988,9 @@ golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0 h1:BrVqGRd7+k1DiOgtnFvAkoQEWQvBc25ouMJM6429SFg=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
Expand Down
12 changes: 11 additions & 1 deletion internal/data/fixtures.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ package data

import (
"context"
"fmt"

"allaboutapps.dev/aw/go-starter/internal/util"
"github.com/volatiletech/sqlboiler/v4/boil"
)

Expand All @@ -14,12 +16,20 @@ type Upsertable interface {
Upsert(ctx context.Context, exec boil.ContextExecutor, updateOnConflict bool, conflictColumns []string, updateColumns, insertColumns boil.Columns) error
}

// Mind the declaration order! The fields get upserted exactly in the order they are declared.
type FixtureMap struct{}

func Fixtures() FixtureMap {
return FixtureMap{}
}

func Upserts() []Upsertable {
return []Upsertable{}
fix := Fixtures()
upsertableIfc := (*Upsertable)(nil)
upserts, err := util.GetFieldsImplementing(&fix, upsertableIfc)
if err != nil {
panic(fmt.Errorf("failed to get upsertable fixture fields: %w", err))
}

return upserts
}
29 changes: 11 additions & 18 deletions internal/test/fixtures.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ package test

import (
"context"
"fmt"
"time"

"allaboutapps.dev/aw/go-starter/internal/models"
"allaboutapps.dev/aw/go-starter/internal/util"
"github.com/volatiletech/null/v8"
"github.com/volatiletech/sqlboiler/v4/boil"
)
Expand All @@ -19,7 +21,8 @@ type Insertable interface {
Insert(ctx context.Context, exec boil.ContextExecutor, columns boil.Columns) error
}

// FixtureMap represents the main definition which fixtures are available though Fixtures()
// The main definition which fixtures are available through Fixtures().
// Mind the declaration order! The fields get inserted exactly in the order they are declared.
type FixtureMap struct {
User1 *models.User
User1AppUserProfile *models.AppUserProfile
Expand Down Expand Up @@ -135,22 +138,12 @@ func Fixtures() FixtureMap {
// Inserts defines the order in which the fixtures will be inserted
// into the test database
func Inserts() []Insertable {
fixtures := Fixtures()

return []Insertable{
fixtures.User1,
fixtures.User1AppUserProfile,
fixtures.User1AccessToken1,
fixtures.User1RefreshToken1,
fixtures.User2,
fixtures.User2AppUserProfile,
fixtures.User2AccessToken1,
fixtures.User2RefreshToken1,
fixtures.UserDeactivated,
fixtures.UserDeactivatedAppUserProfile,
fixtures.UserDeactivatedAccessToken1,
fixtures.UserDeactivatedRefreshToken1,
fixtures.User1PushToken,
fixtures.User1PushTokenAPN,
fix := Fixtures()
insertableIfc := (*Insertable)(nil)
inserts, err := util.GetFieldsImplementing(&fix, insertableIfc)
if err != nil {
panic(fmt.Errorf("failed to get insertable fixture fields: %w", err))
}

return inserts
}
83 changes: 83 additions & 0 deletions internal/util/struct.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package util

import (
"errors"
"reflect"
)

// GetFieldsImplementing returns all fields of a struct implementing a certain interface.
// Returned fields are pointers to a type or interface objects.
//
// Parameter structPtr must be a pointer to a struct.
// Parameter interfaceObject must be given as a pointer to an interface,
// for example (*Insertable)(nil), where Insertable is an interface name.
func GetFieldsImplementing[T any](structPtr interface{}, interfaceObject *T) ([]T, error) {

// Verify if structPtr is a pointer to a struct
inputParamStructType := reflect.TypeOf(structPtr)
if inputParamStructType == nil ||
inputParamStructType.Kind() != reflect.Ptr ||
inputParamStructType.Elem().Kind() != reflect.Struct {
return nil, errors.New("invalid input structPtr param: should be a pointer to a struct")
}

inputParamIfcType := reflect.TypeOf(interfaceObject)
// Verify if interfaceObject is a pointer to an interface
if inputParamIfcType == nil ||
inputParamIfcType.Kind() != reflect.Ptr ||
inputParamIfcType.Elem().Kind() != reflect.Interface {

return nil, errors.New("invalid input interfaceObject param: should be a pointer to an interface")
}

// We need the type, not the pointer to it.
// By using Elem() we can get the value pointed by the pointer.
interfaceType := inputParamIfcType.Elem()
structType := inputParamStructType.Elem()

structValue := reflect.ValueOf(structPtr).Elem()

retFields := make([]T, 0)

// Getting the VisibleFields returns all public fields in the struct
for i, field := range reflect.VisibleFields(structType) {

// Check if the field can be exported.
// Interface() can be called only on exportable fields.
if !field.IsExported() {
continue
}

fieldValue := structValue.Field(i)

// Depending on the field type, different checks apply.
switch field.Type.Kind() {

case reflect.Pointer:

// Let's check if it implements the interface.
if field.Type.Implements(interfaceType) {
// Great, we can add it to the return slice
retFields = append(retFields, fieldValue.Interface().(T))
}

case reflect.Interface:
// If it's an interface, make sure it's not nil.
if fieldValue.IsNil() {
continue
}

// Now we can check if it's the same interface.
if field.Type.Implements(interfaceType) {
// Great, we can add it to the return slice
retFields = append(retFields, fieldValue.Interface().(T))
}

default:
// We can skip any other cases.
continue
}
}

return retFields, nil
}
Loading

0 comments on commit d9979f3

Please sign in to comment.