Add regex support to only and except table filters

[mbezhanov]

Resolves #256

This pull request adds regex support to the only and except table filters.

bobgen.yaml now allows writing expressions such as:

mysql:
  only:
    "/^foo/":
    "/^bar/":
  except:
    "/baz$/":   

These are only made available for the PostgreSQL and MySQL drivers, as SQLite doesn't support regular expressions out of the box (it requires loading an extension).

The proposed regex delimiter syntax (enclosing the regular expressions with forward slashes) borrows its concept from languages such as JavaScript, PHP, and Perl. The idea is to keep the old IN/NOT IN logic intact while complementing it with additional regular expression matching logic whenever explicitly requested.

I've added a few additional test cases to ensure that all filtering combinations work as expected (the tradeoff here, though, is that the number of golden files committed to the repository has increased significantly).

I remain open to feedback and suggestions for further improvements.

[stephenafamo]

This is fantastic! Great work!!!

SQLite doesn't support regular expressions out of the box (it requires loading an extension).

The sqlite driver allows us to register functions. This should enable us add regexp support

func registerRegexpFunction() error {
	return sqlite.RegisterScalarFunction("regexp", 2, func(
		ctx *sqlite.FunctionContext,
		args []sqlDriver.Value,
	) (sqlDriver.Value, error) {
		if len(args) != 2 {
			return nil, fmt.Errorf("expected 2 arguments, got %d", len(args))
		}

		re, ok := args[0].(string)
		if !ok {
			return nil, fmt.Errorf("expected string, got %T", args[0])
		}

		s, ok := args[1].(string)
		if !ok {
			return nil, fmt.Errorf("expected string, got %T", args[1])
		}

		match, err := regexp.MatchString(re, s)
		if err != nil {
			return nil, err
		}

		return match, nil
	})
}

Can you add this so that this is supported across the 3 primary drivers?

[mbezhanov]

Thank you for providing the registerRegexpFunction() implementation; this was a real time saver!

I've added the requested changes to the SQLite driver implementation. However, the newly added tests uncovered a (presumably old) issue: the except filter for columns doesn't seem to be working (the columns are not being excluded).

Would you like this addressed in the current PR, or do you prefer a separate PR for this?

[stephenafamo]

the except filter for columns doesn't seem to be working

• Can you share more details about this? I am not aware of this issue. We can create a separate issue, but fix it in the same PR.
• If it can be fixed in the same PR, then we should fix it. Please remember to mention it in CHANGELOG.md.
• Remember to add (thanks @mbezhanov) to the changelog items. Need to give credit where it is due 😁

[mbezhanov]

Can you share more details about this? I am not aware of this issue.

Sure, basically I have the following table schema:

CREATE TABLE foo_bar (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    secret_col TEXT NOT NULL
);
CREATE TABLE foo_baz (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    secret_col TEXT NOT NULL
);

And I also have the following configuration:

sqlite:
  except:
    "*":
      - secret_col

I expect secret_col to be excluded from the generated models, but it isn't being excluded

---

I'll go ahead and fix it directly in this PR..

[mbezhanov]

All fixed and ready for review I believe 🙂