Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding basic gocron-gorm-lock functionality. #1

Merged
merged 8 commits into from
Nov 14, 2023
Merged

adding basic gocron-gorm-lock functionality. #1

merged 8 commits into from
Nov 14, 2023

Conversation

manuelarte
Copy link
Collaborator

@manuelarte manuelarte commented Aug 5, 2023

What does this do?

Which issue(s) does this PR fix/relate to?

Adding basic gocron-gorm lock functionality. It should be described in the Readme file.

Important topic to discuss:

  • It needs the cron_job_locks table to exists in the database because that's where the cron jobs are stored.
  • it is using a combination of job name + job execution timestamp (precision to miliseconds) to uniquely identify the running job, so it avoids running several times the same jobs.
  • In order to test the implementation, I needed to add a gorm driver, in this case I used postgres, so the driver is included in the go.mod, but I also tested it with another driver and it works, that's actually the magic of Gorm.

List any changes that modify/break current functionality

Have you included tests for your changes?

I added a test following the convention in gocron-redis-lock, I also added an extra test that checks the uniqueness of the job name + timestamp in the database.

Did you document any new/modified functionality?

  • Updated example_test.go
  • Updated README.md

Notes

Any feedback is appreciated.

@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@JohnRoesler
Copy link
Contributor

@manuelarte thanks for your desire to contribute to the open source! See my comments on the design below. Great work!

gormlock.go Outdated
Comment on lines 23 to 24
// This is a complicated thing, we are assuming that the minute precision is enough
ji := time.Now().Truncate(time.Millisecond).Format("2006-01-02 15:04:05.000")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only minute precision? Gocron supports seconds, miliseconds, etc. So locker should be able to handle jobs running sub-minute intervals

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's miliseconds precission, right? .Truncate(time.Millisecond) (I need to fix the stupid comment... sorry about that).

But apart from that, this is a tricky one, and maybe it's worth discussing with an example. In my case I am using the Postgress driver for gorm, in this case Postgres timestamp does not have nanoseconds precission (go-gorm/gorm#3232), so that's why I decided to put it in miliseconds, but I guess this is "driver" dependent. Maybe it's a good idea to allow the user to set it's precission? (and if not, use miliseconds?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think making it configurable. Defaulting to milliseconds seems reasonable though.

What happens if you pass in nanosecond precision to Postgress, will it truncate?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it configurable. Now you can pass how you can create the job identifier. I also added a test and updated the readme.

If you pass nanoseconds to postgres then you get weird behaviours (I think I got the information from here: lib/pq#227)

func TestEnableDistributedLocking_DifferentJob(t *testing.T) {
ctx := context.Background()
postgresContainer, err := testcontainerspostgres.RunContainer(ctx,
testcontainers.WithWaitStrategy(wait.ForLog("database system is ready to accept connections").
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an alias issue here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't reproduce this alias issue, @JohnRoesler is it possible that you tell me how you get it? do you run a command? or you get it in your IDE?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The golangci lint step reports the error https://github.com/go-co-op/gocron-gorm-lock/actions/runs/6825095998/job/18568477606

Let me see if I can see what the issue is locally

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it's because you dont' have a go.sum file. Run go mod tidy and it should solve the issue

Copy link
Collaborator Author

@manuelarte manuelarte Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran go mod tidy, and I also ran the command that is failing in the pipeline (golangci-lint run --out-format=github-actions), and I still don't get the error. I am sure I am missing something...

Copy link
Collaborator Author

@manuelarte manuelarte Nov 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall I commit the go.sum file that I get after running go mod tidy?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. Go expects a mod and sum file to be present

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go.sum pushed

@rfyiamcool
Copy link
Contributor

👍

@manuelarte
Copy link
Collaborator Author

I updated the PR

README.md Outdated Show resolved Hide resolved
manuelarte and others added 3 commits November 10, 2023 21:46
using suggestion

Co-authored-by: John Roesler <[email protected]>
@JohnRoesler
Copy link
Contributor

Looks like the test is failing

@manuelarte
Copy link
Collaborator Author

manuelarte commented Nov 12, 2023

Looks like the test is failing

yes. This is a problem I discovered some time ago. The miliseconds precision (the one I put by default), gives problems since the operation to lock and unlock sometimes takes quite some time. I decided to change the precision to seconds, and change the tests accordingly.

@manuelarte
Copy link
Collaborator Author

I forgot to update one part of one test

@manuelarte
Copy link
Collaborator Author

@JohnRoesler could you run the workflows again?

@manuelarte
Copy link
Collaborator Author

is something else missing?

@JohnRoesler JohnRoesler merged commit 4af7149 into go-co-op:main Nov 14, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants