Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Artifacts bring admin monorepo #4203

Merged
merged 52 commits into from
Oct 30, 2023
Merged

Conversation

wild-endeavor
Copy link
Contributor

No description provided.

wild-endeavor and others added 30 commits June 1, 2023 15:05
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
… functions, update the source field

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Also clean up the logic around resolving parameter and literal map a bit more.
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
@codecov
Copy link

codecov bot commented Oct 11, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c9ce299) 75.92% compared to head (244d3b2) 60.03%.

❗ Current head 244d3b2 differs from pull request most recent head 23d5fc2. Consider uploading reports for the commit 23d5fc2 to get more accurate results

Additional details and impacted files
@@              Coverage Diff               @@
##           artifacts    #4203       +/-   ##
==============================================
- Coverage      75.92%   60.03%   -15.90%     
==============================================
  Files             18      387      +369     
  Lines           1458    26845    +25387     
==============================================
+ Hits            1107    16116    +15009     
- Misses           294     9231     +8937     
- Partials          57     1498     +1441     
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 387 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

wild-endeavor and others added 22 commits October 12, 2023 12:05
Signed-off-by: Yee Hing Tong <[email protected]>
artifact service.
Merge branch 'artifacts-add-svc' into artifacts-bring-admin-monorepo

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
…onfig option to v2 instead of a 'transform', add launch plan id to the cloud events, moved the database option for artifact service into its own application configuration instead of relying on a shared database because i think we should use a different postgres db name, drop the Omit(id) bit in the still non-working Create call, updated the cloud event rst a bit (#4256)

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
* Manually reverting 7df67de ([this branch](master...jeev/sandbox-redis)), not sure why readme isn't updating, will worry about it later.
* Remove the redis publisher.
* Remove values related to redis publisher.

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
@wild-endeavor wild-endeavor marked this pull request as ready for review October 30, 2023 01:47
@wild-endeavor wild-endeavor merged commit 6792f4e into artifacts Oct 30, 2023
31 of 41 checks passed
@wild-endeavor wild-endeavor deleted the artifacts-bring-admin-monorepo branch October 30, 2023 01:47
eapolinario added a commit that referenced this pull request Dec 27, 2023
* use-personal

* tidy

* update port to 30004

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bundle (#7)

* Remove private repos (#4078)

Signed-off-by: Yee Hing Tong <[email protected]>

* nit from monorepo merge

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bring in idl (#4204)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bring admin monorepo (#4203)

Main pr to bring in the boilerplate code for artifact service into the main pr.

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Artf/triggers (#4394)

This PR adds the trigger concept.

Other changes:
* Forgot to make the index on artifactkey unique.
* Fix to create artifact, it wasn't pulling back out the artifact key.
* Updated the Artifact service to handle the passing of artifacts created by the events processor to the trigger handling components. Currently, and this is potentially a design flaw, the events processor has no way of contacting the trigger component directly.

Signed-off-by: Yee Hing Tong <[email protected]>

* make trigger key index unique (#4410)

Signed-off-by: Yee Hing Tong <[email protected]>

* revert data proxy changes (#4411)

Signed-off-by: Yee Hing Tong <[email protected]>

* artf/ fix issue with trigger (#4412)

Signed-off-by: Yee Hing Tong <[email protected]>

* sandbox lite dockerfile (#4413)

Signed-off-by: Yee Hing Tong <[email protected]>

* More pgconn handling fixes (#4420)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/switch event (#4428)

Most of the changes here are generated proto changes. Actual code changes are:
IDL
* Remove the supplemental fields in the CloudEventTaskExecution object and move them to CloudEventNodeExecution object.
* Remove some fields that the artifact service ended up not using (parent_node_execution and scheduled_at)

in the cloudevent publisher, change the code filling in of the aforementioned supplemental information to happen for node execution events instead of task execution events.
* Remove the deleted fields.

On the event handling side, move the logic to the handling of the node event instead of the task event.
Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/lints (#4429)

* lint fixes
* rename sandbox_utils without underscore

Signed-off-by: Yee Hing Tong <[email protected]>

* look for other places where pgconn is cast and add handling (#4430)

Add additional handling of pgconn
* Change datacatalog logic to use flytestdlib function instead.
* Change flytestdlib constants to capitalize
* Add handling of the other type to Admin code directly

Signed-off-by: Yee Hing Tong <[email protected]>

* add artifacts branch to buf push (#4448)

Signed-off-by: squiishyy <[email protected]>

* add dummy comment (#4451)

Signed-off-by: squiishyy <[email protected]>

* add dummy comment again (#4453)

Signed-off-by: squiishyy <[email protected]>

* artf/updates to source (#4443)


Signed-off-by: Yee Hing Tong <[email protected]>

* add a feature gate (#4472)


Signed-off-by: Yee Hing Tong <[email protected]>

* fix bug in execution_manager with lpExpectedInputs, remove the standalone artifact client and move into idl clientset, update local config file, remove a deprecated function (#4473)

Signed-off-by: Yee Hing Tong <[email protected]>

* initial deletion of artifact code

Signed-off-by: Yee Hing Tong <[email protected]>

* running make manifests in docker/sandbox-bundled to update generated files

Signed-off-by: Yee Hing Tong <[email protected]>

* now compiles (#4481)

Signed-off-by: Yee Hing Tong <[email protected]>

* add a debug line

Signed-off-by: Yee Hing Tong <[email protected]>

* more debugging

Signed-off-by: Yee Hing Tong <[email protected]>

* clean up go mod and switch the order of the input extraction to check for uri first

Signed-off-by: Yee Hing Tong <[email protected]>

* change artifacts http mappings in protos

Signed-off-by: squiishyy <[email protected]>

* testing removal of go mod pin

Signed-off-by: Yee Hing Tong <[email protected]>

* update go mod some more

Signed-off-by: Yee Hing Tong <[email protected]>

* copilot go mod

Signed-off-by: Yee Hing Tong <[email protected]>

* change artifacts http mappings again (#4562)

Signed-off-by: squiishyy <[email protected]>

* panic and revert gorm log level change

Signed-off-by: Yee Hing Tong <[email protected]>

* revert local to 5

Signed-off-by: Yee Hing Tong <[email protected]>

* make helm

Signed-off-by: Yee Hing Tong <[email protected]>

* Reclaim space prior to running functional tests in single-binary

Signed-off-by: Eduardo Apolinario <[email protected]>

* Pin flytectl version to 0.8.0

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Pin flytectl version to 0.8.0"

This reverts commit cfa4745.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use flytagent v1.10.2

Signed-off-by: Eduardo Apolinario <[email protected]>

* Debug single-binary

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use an infinite loop instead of watch (term env var is not set)

Signed-off-by: Eduardo Apolinario <[email protected]>

* Get logs from single-binary pod

Signed-off-by: Eduardo Apolinario <[email protected]>

* Debug single-binary pod: use a different way of choosing the pod

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix flytectl version

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use port 5432 to connect to postgres

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Use port 5432 to connect to postgres"

This reverts commit 8ded67b.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Fix flytectl version"

This reverts commit 3968d40.

Signed-off-by: Eduardo Apolinario <[email protected]>

* fix database error

Signed-off-by: Yee Hing Tong <[email protected]>

* revert .github/workflows/single-binary.yml by pulling from master

Signed-off-by: Yee Hing Tong <[email protected]>

* Review feedback

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: squiishyy <[email protected]>
Co-authored-by: Joe Eschen <[email protected]>
Co-authored-by: squiishyy <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants