Bump crossplane runtime version to v0.13.0 #64

zzxwill · 2021-03-18T12:24:16Z

use common extracting credentials function CommonCredentialExtractor
use base64 string as the secret value for Alibaba provider config
bump version of providerconfi from v1alpha1 to v1alpha2
update make demo

Signed-off-by: Zheng Xi Zhou [email protected]

Description of your changes

Fixes https://github.com/crossplane/provider-alibaba/pull/58/files#r595574127

I have:

Read and followed Crossplane's contribution process.
Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

Apply a `ProviderConfig` to provide Alibaba cloud credentials.

export ALICLOUD_ACCESS_KEY=xxx
export ALICLOUD_SECRET_KEY=yyy
export ALICLOUD_REGION=zzz

sh hack/demo/prepare-alibaba-credentials.sh
kubectl apply -f examples/provider.yaml

Test provisioning and destroying managed resources

As this PR lasts a long time, almost all supported cloud products RDS, OSS, Redis, SLB, SLS are related. I test each managed resource in each product. Let's take OSS as an example.

Successfully create a bucket.

➜  examples git:(runtime-upgrade) k apply -f oss/oss.yaml
bucket.oss.alibaba.crossplane.io/example-oss-bucket created

➜  examples git:(runtime-upgrade) k get bucket --watch
NAME                 READY   SYNCED   WARNING   AGE
example-oss-bucket                              0s
example-oss-bucket                              0s
example-oss-bucket                              0s
example-oss-bucket                              1s
example-oss-bucket   False   True               2s
example-oss-bucket   True    True               2s

ACL is not right.

diff --git a/examples/oss/oss.yaml b/examples/oss/oss.yaml
index 8ba35b0..5503dc9 100644
--- a/examples/oss/oss.yaml
+++ b/examples/oss/oss.yaml
@@ -3,7 +3,7 @@ kind: Bucket
 metadata:
   name: example-oss-bucket
 spec:
-  acl: private
+  acl: ppp
   storageClass: Standard
   dataRedundancyType: LRS
   writeConnectionSecretToRef:

➜  examples git:(runtime-upgrade) ✗ k apply -f oss/oss.yaml
bucket.oss.alibaba.crossplane.io/example-oss-bucket created

➜  examples git:(runtime-upgrade) ✗ k get bucket --watch
NAME                 READY   SYNCED   WARNING   AGE
example-oss-bucket                              0s
example-oss-bucket                              1s
example-oss-bucket                              2s
example-oss-bucket                              4s
example-oss-bucket   False   False              5s


➜  examples git:(runtime-upgrade) ✗ k describe bucket example-oss-bucket
Name:         example-oss-bucket
Namespace:
Labels:       <none>
Annotations:  crossplane.io/external-name: example-oss-bucket
API Version:  oss.alibaba.crossplane.io/v1alpha1
Kind:         Bucket
Metadata:
  Creation Timestamp:  2021-07-20T06:23:50Z
  Finalizers:
    finalizer.managedresource.crossplane.io
  Generation:  2
  Managed Fields:
    API Version:  oss.alibaba.crossplane.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:acl:
        f:dataRedundancyType:
        f:storageClass:
        f:writeConnectionSecretToRef:
          .:
          f:name:
          f:namespace:
    Manager:      k
    Operation:    Update
    Time:         2021-07-20T06:23:50Z
    API Version:  oss.alibaba.crossplane.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:crossplane.io/external-name:
        f:finalizers:
          .:
          v:"finalizer.managedresource.crossplane.io":
      f:spec:
        f:providerConfigRef:
          .:
          f:name:
      f:status:
        .:
        f:atProvider:
        f:conditions:
    Manager:         ___go_build_main_go
    Operation:       Update
    Time:            2021-07-20T06:23:55Z
  Resource Version:  507759
  UID:               d3e15d25-6c8f-41e3-a197-95f66059ee78
Spec:
  Acl:                   ppp
  Data Redundancy Type:  LRS
  Provider Config Ref:
    Name:         default
  Storage Class:  Standard
  Write Connection Secret To Ref:
    Name:       example-oss
    Namespace:  default
Status:
  At Provider:
  Conditions:
    Last Transition Time:  2021-07-20T06:23:54Z
    Reason:                Creating
    Status:                False
    Type:                  Ready
    Last Transition Time:  2021-07-20T06:23:54Z
    Message:               create failed: failed to create OSS bucket: bucket ACL ppp is invalid. The ACL could only be public-read-write, public-read, and private
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                        Age                From                                      Message
  ----     ------                        ----               ----                                      -------
  Warning  CannotCreateExternalResource  0s (x14 over 52s)  managed/bucket.oss.alibaba.crossplane.io  failed to create OSS bucket: bucket ACL ppp is invalid. The ACL could only be public-read-write, public-read, and private

update the loss bucket (not allowed)

➜  examples git:(runtime-upgrade) ✗ k apply -f oss/oss.yaml
bucket.oss.alibaba.crossplane.io/example-oss-bucket configured

➜  examples git:(runtime-upgrade) k get bucket --watch
NAME                 READY   SYNCED   WARNING   AGE
example-oss-bucket                              0s
example-oss-bucket                              0s
example-oss-bucket                              1s
example-oss-bucket                              1s
example-oss-bucket   False   True               2s
example-oss-bucket   True    True               3s
example-oss-bucket   True    True               75s


diff --git a/examples/oss/oss.yaml b/examples/oss/oss.yaml
index 8ba35b0..e58fe87 100644
--- a/examples/oss/oss.yaml
+++ b/examples/oss/oss.yaml
@@ -4,7 +4,7 @@ metadata:
   name: example-oss-bucket
 spec:
   acl: private
-  storageClass: Standard
+  storageClass: Archive
   dataRedundancyType: LRS
   writeConnectionSecretToRef:
     name: example-oss

➜  examples git:(runtime-upgrade) ✗ k apply -f oss/oss.yaml
bucket.oss.alibaba.crossplane.io/example-oss-bucket configured


➜  examples git:(runtime-upgrade) ✗ k get bucket --watch
NAME                 READY   SYNCED   WARNING                                                            AGE
example-oss-bucket   True    True     [Warning] StorageClass is not allowed to update after creation;    2m9s```

pkg/controller/database/rdsinstance.go

negz

A few nitpicks, but otherwise this looks good to me.

apis/v1alpha2/types.go

pkg/controller/database/rdsinstance.go

negz

This looks good to me except for the introduction of testify for testing, which I would prefer to avoid.

pkg/clients/rds/rds_test.go

zzxwill · 2021-04-25T09:14:53Z

Hi @negz @hasheddan, please help review this PR again.
@xiqiu618 and I are hoping to power these two managed resource Redis (#68) and OSS bucket (#59) by Crossplane new runtime:)

hasheddan

This looks very close, nice work @zzxwill! A few small comments below.

apis/v1alpha2/types.go

pkg/clients/rds/rds.go

pkg/controller/database/rdsinstance.go

ulucinar

Thank you @zzxwill for the PR.
I will continue & finalize the review later in the day.

pkg/util/provider.go

apis/v1alpha2/types.go

examples/provider.yaml

apis/v1alpha2/types.go

hack/demo/prepare-alibaba-credentials.sh

pkg/clients/rds/rds_test.go

ulucinar

Hi @zzxwill,
Thank you for the PR. This concludes my initial review of this PR.

pkg/controller/oss/oss_controller.go

pkg/controller/sls/project_controller.go

pkg/controller/oss/oss_controller.go

pkg/controller/sls/project_controller.go

ulucinar

Thank you @zzxwill for the PR! This looks good to me apart from a few nits.

pkg/util/client.go

apis/v1beta1/types.go

- use common extracting credentials function `CommonCredentialExtractor` - use base64 string as the secret value for Alibaba provider config - bump version of providerconfi from v1alpha1 to v1alpha2 - update `make demo` Signed-off-by: Zheng Xi Zhou <[email protected]>

Signed-off-by: Zheng Xi Zhou <[email protected]>

- added kubebuilder catagories field - removed check on cr.Spec.ProviderConfigReference Signed-off-by: Zheng Xi Zhou <[email protected]>

Signed-off-by: Zheng Xi Zhou <[email protected]>

Due to the upgrade of runtime, I deleted the manually Creating/Deleting status setting for SLS project after merge from master Signed-off-by: Zheng Xi Zhou <[email protected]>

I reverted some TODO and comments from master when upgrading crossplane runtime by abandoning Provider object and copy latest practive from provider-template Signed-off-by: Zheng Xi Zhou <[email protected]>

Signed-off-by: Zheng Xi Zhou <[email protected]>

Add a common PrepareClient to prepare all information to establish a SDK client Signed-off-by: Zheng Xi Zhou <[email protected]>

Signed-off-by: Zheng Xi Zhou <[email protected]>

co-authored-by: Alper Ulucinar <[email protected]> Signed-off-by: Zheng Xi Zhou <[email protected]>

Co-authored-by: Alper Ulucinar <[email protected]> Signed-off-by: Zheng Xi Zhou <[email protected]>

Signed-off-by: Zheng Xi Zhou <[email protected]>

wonderflow · 2021-07-07T09:41:37Z

@negz Hi Nic, can you help approve this PR. Let's merge it.

negz · 2021-07-14T21:00:56Z

Hey folks! I'm happy to merge this, but I'd love to see the "How has this been tested" section filled out with some more detail. I think it was misinterpreted as "Was this tested" because it currently just says "Yes". :)

zzxwill · 2021-07-20T06:34:40Z

@negz It's done.

negz · 2021-07-22T00:27:46Z

@zzxwill Thank you for the very thorough testing info!

negz · 2021-07-22T00:34:34Z

Hrm I think merging this PR broke the main branch - see https://github.com/crossplane/provider-alibaba/actions/runs/1054471865. Did you rebase before running your tests?

https://github.com/crossplane/provider-alibaba/blob/8131deba22c8dd853a2c0311c0803de95de9ae0b/pkg/controller/sls/machineGroup_controller.go#L35

It looks like the problem is that the SLS controller/sls package is trying to import the v1alpha1 ProviderConfig type, that no longer exists.

By upgrading the version of ProviderConfig to v1beta1 can fix the build issue and simplify the process of managed resource connection. Fixed the issue mentioned in crossplane-contrib#64 (comment)

zzxwill · 2021-07-22T07:13:46Z

@negz SLS MachineGroup and MachineGroupBinding are the last two managed resources I supported. In the PR, there should be conflicts. And, indeed, these two are missed out. I have fixed it in #95

By upgrading the version of ProviderConfig to v1beta1 can fix the build issue and simplify the process of managed resource connection. Fixed the issue mentioned in crossplane-contrib#64 (comment) Signed-off-by: zzxwill <[email protected]>

zzxwill mentioned this pull request Mar 18, 2021

Implement Alibaba cloud SLS project #58

Merged

2 tasks

zzxwill commented Mar 18, 2021

View reviewed changes

pkg/controller/database/rdsinstance.go Outdated Show resolved Hide resolved

hasheddan reviewed Mar 18, 2021

View reviewed changes

pkg/controller/database/rdsinstance.go Outdated Show resolved Hide resolved

pkg/controller/database/rdsinstance.go Show resolved Hide resolved

negz approved these changes Mar 23, 2021

View reviewed changes

apis/v1alpha2/types.go Outdated Show resolved Hide resolved

apis/v1alpha2/types.go Outdated Show resolved Hide resolved

apis/v1alpha2/types.go Outdated Show resolved Hide resolved

pkg/controller/database/rdsinstance.go Outdated Show resolved Hide resolved

zzxwill mentioned this pull request Apr 6, 2021

Add kubebuilder catagories for ProviderConfig and MyType crossplane/provider-template#27

Merged

negz requested changes Apr 6, 2021

View reviewed changes

pkg/clients/rds/rds_test.go Outdated Show resolved Hide resolved

zzxwill force-pushed the runtime-upgrade branch from 57748b5 to cd160f9 Compare April 7, 2021 03:22

zzxwill requested a review from negz April 27, 2021 09:08

zzxwill force-pushed the runtime-upgrade branch 3 times, most recently from d122e9c to 5b2932f Compare June 21, 2021 06:33

wonderflow approved these changes Jun 21, 2021

View reviewed changes

hasheddan mentioned this pull request Jun 22, 2021

Support SLS logstore #77

Merged

2 tasks

zzxwill force-pushed the runtime-upgrade branch from 5b2932f to 592c395 Compare June 23, 2021 03:19

zzxwill mentioned this pull request Jun 23, 2021

Add e2e tests #78

Open

hasheddan reviewed Jun 23, 2021

View reviewed changes

apis/v1alpha2/types.go Outdated Show resolved Hide resolved

pkg/clients/rds/rds.go Outdated Show resolved Hide resolved

pkg/controller/database/rdsinstance.go Outdated Show resolved Hide resolved

ulucinar reviewed Jun 24, 2021

View reviewed changes

pkg/controller/oss/oss_controller.go Outdated Show resolved Hide resolved

pkg/controller/sls/project_controller.go Outdated Show resolved Hide resolved

zzxwill force-pushed the runtime-upgrade branch 4 times, most recently from 0ecf797 to 6b18a27 Compare June 28, 2021 08:42

ulucinar reviewed Jul 1, 2021

View reviewed changes

pkg/controller/oss/oss_controller.go Outdated Show resolved Hide resolved

pkg/controller/sls/project_controller.go Outdated Show resolved Hide resolved

ulucinar approved these changes Jul 1, 2021

View reviewed changes

pkg/util/client.go Show resolved Hide resolved

ulucinar approved these changes Jul 1, 2021

View reviewed changes

apis/v1beta1/types.go Outdated Show resolved Hide resolved

zzxwill force-pushed the runtime-upgrade branch from 55dd5f3 to 732f5a3 Compare July 2, 2021 03:06

zzxwill added 3 commits July 6, 2021 11:13

Add unit-test

8819304

Signed-off-by: Zheng Xi Zhou <[email protected]>

Addressed comments

2760cc9

- added kubebuilder catagories field - removed check on cr.Spec.ProviderConfigReference Signed-off-by: Zheng Xi Zhou <[email protected]>

zzxwill and others added 17 commits July 6, 2021 11:13

Added more unit-tests to meet the coverage requirements

381dd61

Signed-off-by: Zheng Xi Zhou <[email protected]>

Convert all unit-tests to table-driven style

1cc7d9f

Signed-off-by: Zheng Xi Zhou <[email protected]>

Delete manually Creating/Deleting status setting

4b6f6c5

Due to the upgrade of runtime, I deleted the manually Creating/Deleting status setting for SLS project after merge from master Signed-off-by: Zheng Xi Zhou <[email protected]>

Revert some TODO and commetns from master

267f085

I reverted some TODO and comments from master when upgrading crossplane runtime by abandoning Provider object and copy latest practive from provider-template Signed-off-by: Zheng Xi Zhou <[email protected]>

Fix connect for newly merged SLS managed resource

84db8d6

Signed-off-by: Zheng Xi Zhou <[email protected]>

fix ci issues

8abf1d5

Signed-off-by: Zheng Xi Zhou <[email protected]>

fix ut issue

65d88b3

Signed-off-by: Zheng Xi Zhou <[email protected]>

Fix rebase issues

7b11e19

Signed-off-by: Zheng Xi Zhou <[email protected]>

Upgrade version of ProviderConfig from v1alph2 to v1beta1

4651fb4

Signed-off-by: Zheng Xi Zhou <[email protected]>

Extract all common logic from Connect

4d256ac

Add a common PrepareClient to prepare all information to establish a SDK client Signed-off-by: Zheng Xi Zhou <[email protected]>

Check whether extracted AccessKeyID and AccessKeySecret are complete

29d4de4

Signed-off-by: Zheng Xi Zhou <[email protected]>

Update ProviderConfig version

43cc21f

Signed-off-by: Zheng Xi Zhou <[email protected]>

Fix unit tests issues

1be4d0c

Signed-off-by: Zheng Xi Zhou <[email protected]>

Address comments

04a2d91

co-authored-by: Alper Ulucinar <[email protected]> Signed-off-by: Zheng Xi Zhou <[email protected]>

Remove InjectedIndentity

8c160a9

Co-authored-by: Alper Ulucinar <[email protected]> Signed-off-by: Zheng Xi Zhou <[email protected]>

Make sls/oss/redis/slb to adapt to newer Crossplane runtime

604591b

Signed-off-by: Zheng Xi Zhou <[email protected]>

upgrade SLS index based on new runtime

ba64de0

Signed-off-by: Zheng Xi Zhou <[email protected]>

zzxwill force-pushed the runtime-upgrade branch from 8907d05 to ba64de0 Compare July 6, 2021 03:21

negz approved these changes Jul 22, 2021

View reviewed changes

negz merged commit ef3a3a2 into crossplane-contrib:master Jul 22, 2021

zzxwill deleted the runtime-upgrade branch July 22, 2021 07:10

zzxwill mentioned this pull request Jul 22, 2021

Upgrade ProviderConfig version to v1beta1 #95

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump crossplane runtime version to v0.13.0 #64

Bump crossplane runtime version to v0.13.0 #64

zzxwill commented Mar 18, 2021 •

edited

Loading

negz left a comment

negz left a comment

zzxwill commented Apr 25, 2021

hasheddan left a comment

ulucinar left a comment

ulucinar left a comment

ulucinar left a comment

wonderflow commented Jul 7, 2021

negz commented Jul 14, 2021

zzxwill commented Jul 20, 2021

negz commented Jul 22, 2021

negz commented Jul 22, 2021

zzxwill commented Jul 22, 2021

Bump crossplane runtime version to v0.13.0 #64

Bump crossplane runtime version to v0.13.0 #64

Conversation

zzxwill commented Mar 18, 2021 • edited Loading

Description of your changes

How has this code been tested

Apply a ProviderConfig to provide Alibaba cloud credentials.

Test provisioning and destroying managed resources

negz left a comment

Choose a reason for hiding this comment

negz left a comment

Choose a reason for hiding this comment

zzxwill commented Apr 25, 2021

hasheddan left a comment

Choose a reason for hiding this comment

ulucinar left a comment

Choose a reason for hiding this comment

ulucinar left a comment

Choose a reason for hiding this comment

ulucinar left a comment

Choose a reason for hiding this comment

wonderflow commented Jul 7, 2021

negz commented Jul 14, 2021

zzxwill commented Jul 20, 2021

negz commented Jul 22, 2021

negz commented Jul 22, 2021

zzxwill commented Jul 22, 2021

zzxwill commented Mar 18, 2021 •

edited

Loading

Apply a `ProviderConfig` to provide Alibaba cloud credentials.