Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Organize the observability components of Cloudwego #2

Merged
merged 81 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
b196d00
combine logging for kitex and hertz
smx-Morgan Jul 23, 2024
878aba7
combine logging for kitex and hertz V2
smx-Morgan Jul 23, 2024
e1e40e6
combine logging for kitex and hertz V2
smx-Morgan Jul 23, 2024
ae5509f
combine metrics
smx-Morgan Jul 23, 2024
a0fc887
otel V2
smx-Morgan Jul 27, 2024
446d328
logging down
smx-Morgan Jul 28, 2024
4685ebb
logging V2
smx-Morgan Jul 30, 2024
9e71cff
Merge pull request #1 from smx-Morgan/mainV3
smx-Morgan Jul 30, 2024
b3729e2
propagtor V1
smx-Morgan Jul 31, 2024
6597632
propagtor V2
smx-Morgan Aug 1, 2024
89c2ad3
fix
smx-Morgan Aug 1, 2024
f5b2114
可观测组件整合 V2
smx-Morgan Aug 2, 2024
c55b0a3
抽象otel和promth共性逻辑
smx-Morgan Aug 4, 2024
327a8f9
抽象otel和promth共性逻辑 update
smx-Morgan Aug 4, 2024
af34e39
Merge pull request #2 from smx-Morgan/mainV3
smx-Morgan Aug 4, 2024
55b9056
fix bug
smx-Morgan Aug 4, 2024
830ef57
fix Reference Path
smx-Morgan Aug 4, 2024
a0801d1
add promProvider
smx-Morgan Aug 8, 2024
bd21d3f
add labelControl
smx-Morgan Aug 12, 2024
8b63a53
fix bugs
smx-Morgan Aug 12, 2024
0e8ecb7
fix format
smx-Morgan Aug 12, 2024
458b34a
fix CI
smx-Morgan Aug 15, 2024
446e393
fix labelControl name, fix promProvider,promTracer
smx-Morgan Aug 16, 2024
d5dbc22
fix labelControl name, fix promProvider,promTracer
smx-Morgan Aug 16, 2024
9cfcc20
move Measure build to promProvider
smx-Morgan Aug 17, 2024
7b61315
Unified Label Standard
smx-Morgan Aug 19, 2024
7430b81
fix CI
smx-Morgan Aug 19, 2024
2961603
fix CI bugs
smx-Morgan Aug 19, 2024
29621e2
fix CI bugs
smx-Morgan Aug 19, 2024
611aefc
fix CI bug
smx-Morgan Aug 20, 2024
afa535b
fix remove registry from tracer
smx-Morgan Aug 20, 2024
a406cd2
fix CI gofumpt
smx-Morgan Aug 20, 2024
7770544
fix change name to telemetry
smx-Morgan Aug 20, 2024
bbedf5e
fix remove promtracer
smx-Morgan Aug 21, 2024
991e025
fix tracer
smx-Morgan Aug 26, 2024
f1ad26b
fix file_name
smx-Morgan Aug 26, 2024
a9c2581
remove labelcontrol
smx-Morgan Aug 27, 2024
c3889ae
Standardized code
smx-Morgan Aug 30, 2024
e022803
add recorder for retry
smx-Morgan Aug 30, 2024
5a96cf2
update test
smx-Morgan Sep 2, 2024
57a8f2d
fix Data Race
smx-Morgan Sep 2, 2024
0e50950
fix Data Race
smx-Morgan Sep 2, 2024
95901c1
remove promProvider test add example
smx-Morgan Sep 2, 2024
be0e2ea
fix format
smx-Morgan Sep 2, 2024
2358778
move go.mod to telemetry
smx-Morgan Sep 3, 2024
d6cfb97
update makefile
smx-Morgan Sep 3, 2024
0a83f00
update makefile, update file name
smx-Morgan Sep 3, 2024
3de654b
update makefile,update labels
smx-Morgan Sep 4, 2024
3c59ee5
update measure structure
smx-Morgan Sep 7, 2024
971f968
update measure structure
smx-Morgan Sep 8, 2024
f662adb
update formate
smx-Morgan Sep 9, 2024
ced4246
update TelemetryProvider
smx-Morgan Sep 16, 2024
634b01d
update Code Formatting
smx-Morgan Sep 16, 2024
91f7ee3
update build name func
smx-Morgan Sep 16, 2024
7f07ace
Optimize logical structure
smx-Morgan Sep 17, 2024
54b55b6
Support simultaneous monitoring of HTTP and RPC
smx-Morgan Sep 19, 2024
8d527a0
remove measure form promProvider
smx-Morgan Sep 26, 2024
a3d2728
update labels
smx-Morgan Sep 29, 2024
3ccb78c
otel log component compatible with Kitex
smx-Morgan Sep 30, 2024
bc37466
fix path
smx-Morgan Oct 3, 2024
9d2204d
replace logging to hlog
smx-Morgan Oct 8, 2024
644816b
Replace logging with hlog
smx-Morgan Oct 8, 2024
d311634
fix
smx-Morgan Oct 8, 2024
0ebc361
fix
smx-Morgan Oct 8, 2024
8fa3353
fix log && Optimize promProvider
smx-Morgan Oct 8, 2024
9c72b1b
update example for promProvider
smx-Morgan Oct 8, 2024
1e6d032
update go work and fix CI
smx-Morgan Oct 8, 2024
42937e7
fix log
smx-Morgan Oct 8, 2024
5697728
Fix duplicate naming for prometheus
smx-Morgan Oct 8, 2024
3580afa
fix ci
smx-Morgan Oct 8, 2024
cd07dd7
update server
smx-Morgan Oct 9, 2024
133c326
update to metadata, support shell
smx-Morgan Oct 11, 2024
0e71ce3
update CI
smx-Morgan Oct 11, 2024
278a6e0
add examples and support shell
smx-Morgan Oct 12, 2024
77c616b
fix bugs remove the Local dependencies
smx-Morgan Oct 12, 2024
b935878
fix CI
smx-Morgan Oct 12, 2024
b6d6c17
fix CI
smx-Morgan Oct 12, 2024
6f2a8d2
fix ci & add counter to otelkitex
smx-Morgan Oct 14, 2024
1dba93f
fix unit test ci
smx-Morgan Oct 14, 2024
4631375
fix unit test ci
smx-Morgan Oct 14, 2024
72bfeb2
fix unit test ci
smx-Morgan Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ A clear and concise description of what you expected to happen.

If applicable, add screenshots to help explain your problem.

**Kitex version:**
**Version:**

Please provide the version of Kitex you are using.
Please provide the version of {cwgo-pkg} you are using.

**Environment:**

Expand Down
24 changes: 11 additions & 13 deletions .github/workflows/pr-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,24 @@ on: [ pull_request ]

jobs:
compliant:
runs-on: [ self-hosted, X64 ]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Check License Header
uses: apache/skywalking-eyes/header@main
uses: apache/skywalking-eyes/header@v0.4.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Check Spell
uses: crate-ci/typos@master

staticcheck:
runs-on: [ self-hosted, X64 ]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: 1.19
go-version: "1.21"

- uses: actions/cache@v3
with:
Expand All @@ -45,13 +43,13 @@ jobs:
staticcheck_flags: -checks=inherit,-SA1029

lint:
runs-on: [ self-hosted, X64 ]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: 1.19
go-version: "1.22"

- name: Golangci Lint
# https://golangci-lint.run/
Expand Down
25 changes: 5 additions & 20 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,14 @@ on: [ push, pull_request ]

jobs:
unit-benchmark-test:
strategy:
matrix:
go: [ 1.17, 1.18, 1.19 ]
os: [ X64, ARM64 ]
runs-on: ${{ matrix.os }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go }}

# block scenario, comment temporarily
# - uses: actions/cache@v3
# with:
# path: ~/go/pkg/mod
# key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
# restore-keys: |
# ${{ runner.os }}-go-
go-version: "1.21"

- name: Unit Test
run: go test -race -covermode=atomic -coverprofile=coverage.out ./...
run: make test

- name: Benchmark
run: go test -bench=. -benchmem -run=none ./...
12 changes: 5 additions & 7 deletions .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,10 @@
run:
# include `vendor` `third_party` `testdata` `examples` `Godeps` `builtin`
skip-dirs-use-default: true
skip-dirs:
- kitex_gen
skip-files:
- ".*\\.mock\\.go$"
# output configuration options
output:
# Format: colored-line-number|line-number|json|tab|checkstyle|code-climate|junit-xml|github-actions
format: colored-line-number
formats: colored-line-number
# All available settings of specific linters.
# Refer to https://golangci-lint.run/usage/linters
linters-settings:
Expand All @@ -25,12 +21,14 @@ linters-settings:
linters:
enable:
- gofumpt
- goimports
- gofmt
disable:
- errcheck
- typecheck
- deadcode
- varcheck
- staticcheck
issues:
exclude-use-default: true
exclude-files:
- ".*\\.mock\\.go$"
exclude-dirs:
3 changes: 3 additions & 0 deletions .licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@ header:
spdx-id: Apache-2.0
copyright-owner: CloudWeGo Authors


paths:
- '**/*.go'
- '**/*.s'
paths-ignore:
- example/prom/promWithkitex/kitex_gen/**

comment: on-failure
18 changes: 7 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# How to Contribute

## Your First Pull Request
We use github for our codebase. You can start by reading [How To Pull Request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests).
We use GitHub for our codebase. You can start by reading [How To Pull Request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests).

## Branch Organization
We use [git-flow](https://nvie.com/posts/a-successful-git-branching-model/) as our branch organization, as known as [FDD](https://en.wikipedia.org/wiki/Feature-driven_development)

## Bugs
### 1. How to Find Known Issues
We are using [Github Issues](https://github.com/cloudwego/kitex/issues) for our public bugs. We keep a close eye on this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your problem doesn’t already exist.
We are using [Github Issues](https://github.com/cloudwego/{project_name}/issues) for our public bugs. We keep a close eye on this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your problem doesn’t already exist.

### 2. Reporting New Issues
Providing a reduced test code is a recommended way for reporting issues. Then can placed in:
Providing a reduced test code is a recommended way for reporting issues. Then can place in:
- Just in issues
- [Golang Playground](https://play.golang.org/)

Expand All @@ -23,9 +23,9 @@ Please do not report the safe disclosure of bugs to public issues. Contact us by

## Submit a Pull Request
Before you submit your Pull Request (PR) consider the following guidelines:
1. Search [GitHub](https://github.com/cloudwego/kitex/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate existing efforts.
1. Search [GitHub](https://github.com/cloudwego/{project_name}/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate existing efforts.
2. Be sure that an issue describes the problem you're fixing, or documents the design for the feature you'd like to add. Discussing the design upfront helps to ensure that we're ready to accept your work.
3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the cloudwego/kitex repo.
3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the cloudwego {project_name} repo.
4. In your forked repository, make your changes in a new git branch:
```
git checkout -b my-fix-branch develop
Expand All @@ -38,18 +38,14 @@ Before you submit your Pull Request (PR) consider the following guidelines:
```
git push origin my-fix-branch
```
9. In GitHub, send a pull request to `kitex:develop`
9. In GitHub, send a pull request to `{project_name}:develop`

## Contribution Prerequisites
- Our development environment keeps up with [Go Official](https://golang.org/project/).
- You need fully checking with lint tools before submit your pull request. [gofmt](https://golang.org/pkg/cmd/gofmt/) and [golangci-lint](https://github.com/golangci/golangci-lint)
- You are familiar with [Github](https://github.com)
- You are familiar with [GitHub](https://github.com)
- Maybe you need familiar with [Actions](https://github.com/features/actions)(our default workflow tool).

## Code Style Guides
Also see [Pingcap General advice](https://pingcap.github.io/style-guide/general.html).

Good resources:
- [Effective Go](https://golang.org/doc/effective_go)
- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
- [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md)
15 changes: 15 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
TOOLS_SHELL="./hack/tools.sh"

.PHONY: test
test:
chmod +x ${TOOLS_SHELL}
@${TOOLS_SHELL} test
@echo "go test finished"



.PHONY: vet
vet:
chmod +x ${TOOLS_SHELL}
@${TOOLS_SHELL} vet
@echo "vet check finished"
20 changes: 20 additions & 0 deletions example/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
* Copyright 2024 CloudWeGo Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package main

func main() {
}
66 changes: 66 additions & 0 deletions example/otel/otelwithhertz/client/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
/*
* Copyright 2024 CloudWeGo Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package main

import (
"context"
"time"

"github.com/cloudwego-contrib/cwgo-pkg/telemetry/instrumentation/otelhertz"
"github.com/cloudwego-contrib/cwgo-pkg/telemetry/provider/otelprovider"
"github.com/cloudwego/hertz/pkg/app/client"
"github.com/cloudwego/hertz/pkg/common/hlog"
hertzlogrus "github.com/hertz-contrib/obs-opentelemetry/logging/logrus"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
)

func main() {
hlog.SetLogger(hertzlogrus.NewLogger())
hlog.SetLevel(hlog.LevelDebug)

serviceName := "demo-hertz-client"

p := otelprovider.NewOpenTelemetryProvider(
otelprovider.WithServiceName(serviceName),
// Support setting ExportEndpoint via environment variables: OTEL_EXPORTER_OTLP_ENDPOINT
otelprovider.WithExportEndpoint("localhost:4317"),
otelprovider.WithHttpServer(),
otelprovider.WithInsecure(),
)
defer p.Shutdown(context.Background())

c, _ := client.NewClient()
c.Use(otelhertz.ClientMiddleware())

for {
ctx, span := otel.Tracer("github.com/hertz-contrib/obs-opentelemetry").
Start(context.Background(), "loop")

_, b, err := c.Get(ctx, nil, "http://0.0.0.0:8888/ping?foo=bar")
if err != nil {
hlog.CtxErrorf(ctx, err.Error())
}

span.SetAttributes(attribute.String("msg", string(b)))

hlog.CtxInfof(ctx, "hertz client %s", string(b))
span.End()

<-time.After(time.Second)
}
}
15 changes: 15 additions & 0 deletions example/otel/otelwithhertz/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
module otelwithhertz

go 1.21

require (
github.com/cloudwego/hertz v0.9.3
github.com/hertz-contrib/obs-opentelemetry/logging/logrus v0.1.1
)

require (
github.com/sirupsen/logrus v1.8.1 // indirect
go.opentelemetry.io/otel v1.4.1 // indirect
go.opentelemetry.io/otel/trace v1.4.1 // indirect
golang.org/x/sys v0.24.0 // indirect
)
58 changes: 58 additions & 0 deletions example/otel/otelwithhertz/server/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/*
* Copyright 2024 CloudWeGo Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package main

import (
"context"
"exampleprom/promWithkitex/kitex_gen/api"

"github.com/cloudwego-contrib/cwgo-pkg/telemetry/instrumentation/otelhertz"
"github.com/cloudwego-contrib/cwgo-pkg/telemetry/provider/otelprovider"
"github.com/cloudwego/hertz/pkg/app"
"github.com/cloudwego/hertz/pkg/app/server"
"github.com/cloudwego/hertz/pkg/common/hlog"
"github.com/cloudwego/hertz/pkg/protocol/consts"
hertzlogrus "github.com/hertz-contrib/obs-opentelemetry/logging/logrus"
)

func main() {
hlog.SetLogger(hertzlogrus.NewLogger())
hlog.SetLevel(hlog.LevelDebug)

serviceName := "demo-hertz-server"
p := otelprovider.NewOpenTelemetryProvider(
otelprovider.WithServiceName(serviceName),
// Support setting ExportEndpoint via environment variables: OTEL_EXPORTER_OTLP_ENDPOINT
otelprovider.WithExportEndpoint("localhost:4317"),
otelprovider.WithHttpServer(),
otelprovider.WithInsecure(),
)
defer p.Shutdown(context.Background())

tracer, cfg := otelhertz.NewServerOption()
h := server.Default(tracer)
h.Use(otelhertz.ServerMiddleware(cfg))

h.GET("/ping", func(c context.Context, ctx *app.RequestContext) {
req := &api.Request{Message: "my request"}

hlog.CtxDebugf(c, "message received successfully: %s", req.Message)
ctx.JSON(consts.StatusOK, "resp")
})

h.Spin()
}
Loading
Loading