Skip to content

Commit

Permalink
misc: move all importable code to internal/ (#6552)
Browse files Browse the repository at this point in the history 
Our backwards compatibility guarantees[1] cover importable Go code. This applies once we cut the 1.0 release, which we are rapidly approaching. 

The Grafana Agent maintainers have agreed that it is more important for 1.0 to focus on stability of the configuration file and usage of the Grafana Agent binaries rather than on the stability of importable code. 

This PR moves all importable Go code to `internal/` so that it is no longer externally importable. This gives us time to finalize Go APIs without blocking the 1.0 release or removing our commitment to having Go APIs follow semantic versioning. Some time after the 1.0 release, we will have public Go APIs available for consumption. 

There are some slightly odd artifacts in this PR for the purposes of making sure we have no public Go API surface:

* Non-binary tools have been moved to `internal/tools`:
  * Code for generating documentation has been moved to `internal/tools/docs_generator`. 
  * Code for integration testing system packages has been moved to `internal/tools/packaging_test`. 
* Binaries (i.e., `package main`) not a part of our release assets have similarly moved to `internal/cmd`:
  * Code for integration testing Flow components has been moved to `internal/cmd/integration-tests` 
  * Code for linting the codebase has been moved to `internal/cmd/agentlint` 
  * Code for evaluating River files has been moved to `internal/cmd/rivereval`
* All flow mode UI code has been moved from `web` to `internal/web`. 

I don't know if all of these changes are necessary, or if they have been moved to a sensible location. However, for the scope of this PR, I would like to keep the folders in their new location, and follow-up PRs and discussions can be used to reorganize `internal/` as needed. 

[1]: https://github.com/grafana/agent/blob/main/docs/rfcs/0008-backwards-compatibility.md
  • Loading branch information
@rfratto
rfratto authored Feb 29, 2024
1 parent bcc9b0a commit 0e36aa2
Show file tree
Hide file tree
Showing 1,816 changed files with 2,731 additions and 2,726 deletions.
18 changes: 9 additions & 9 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ GO_ENV := GOOS=$(GOOS) GOARCH=$(GOARCH) GOARM=$(GOARM) CGO_ENABLED=$(CGO_ENABLED
VERSION ?= $(shell bash ./tools/image-tag)
GIT_REVISION := $(shell git rev-parse --short HEAD)
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD)
VPREFIX := github.com/grafana/agent/pkg/build
VPREFIX := github.com/grafana/agent/internal/build
GO_LDFLAGS := -X $(VPREFIX).Branch=$(GIT_BRANCH) \
-X $(VPREFIX).Version=$(VERSION) \
-X $(VPREFIX).Revision=$(GIT_REVISION) \
Expand All @@ -150,7 +150,7 @@ endif
# Targets for running tests
#
# These targets currently don't support proxying to a build container due to
# difficulties with testing ./pkg/util/k8s and testing packages.
# difficulties with testing ./internal/util/k8s and testing packages.
#

.PHONY: lint
Expand All @@ -163,15 +163,15 @@ lint: agentlint
# more without -race for packages that have known race detection issues.
test:
$(GO_ENV) go test $(GO_FLAGS) -race $(shell go list ./... | grep -v /integration-tests/)
$(GO_ENV) go test $(GO_FLAGS) ./pkg/integrations/node_exporter ./pkg/logs ./pkg/operator ./pkg/util/k8s ./component/otelcol/processor/tail_sampling ./component/loki/source/file ./component/loki/source/docker
$(GO_ENV) go test $(GO_FLAGS) ./internal/static/integrations/node_exporter ./internal/static/logs ./internal/static/operator ./internal/util/k8s ./internal/component/otelcol/processor/tail_sampling ./internal/component/loki/source/file ./internal/component/loki/source/docker

test-packages:
docker pull $(BUILD_IMAGE)
go test -tags=packaging ./packaging
go test -tags=packaging ./internal/tools/packaging_test

.PHONY: integration-test
integration-test:
cd integration-tests && $(GO_ENV) go run .
cd internal/cmd/integration-tests && $(GO_ENV) go run .

#
# Targets for building binaries
Expand Down Expand Up @@ -235,7 +235,7 @@ agentlint:
ifeq ($(USE_CONTAINER),1)
$(RERUN_IN_CONTAINER)
else
cd ./tools/agentlint && $(GO_ENV) go build $(GO_FLAGS) -o ../../$(AGENTLINT_BINARY) .
cd ./internal/cmd/agentlint && $(GO_ENV) go build $(GO_FLAGS) -o ../../../$(AGENTLINT_BINARY) .
endif

#
Expand Down Expand Up @@ -272,7 +272,7 @@ ifeq ($(USE_CONTAINER),1)
$(RERUN_IN_CONTAINER)
else
bash ./tools/generate-crds.bash
gen-crd-api-reference-docs -config tools/gen-crd-docs/config.json -api-dir "github.com/grafana/agent/pkg/operator/apis/monitoring/" -out-file docs/sources/operator/api.md -template-dir tools/gen-crd-docs/template
gen-crd-api-reference-docs -config tools/gen-crd-docs/config.json -api-dir "github.com/grafana/agent/internal/static/operator/apis/monitoring/" -out-file docs/sources/operator/api.md -template-dir tools/gen-crd-docs/template
endif

generate-drone:
Expand Down Expand Up @@ -304,14 +304,14 @@ generate-protos:
ifeq ($(USE_CONTAINER),1)
$(RERUN_IN_CONTAINER)
else
go generate ./pkg/agentproto/
go generate ./internal/static/agentproto/
endif

generate-ui:
ifeq ($(USE_CONTAINER),1)
$(RERUN_IN_CONTAINER)
else
cd ./web/ui && yarn --network-timeout=1200000 && yarn run build
cd ./internal/web/ui && yarn --network-timeout=1200000 && yarn run build
endif

generate-versioned-files:
Expand Down
6 changes: 3 additions & 3 deletions cmd/grafana-agent-flow/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@
package main

import (
"github.com/grafana/agent/cmd/internal/flowmode"
"github.com/grafana/agent/pkg/build"
"github.com/grafana/agent/internal/build"
"github.com/grafana/agent/internal/flowmode"
"github.com/prometheus/client_golang/prometheus"

// Register Prometheus SD components
_ "github.com/grafana/loki/clients/pkg/promtail/discovery/consulagent"
_ "github.com/prometheus/prometheus/discovery/install"

// Register integrations
_ "github.com/grafana/agent/pkg/integrations/install"
_ "github.com/grafana/agent/internal/static/integrations/install"

// Embed a set of fallback X.509 trusted roots
// Allows the app to work correctly even when the OS does not provide a verifier or systems roots pool
Expand Down
8 changes: 4 additions & 4 deletions cmd/grafana-agent-operator/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ import (

"github.com/go-kit/log"
"github.com/go-kit/log/level"
"github.com/grafana/agent/pkg/build"
"github.com/grafana/agent/pkg/operator"
"github.com/grafana/agent/pkg/operator/logutil"
util_log "github.com/grafana/agent/pkg/util/log"
"github.com/grafana/agent/internal/build"
"github.com/grafana/agent/internal/static/operator"
"github.com/grafana/agent/internal/static/operator/logutil"
util_log "github.com/grafana/agent/internal/util/log"
controller "sigs.k8s.io/controller-runtime"

// Needed for clients.
Expand Down
4 changes: 2 additions & 2 deletions cmd/grafana-agent-service/service_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ import (
"testing"

"github.com/go-kit/log"
"github.com/grafana/agent/pkg/flow/componenttest"
"github.com/grafana/agent/pkg/util"
"github.com/grafana/agent/internal/flow/componenttest"
"github.com/grafana/agent/internal/util"
"github.com/phayes/freeport"
"github.com/stretchr/testify/require"
)
Expand Down
16 changes: 8 additions & 8 deletions cmd/grafana-agent/entrypoint.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@ import (
"github.com/go-kit/log/level"
"github.com/gorilla/mux"
"github.com/grafana/agent/internal/agentseed"
"github.com/grafana/agent/pkg/config"
"github.com/grafana/agent/pkg/logs"
"github.com/grafana/agent/pkg/metrics"
"github.com/grafana/agent/pkg/metrics/instance"
"github.com/grafana/agent/pkg/server"
"github.com/grafana/agent/pkg/supportbundle"
"github.com/grafana/agent/pkg/traces"
"github.com/grafana/agent/pkg/usagestats"
"github.com/grafana/agent/internal/static/config"
"github.com/grafana/agent/internal/static/logs"
"github.com/grafana/agent/internal/static/metrics"
"github.com/grafana/agent/internal/static/metrics/instance"
"github.com/grafana/agent/internal/static/server"
"github.com/grafana/agent/internal/static/supportbundle"
"github.com/grafana/agent/internal/static/traces"
"github.com/grafana/agent/internal/usagestats"
"github.com/grafana/dskit/signals"
"github.com/oklog/run"
"github.com/prometheus/client_golang/prometheus"
Expand Down
14 changes: 7 additions & 7 deletions cmd/grafana-agent/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ import (
"os"

"github.com/go-kit/log/level"
"github.com/grafana/agent/cmd/internal/flowmode"
"github.com/grafana/agent/pkg/boringcrypto"
"github.com/grafana/agent/pkg/build"
"github.com/grafana/agent/pkg/config"
"github.com/grafana/agent/pkg/server"
util_log "github.com/grafana/agent/pkg/util/log"
"github.com/grafana/agent/internal/boringcrypto"
"github.com/grafana/agent/internal/build"
"github.com/grafana/agent/internal/flowmode"
"github.com/grafana/agent/internal/static/config"
"github.com/grafana/agent/internal/static/server"
util_log "github.com/grafana/agent/internal/util/log"

"github.com/prometheus/client_golang/prometheus"

Expand All @@ -20,7 +20,7 @@ import (
_ "github.com/prometheus/prometheus/discovery/install"

// Register integrations
_ "github.com/grafana/agent/pkg/integrations/install"
_ "github.com/grafana/agent/internal/static/integrations/install"

// Embed a set of fallback X.509 trusted roots
// Allows the app to work correctly even when the OS does not provide a verifier or systems roots pool
Expand Down
6 changes: 3 additions & 3 deletions cmd/grafana-agent/service_windows.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (
"os"

"github.com/go-kit/log/level"
"github.com/grafana/agent/pkg/config"
"github.com/grafana/agent/pkg/server"
util_log "github.com/grafana/agent/pkg/util/log"
"github.com/grafana/agent/internal/static/config"
"github.com/grafana/agent/internal/static/server"
util_log "github.com/grafana/agent/internal/util/log"

"golang.org/x/sys/windows/svc"
)
Expand Down
14 changes: 7 additions & 7 deletions cmd/grafana-agentctl/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,24 @@ import (
"strings"
"syscall"

"github.com/grafana/agent/pkg/agentctl/waltools"
"github.com/grafana/agent/pkg/build"
"github.com/grafana/agent/pkg/config"
"github.com/grafana/agent/pkg/logs"
"github.com/grafana/agent/internal/build"
"github.com/grafana/agent/internal/static/agentctl/waltools"
"github.com/grafana/agent/internal/static/config"
"github.com/grafana/agent/internal/static/logs"
"github.com/olekukonko/tablewriter"
"github.com/prometheus/client_golang/prometheus"

"github.com/go-kit/log"
"github.com/go-kit/log/level"
"github.com/grafana/agent/pkg/agentctl"
"github.com/grafana/agent/pkg/client"
"github.com/grafana/agent/internal/static/agentctl"
"github.com/grafana/agent/internal/static/client"
"github.com/spf13/cobra"

// Register Prometheus SD components
_ "github.com/prometheus/prometheus/discovery/install"

// Register integrations
_ "github.com/grafana/agent/pkg/integrations/install"
_ "github.com/grafana/agent/internal/static/integrations/install"

// Needed for operator-detach
"k8s.io/apimachinery/pkg/fields"
Expand Down
Loading

0 comments on commit 0e36aa2

Please sign in to comment.