Skip to content

Commit

Permalink
AIP-157 Partial response implementation
Browse files Browse the repository at this point in the history 
This feature add capabilities to filter the response message from all the APIs.
AIP detail:
https://google.aip.dev/157
  • Loading branch information
@sayan-biswas
sayan-biswas committed May 2, 2024
1 parent 714dfd2 commit 0acbbac
Show file tree
Hide file tree
Showing 27 changed files with 6,331 additions and 61 deletions.
66 changes: 30 additions & 36 deletions cmd/api/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,42 +2,36 @@

## Variables

| Environment Variable | Description | Example |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
| DB_USER | Postgres Database user | user |
| DB_PASSWORD | Postgres Database Password | hunter2 |
| DB_HOST | Postgres Database host | /cloudsql/my-project:us-east1:tekton-results |
| DB_NAME | Postgres Database name | tekton_results |
| DB_SSLMODE | Database SSL mode | verify-full |
| DB_SSLROOTCERT | Path to CA cert used to validate Database cert | /etc/tls/db/ca.crt |
| DB_ENABLE_AUTO_MIGRATION | Auto-migrate the database on startup (create/update schemas). For further details, refer to <https://gorm.io/docs/migration.html> | true (default) |
| PROFILING | Enable profiling server | false (default) |
| PROFILING_PORT | Profiling Server Port | 6060 (default) |
| DB_MAX_IDLE_CONNECTIONS | The number of idle database connections to keep open | 2 (default for golang, but specific database drivers may have settings for this too) |
| DB_MAX_OPEN_CONNECTIONS | The maximum number of database connections, for best performance it should equal DB_MAX_IDLE_CONNECTIONS | unlimited (default for golang, but specific database drivers may have settings for this too) |
| GRPC_WORKER_POOL | The maximum number of goroutines pre-allocated for process GRPC requests. The GRPC server will also dynamically create threads. | 2 (default) |
| K8S_QPS | The QPS setting for the kubernetes client created. | 5 (default) |
| K8S_BURST | The burst setting for the kubernetes client created. | 10 (default) |
| SERVER_PORT | gRPC and REST Server Port | 8080 (default) |
| PROMETHEUS_PORT | Prometheus Port | 9090 (default) |
| PROMETHEUS_HISTOGRAM | Enable Prometheus histogram metrics to measure latency distributions of RPCs | false (default) |
| TLS_PATH | Path to TLS files | /etc/tls |
| AUTH_DISABLE | Disable RBAC check for resources | false (default) |
| AUTH_IMPERSONATE | Enable RBAC impersonation | true (default) |
| LOG_LEVEL | Log level for api server | info (default) |
| LOGS_API | Enable logs storage service | false (default) |
| LOGS_TYPE | Determine Logs storage backend type | File (default) |
| LOGS_BUFFER_SIZE | Buffer for streaming logs | 32768 (default) |
| LOGS_PATH | Logs storage path | logs (default) |
| S3_BUCKET_NAME | S3 Bucket name | <S3 Bucket Name> |
| S3_ENDPOINT | S3 Endpoint | https://s3.ap-south-1.amazonaws.com |
| S3_HOSTNAME_IMMUTABLE | S3 Hostname immutable | false (default) |
| S3_REGION | S3 Region | ap-south-1 |
| S3_ACCESS_KEY_ID | S3 Access Key ID | <S3 Acces Key> |
| S3_SECRET_ACCESS_KEY | S3 Secret Access Key | <S3 Access Secret> |
| S3_MULTI_PART_SIZE | S3 Multi part size | 5242880 (default) |
| GCS_BUCKET_NAME | GCS Bucket Name | <GCS Bucket Name> |
| STORAGE_EMULATOR_HOST | GCS Storage Emulator Server | http://localhost:9004 |
| Environment Variable | Description | Example |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------|
| DB_USER | Postgres Database user | user |
| DB_PASSWORD | Postgres Database Password | hunter2 |
| DB_HOST | Postgres Database host | /cloudsql/my-project:us-east1:tekton-results |
| DB_NAME | Postgres Database name | tekton_results |
| DB_SSLMODE | Database SSL mode | verify-full |
| DB_SSLROOTCERT | Path to CA cert used to validate Database cert | /etc/tls/db/ca.crt |
| DB_ENABLE_AUTO_MIGRATION | Auto-migrate the database on startup (create/update schemas). For further details, refer to <https://gorm.io/docs/migration.html> | true (default) |
| SERVER_PORT | gRPC and REST Server Port | 8080 (default) |
| FEATURE_GATES | Enables server features | FeatureA=true,FeatureB=false | |
| PROMETHEUS_PORT | Prometheus Port | 9090 (default) |
| PROMETHEUS_HISTOGRAM | Enable Prometheus histogram metrics to measure latency distributions of RPCs | false (default) |
| TLS_PATH | Path to TLS files | /etc/tls |
| AUTH_DISABLE | Disable RBAC check for resources | false (default) |
| AUTH_IMPERSONATE | Enable RBAC impersonation | true (default) |
| LOG_LEVEL | Log level for api server | info (default) |
| LOGS_API | Enable logs storage service | false (default) |
| LOGS_TYPE | Determine Logs storage backend type | File (default) |
| LOGS_BUFFER_SIZE | Buffer for streaming logs | 32768 (default) |
| LOGS_PATH | Logs storage path | logs (default) |
| S3_BUCKET_NAME | S3 Bucket name | <S3 Bucket Name> |
| S3_ENDPOINT | S3 Endpoint | https://s3.ap-south-1.amazonaws.com |
| S3_HOSTNAME_IMMUTABLE | S3 Hostname immutable | false (default) |
| S3_REGION | S3 Region | ap-south-1 |
| S3_ACCESS_KEY_ID | S3 Access Key ID | <S3 Acces Key> |
| S3_SECRET_ACCESS_KEY | S3 Secret Access Key | <S3 Access Secret> |
| S3_MULTI_PART_SIZE | S3 Multi part size | 5242880 (default) |
| GCS_BUCKET_NAME | GCS Bucket Name | <GCS Bucket Name> |
| STORAGE_EMULATOR_HOST | GCS Storage Emulator Server | http://localhost:9004 |

These values can also be set in the config file located in the `config/env/config` directory.

Expand Down
15 changes: 14 additions & 1 deletion cmd/api/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,14 @@ import (
"crypto/tls"
"database/sql"
"fmt"
"github.com/tektoncd/results/pkg/api/server/features"
"net/http"
"path"
"strings"
"time"

"github.com/tektoncd/results/internal/fieldmask"

"github.com/tektoncd/results/pkg/api/server/v1alpha2/auth/impersonation"
"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
Expand Down Expand Up @@ -76,6 +79,12 @@ func main() {
// any data in the event of an unhandled error.
defer log.Sync() //nolint:errcheck

// Load server features
f := features.NewFeatureGate()
if err := f.Set(serverConfig.FEATURE_GATES); err != nil {
log.Errorf("Failed to load feature gates: %v", err)
}

// Load server TLS
certFile := path.Join(serverConfig.TLS_PATH, "tls.crt")
keyFile := path.Join(serverConfig.TLS_PATH, "tls.key")
Expand Down Expand Up @@ -208,6 +217,7 @@ func main() {
grpc_zap.UnaryServerInterceptor(grpcLogger, zapOpts...),
grpc_auth.UnaryServerInterceptor(determineAuth),
prometheus.UnaryServerInterceptor,
fieldmask.UnaryServerInterceptor(f.Get(features.PartialResponse)),
recovery.UnaryServerInterceptor(recovery.WithRecoveryHandler(recoveryHandler)),
),
grpc_middleware.WithStreamServerChain(
Expand Down Expand Up @@ -274,7 +284,10 @@ func main() {
if err != nil {
log.Fatalf("Error dialing gRPC endpoint: %v", err)
}
serverMuxOptions = append(serverMuxOptions, runtime.WithHealthzEndpoint(healthpb.NewHealthClient(clientConn)))
serverMuxOptions = append(serverMuxOptions,
runtime.WithHealthzEndpoint(healthpb.NewHealthClient(clientConn)),
runtime.WithMetadata(fieldmask.MetadataAnnotator),
)

// Create server for gRPC gateway
ctx := context.Background()
Expand Down
8 changes: 1 addition & 7 deletions config/base/env/config
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,7 @@ DB_NAME=tekton-results
DB_SSLMODE=disable
DB_SSLROOTCERT=
DB_ENABLE_AUTO_MIGRATION=true
DB_MAX_IDLE_CONNECTIONS=10
DB_MAX_OPEN_CONNECTIONS=10
GRPC_WORKER_POOL=2
K8S_QPS=5
K8S_BURST=10
PROFILING=false
PROFILING_PORT=6060
FEATURE_GATES=
SERVER_PORT=8080
PROMETHEUS_PORT=9090
PROMETHEUS_HISTOGRAM=false
Expand Down
Loading

0 comments on commit 0acbbac

Please sign in to comment.