v0.4.0

Notable changes since the last v0.3.0 release

Architectural changes

• Introduced the “Route” custom resource. The Cloud Controller now creates and updates “Route” CRD objects directly as part of the cf map-route and cf unmap-route workflows. Please review the networking team’s proposal to learn more about “Route” CRD and the problems it solves.
  • This change also removes the need for the meta controller component.
• Introduced the “CustomBuilder”, “Store” and “Stack” custom resources. The new resources are stepping stones towards integrating Paketo buildpacks and setting up auto-consumption of stack and buildpacks when new releases are available.
• Added the remaining namespaces cf-workloads-staging and kpack to the Istio service mesh.

New Features / Bug fixes

• 🎉 🎉 App developers can now push Ruby and Python based applications to cf-for-k8s. Both Ruby and Python language buildpacks are community versions ^[1] but we encourage you to take it for a spin and provide feedback to us. The current buildpacks in cf-for-k8s are as follow,

  • Official buildpacks - Java, Nodejs, Go, DotNet, PHP, HTTPD, Nginx, Procfile

  • Community buildpacks - Ruby, Python

    Go ahead, push an app of your choice!!

    ^[1] All net-new buildpacks are brought in as "Community" buildpacks. After some period of seeking feedback, the buildpacks go through an RFC process of promoting to an official Paketo Buildpack.

• Platform operators can optionally install metrics-server on K8s clusters that don’t come with metrics-server pre-installed Issue 210.

• Resolved an issue with route-controller deletion order Issue 224.

• Resolved issue with how kpack image pull secrets being set and moved it to the right namespace.

Release Updates

Release	Old Version	New Version
minio	5.0.22	5.0.30
kpack	0.0.8-rc.2	0.0.9-rc.41
uaa	v74.17.0	v74.21.0
postgres	8.9.0	8.10.6
kapp	0.26.0	0.29.0

Kubernetes version update

cf-for-k8s has dropped support for Kubernetes version 1.14. The oldest version supported is bumped to 1.15 and the newest version is 1.18.

Testing updates

Our primary goal is to shorten the feedback loops with the component teams.

• Added uptime measurements to our master upgrade pipeline. The intent is to observe and report any anomalies to component teams.
• Auto consume kpack, uaa, cf-k8s-logging, erini, capi-k8s-release.
• Added upgrade tests to kpack, uaa, cf-k8s-loggin, eirini, capi-k8s-release.

Doc updates

• Updated documentation to indicate that the load balancer IP can be surfaced as hostname 209
• Updated documentation to make it easier for contributors to find good first issues.

What we are working on next

• Continue to improve cf-for-k8s upgradability. We now have upgradability tests in component bumps and soon on pull requests.
• Run cf-acceptance tests in cf-for-k8s pipelines.
• Collaborate with CredHub and component teams on Secret management.

Have a question, reach out to us

Our slack channels

• #cf-for-k8s
• #release-integration

Interested in contributing?

• The easiest way to get involved is to start attending the SIG meetings, join the #cf-for-k8s slack channel, and subscribe to the [email protected] mailing list.
• You can also start by improving the docs. Install cf-for-k8s using the deploy docs and if you notice issues or discrepancies in the docs, you can submit a PR.

v0.3.0

Feedback Survey

We would love to hear from the CF community on your experience with cf-for-k8s. Your feedback helps us improve our product and processes. Please take a moment to submit your feedback with this survey. Thanks in advance!

Notable changes since the last (v0.2.0) release

New features and Bug fixes

• 🔥 Platform operators can now set ingress certs for cf apps using the new property workloads_certificate. 🔥
  • By separating certificate properties for CF API domain (system_certificate), app domains (workloads_certificate), and internal system components (internal_certificate), operators get granular control in creating certificates for different domains.
  • Docs with instructions on how to set up real certs with LetsEncrypt.
• Resolved an issue when upgrading the internal Minio blobstore to new versions (Issue #164).
• Resolved an issue when upgrading the internal Postgres database (Bitnami Postgres helm chart) (Issue #212). Please note that kapp exits ^[1] before the Postgres statefulset is available. In addition, rotating database passwords on an existing install do not currently work.
• Platform operators can now install cf-for-k8s on Kubernetes 1.18 version. You can follow the newest (1.18) and the oldest version (1.14) changes by watching supported_k8s_versions.yml file.
• Platform operators and app developers can see metrics after pushing the app using the cf cli v7 rolling strategy (Issue #177).
• Platform operators will now get an error message if the kapp CLI version does not meet the minimum kapp version required to install cf-for-k8s. (This was already the case for vendir and ytt.)
• Platform operators should see a reduction in insufficient CPU errors when installing cf-for-k8s (Issue #103). With PriorityClass for DaemonSet, K8s will schedule DaemonSet first before scheduling pods.
• Platform operators can now consume Istio pilot metrics.
• App Developers can pass environment variables with cf push (Issue #163).

^[1] This is a known issue. You will have to manually check for the statefulset pod health before running cf-cli commands.

Other updates

• CAPI now includes API readiness configuration, so that it can receive traffic only when it’s ready to accept.
• Changed prefix for user-facing CAPI names to "cf-api".

What we are working on next

• Continue to resolve issues pertaining to cf-for-k8s upgradability. With v0.3.0 release, Operators can now upgrade Minio, PostgresDB, and config-maps changes. Our next milestone is to resolve upgrade issues with system components, K8s cluster version, and 3rd party components.
• Incorporate the full set of Paketo Cloud Native Buildpacks into cf-for-k8s.
• Collaborate with contributing projects in establishing a workflow on consuming releases.

Have a question, reach out to us

Our slack channels

• #cf-for-k8s
• #release-integration

Interested in contributing?

• The easiest way to get involved is to start attending the SIG meetings, join the #cf-for-k8s slack channel, and subscribe to the [email protected] mailing list.
• You can also start by improving the docs. Install cf-for-k8s using the deploy docs and if you notice issues or discrepancies in the docs, you can submit a PR.

v0.2.0

Notable changes since the last v0.1.0 release.

New features

• 🔥 Users can now see streaming logs during cf push command.:fire: (ISSUE-80).
• Users can get logs from cf log APP_NAME (ISSUE-157).
• Users can now see app metrics (ISSUE-122).
  • Most cloud providers embed metrics-server in the cluster. See deployment docs if you need to install it manually.
• Users can now connect to nodes without requiring a load balancer. This is ideal for non-production use cases to just connect to CC API and apps without needing an LB.
  • Given users can connect to nodes, config-optional/use-nodeport-for-ingress.yml is no longer required and is removed in this version.
  • For kind install, the new config file, remove-ingressgateway-service.yml, will remove LB resource because kind doesn't support LoadBalancer services.
• Users can set a static Loadbalancer IP when installing cf-for-k8s. This is ideal for users(or CI pipelines) who repeatedly re-install cf-for-k8s,
  • Just add istio_static_ip: <reserved IP> to your cf-values.yaml. It requires you to reserve IP address from your cloud provider
• HTTP requests are now routed to subdomains of UAA. (ISSUE-170)
• Users can restage apps after cups binding (ISSUE-119). Also, general restage works (ISSUE-100).
• Users do not need to enable diego_docker feature to push a source code app (ISSUE-70).

Other updates

• Removed install-cf.sh script (ISSUE-165).
  • Users now have the freedom to use ytt and kapp independently and pass any number of config files.
• Pinned system components images (e.g. capi, logging etc) and 3rd party images (e.g. meta controller etc) to specific digest (ISSUE-106), (ISSUE-107).
• All system components now use the local DNS (instead of hair pinning previously) to communicate with each other.
• Updated deploy documentation to incorporate the new changes and add clarity for new users.
• Added AKS to the Kubernetes testing matrix.

What we are working on next

• Ability for users to add an external database and blobstore. The experience will be similar to how users configured an external database/blobstore in cf-deployment (Planned for 0.3.0 alpha release).
• Add the newly released paketo buildpacks to cf-for-k8s. (Planned for 0.3.0 alpha release).
• Ability to upgrade to new versions of cf-for-k8s. In addition, rotate passwords/certs with minimal downtime (ISSUE-45) etc.
• Add tests to PR gates and master pipeline that will validate cf-for-k8s is compatible with the current Kubernetes versions (highlighted in the deploy docs)

Have a question, reach out to us,

• Our slack channels
  • #cf-for-k8s
  • #release-integration

Interested in contributing?

• Easiest way to get involved is to start attending the SIG meetings, join the #cf-for-k8s slack channel, and subscribe to the [email protected] mailing list.
• You can also start by improving the docs. Install cf-for-k8s using the deploy docs and if you notice issues or discrepancies in the docs, you can submit a PR.

v0.1.0

Introducing the v0.1.0 alpha version of Cloud Foundry for Kubernetes

Installation

Follow the instructions in the How to Deploy document.

Background

The Release Integration team has been developing a Cloud Foundry deployment artifact to install Cloud Foundry on a Kubernetes cluster. The deployment artifact contains the new Kubernetes-native CF components, which are built on top of popular Kubernetes projects like kpack, fluentd, and Istio.

The deployment artifact has been available to the community for some time with limited capabilities. The primary goal was to enable CF contributing projects to rapidly iterate on their components.

With the recent integration of buildpacks support, we are excited to release the first version, 0.1.0, of this product "Cloud Foundry for Kubernetes" (aka "cf-for-k8s"). We will continue to create 0.X releases as new capabilities are added and we improve stability of cf-for-k8s. We plan to share our release cadence soon.

Highlights

App staging with kpack

With the 0.1.0 release, users can now push an app with source code. In the Cloud Foundry for Bosh, the Cloud Controller issues a staging request to Diego, which detects and builds a droplet with the right Cloud Foundry build packs. Once the droplet is built, it then schedules the app on one or more Diego cells.

In cf-for-k8s, CAPI issues the request to kpack, which uses cloud-native buildpacks to detect the app language and then build the app image. Once the app image is available, the image is pushed to the app registry and a request is sent to Eirini to schedule the app workloads, which are scheduled as one or more K8s pod deployments. Once the app workloads are available, users can curl the app.

Encrypted communication

The v0.1.0 release comes with Istio, which enforces encrypted communication between Cloud Foundry components. In the Cloud Foundry for Bosh, each component owned the responsibility of managing and enforcing encrypted communication. All of this responsibility is now delegated to Istio.

Istio uses sidecar, which is deployed to every pod, to encrypt and mTLS communication between all CF components. In addition, Istio will rotate certificates automatically without requiring any intervention from the component teams or platform engineers.

Manage cf-for-k8s lifecycle with kapp

In the Cloud Foundry for Bosh, Platform engineers and Contributing teams relied on bosh deploy command to install, upgrade or remove Cloud Foundry foundation on VMs.

kapp provides a similar experience where users can install, upgrade or remove Cloud Foundry on Kubernetes. Unlike kubectl apply command that exits before resources are created in the cluster, kapp waits until all resources are created and continuously provides status updates on the resource availability. In addition, kapp can delete all cf-for-k8s resources in one swoop.

Furthermore, kapp provides resource differences when upgrading to new versions of cf-for-k8s. Platform engineers can audit the differences (new resources, updates to existing resources) between their current foundation and the new version (e.g. new version of cf-for-k8s may bump cluster resource needs).

Templating with ytt

ytt (pronounced spelled out) is a templating tool that understands YAML structure. Product delivery teams can use it to create reusable YAML templates that operators can use for product configuration.

Reusable configuration and built in full-featured programming language help ease the burden of configuring complex software. The built in YAML structure helps reduce the mental overhead of YAML construction. You can reuse the same templates in different environments by injecting environment-specific values (via cf-install-values.yml) at deploy time. For example, configuring the app registry, your domain certificates and so on.

Furthermore, with the custom validations, and fast and deterministic execution, you can take advantage of faster feedback loops when creating, testing, and deploying templates.
ytt ‘s “overlay” functionality helps users manage the customization of complex software configuration by providing advanced configuration. Using an overlay, you can replace parts or whole of cf-for-k8s templates. For e.g. see remove-resource-requirements.yml, which reduces needs for matching resources, so cf-for-k8s can be installed on smaller environments.

Documentation

The main documentation page for cf-for-k8s contains a variety of resources to help get you started. You can find instructions on deploying CF, guidelines for contributors, and other helpful resources. We eagerly accept PRs if you have corrections, suggestions, etc.

Configuration options

Platform operators define their configuration using a cf-for-k8s “values” file. See the sample-cf-install-values.yml file as suggested by the deploy documentation.

• cf-for-k8s has been shown to run on multiple distributions of Kubernetes, including GKE, PKS, AKS, EKS, Minikube and Kind.
• Both Docker Hub and Google’s Container Registry can be used as the App Registry.

If there are any missing configuration options, we recommend you create a feature request issue in the cf-for-k8s repository. We would like to know more about your usecases.

Compatibility

Kubernetes 1.15 or 1.16

Known Issues

For a list of known issues with this release, please visit the issues page in the repository. A few notable issues are listed below,

• cf push does not stream app staging logs.
• Upgrading cf-for-k8s is not yet supported.
• Custom buildpacks and cf cli buildpacks related commands are not supported.

Feedback

We love feedback. Please file a GitHub issue for bugs, feature requests, or suggestions. Or reach out to us on Cloud Foundry Slack in #cf-for-k8s.

Coming Up Next

You can see our upcoming prioritized work in our CF Release Integration tracker project.

Resources

• cf-for-k8s repository.
• cf-for-k8s known issues.
• K14 tools - ytt, kapp, kbld
• KubeCF - Users also have the option of deploying Cloud Foundry on Kubernetes with KubeCF. Under the hood, KubeCF is based on cf-deployment and it uses cf-operator to deploy component bosh-releases. It uses eirini or diego to schedule the app workloads. It is now an independent project in runtime pmc and has reached 1.0 milestone.