diff --git a/README.md b/README.md index e728f9ac..61cdc7c0 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,29 @@ ![CI workflow](https://github.com/playfab/thundernetes/actions/workflows/main.yml/badge.svg) [![Software License](https://img.shields.io/badge/license-Apache-brightgreen.svg?style=flat-square)](LICENSE) [![GitHub release](https://img.shields.io/github/release/playfab/thundernetes.svg)](https://github.com/playfab/thundernetes/releases) -![](https://img.shields.io/badge/status-alpha-red.svg) +![](https://img.shields.io/badge/status-beta-lightgreen.svg) [![CodeQL](https://github.com/PlayFab/thundernetes/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/PlayFab/thundernetes/actions/workflows/codeql-analysis.yml) -# (code name) thundernetes +# Thundernetes -> Thundernetes is an experimental project and not recommended for production use. However, feel free to try it and let us know of any feedback! Thundernetes is a [code name](https://github.com/PlayFab/thundernetes/issues/177), project will soon have a new name! +Thundernetes makes it easy to run your game servers on Kubernetes. -## Description +> Thundernetes is a project still in development, feel free to try it and let us know of any [feedback](#feedback---community)! -Thundernetes is a project from the [Azure PlayFab Multiplayer Servers (MPS)](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/) team that enables you to run Linux game servers on your Kubernetes cluster. Thundernetes can be useful in the following scenarios: +## ℹ️ Description -- host your game servers on a Kubernetes cluster, either on a public cloud provider or on-premises -- do manual allocations of game server sessions -- validate your game server integration with GSDK -- as part of your iterative development process, you can use thundernetes to test your game server code before pushing it to the MPS service -- as part of your CI/CD pipeline, you can publish the game server to a container registry and then have it deploy to a Kubernetes cluster where you can run your tests +Thundernetes is a project originating from the [Azure PlayFab Multiplayer Servers (MPS)](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/) team and other teams in Azure/XBOX that enables you to run Linux game servers on your Kubernetes cluster. Thundernetes can be useful in the following scenarios: -## Usage +- host your game servers on a Kubernetes cluster, either on a public cloud provider or on-premise and allow your users to connect from everywhere +- pre-warm game servers so that they are ready to accept players within seconds, when the game is about to start +- as part of your iterative development process, you can use Thundernetes to test your game server code -Check [our website](https://playfab.github.io/thundernetes) for more information. +## 📚 Documentation +Check 🔥[our website](https://playfab.github.io/thundernetes)🔥 for more information. -## Feedback +## 💬❓Feedback - Community -As mentioned, thundernetes is in preview and a work in progress. If you find a bug or have a feature request, please file an issue [here](https://github.com/PlayFab/thundernetes/issues) and we will try to get back to you as soon as possible. You can also reach us directly on [Game Stack server on Discord](https://discord.gg/gamestack). +As mentioned, Thundernetes is in preview and a work in progress. If you find a bug or have a feature request, please file an issue [here](https://github.com/PlayFab/thundernetes/issues) and we will try to get back to you as soon as possible. You can also reach us directly on [Game Dev server on Discord](https://aka.ms/msftgamedevdiscord). +> Project will soon have a new name (https://github.com/PlayFab/thundernetes/issues/177)! \ No newline at end of file diff --git a/docs/FAQ.md b/docs/FAQ.md index 6c66614b..062dcec2 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -1,14 +1,14 @@ --- layout: default title: Frequently Asked Questions -nav_order: 11 +nav_order: 13 --- # Frequently Asked Questions -## Can I run a Unity or Unreal game server on thundernetes? +## Can I run a Unity or Unreal game server on Thundernetes? -You can run any game server that supports the [PlayFab GameServer SDK](https://github.com/PlayFab/gsdk). Check a Unity sample [here](https://github.com/PlayFab/thundernetes/tree/main/samples/unity/README.md). On [this](https://github.com/PlayFab/MpsSamples) repository you can find samples for all programming languages GSDK supports, like C#/Java/C++/Go/Unity/Unreal. +You can run any game server that supports the [PlayFab GameServer SDK](https://github.com/PlayFab/gsdk). Check a Unity sample [here](https://github.com/PlayFab/thundernetes/tree/main/samples/unity/README.md) and documentation for the Unreal plugin [here](https://github.com/PlayFab/gsdk/tree/main/UnrealPlugin). On [this](https://github.com/PlayFab/MpsSamples) repository you can find samples for all programming languages GSDK supports, like C#/Java/C++/Go/Unity/Unreal. ## How can I add custom Annotations and/or Labels to my GameServer Pods? @@ -45,62 +45,13 @@ spec: ## Can I run my game server pods in a non-default namespace? -You don't need to do anything special to run your game server Pods in a namespace different than "default". Old versions of thundernetes (up to 0.1) made use of a sidecar to access the Kubernetes API Server, so you needed to create special RoleBinding and ServiceAccount in the non-default namespace. With the transition to DaemonSet NodeAgent in 0.2, this is no longer necessary. - -## How do I schedule thundernetes Pods and GameServer Pods into different Nodes? - -In production environments, you would like to have system and thundernetes Pods (Pods that are created on the kube-system and thundernetes-system namespaces) scheduled on a different set Nodes other than the GameServer Pods. One reason for this might be that you want special Node types for your GameServers. For example, you might want to have dedicated Nodes with special GPUs for your GameServers. Another reason might be that you don't want any interruption whatsoever to Pods that are critical for the cluster to run properly (system and thundernetes Pods). One approach to achieve this isolation on public cloud providers is by using multiple Node Pools. A Node Pool is essentially a group of Nodes that share the same configuration (CPU type, memory, etc) and can be scaled independently of the others. In production scenarios, it is recommended to use three Node Pools: - -- one Node Pool for Kubernetes system resources (everything in kube-system namespace) and thundernetes system resources (everything in thundernetes-system namespace) -- one Node Pool for telemetry related Pods (Prometheus, Grafana, etc) -- one Node Pool to host your GameServer Pods - -Let's discuss on how to create and use a Node Pool to host the GameServer Pods. - -1. First, you would need to create a separate NodePool for the GameServer Pods. Check [here](https://docs.microsoft.com/azure/aks/use-multiple-node-pools) on how to do it on Azure Kubernetes Service. Create this on "user" mode so that "kube-system" Pods are not scheduled on this NodePool. Most importantly, when creating a NodePool, you can specify custom Labels for the Nodes. Let's assume that you apply the `agentpool=gameserver` Label. -1. Use the `nodeSelector` field on your GameServer Pod spec to request that the GameServer Pod is scheduled on Nodes that have the `agentpool=gameserver` Label. Take a look at this [sample YAML file](https://github.com/PlayFab/thundernetes/tree/main/samples/netcore/sample_second_node_pool.yaml) for an example. -1. When you create your GameServerBuild, the GameServer Pods will be scheduled on the NodePool you created. -1. Moreover, you should modify the `nodeSelector` field on the controller Pod spec to make sure it will be scheduled on the system Node Pool. On AKS, if the system Node Pool is called `nodepool1`, you should add this YAML snippet to the `thundernetes-controller-manager` Deployment on the [YAML install file](https://github.com/PlayFab/thundernetes/tree/main/installfiles/operator.yaml): - -```YAML -nodeSelector: - agentpool: nodepool1 -``` - -You should add the above YAML snippet to any workloads you don't want to be scheduled on the GameServer NodePool. Check [here](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/) for additional information on assigning pods to Nodes and check [here](https://docs.microsoft.com/azure/aks/use-system-pools#system-and-user-node-pools) for more information on AKS system and user node pools. - -### Schedule DaemonSet Pods on GameServer Nodes - -> For more information on the NodeAgent process running in the DaemonSet, check the architecture document [here](architecture.md#gsdk-integration). - -Now that we've shown how to run multiple Node Pools, let's discuss how to schedule DaemonSet Pods running NodeAgent process to run only on Nodes that run game server Pods. Since NodeAgent's only concern is to work with game server Pods on Node's it's been scheduled, it's unnecessary to run in on Nodes that run system resources and/or telemetry. Since we have already split the cluster into multiple Node Pools, we can use the `nodeSelector` field on the DaemonSet Pod spec to request that the DaemonSet Pod is scheduled on Nodes that have the `agentpool=gameserver` Label (or whatever Label you have added to your game server Node Pool). Take a look at the following example to see how you can modify your DaemonSet YAML for this purpose: - -```YAML -apiVersion: apps/v1 -kind: DaemonSet -metadata: - name: thundernetes-nodeagent - namespace: thundernetes-system -spec: - selector: - matchLabels: - name: nodeagent - template: - metadata: - labels: - name: nodeagent - spec: - nodeSelector: # add this line - agentpool: gameserver # add this line as well - containers: - ... -``` +Yes. You don't need to do anything special to run your game server Pods in a namespace different than "default". ## How do I make GameServer Pods start before DaemonSet Pods? When a new Node is added to the Kubernetes cluster, a NodeAgent Pod (part of DaemonSet) will be created there. However, if there were pending GameServer Pods before the Node's addition to the cluster, they will also be scheduled on the new Node. Consequently, GameServer Pods might start at the same time as the NodeAgent Pod. GameServer Pods are heartbeating to the NodeAgent process so there is a chance that some heartbeats will be lost and, potentially, a state change from "" to "Initializing" will not be tracked (however, the GameServer Pod should have no trouble transitioning to StandingBy when the NodeAgent Pod is up and can process heartbeats). -There will be no impact from these lost heartbeats. However, you can tell Kubernetes to schedule NodeAgent Pods before the GameServer Pods by assigning Pod Priorities to the NodeAgent Pods. You can read more about Pod priority [here](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption) and specifically about the impact of Pod priority on scheduling order [here](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#effect-of-pod-priority-on-scheduling-order). +There will be no impact from these lost heartbeats. However, you can ask Kubernetes to schedule NodeAgent Pods before the GameServer Pods by assigning Pod Priorities to the NodeAgent Pods. You can read more about Pod priority [here](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption) and specifically about the impact of Pod priority on scheduling order [here](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#effect-of-pod-priority-on-scheduling-order). ## How can I add resource constraints to my GameServer Pods? @@ -127,13 +78,10 @@ template: For a full sample, you can check [here](https://github.com/PlayFab/thundernetes/tree/main/samples/netcore/sample-requestslimits.yaml). -## Not supported features (compared to MPS) - -There are some features of MPS that are not yet supported on Thundernetes. +## How can I run custom code on my container to respond to a terminated event from Kubernetes? -1. Thundernetes, for the time being, supports only Linux game servers. Work to support Windows is tracked [here](https://github.com/PlayFab/thundernetes/issues/8), please leave a comment if that's important for you. If you want to host Windows game servers, you can always use [MPS](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/). -1. On PlayFab MPS, you can upload a zip file that contains parts of your game server (referred to as assets). This is decompressed on the VM that your game server runs and is automatically mounted. You cannot do that on Thundernetes, however you can always mount a storage volume onto your Pod (e.g. check [here](https://kubernetes.io/docs/concepts/storage/volumes/#azuredisk) on how to mount an Azure Disk). +You can use the [PreStop container hook](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks). Effectively, this hook is executed when the container is terminated. You can use this hook to perform custom cleanup operations. Read [here](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination) for more information about Pod termination. -## Where does the name 'thundernetes' come from? +## Where does the name 'Thundernetes' come from? It's a combination of the words 'thunderhead' and 'kubernetes'. 'Thunderhead' is the internal code name for the Azure PlayFab Multiplayer Servers service. Credits to [Andreas Pohl](https://github.com/Annonator) for the naming idea! diff --git a/docs/README.md b/docs/README.md index f5f09f62..1123ade1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,51 +8,44 @@ permalink: / # Thundernetes -Welcome to (code name) Thundernetes! Please use the links on the left to navigate the documentation. +Welcome to Thundernetes, an open source project from teams in Azure/XBOX that enables you to run Linux game servers on your Kubernetes cluster! -## Description +:exclamation: Latest release: [![GitHub release](https://img.shields.io/github/release/playfab/thundernetes.svg)](https://github.com/playfab/thundernetes/releases) -Thundernetes is a project from the Azure PlayFab Multiplayer Servers (MPS) team that enables you to run Linux game servers on your Kubernetes cluster. Thundernetes can be useful in the following scenarios: +## Prerequisite knowledge -- host your game servers on a Kubernetes cluster, either on a public cloud provider or on-premises -- do manual allocations of game server sessions -- validate your game server integration with GSDK -- as part of your iterative development process, you can use thundernetes to test your game server code before pushing it to the MPS service -- as part of your CI/CD pipeline, you can publish the game server to a container registry and then have it deploy to a Kubernetes cluster where you can run your tests -- For a game server to be able to run in thundernetes, it must use the PlayFab Game Server SDK (GSDK) either directly on the game server binary or indirectly, via a wrapper. +New to Kubernetes or containers? Check our [prerequisites](prerequisites.md) document that has resources that will fill the knowledge gaps when working with technologies within Thundernetes. -Thundernetes requires a Kubernetes cluster with Public IP per Node. We've tested it extensively on [Azure Kubernetes Service - AKS](https://docs.microsoft.com/azure/aks/intro-kubernetes) as well as in local clusters using [kind](https://kind.sigs.k8s.io/). You also need to have ports 10000-12000 open in your cluster, since these are the ports that Thundernetes by default will set up on your Kubernetes Nodes so they can receive game network traffic and forward to your game server Pod. +## Requirements -> This port range is configurable, check [here](howtos/configureportrange.md) for details. -> You can use a Kubernetes cluster without a Public IP. However, you would need to configure your own network architecture if you want to access your game servers. For example, if you use a cloud provider's Load Balancer, you would need to configure routes from Load Balancer's public endpoints to the internal ones on your Kubernetes cluster. -> You can try Azure Kubernetes Service for free [azure.com/free](https://azure.com/free). +Thundernetes requires: -## Prerequisites +- A Kubernetes cluster, either on-premise or on a public cloud provider. Ideally, the cluster should support having a Public IP per Node to allow external incoming connections +- A game server + - integrated with the open source [Game Server SDK](https://playfab.com/gsdk) (GSDK). GSDK has been battle-tested by multiple AAA titles for years on the [Azure PlayFab Multiplayer Servers service](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/) and supports multiple popular programming languages and game engines like Unity, Unreal, C#, C++, Java, Go. + - built as a Linux container image. This image should be deployed to a container registry that your Kubernetes cluster can access. -Check our [prerequisites](docs/prerequisites.md) document that has resources that will fill the knowledge gaps when working with technologies within thundernetes. +> **_NOTE_**: You can avoid having to integrate with GSDK by using the [wrapper sample](usingwrapper.md). This sample is great if you want to experiment with Thundernetes, however proper GSDK integration is highly recommended. ## Quickstart -Check the [quickstart](docs/quickstart.md) document on how to install thundernetes on your cluster and run the sample game server. +Check the [quickstart](quickstart.md) document on how to install Thundernetes on your cluster and run a sample game server to verify that Thundernetes is working properly. -### Installing on Azure Kubernetes Service +Click on the following image to see how easy it is to install and use Thundernetes: -Click on the following image for a quick preview of the quickstart: +[![asciicast](https://asciinema.org/a/438455.svg)](https://asciinema.org/a/438455) -[![asciicast](https://asciinema.org/a/438455.png)](https://asciinema.org/a/438455) +## Recommended links -## Links - -- [Prerequisites](prerequisites.md) - resources that will fill the knowledge gaps when working with technologies within thundernetes -- [Quickstart](quickstart.md) - Recommended - how to install thundernetes on your cluster and run the sample game server -- [Defining a GameServerBuild](gameserverbuild.md) - Recommended - how to define a GameServerBuild in YAML -- [Your game server](yourgameserver.md) - how to use thundernetes with your own game server +- [Using a wrapper](usingwrapper.md) - use a wrapper process to launch your game server in Thundernetes, without integrating with GSDK +- [Your game server](yourgameserver.md) - how to use Thundernetes with your own game server +- [Defining a GameServerBuild](gameserverbuild.md) - how to define a GameServerBuild in YAML - [Game Server lifecycle](gameserverlifecycle.md) - game server process lifecycle -- [Architecture](architecture.md) -- [Frequently Asked Questions](FAQ.md) -- [Troubleshooting Guide](troubleshooting/README.md) - public repository for all of thundernetes Troubleshooting guides -- [Development notes](development.md) - useful if you are working on thundernetes development +- [Architecture](architecture.md) - overview of Thundernetes architecture +- [Troubleshooting Guide](troubleshooting/README.md) - public repository for all of Thundernetes troubleshooting guides +- [Development notes](development.md) - useful development notes if you are plan on contributing to Thundernetes +- [Frequently Asked Questions](FAQ.md) - frequently asked questions ## Contributing -If you are interested in contributing to thundernetes, please read our [Contributing Guide](contributing.md) and open a PR. We'd be more than happy to help you out! +If you are interested in contributing to Thundernetes, please read our [Contributing Guide](contributing.md) and open a PR. We'd be more than happy to help you out! diff --git a/docs/_config.yml b/docs/_config.yml index f667ae73..ab22c13d 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -1,9 +1,16 @@ remote_theme: just-the-docs/just-the-docs +title: Thundernetes +description: Thundernetes makes it easy to host game servers on Kubernetes +baseurl: "/thundernetes" # the subpath of your site, e.g. /blog +url: "https://playfab.github.io" # the base hostname & protocol for your site, e.g. http://example.com + +# permalink: pretty +# exclude: ["node_modules/", "*.gemspec", "*.gem", "Gemfile", "Gemfile.lock", "package.json", "package-lock.json", "script/", "LICENSE.txt", "lib/", "bin/", "README.md", "Makefile", "docs/tests/"] + # Enable or disable the site search # Supports true (default) or false search_enabled: true - search: # Split pages into sections that can be searched individually # Supports 1 - 6, default: 2 @@ -32,6 +39,10 @@ search: # appears at the bottom of every page's main content footer_content: "Copyright © 2022 Microsoft. Distributed with Apache license." +# Back to top link +back_to_top: true +back_to_top_text: "Back to top" + # Footer last edited timestamp last_edit_timestamp: true # show or hide edit time - page must have `last_modified_date` defined in the frontmatter last_edit_time_format: "%b %e %Y at %I:%M %p" # uses ruby's time format: https://ruby-doc.org/stdlib-2.7.0/libdoc/time/rdoc/Time.html @@ -50,8 +61,35 @@ gh_edit_view_mode: "tree" # "tree" or "edit" if you want the user to jump into t # Supports true (default) or false heading_anchors: true +# Aux links for the upper right navigation +aux_links: + "Thundernetes on GitHub": + - "//github.com/playfab/thundernetes" + "Microsoft Game Dev on Discord": + - "https://aka.ms/msftgamedevdiscord" + +# Makes Aux links open in a new tab. Default is false +aux_links_new_tab: true # Google Analytics Tracking (optional) # e.g, UA-1234567-89 ga_tracking: G-1MY0MFCQR4 -ga_tracking_anonymize_ip: true # Use GDPR compliant Google Analytics settings (true by default) \ No newline at end of file +ga_tracking_anonymize_ip: true # Use GDPR compliant Google Analytics settings (true by default) + +plugins: + - jekyll-seo-tag + - jekyll-sitemap + - jemoji + +kramdown: + syntax_highlighter_opts: + block: + line_numbers: false + +compress_html: + clippings: all + comments: all + endings: all + startings: [] + blanklines: false + profile: false \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md index c2eb37ba..3b6c853b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,20 +1,20 @@ --- layout: default title: Architecture -nav_order: 7 +nav_order: 8 --- # Architecture -This document describes the architecture of thundernetes as well as various design notes that are relevant to its implementation. +This document describes the architecture of Thundernetes as well as various design notes that are relevant to its implementation. ![Architecture diagram](diagram.png) ## Goal -End goal is to create a utility that will allow game developers to host their GSDK enabled Linux game servers on a Kubernetes cluster, either on-premises or on a public cloud provider. This utility will enable GameServer auto-scaling and GameServer allocations. With the term "allocation" we mean that pre-warmed game servers can transition into an "Active" state so that they can have players connect to them. On a potential scale down, thundernetes will never take down Active game servers or Nodes that have Active game servers running. +End goal is to create a utility that will allow game developers to host their Game Server SDK ([gsdk](http://github.com/playfab/gsdk)) enabled Linux game servers on a Kubernetes cluster, either on-premise or on a public cloud provider. This utility will enable GameServer auto-scaling and GameServer allocations. With the term "allocation" we mean that pre-warmed game servers (we call this state "StandingBy") can transition into an "Active" state so that players (game clients) can connect to them. On a potential scale down, Thundernetes will never take down Active game servers or Nodes that have Active game servers running. -We are extending Kubernetes by creating an [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). This involves creating a custom [controller](https://kubernetes.io/docs/concepts/architecture/controller/) and some [Custom Resource Definitions (CRDs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). We are using the open source tool [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) to scaffold our operator files which use [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime). +We are extending Kubernetes by creating an [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/). This involves creating a custom [controller](https://kubernetes.io/docs/concepts/architecture/controller/) and some [Custom Resource Definitions (CRDs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) that describe various game server related entities to Kubernetes. We are using the open source tool [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) to scaffold our operator files which use [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime). Specifically, we have three core entities in our project, which are represented by the respective CRDs: @@ -24,13 +24,13 @@ Specifically, we have three core entities in our project, which are represented ## GSDK integration -We have created a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) which spawns Pods that run on all Nodes in the cluster (or in a subset of them, if the user configures the DaemonSet with NodeSelectors). The process running in the DaemonSet Pod is called NodeAgent. NodeAgent sets up a web server that receives all the GSDK calls from the GameServer Pods on the Node it runs and modifies the GameServer state accordingly. In essense, every game server process heartbeats (via GSDK) to the NodeAgent process on the same Node. NodeAgent is also responsible for "watching" (via a Kubernetes watch) the state of these GameServer objects, getting a notification when it changes. This is particularly useful to track when the GameServer has been allocated (its game state was transitioned to Active). +We have created a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) which spawns Pods that run on every Node in the cluster (or in a subset of them, if the user configures the DaemonSet with NodeSelectors). The process running in the DaemonSet Pod is called NodeAgent. NodeAgent sets up a web server that receives all the GSDK calls from the GameServer Pods on the Node it runs and modifies the GameServer state accordingly. In essense, every game server process heartbeats (via GSDK) to the NodeAgent process on the same Node. NodeAgent is also responsible for "watching" (via a Kubernetes watch) the state of these GameServer objects, getting a notification when it changes. This is particularly useful to track when the GameServer has been allocated (its game state was transitioned to Active). -GSDK is [open source](https://github.com/PlayFab/gsdk), it supports the most popular game programming languages and environments. You can check the GSDK docs [here](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/integrating-game-servers-with-gsdk) and find ready-to-use samples [here](https://github.com/PlayFab/MpsSamples). The GSDK is also used by game servers running in Azure PlayFab Multiplayer Servers (MPS), thus making the migration from thundernetes to MPS (and vice versa) pretty seamless. To verify GSDK integration on your game server process, we recommend using the [LocalMultiplayerAgent](https://github.com/PlayFab/MpsAgent) tool. +GSDK is [open source](https://github.com/PlayFab/gsdk), it supports the most popular game programming languages and environments. You can check the GSDK docs [here](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/integrating-game-servers-with-gsdk) and find ready-to-use samples [here](https://github.com/PlayFab/MpsSamples). The GSDK is also used by game servers running in Azure PlayFab Multiplayer Servers (MPS), thus making the migration from Thundernetes to MPS service (and vice versa) pretty seamless. To verify GSDK integration on your game server process, we recommend using the [LocalMultiplayerAgent](https://github.com/PlayFab/MpsAgent) tool, check [here](howtos/runlocalmultiplayeragent.md) for details. ### initcontainer -GSDK libraries need to read configuration from a file (GSDKConfig.json). In order to create this file and make it readable by the GameServer Pod, we have created a Kubernetes [Init Container](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that shares a volume mount with the GameServer container and will create the configuration file for it to read. Both the initcontainer and the GameServer container are part of the same Pod. +GSDK libraries need to read configuration from a file (GSDKConfig.json). In order to create this file and make it readable by the GameServer Pod, we have created a lightweight Kubernetes [Init Container](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that shares a volume mount with the GameServer container and will create the configuration file for it to read. Both the initcontainer and the GameServer container are part of the same Pod. ## End to end (e2e) testing @@ -50,17 +50,20 @@ If you currently have [Prometheus](https://prometheus.io) & [Grafana](https://gr ## Port allocation -Thundernetes requires ports in the range 10000-12000 to be open in the cluster (i.e. in the case of Azure Kubernetes Service, this port range must allow incoming traffic in the corresponding Network Security Group). This port range is configurable, check [here](howtos/configureportrange.md) for details. +Thundernetes requires a Kubernetes cluster with Public IP per Node. We've tested it extensively on [Azure Kubernetes Service - AKS](https://docs.microsoft.com/azure/aks/intro-kubernetes) as well as in local clusters using [kind](https://kind.sigs.k8s.io/). You also need to have ports 10000-12000 open in your cluster, since these are the ports that Thundernetes by default will set up on your Kubernetes Nodes so they can receive game network traffic and forward to your game server Pod. -Each GameServerBuild contains the *portsToExpose* field, which contains information about the port(s) that each GameServer listens to. When the GameServer Pod is created, each port in the *portsToExpose* field will be assigned a port in the (default) range 10000-12000 (let's call it an external port) via a PortRegistry mechanism in the thundernetes controller. Game clients can send traffic to this external port and this will be forwarded to the game server container port. Once the GameServer session ends, the port is returned back to the pool of available ports and may be re-used in the future. +> You can use a Kubernetes cluster without a Public IP. However, you would need to configure your own network architecture if you want to access your game servers. For example, if you use a cloud provider's Load Balancer, you would need to configure routes from Load Balancer's public endpoints to the internal ones on your Kubernetes cluster. Check [here](https://github.com/dgkanatsios/thundernetescontrib/tree/main/traefikingress) for an example controller with [traefik](https://github.com/traefik/traefik) ingress controller for HTTP-based game servers (e.g. WebGL). -> Each port that is allocated by the PortRegistry is assigned to HostPort field of the Pod's definition. The fact that Nodes in the cluster have a Public IP makes this port accessible outside the cluster. +Moreover, Thundernetes requires ports in the range 10000-12000 to be open in the cluster for external connections (i.e. in the case of Azure Kubernetes Service, this port range must allow incoming traffic in the corresponding Network Security Group). This port range is configurable, check [here](howtos/configureportrange.md) for details. + +Each GameServerBuild contains the *portsToExpose* field, which contains information about the port(s) that each GameServer listens to for incoming client connections. When the GameServer Pod is created, each port in the *portsToExpose* field will be assigned a port in the (default) range 10000-12000 (let's call it an external port) via a PortRegistry mechanism in the Thundernetes controller. Game clients can send traffic to this external port and this will be forwarded to the game server container port. Once the GameServer session ends, the port is returned back to the pool of available ports and may be re-used in the future. -Noteworthy is the fact that this port range is used for all GameServerBuilds and for all Nodes in the cluster. This way, thundernetes can support up to 40000/number_of_exposed_ports GameServers per cluster game servers, i.e. if your GameServer needs only a single port, you can have up to 40k GameServers in the cluster. +> Each port that is allocated by the PortRegistry is assigned to HostPort field of the Pod's definition. The fact that Nodes in the cluster have a Public IP makes this port accessible outside the cluster. +> Thundernetes supports `hostNetwork` networking for the GameServer Pods, if requested in the GameServerBuild Pod definition. ## GameServer allocation -When you allocate a GameServer so players can connect to it, thundernetes needs to do two things: +Make sure you familiarize yourself with the [GameServer lifecycle](gameserverlifecycle.md) document. When you allocate a GameServer so players can connect to it (transition it to the Active state), Thundernetes needs to do two things: - Find a GameServer instance for the requested GameServerBuild in the StandindBy state and update it to the Active state. - Inform the corresponding GameServer Pod (specifically, the NodeAgent that handles GSDK calls for that Pod) that the GameServer state is now Active. NodeAgent will pass this information to the GameServer container. The way that this is accomplished is the following: each GameServer process/container regularly heartbeats (sends a JSON HTTP request) to the NodeAgent. When the NodeAgent is notified that the GameServer state has transitioned to Active, it will respond send information about the change of state back to the GameServer container. @@ -74,4 +77,4 @@ For communicating with the NodeAgent, we eventually picked the first approach. T Moreover, when the user allocates a GameServer, we create an instance of the GameServerDetail custom resource which stores the InitialPlayers (if any). The GameServerDetail has a 1:1 relationship with the GameServer CR and share the same name. Moreover, GameServer is the owner of the GameServerDetail instance so both will be deleted, upon GameServer's deletion. The GameServerDetail also tracks the ConnectedPlayersCount and ConnectedPlayers names/IDs, which can be updated by the UpdateConnectedPlayers GSDK method. -Worth mentioning here is the fact that up to 0.1, the NodeAgent process was a sidecar container that lived inside the GameServer Pod. However, on version 0.2 we transitioned to a NodeAgent Pod that runs on all Nodes in the cluster. This was done to avoid the need for a sidecar container. +Worth mentioning here is the fact that up to 0.1, the NodeAgent process was a sidecar container that lived inside the GameServer Pod. However, on version 0.2 we transitioned to a NodeAgent Pod that runs on all Nodes in the cluster. This was done to avoid the need for a sidecar container and also made `hostNetwork` functionality available. diff --git a/docs/contributing.md b/docs/contributing.md index 5476ab19..80506922 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -1,3 +1,9 @@ +--- +layout: default +title: Contributing +nav_order: 11 +--- + # Contributing Guide Welcome! We are glad that you want to contribute to our project! 💖 @@ -22,7 +28,6 @@ bug report and let us know! * [Pull Request Workflow](#pull-request-workflow) * [Pull Request Checklist](#pull-request-checklist) * [Sign Your Commits](#sign-your-commits) - * [DCO](#dco) * [CLA](#cla) ## Ways to Contribute @@ -107,42 +112,10 @@ before you submit your code: * Any new or changed code include test additions or changes, as appropriate. * Documentation is updated as required by the scope of the change. * CHANGELOG.md is updated to reflect this change. -* Commits are signed with DCO * Our CLA has been signed ## Sign Your Commits -### DCO - -Licensing is important to open source projects. It provides some assurances that -the software will continue to be available based under the terms that the -author(s) desired. We require that contributors sign off on commits submitted to -our project's repositories. The [Developer Certificate of Origin -(DCO)](https://developercertificate.org/) is a way to certify that you wrote and -have the right to contribute the code you are submitting to the project. - -You sign-off by adding the following to your commit messages. Your sign-off must -match the git user and email associated with the commit. - -```text - This is my commit message - - Signed-off-by: Your Name -``` - -Git has a `-s` command line option to do this automatically: - -```text - git commit -s -m 'This is my commit message' -``` - -If you forgot to do this and have not yet pushed your changes to the remote -repository, you can amend your commit with the sign-off by running - -```text - git commit --amend -s -``` - ### CLA We require that contributors have signed our Contributor License Agreement (CLA). diff --git a/docs/development.md b/docs/development.md index a7b7f7c4..83768a92 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,21 +1,24 @@ --- layout: default title: Development -nav_order: 9 +nav_order: 12 --- # Development -## Release new thundernetes version +This document contains development notes and tips for working with Thundernetes source code. + +## Release new Thundernetes version This will require 2 PRs. -- Make sure you update `.version` file on the root of this repository with the new version +- Make sure you update `.versions` file on the root of this repository with the new version - Run `make clean` to ensure any cached artifacts of old builds are deleted. - Push and merge -- Run the GitHub Actions workflow [here](https://github.com/PlayFab/thundernetes/actions/workflows/publish.yml) to create the new images +- Manually run the GitHub Actions workflow [here](https://github.com/PlayFab/thundernetes/actions/workflows/publish.yml) to create the new images +- Git pull the latest changes from the main branch - Run `make create-install-files` to generate the operator install files -- Replace the image on the [netcore-sample YAML files](../samples/netcore) +- Replace the image on the [netcore-sample YAML files](https://github.com/PlayFab/thundernetes/samples/netcore) - Push and merge ## Metrics @@ -25,7 +28,7 @@ This will require 2 PRs. ## Running end to end tests on macOS -First of all, end to end tests require `envsubst` utility, assuming that you have Homebrew installed you can get it via `brew install gettext &&brew link --force gettext`. +First of all, end to end tests require `envsubst` utility, assuming that you have Homebrew installed you can get it via `brew install gettext && brew link --force gettext`. We assume that you have installed Go, then you should install kind with `go install sigs.k8s.io/kind@latest`. Kind will be installed in `$(go env GOPATH)/bin` directory. Then, you should move kind to the `/operator/testbin/bin/` folder with a command like `cp $(go env GOPATH)/bin/kind ./operator/testbin/bin/kind`. You can run end to end tests with `make builddockerlocal createkindcluster e2elocal`. ## Various scripts @@ -271,4 +274,5 @@ To test your changes to thundernetes to a Kubernetes cluster, you can use the fo - Run `create-install-files-dev` to create the install files for the cluster - Checkout the `installfilesdev` folder for the generated install files. This file is included in .gitignore so it will never be committed. - Test your changes as required. +- single command: `NS=docker.io/dgkanatsios/ make clean build push create-install-files-dev` \ No newline at end of file diff --git a/docs/gameserverbuild.md b/docs/gameserverbuild.md index 5109a0d8..749f01f3 100755 --- a/docs/gameserverbuild.md +++ b/docs/gameserverbuild.md @@ -1,16 +1,16 @@ --- layout: default -title: GameServerBuild -nav_order: 6 +title: GameServerBuild definition +nav_order: 7 --- # GameServerBuild definition -GameServerBuild defines the amount and the specification of the GameServers that you want to run in the cluster. +GameServerBuild defines the specification and auto-scaling configuration of the GameServers that you want to run in the cluster. Each version of your game server should have its own GameServerBuild. -> A GameServerBuild is equivalent to a Build region in MPS. GameServer containers that work in thundernetes should work in a similar way on PlayFab Multiplayer Servers service. +> A GameServerBuild is equivalent to a Build region in MPS. GameServer containers that work in Thundernetes should work in a similar way on PlayFab Multiplayer Servers service. -Here you can see the YAML that can be used to create a GameServerBuild in thundernetes. The only fields that you should change after the GameServerBuild is created are the *standingBy* and the *max* ones. The other fields should be considered immutable. +Here you can see the YAML that can be used to create a GameServerBuild in Thundernetes. The only fields that you should change after the GameServerBuild is created are the *standingBy* and the *max* ones. The other fields should be considered immutable. ```yaml apiVersion: mps.playfab.com/v1alpha1 @@ -18,35 +18,39 @@ kind: GameServerBuild metadata: name: gameserverbuild-sample # required, name of the GameServerBuild spec: - titleID: "1E03" # required, corresponds to the PlayFab TitleID your game server is using. Can be an arbitrary string - buildID: "85ffe8da-c82f-4035-86c5-9d2b5f42d6f5" # required, build ID of your game, must be GUID. Will be used for allocations + titleID: "1E03" # required, corresponds to a unique ID for your game. Can be an arbitrary string + buildID: "85ffe8da-c82f-4035-86c5-9d2b5f42d6f5" # required, build ID of your game, must be GUID. Will be used for allocations, must be unique for each Build/version of your game server standingBy: 2 # required, number of standing by servers to create - max: 4 # reqired, max number of servers to create. Active+StandingBy servers will never be larger than max + max: 4 # required, max number of servers to create. Sum of active+standingBy servers will never be larger than max crashesToMarkUnhealthy: 5 # optional, default is 5. It is the number of crashes needed to mark the GameServerBuild unhealthy. Once this happens, no other operation will take place buildMetadata: # optional. Retrievable via GSDK, used to customize your game server - key: "buildMetadataKey1" value: "buildMetadataValue1" - - key: "buildMetadataKey1" - value: "buildMetadataValue1" + - key: "buildMetadataKey2" + value: "buildMetadataValue2" portsToExpose: # port names that you need to expose for your game server, read more below - - containerName: gameserver-sample # name of the container that you want its port exposed - portName: gameport # name of the port that you want to expose + - containerName: gameserver-sample # name of the container that you want its port exposed. Must be the same as containers.containerName + portName: gameport # name of the port that you want to expose. Must be the same as containers.ports.name template: spec: containers: - - image: youGameServerImage:tag # image of your game server - name: gameserver-sample # name of the container. Must be the same as portsToExpose.containerName + - image: registryrepo/youGameServerImage:tag # image of your game server + name: gameserver-sample # name of the container. ports: - containerPort: 7777 # port that you want to expose - name: gameport # name of the port that you want to expose. Must be the same as portsToExpose.portName + name: gameport # name of the port that you want to expose. ``` -The template.spec contains the definition for a [Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/). As a result, you should include here whatever is needed for your game server (environment variables, storage, etc) to run. +The template.spec contains the definition for a [Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/). As a result, you should include here whatever is needed for your game server to run (environment variables, storage, etc). ## PortsToExpose -This is a list of containerName/portName tuples: These are the ports that you want to be exposed in the [Worker Node/VM](https://kubernetes.io/docs/concepts/architecture/nodes/) when the Pod is created. The way this works is that each Pod you create will have >=1 number of containers. There, each container will have its own *Ports* definition. If a port in this definition is included in the *portsToExpose* array, this port will be publicly exposed in the Node/VM. This is accomplished by the setting of a **hostPort** value for each of the container ports you want to expose. The reason we need this is that because i) you may want to use some ports on your Pod containers for other purposes rather than players connecting to it and ii) a portName must be unique within a container. Ports assigned are in the port range 10000-12000 by default. This port range is configurable, check [here](howtos/configureportrange.md) for details. +This is a list of containerName/portName tuples: These are the ports that you want to be exposed in the [Worker Node/VM](https://kubernetes.io/docs/concepts/architecture/nodes/) when the Pod is created. The way this works is that each Pod you create will have >=1 number of containers. There, each container will have its own *Ports* definition. If a port in this definition is included in the *portsToExpose* array, this port will be publicly exposed in the Node/VM. This is accomplished by the setting of a **hostPort** value for each of the container ports you want to expose. + +The reasons we need this functionality are: +i) you may want to use some ports on your Pod containers for other purposes rather than players connecting to it +ii) a portName must be unique within a container. Ports assigned are in the port range 10000-12000 by default. This port range is configurable, check [here](howtos/configureportrange.md) for details. ## CrashesToMarkUnhealthy -CrashesToMarkUnhealthy (integet) is the number of crashes that you want to trigger the GameServerBuild to become Unhealthy. Once this happens, no other operation will take place on the GameServerBuild. To allow thundernets to continue performing reconciliations on the GameServerBuild after it has become Unhealthy, you can increase the value of the CrashesToMarkUnhealthy field. The GameServerBuild will be marked as Healthy again till the number of crashes reaches the value of CrashesToMarkUnhealthy. +CrashesToMarkUnhealthy (integer) is the number of crashes that you want to trigger the GameServerBuild to become Unhealthy. Once this happens, no other operation will take place on the GameServerBuild. To allow Thundernetes to continue performing reconciliations on the GameServerBuild after it has become Unhealthy, you can increase the value of the CrashesToMarkUnhealthy field. The GameServerBuild will be marked as Healthy again till the number of crashes reaches the value of CrashesToMarkUnhealthy. diff --git a/docs/gameserverlifecycle.md b/docs/gameserverlifecycle.md index fd1ffed8..57252bd7 100755 --- a/docs/gameserverlifecycle.md +++ b/docs/gameserverlifecycle.md @@ -1,37 +1,39 @@ --- layout: default title: Game Server lifecycle -nav_order: 5 +nav_order: 6 --- # Game Server lifecycle -This document contains information about GSDK and how it affects the game server lifecycle on thundernetes. +This document contains information about GSDK and how it affects the game server lifecycle on Thundernetes. ## Game Server SDK (GSDK) -Game server lifecycle is dependent on the [GSDK](https://github.com/PlayFab/gsdk) calls as well as the state of the game server process, running in the thundernetes Pod. For more information on integrating your game server with GSDK, check the docs [here](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/integrating-game-servers-with-gsdk). +Game server lifecycle is dependent on the [GSDK](https://github.com/PlayFab/gsdk) calls as well as the state of the game server process. For more information on integrating your game server with GSDK, check the docs [here](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/integrating-game-servers-with-gsdk). -Your game server can integrate GSDK and run great on thundernetes just by calling the **ReadyForPlayers** method. However, we recommend calling **Start** as well when your game server process starts so thundernetes is aware that the game server is initializing. +Your game server can integrate GSDK and run great on Thundernetes just by calling the **ReadyForPlayers** method. However, we recommend calling **Start** as well when your game server process starts so thundernetes is aware that the game server is initializing. -## Verifying GSDK integration with your game server +## States of the game server -MPS has a developer tool to test GSDK integration for game servers. Tool is called **LocalMultiplayerAgent**, you can download it [here](https://github.com/PlayFab/MpsAgent) and you can check the docs [here](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/locally-debugging-game-servers-and-integration-with-playfab#using-localmultiplayeragent-with-linux-containers) for more information. +You can view the states of your GameServers by typing `kubectl get gs`. The states are: -## GSDK samples +- **Empty**: when the GameServer is created, the status is empty since we are waiting for the game server process to start and call the necessary GSDK methods. +- **Initializing**: GameServer transitions to this state when it calls the **Start()** GSDK method. In this state, the game server should be starting to load the necessary assets. +- **StandingBy**: GameServer transitions to this state when it calls the **ReadyForPlayers()** GSDK method. This state implies that the GameServer has loaded all the necessary assets and its ready for allocation. +- **Active**: GameServer transitions to this state when it is [allocated](quickstart.md#allocate-a-game-server) by an external call to the allocation API service. Usually it's the responsibility of your matchmaker or lobby service to make this API call. This state implies that players can connect to the game server. When the server is in this state, it can never go back to the **Initializing** or **StandingBy** state. +- **Terminated**: GameServer process can reach this state by terminating, either gracefully or via a crash. Thundernetes monitors all containers in the Pod you specify and will consider a termination of any of them as the termination of the game server. This can happen at any GameServer state. When this happens, Thundernetes will remove the Pod running this GameServer and will create a new one in its place, which will start from the **Empty** state. -To check GSDK samples demonstrating integration with popular game engines, you can check the MpsSamples repository [here](https://github.com/PlayFab/MpsSamples). +GameServer Pods in the Initializing or StandingBy state can be taken down during a cluster scale-down. Thundernetes makes every effort to prevent Active GameServer Pods from being taken down, since this would have the undesirable effect of breaking an existing game. Moreover, as mentioned, GameServer can never transition back to StandingBy state from Active state. The only way to get a new game server in StandingBy state is if the GameServer process exits. You should gracefully exit your game server process when the game session is done and the last connected player has exited the game. -## States of the game server +> If the game server process crashes for more than `crashesToMarkUnhealthy` times (specified in the GameServerBuild spec, default value 5), then no more operations will be performed on the GameServerBuild. -You can view the states of your GameServers by typing `kubectl get gs`. The states are: +> User is responsible for collecting logs while the Pod is running. Check [here](howtos/gameserverlogs.md) on how to do this. -- Empty: when the GameServer is starting, the status is empty -- **Initializing**: GameServer transitions to this state when it calls the **Start()** GSDK method. In this state, the game server is starting to load the necessary assets. -- **StandingBy**: GameServer transitions to this state when it calls the **ReadyForPlayers()** GSDK method. This state implies that the GameServer has loaded all the necessary assets and its ready for allocation. -- **Active**: GameServer transitions to this state when it is [allocated](quickstart.md#allocate-a-game-server). This state implies that players can connect to the game server. +## Verifying GSDK integration with your game server -GameServer process can terminate either gracefully or via a crash. This can happen at any state. Thundernetes will remove the Pod running this GameServer and will create a new one in its place. User is responsible for collecting logs while the Pod is running. Check [here](howtos/gameserverlogs.md) on how to do this. +You can use the open source tool **LocalMultiplayerAgent** to test GSDK integration and maybe test the game server in your local environment. You can download it [here](https://github.com/PlayFab/MpsAgent) and you can check the docs [here](howtos/runlocalmultiplayeragent.md) for more information. + +## GSDK samples -> GameServer can never transition to StandingBy state from Active state. The only way to get a new game server in StandingBy state is if the GameServer process exits. -> If the game server process crashes for more than `crashesToMarkUnhealthy` times (specified in the GameServerBuild spec, default value 5), then no more operations will be performed on the GameServerBuild. \ No newline at end of file +To check GSDK samples demonstrating integration with popular game engines, you can check the MpsSamples repository [here](https://github.com/PlayFab/MpsSamples). \ No newline at end of file diff --git a/docs/howtos/README.md b/docs/howtos/README.md index 650c8394..971daddf 100644 --- a/docs/howtos/README.md +++ b/docs/howtos/README.md @@ -1,7 +1,7 @@ --- layout: default title: How to's -nav_order: 8 +nav_order: 9 has_children: true --- diff --git a/docs/howtos/clusterautoscaling.md b/docs/howtos/clusterautoscaling.md index b6362441..665741e5 100644 --- a/docs/howtos/clusterautoscaling.md +++ b/docs/howtos/clusterautoscaling.md @@ -7,4 +7,10 @@ nav_order: 5 # Cluster Autoscaling -Thundernetes natively supports GameServer autoscaling via its standingBy/max mechanism. However, scaling Pods is just one part of the process. The other part is about scaling the Kubernetes Nodes in the cluster. For Node autoscaling, thundernetes can work with the open source [Kubernetes cluster autoscaler](https://github.com/kubernetes/autoscaler). We also recommend using the [overprovisioning feature](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler) so you can spin up Nodes as soon as possible. Each cloud provider has its own documentation for using the cluster autoscaler. If you are using Azure Kubernetes Service, you can easily enable cluster autoscaler using the documentation [here](https://docs.microsoft.com/azure/aks/cluster-autoscaler). +Thundernetes natively supports GameServer autoscaling via its standingBy/max mechanism. However, scaling Pods is just one part of the process. The other part is about scaling the Kubernetes Nodes in the cluster. The challenges with Node autoscaling are mostly with scaling down. Nodes that have at least one GameServer in the Active state (where players are connected to the GameServer and playing the game) should not be removed. However, Nodes that all their GameServers are on Initializing or StandingBy state are eligible to be removed. We should also keep in mind that multiplayer game sessions are usually short lived, so Active game servers will eventually be terminated and new game servers in Initializing or StandingBy state will be created. + +For Node autoscaling, Thundernetes can work with the open source [Kubernetes Cluster Autoscaler](https://github.com/kubernetes/autoscaler). To let Cluster Autoscaler be aware of the state of the Pods, Thundernetes adds the `[safe-to-evict=false](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-types-of-pods-can-prevent-ca-from-removing-a-node)` annotation to Active game server Pods and `safe-to-evict=true` to Pods in the Initializing or StandingBy state. + +We also recommend using the [overprovisioning feature](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler) so you can spin up Nodes as soon as possible. Since the Cluster Autoscaler will create a new Node only when there are Pods in the Pending state and a new Node addition might take a couple of minutes, it might be desirable to have some Pods that just reserve resources with a negative PriorityClass, so these are the first ones that will end up on a Pending State. + +Each cloud provider has its own documentation for using the cluster autoscaler. If you are using Azure Kubernetes Service, you can easily enable cluster autoscaler using the documentation [here](https://docs.microsoft.com/azure/aks/cluster-autoscaler). diff --git a/docs/howtos/configureportrange.md b/docs/howtos/configureportrange.md index efc50595..322c0258 100644 --- a/docs/howtos/configureportrange.md +++ b/docs/howtos/configureportrange.md @@ -7,4 +7,8 @@ nav_order: 7 # Configure Thundernetes port range -By default, Thundernetes will allocate ports in the range 10000-12000 to your GameServers. These ports are allocated to the entire set of VMs in the cluster and are open for each and every VM. If you need more or just a different port range, you can configure it via changing the `MIN_PORT` and the `MAX_PORT` environment variables in the controller deployment YAML file. However, do not modify the port range when there game servers running on the cluster, since this will probably corrupt the port registry, especially if the new and the old range are different. \ No newline at end of file +By default, Thundernetes will allocate ports in the range 10000-12000 to your GameServers. These ports are allocated to the entire set of VMs in the cluster and are open for each and every VM. If you need more or just a different port range, you can configure it via changing the `MIN_PORT` and the `MAX_PORT` environment variables in the controller deployment YAML file. + +Additionally, since by default Thundernetes requires each Node in the cluster to have a Public IP, you would need to allow external traffic on this port range on your cluster. For instructions on how to do this, check your cloud provider's documentation. For Azure, you would need to open the port range in the Azure Network Security Group of your cluster. + +> _**IMPORTANT**_: Do not modify the port range when there game servers running on the cluster, since this will probably corrupt the port registry, especially if the new and the old range are different. \ No newline at end of file diff --git a/docs/howtos/gameserverlogs.md b/docs/howtos/gameserverlogs.md index a5e972d2..3c0f23b6 100644 --- a/docs/howtos/gameserverlogs.md +++ b/docs/howtos/gameserverlogs.md @@ -1,17 +1,17 @@ --- layout: default -title: game server logging +title: Game server logging parent: How to's nav_order: 4 --- -# Get GameServer logs +# Game server logging -Thundernetes does not do anything special to obtain the logs for your GameServer Pods, since there already are a lot of solutions in the Kubernetes ecosystem for this purpose. One of easiest ways to do this is to use [fluentbit](https://fluentbit.io/) to capture logs and send them to [Azure Blob Storage](https://docs.microsoft.com/azure/storage/blobs/storage-blobs-overview) or on a Storage provide of your choice (you can see output providers for fluentbit [here](https://docs.fluentbit.io/manual/pipeline/outputs)). +Thundernetes does not do anything special to obtain the logs for your GameServer Pods, since there already are a lot of existing solutions in the Kubernetes ecosystem. One easy ways to accomplish this is to use [fluentbit](https://fluentbit.io/) to capture logs and send them to [Azure Blob Storage](https://docs.microsoft.com/azure/storage/blobs/storage-blobs-overview) or on a Storage provide of your choice. Fluentbit supports multiple output providers, you can can see them [here](https://docs.fluentbit.io/manual/pipeline/outputs). You can use the following steps to setup fluentbit to capture logs from your GameServer Pods and send them to Azure Storage: - Set up an Azure Storage Account. Check [here](https://docs.microsoft.com/azure/storage/common/storage-account-create?tabs=azure-portal) on how to do it using the Azure Portal. -- Install fluentbit on your Kubernetes cluster. Check [here](https://docs.fluentbit.io/manual/installation/kubernetes) on how to do it using the Azure Portal. -- As soon as you create the namespace and roles/role bindings, you should create the fluentbit ConfigMap containing the fluentbit configuration file. You can see a sample [here](../samples/fluentbit/fluent-bit-configmap.yaml). Remember to replace the values with your Azure Storage Account name and key. -- Finally, you should create the fluentbit DaemonSet, so a fluentbit Pod runs on every Node in your cluster and grabs the logs. You can find a sample [here](../samples/fluentbit/fluent-bit-ds.yaml). \ No newline at end of file +- Install fluentbit on your Kubernetes cluster. Check [here](https://docs.fluentbit.io/manual/installation/kubernetes) on how to do it. +- As soon as you create the namespace and roles/role bindings, you should create the fluentbit ConfigMap containing the fluentbit configuration file. You can see a sample [here](https://github.com/PlayFab/thundernetes/blob/main/samples/fluentbit/fluent-bit-configmap.yaml). Remember to replace the values with your Azure Storage Account name and key. +- Finally, you should create the fluentbit DaemonSet, so a fluentbit Pod runs on every Node in your cluster and grabs the logs. You can find a sample [here](https://github.com/PlayFab/thundernetes/blob/main/samples/fluentbit/fluent-bit-ds.yaml). \ No newline at end of file diff --git a/docs/howtos/nodepools.md b/docs/howtos/nodepools.md index a86be8e9..43f53886 100644 --- a/docs/howtos/nodepools.md +++ b/docs/howtos/nodepools.md @@ -1,19 +1,69 @@ --- layout: default -title: separate node pools +title: Separate node pools parent: How to's nav_order: 2 --- -# How do I schedule thundernetes Pods and GameServer Pods into different Nodes? +# How do I schedule Thundernetes Pods and GameServer Pods into different Node pools/groups? -There might be cases in which you would like to have system and operator Pods (Pods that are created on the kube-system and thundernetes-system namespaces) and your GameServer Pods scheduled on different Nodes. One reason for this might be that you want special Node types for your GameServers. For example, you might want to have a dedicated Node for your GameServers that are dependent on a special GPU. Another reason might be that you don't want any interruption whatsoever to Pods that are critical for the cluster to run properly. One approach to achieve this isolation is: +In production environments, it would be ideal to have system and Thundernetes Pods (Pods that are created on the kube-system and thundernetes-system namespaces) scheduled on a different set of Nodes other than the GameServer Pods. +- One reason for this might be that you want special Node types for your GameServers. For example, you might want to have dedicated Nodes with special GPUs for your GameServers or just bigger (in terms of CPU/RAM). +- Moreover, you might not want any interruption whatsoever to Pods that are critical for the cluster to run properly (system and thundernetes Pods). +- Public IP per Node feature (required for Thundernetes game servers) can only be set on the Node pool that the GameServer Pods are scheduled on, thus making the system pods more secure. +- Last but not least, you might want to scale the GameServer Nodes independently of the system Nodes. + +One approach to achieve this isolation on public cloud providers is by using multiple Node Pools/Groups. A Node Pool is essentially a group of Nodes that share the same configuration (CPU type, memory, etc) and can be scaled independently of the others. In production scenarios, it is recommended to use three Node Pools: + +- one Node Pool for Kubernetes system resources (everything in kube-system namespace) and Thundernetes system resources (everything in thundernetes-system namespace) +- one Node Pool for telemetry related Pods (Prometheus, Grafana, etc) +- one Node Pool to host your GameServer Pods + +For example, on Azure Kubernetes Service you can use the following guidelines: + +1. Create a separate NodePool to host your GameServer Pods. Check [here](https://docs.microsoft.com/azure/aks/use-multiple-node-pools) on how to do it on Azure Kubernetes Service. Create this on "user" mode so that "kube-system" Pods are not scheduled on this NodePool. Moreover, when creating a NodePool, you can specify custom Labels for the Nodes. Let's assume that you apply the ```agentpool=gameserver``` Label. +2. Use the ```nodeSelector``` field on your GameServer Pod spec to request that the GameServer Pod is scheduled on Nodes that have the ```agentpool=gameserver``` Label. Take a look at this [sample YAML file](https://github.com/PlayFab/thundernetes/tree/main/samples/netcore/sample_second_node_pool.yaml) for an example. +3. When you create your GameServer Pods, these will be scheduled on the NodePool you created. +4. You should also modify the ```nodeSelector``` field on the controller Pod spec to make it will be scheduled on the system Node Pool. On AKS, if the NodePool is called ```nodepool1```, you should add this YAML snippet to the ```thundernetes-controller-manager``` Deployment on the [YAML install file](https://github.com/PlayFab/thundernetes/tree/main/installfiles/operator.yaml): -1. Create a separate NodePool to host your GameServer Pods. Check here on how to do it on Azure Kubernetes Service. Create this on "user" mode so that "kube-system" Pods are not scheduled on this NodePool. When creating a NodePool, you can specify custom Labels for the Nodes. Let's assume that you apply the ```agentpool=gameserver``` Label. -2. Use the ```nodeSelector``` field on your GameServer Pod spec to request that the GameServer Pod is scheduled on Nodes that have the ```agentpool=gameserver``` Label. Take a look at this sample YAML file for an example. -3. When you create your GameServer Pods, those will be scheduled on the NodePool you created. -4. You should also modify the ```nodeSelector``` field on the controller Pod spec to make it will be scheduled on the system Node Pool. On AKS, if the NodePool is called nodepool1, you should add this YAML snippet to the ```thundernetes-controller-manager``` Deployment on the [YAML install file](https://github.com/PlayFab/thundernetes/tree/main/installfiles/operator.yaml): ```nodeSelector: - agentpool: nodepool1 + agentpool: nodepool1 +``` + +You should add the above YAML snippet to any workloads you don't want to be scheduled on the GameServer NodePool. Check [here](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/) for additional information on assigning pods to Nodes and check [here](https://docs.microsoft.com/azure/aks/use-system-pools#system-and-user-node-pools) for more information on AKS system and user node pools. + +## Schedule DaemonSet Pods on GameServer Nodes + +Now that we've shown how to run multiple Node Pools, let's discuss how to schedule DaemonSet Pods running NodeAgent process to run only on Nodes that run game server Pods. NodeAgent's goal is to listen and respond to heartbeats from the gameservers and notify Kubernetes API server for any updates on their status. Since each NodeAgent instance has to communicate with game server Pods on the Node it's been scheduled, it's unnecessary to run it on Nodes that run system resources and/or telemetry. + +Since we have already split the cluster into multiple Node Pools, we can use the `nodeSelector` field on the DaemonSet Pod spec to request that the DaemonSet Pod is scheduled on Nodes that have the `agentpool=gameserver` Label (or whatever Label you have added to your game server Node Pool). Take a look at the following example to see how you can modify your DaemonSet YAML for this purpose: + +```YAML +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: thundernetes-nodeagent + namespace: thundernetes-system +spec: + selector: + matchLabels: + name: nodeagent + template: + metadata: + labels: + name: nodeagent + spec: + nodeSelector: # add this line + agentpool: gameserver # add this line as well + containers: + ... ``` -You should add this YAML snippet to any workloads you don't want to be scheduled on the GameServer NodePool. Check here for additional information on assigning pods to Nodes and check here for more information on AKS system and user node pools. \ No newline at end of file + +> For more information on the NodeAgent process running in the DaemonSet, check the architecture document [here](../architecture.md#gsdk-integration). + +## Limit the number of available ports + +Thundernetes needs to provide dedicated port numbers for the GameServer ports that you need exposed to the internet (i.e. the ports that game clients will connect to). For each VM, Thundernetes provides a single port in the range of 10000-12000. By default, the PortRegistry mechanism inside Thundernetes will allocate a set of 10000-12000 range for every VM in the cluster. However, if you are using a dedicated Node Pool for your GameServers, you'd need to specify this to the Thundernetes controller to avoid assigning ports for more VMs that you can have and lead to pending Pods. There are two steps you need to to perform this: +- Label the GameServer Nodes with the ```mps.playfab.com/gameservernode=true``` Label. +- Controller YAML deployment needs to be updated by adding the ```PORT_REGISTRY_EXCLUSIVELY_GAMESERVER_NODES``` environment variable with the value of ```"true"```. + diff --git a/docs/howtos/publicipaddress.md b/docs/howtos/publicipaddress.md index 1f48dda6..923d535b 100644 --- a/docs/howtos/publicipaddress.md +++ b/docs/howtos/publicipaddress.md @@ -1,15 +1,15 @@ --- layout: default -title: get public IP address +title: Get public IP address from inside a GameServer parent: How to's nav_order: 1 --- # How can I find the Public IP address from inside a GameServer? -External code can easily get the Public IP for each game server by querying the Kubernetes API, e.g. you can easily do `kubectl get gs` and you will get IP:port for all your game servers. However, what if you want to find the Public IP from inside a GameServer process? +External code (e.g. your matchmaker or lobby service) can easily get the Public IP for each game server by querying the Kubernetes API, e.g. you can easily do `kubectl get gs` and you will get IP:port for all your game servers. However, what if you want to find the Public IP from the code in your GameServer process? -You can easily get the Public IP address by using one of the following web sites from your game server: +You can easily get the Public IP address by calling one of the following web sites from inside your game server: ``` curl http://canhazip.com @@ -26,4 +26,4 @@ curl bot.whatismyipaddress.com wget -q -O - checkip.dyndns.org | sed -e 's/[^[:digit:]\|.]//g' ``` -The above methods work since the Node hosting your Pod has a Public IP. \ No newline at end of file +The above methods will work since the Node hosting your Pod has a Public IP. \ No newline at end of file diff --git a/docs/howtos/runlocalmultiplayeragent.md b/docs/howtos/runlocalmultiplayeragent.md index 6d87a168..4dd4478c 100644 --- a/docs/howtos/runlocalmultiplayeragent.md +++ b/docs/howtos/runlocalmultiplayeragent.md @@ -1,10 +1,84 @@ --- layout: default -title: running LocalMultiplayerAgent +title: Using LocalMultiplayerAgent parent: How to's nav_order: 8 --- -# Verifying GSDK integration with LocalMultiplayerAgent +# Using LocalMultiplayerAgent -To verify the GSDK integration with your game server, you can use the LocalMultiplayerAgent utility. This utility can be found [here](https://github.com/PlayFab/MpsAgent). To use it with your game server, check the instructions [here](https://docs.microsoft.com/en-us/gaming/playfab/features/multiplayer/servers/locally-debugging-game-servers-and-integration-with-playfab#using-localmultiplayeragent-with-linux-containers). \ No newline at end of file +LocalMultiplayerAgent is an open source utility from the Azure PlayFab MPS team that can be used to test game server integration with GSDK and potentially local debugging of a game server. Since MPS and Thundernetes game server container images are identical, the same tool can be used as you develop your game server for Thundernetes. LocalMultiplayerAgent works on Windows operating system. + +## Setting up LocalMultiplayerAgent + +To run LocalMultiplayerAgent to test your game servers for Thundernetes, you'll need to perform the following steps: + +- Download latest version of LocalMultiplayerAgent from the [Releases](https://github.com/PlayFab/MpsAgent/releases) page on GitHub +- [Install Docker Desktop on Windows](https://docs.docker.com/docker-for-windows/install/) +- Make sure it's running on [Linux Containers](https://docs.docker.com/docker-for-windows/#switch-between-windows-and-linux-containers) +- Your game server image can be can be locally built or published on a container registry. +- You should run `SetupLinuxContainersOnWindows.ps1` Powershell file which will create a Docker network called "PlayFab". This is necessary so that your containers running on a separate network namespace can communicate with LocalMultiplayerAgent, running on the host's network namespace. +- You should properly configure your *LocalMultiplayerSettings.json* file. Below you can see a sample, included in `MultiplayerSettingsLinuxContainersOnWindowsSample.json`: + +```json +{ + "RunContainer": true, + "OutputFolder": "C:\\output\\UnityServerLinux", + "NumHeartBeatsForActivateResponse": 10, + "NumHeartBeatsForTerminateResponse": 60, + "TitleId": "", + "BuildId": "00000000-0000-0000-0000-000000000000", + "Region": "WestUs", + "AgentListeningPort": 56001, + "ContainerStartParameters": { + "ImageDetails": { + "Registry": "mydockerregistry.io", + "ImageName": "mygame", + "ImageTag": "0.1", + "Username": "", + "Password": "" + } + }, + "PortMappingsList": [ + [ + { + "NodePort": 56100, + "GamePort": { + "Name": "game_port", + "Number": 7777, + "Protocol": "TCP" + } + } + ] + ], + "SessionConfig": { + "SessionId": "ba67d671-512a-4e7d-a38c-2329ce181946", + "SessionCookie": null, + "InitialPlayers": [ "Player1", "Player2" ] + } +} +``` + +> Some notes: +> 1. You must set `RunContainer` to true. +> 2. Modify `imageDetails` with your game server Docker image details. Image may be built locally (using [docker build](https://docs.docker.com/engine/reference/commandline/build/) command) or be hosted in a remote container registry. If you host it on a remote container registry, you must provide the username and password for the registry. +> 3. `StartGameCommand` and `AssetDetails` are optional. You don't normally use them when you use a Docker container since all game assets + start game server command can be packaged in the corresponding [Dockerfile](https://docs.docker.com/engine/reference/builder/). +> 4. Last, but definitely not least, pay attention to the casing on your `OutputFolder` variable, since Linux containers are case sensitive. If casing is wrong, you might see a Docker exception similar to *error while creating mount source path '/host_mnt/c/output/UnityServerLinux/PlayFabVmAgentOutput/2020-01-30T12-47-09/GameLogs/a94cfbb5-95a4-480f-a4af-749c2d9cf04b': mkdir /host_mnt/c/output: file exists* + +## Verifying GSDK integration + +- After you perform all the previous steps, you can then run the LocalMultiPlayerAgent with the command `LocalMultiplayerAgent.exe -lcow` (lcow stands for *Linux Containers On Windows*) +- If the GSDK is integrated correctly, **LocalMultiplayerAgent** prints the following outputs: + - `CurrentGameState - Initializing` (this is optional and may not show up if your game server directly calls `GSDK::ReadyForPlayers` and does not call `GSDK::Start`) + - `CurrentGameState - StandingBy` + - `CurrentGameState - Active` + - `CurrentGameState - Terminating` +- At this point, LocalMultiplayerAgent will try to execute the GSDK shutdown callback which **is not** required for Thundernetes. You can either implement it (by exiting the game server process manually) or simply cancel LocalMultiplayerAgent execution and then manually delete the game server container (by executing `docker rm -f `). + +### Testing connection to your game + +When your game server executable is running and **LocalMultiplayerAgent** prints `CurrentGameState - Active`, you can connect to your game server using IP address **127.0.0.1** and port `NodePort` on which your game server is listening. + +After `NumHeartBeatsForActivateResponse` heartbeats, **LocalMultiplayerAgent** requests the game server to move from standby to active. + +> You can also check the (very similar) MPS instructions [here](https://docs.microsoft.com/en-us/gaming/playfab/features/multiplayer/servers/locally-debugging-game-servers-and-integration-with-playfab#using-localmultiplayeragent-with-linux-containers). \ No newline at end of file diff --git a/docs/howtos/scheduling.md b/docs/howtos/scheduling.md index 555a03ff..a8cafab0 100644 --- a/docs/howtos/scheduling.md +++ b/docs/howtos/scheduling.md @@ -1,13 +1,13 @@ --- layout: default -title: tight scheduling +title: Efficient scheduling parent: How to's nav_order: 3 --- -# Pod scheduling +# Efficient scheduling -By default, Pods are scheduled using the Kubernetes scheduler. Its behavior is to spread the Pods into as many Nodes as possible. However, if you are using a cloud provider (e.g. Azure Kubernetes Service), you'd want to schedule your Game Server Pods into the less amount of Nodes possible. For example, if you have two VMs, you'll want to schedule the Pods on VM 1 till it can't host any more, then you'll schedule the Pods to VM 2. The reason for doing that is that on a potential cluster scale-down you will want to have Nodes with zero (or close to zero) Pods, so they can be effiently reclaimed by the underlying cloud provider. To accomplish this type of tight scheduling, you can use the [Kubernetes inter-pod affinity strategy](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity) when defining your Pod. +By default, Pods are scheduled using the [Kubernetes scheduler](https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/). Generally, its behavior is to spread the Pods into as many Nodes as possible. However, if you are using a cloud provider (e.g. Azure Kubernetes Service), you'd want to schedule your Game Server Pods into the less amount of Nodes possible, to save on costs. For example, if you have two VMs, you'll want to schedule the Pods on VM 1 till it can't host any more, then you'll start scheduling the Pods to VM 2. The reason for doing that is that on a potential cluster scale-down you will want to have Nodes with zero (or close to zero) Pods in the Active, so they can be efficiently reclaimed by the underlying cloud provider. To accomplish this type of tight scheduling, you can use the [Kubernetes inter-pod affinity strategy](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity) when defining your Pod on the GameServerBuild. To instruct the Kubernetes scheduler to try and schedule Pods into as few Nodes as possible you can use something like the following: diff --git a/docs/howtos/upgradebuild.md b/docs/howtos/upgradebuild.md new file mode 100644 index 00000000..02ea1fed --- /dev/null +++ b/docs/howtos/upgradebuild.md @@ -0,0 +1,20 @@ +--- +layout: default +title: Upgrading your game server +parent: How to's +nav_order: 8 +--- + +# Upgrading your game server + +Thundernetes does not support the concept of rolling upgrades to the GameServerBuild. The allocation API call uses the BuildID to allocate a game server on a specific GameServerBuild, so a new one should be used to eventually replace the old GameServerBuild. GameServerBuild specification should be considered immutable, except of course for the `standingBy` and `max` numbers. + +To upgrade your game server process, you can follow the steps below: + +- create a new GameServerBuild with a different BuildID and scale it to a high enough `standingBy` number of servers. +- modify your matchmaker/lobby service to allocate game servers using the new BuildID. +- at the same time, set the `standingBy` number of the old Build to zero. Existing sessions on the old Build will eventually finish, but since the `standingBy` number is zero, no new game servers will be created. +- as the number of Actives on the new GameServerBuild increases, you should increase the `standingBy`/`max` numbers. Cluster Autoscaler should be on so that the number of Nodes in the cluster will increase as needed. +- once the number of Actives on the old GameServerBuild is zero, you can delete it. + +> _**NOTE**_: At this point, Thundernetes does not do anything to prevent you from modifying any part of the GameServerBuild specification. However, this scenario is not recommended, as it can lead to unexpected behavior. \ No newline at end of file diff --git a/docs/prerequisites.md b/docs/prerequisites.md index 21d3e111..1afc6326 100644 --- a/docs/prerequisites.md +++ b/docs/prerequisites.md @@ -1,7 +1,7 @@ --- layout: default title: Prerequisites -nav_order: 10 +nav_order: 3 --- # Prerequisites @@ -10,7 +10,7 @@ nav_order: 10 Here you will find resources that will fill the knowledge gaps when working with technologies within thundernetes. -👉**thundernetes** was named after a combination of the words *thunderhead* and *kubernetes*. +👉**Thundernetes** was named after a combination of the words *thunderhead* and *kubernetes*. Thunderhead is the internal code name for the [Azure PlayFab Multiplayer Servers](https://azure.microsoft.com/services/playfab/multiplayer-services/) service. ## Docker and Containerisation 🚢 @@ -32,15 +32,14 @@ Kubernetes is built on top of Docker to run containers at scale across many mach ## Game Servers 👾 -Thundernetes is a preview project from teams from Azure and XBox that enables you to run Linux game servers that use the PlayFab Game Server SDK (GSDK) on your Kubernetes cluster. +Thundernetes is a preview project from teams from Azure and Xbox that enables you to run Linux game servers that use the open source Azure PlayFab Game Server SDK (GSDK) on your Kubernetes cluster. - [Integrating Game Servers with Game Server SDK (GSDK)](https://docs.microsoft.com/gaming/playfab/features/multiplayer/servers/integrating-game-servers-with-gsdk) -- [Azure PlayFab Multiplayer Servers Gameserver Samples](https://github.com/PlayFab/MpsSamples) -- [PlayFab Game Server SDK](https://github.com/PlayFab/gsdk) +- [Azure PlayFab Game Server SDK](https://github.com/PlayFab/gsdk) ## Azure Account ☁️ -You can host you Kubernetes clusters in Azure. If you don't have an account you can create one for free. +You can host your Kubernetes clusters in Azure. If you don't have an account you can create one for free. - [Create your Azure Account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) ### Azure Blob Storage @@ -49,10 +48,11 @@ You can host you Kubernetes clusters in Azure. If you don't have an account you ## Game Engine 🎮 -Using your multiplayer game to intergrate with thundernetes. +Using your multiplayer game to integrate with Thundernetes. - [Unreal Engine](https://www.unrealengine.com/) - [Unity](https://unity.com/) +- [Azure PlayFab Multiplayer Servers Gameserver Samples](https://github.com/PlayFab/MpsSamples) ## Metrics 💹 diff --git a/docs/quickstart.md b/docs/quickstart.md index 26dd75eb..dab92aec 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -6,9 +6,9 @@ nav_order: 2 # Quickstart -We've tested thundernetes on Azure Kubernetes Service (AKS) version 1.20.7 and 1.20.9 and [kind](https://kind.sigs.k8s.io/) but it can theoretically be installed on any Kubernetes cluster, optionally supporting Public IP per Node (which is something you want if you want to expose your game servers outside the cluster). Read the relevant section depending on where you want to install thundernetes. +We've tested Thundernetes on the latest versions of Azure Kubernetes Service (AKS) and [kind](https://kind.sigs.k8s.io/) but it can theoretically be installed on any Kubernetes cluster supporting Public IP per Node (which is something you want if you want to expose your game servers outside the cluster). Read the relevant section depending on where you want to install Thundernetes. -> If you are using Windows, we recommend using [Windows Subsystem for Linux](https://docs.microsoft.com/windows/wsl/install-win10) to run the CLI commands listed below. +> If you are using Windows, we recommend using [Windows Subsystem for Linux](https://docs.microsoft.com/windows/wsl/install) to run the CLI commands listed below. ## Create an Azure Kubernetes Service cluster with a Public IP per Node @@ -26,7 +26,7 @@ AKS_VERSION=1.22.4 # replace with the Kubernetes version that is supported in th az group create --name $AKS_RESOURCE_GROUP --location $AKS_LOCATION # create a new AKS cluster enabling the feature of Public IP per Node az aks create --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME --ssh-key-value ~/.ssh/id_rsa.pub --kubernetes-version $AKS_VERSION --enable-node-public-ip -# get credentials for this cluster +# get credentials for this cluster, saving them in a separate file az aks get-credentials --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME --file ~/.kube/config-thundernetes # check that cluster is up and running export KUBECONFIG=~/.kube/config-thundernetes @@ -41,9 +41,9 @@ Thundernetes requires VMs to have Public IPs (so game servers can be accessible) > This port range is configurable, check [here](howtos/configureportrange.md) for details. -To allow this you need to perform the following steps *after your AKS cluster gets created*: +To open the ports you need to perform the following steps *after your AKS cluster gets created*: -* Login to the Azure Portal +* Login to the [Azure Portal](https://portal.azure.com) * Find the resource group where the AKS resources are kept, it should have a name like `MC_resourceGroupName_AKSName_location`. Alternative, you can type `az resource show --namespace Microsoft.ContainerService --resource-type managedClusters -g $AKS_RESOURCE_GROUP -n $AKS_NAME -o json | jq .properties.nodeResourceGroup` on your shell to find it. * Find the Network Security Group object, which should have a name like `aks-agentpool-********-nsg` * Select **Inbound Security Rules** @@ -74,7 +74,7 @@ You can use a variety of options to run Kubernetes locally, either [kind](https: * Install kind using the instructions [here](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) * Create a "kind-config.yaml" file to configure the cluster, using the contents listed below. -Special attention is needed on the ports you will forward (the "containerPort" listed below). First of all, you need to expose port 5000 since this is the port used by the thundernetes GameServer allocation API service. You will use this port to do game server allocations. +Special attention is needed on the ports you will forward (the "containerPort" listed below). First of all, you need to expose port 5000 since this is the port used by the Thundernetes GameServer allocation API service. You will use this port to do game server allocations. After that, you can optionally specify ports to test your game server by sending traffic to it. Thundernetes dynamically allocates ports for your game server, ranging from 10000 to 12000. Port assignment from this range is sequential. For example, if you use two game servers with each one having a single port, the first game server port will be mapped to port 10000 and the second will be mapped to port 10001. Be aware that if you scale down your GameServerBuild and scale it up again, you probably will not get the same port. Consequently, pay special attention to the ports that you will use in your kind configuration. Save this content to a file called `kind-config.yaml`. @@ -108,19 +108,17 @@ Kubernetes control plane is running at https://127.0.0.1:34253 CoreDNS is running at https://127.0.0.1:34253/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy ``` -## Install thundernetes with the installation script +## Install Thundernetes with the installation script -Once you have a Kubernetes cluster up and running, you can run the following command to install thundernetes. This will install thundernetes *without* TLS authentication for the allocation API service, which should only be used on test environments. +Once you have a Kubernetes cluster up and running, you can run the following command to install Thundernetes. This will install thundernetes *without* TLS authentication for the allocation API service, which should only be used on test environments. ```bash kubectl apply -f https://raw.githubusercontent.com/PlayFab/thundernetes/main/installfiles/operator.yaml ``` -Read the following section if you want to have TLS based authentication for the thundernetes API service. - ### Install thundernetes with TLS authentication -You need to create/configure the certificate that will be used to protect the allocation API service. +You need to create/configure the certificate that will be used to protect the allocation API service. A properly configured certificate (signed by a well-known CA) is recommended for production environments. For testing purposes, you can generate a self-signed certificate and use it to secure the allocation API service. You can use OpenSSL to create a self-signed certificate and key (of course, this scenario is not recommended for production). @@ -136,7 +134,7 @@ kubectl create namespace thundernetes-system kubectl create secret tls tls-secret -n thundernetes-system --cert=/path/to/public.pem --key=/path/to/private.pem ``` -Then, you can run the following script to install thundernetes with TLS security for the allocation API service. +Then, you can run the following script to install Thundernetes with TLS security for the allocation API service. ```bash kubectl apply -f https://raw.githubusercontent.com/PlayFab/thundernetes/main/installfiles/operator_with_security.yaml @@ -144,15 +142,15 @@ kubectl apply -f https://raw.githubusercontent.com/PlayFab/thundernetes/main/ins The two installation files (operator.yaml and operator_with_security.yaml) are identical except for the API_SERVICE_SECURITY environment variable that is passed into the controller container. -At this point, you are ready to run your game server on thundernetes. If you want to run one of our sample game servers, please read on. Otherwise, if you want to run your own game server, please go to [this document](developertool.md). +At this point, you are ready to run a test game server on Thundernetes to verify that the system is working as expected. If you want to run one of our sample game servers, read on. Otherwise, if you want to run your own game server, go to [this document](developertool.md). ## Run sample game servers -Thundernetes comes with two sample game server projects that are integrated with [GSDK](https://github.com/PlayFab/gsdk). You can use either one of them to validate your thundernetes installation. +Thundernetes comes with two sample game server projects that are already integrated with [GSDK](https://github.com/PlayFab/gsdk). You can use either one of them to validate your Thundernetes installation. ### .NET Core Fake game server -This sample, located [here](../samples/netcore), is a simple .NET Core Web API app that implements GSDK. You can install it on your Kubernetes cluster by runnning the following command: +This sample, located [here](https://github.com/playfab/thundernetes/samples/netcore), is a simple .NET Core Web API app that implements GSDK. You can install it on your Kubernetes cluster by runnning the following command: ```bash kubectl apply -f https://raw.githubusercontent.com/PlayFab/thundernetes/main/samples/netcore/sample.yaml @@ -160,7 +158,7 @@ kubectl apply -f https://raw.githubusercontent.com/PlayFab/thundernetes/main/sam > To read about the fields that you need to specify for a GameServerBuild, you can check [this document](gameserverbuild.md). -Try using `kubectl get gs` to see the running game servers: +Try using `kubectl get gs` to see the running game servers, you should see something similar to this: ```bash dgkanatsios@desktopdigkanat:thundernetes$ kubectl get gs @@ -183,7 +181,7 @@ To scale your GameServerBuild, you can do `kubectl edit gsb gameserverbuild-samp #### Allocate a game server -Allocating a GameServer will transition its state from "StandingBy" to "Active" and will unblock the "ReadyForPlayers" GSDK call. +Allocating a GameServer will transition its state from "StandingBy" to "Active" and will unblock the "ReadyForPlayers" GSDK call. You can read [this document](gameserverlifecycle.md) to learn more about the lifecycle of a GameServer. If you are running on Azure Kubernetes Service, you can use the following command to allocate a game server: @@ -191,13 +189,16 @@ If you are running on Azure Kubernetes Service, you can use the following comman # grab the IP of the external load balancer that is used to route traffic to the allocation API service IP=$(kubectl get svc -n thundernetes-system thundernetes-controller-manager -o jsonpath='{.status.loadBalancer.ingress[0].ip}') # do the allocation call. Make sure the buildID is the same as the one that you created your Build with +# the sessionID is a unique identifier (GUID) that you can use to track the game server session curl -H 'Content-Type: application/json' -d '{"buildID":"85ffe8da-c82f-4035-86c5-9d2b5f42d6f6","sessionID":"ac1b7082-d811-47a7-89ae-fe1a9c48a6da"}' http://${IP}:5000/api/v1/allocate ``` -As you can see, the arguments to the allocation call are two: +The arguments to the allocation call are two: * buildID: this must be the same as the buildID configured in the GameServerBuild -* sessionID: a GUID that you can use to identify the game server session. Must be unique for each game server you allocate. If you try to allocate using a sessionID that is in use, the call will return the details of the existing game server. This call is equivalent to calling [RequestMultiplayerServer](https://docs.microsoft.com/rest/api/playfab/multiplayer/multiplayer-server/request-multiplayer-server) in PlayFab Multiplayer Servers. +* sessionID: a GUID that you can use to identify the game server session. Must be unique for each game server you allocate. If you try to allocate using a sessionID that is in use, the call will return the details of the existing game server. + +> The allocation call is equivalent to calling [RequestMultiplayerServer](https://docs.microsoft.com/rest/api/playfab/multiplayer/multiplayer-server/request-multiplayer-server) in PlayFab Multiplayer Servers. Result of the allocate call is the IP/Port of the server in JSON format. @@ -205,14 +206,14 @@ Result of the allocate call is the IP/Port of the server in JSON format. {"IPV4Address":"52.183.89.4","Ports":"80:10000","SessionID":"ac1b7082-d811-47a7-89ae-fe1a9c48a6da"} ``` -You can now use the IP/Port to connect to the allocated game server. The fake game server exposes a /hello endpoint that returns the hostname of the container. +You can now use the IP/Port to connect to the allocated game server. The fake game server exposes a `/Hello` endpoint that returns the hostname of the container. ```bash dgkanatsios@desktopdigkanat:thundernetes$ curl 52.183.89.4:10000/Hello Hello from fake GameServer with hostname gameserverbuild-sample-netcore-mveex ``` -At the same time, you can check your game servers again. Since the original request was for 2 standingBy and 4 maximum servers, we can now see that we have 2 standingBy and 1 active. +At the same time, you can check your game servers again. Since the original request was for 2 standingBy and 4 maximum servers and the allocation call converted one StandingBy to Active, Thundernetes created another GameServer, which reached the StandingBy state. We can now see that we have 2 StandingBy and 1 Active. We can also see the SessionID of the Active GameServer. ```bash dgkanatsios@desktopdigkanat:thundernetes$ kubectl get gs @@ -224,11 +225,11 @@ gameserverbuild-sample-netcore-pxrqx Healthy StandingBy 52.183.89.4 80:1 #### Lifecycle of a game server -The game server will remain in Active state as long as the game server process is running. Once the game server process exits, the game server pod will be deleted and a new one will be created in its place. For more information on the GameServer lifecycle, please check [here](gameserverlifecycle.md). +The game server will remain in Active state as long as the game server process is running. Once the game server process exits, the GameServer Custom Resource will be deleted. This will make the game server pod to be deleted and a new one will be created in its place (provided we are not beyond the GameServerBuild's maximum). For more information on the GameServer lifecycle, please check [here](gameserverlifecycle.md). ### Openarena -This sample, located [here](../samples/openarena), is based on the popular open source FPS game [OpenArena](http://www.openarena.ws/smfnews.php). You can install it using this script +This sample, located [here](https://github.com/playfab/thundernetes/samples/openarena), is based on the popular open source FPS game [OpenArena](http://www.openarena.ws/smfnews.php). You can install it using this script: ```bash kubectl apply -f https://raw.githubusercontent.com/PlayFab/thundernetes/main/samples/openarena/sample.yaml diff --git a/docs/troubleshooting/README.md b/docs/troubleshooting/README.md index b8cdf7d7..fa7d3785 100644 --- a/docs/troubleshooting/README.md +++ b/docs/troubleshooting/README.md @@ -1,7 +1,7 @@ --- layout: default title: Troubleshooting -nav_order: 9 +nav_order: 10 has_children: true --- diff --git a/docs/troubleshooting/controllernodeagent.md b/docs/troubleshooting/controllernodeagent.md new file mode 100644 index 00000000..63c4430c --- /dev/null +++ b/docs/troubleshooting/controllernodeagent.md @@ -0,0 +1,27 @@ +--- +layout: default +title: Controller/NodeAgent status +parent: Troubleshooting +nav_order: 4 +--- + +# How can I get access to the controller and the NodeAgent status and logs? + +Thundernetes controller Pod and NodeAgent Pods are installed in the `thundernetes-system` namespace by default. + +- You can see the status of the controller Deployment with `kubectl get deploy -n thundernetes-system thundernetes-controller-manager`. This will give you a result like: +```bash +NAME READY UP-TO-DATE AVAILABLE AGE +thundernetes-controller-manager 1/1 1 1 2h +``` +- To get the controller Pod logs, you need to find the Pod name. Run `kubectl get pods -n thundernetes-system` and look for the name of the controller Pod. It should be something like `thundernetes-controller-manager-774c99cd4-6tq8w`. Then you can do `kubectl logs -n thundernetes-system thundernetes-controller-manager-774c99cd4-6tq8w` to see the controller logs. +- To see the NodeAgent status you can run `kubectl get ds -n thundernetes-system`, since NodeAgent is installed as a Kubernetes DaemonSet. You will see an output similar to this: +```bash +NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE +thundernetes-nodeagent 3 3 3 3 3 2h +``` +- To see the logs for a NodeAgent Pod, you need to find the Pod name. Run `kubectl get pods -n thundernetes-system` and look for the name of the NodeAgent Pod. It should be something like `thundernetes-nodeagent-4fdb9`. Then you can do `kubectl logs -n thundernetes-system thundernetes-nodeagent-4fdb9` to see the NodeAgent logs. +- If you want to see the logs for a NodeAgent in a particular Node (e.g. if you need to debug communication between the NodeAgent and a GameServer Pod), you need to first find out the Node name. + - Run `kubectl get pods -owide` so you can see the Node name that the Pod you want to debug is running on. + - Run `kubectl get pods -n thundernetes-system -owide` so you can see the Node name along with all the NodeAgent Pods. + - As soon as you find out the NodeAgent Pod you want to get its logs, you can run `kubectl logs -n thundernetes-system thundernetes-nodeagent-XXXXX` to get this NodeAgent Pod logs. \ No newline at end of file diff --git a/docs/troubleshooting/stuckgameserver.md b/docs/troubleshooting/stuckgameserver.md index eccfa6ef..c2d2dadb 100644 --- a/docs/troubleshooting/stuckgameserver.md +++ b/docs/troubleshooting/stuckgameserver.md @@ -1,13 +1,16 @@ --- layout: default -title: Stuck game servers +title: Game server is stuck parent: Troubleshooting nav_order: 2 --- -# Help! What should we do if a GameServer gets stuck? +# What should we do if a GameServer gets stuck? -You can run ```kubectl delete gs ``` since this will take down both the GameServer instance, the corresponding Pod as well as the GameServerDetail CRD instance (in case the game server was allocated). +Sometimes the GameServer process might not responding, due to a programming bug or a misconfiguration. You can check the logs by using the command ```kubectl logs ``` and running a command shell into the Pod with ```kubectl exec -it -- sh```. It might be useful to also check the NodeAgent logs for the Node that this Pod is running on, check [here](controllernodeagent.md) for more information. -:exclamation: Do not delete the Pod. It will be deleted automatically when the GameServer is deleted. -:exclamation: Do not manually override GameServer.status details, this will create issues during controller reconciliation. \ No newline at end of file +If you want to delete the Pod, you can use the kubectl command ```kubectl delete gs ``` since this will take down both the GameServer instance, the corresponding Pod as well as the GameServerDetail CR instance (in case the game server was allocated). + +❗ Do not directly delete the Pod. It will be deleted automatically when the GameServer is deleted. + +❗ Do not manually overwrite GameServer.status details, this will create issues during controller reconciliation. \ No newline at end of file diff --git a/docs/troubleshooting/stuckinitializing.md b/docs/troubleshooting/stuckinitializing.md index a2d6740a..b8b541a0 100644 --- a/docs/troubleshooting/stuckinitializing.md +++ b/docs/troubleshooting/stuckinitializing.md @@ -1,10 +1,18 @@ --- layout: default -title: Stuck game server in initializing +title: Game Server does not transition to StandingBy state parent: Troubleshooting nav_order: 3 --- -# My GameServer state does not transition into StandingBy +# My GameServer state stays in empty or Initializing state and does not transition into StandingBy -First of all, you should check if the GameServer has created a Pod. Try `kubectl get gs` to see the GameServer name, then use `kubectl get pod` to see if there is a Pod with same name as the GameServer. You can use `kubectl get pod` to see the high-level status of the Pod. Try running `kubectl logs ` to see logs from your game server process. Also, try running `kubectl describe pod` to see the status of the Pod. Check there for some obvious failures, like failure to access the container registry, Pod creation failure because of resource constraints, etc. If everything works great, then probably there is an issue with the GSDK integration of your GameServer. You should take a look at [LocalMultiplayerAgent](../howtos/runlocalmultiplayeragent.md) to see how to run/test your GameServer locally and test its GSDK integration. \ No newline at end of file +> Check [here](../gameserverlifecycle.md) for more information on GameServer lifecycle. + +> You can check for the state of your GameServer by typing `kubectl get gs` + +You might see your GameServer stuck in the `Initializing` state (or in the empty state) and not transitioning to `StandingBy`. This can be a problem since eventually this GameServer cannot be allocated (converted to Active). Thundernetes only looks for `StandingBy` servers when looking for a GameServer to allocate. + +If the GameServer state is empty, tou should check if a corresponding Pod has been created for this GameServer. Pods have the same name and are created in the same namespace as their corresponding GameServer so they are easy to locate. Try `kubectl get gs` to see the GameServer name, then use `kubectl get pod` to see if there is a Pod with same name as the GameServer. You can use `kubectl get pod` to see the high-level status of the Pod. Try running `kubectl logs ` to see logs from your game server process. Also, try running `kubectl describe pod` to see the status of the Pod. Check there for some obvious failures, like failure to access the container registry, Pod creation failure because of resource constraints, etc. You can also try `kubectl exec -it -- sh` to open a shell in the game server Pod and investigate. + +If everything looks OK, then probably there is an issue with the GSDK integration of your GameServer. You should take a look at [LocalMultiplayerAgent](../howtos/runlocalmultiplayeragent.md) to see how to run/test your GameServer locally and test its GSDK integration. \ No newline at end of file diff --git a/docs/troubleshooting/stucknamespace.md b/docs/troubleshooting/stucknamespace.md index 9d4c08f8..1f5490a5 100644 --- a/docs/troubleshooting/stucknamespace.md +++ b/docs/troubleshooting/stucknamespace.md @@ -1,13 +1,13 @@ --- layout: default -title: Stuck terminating namespace +title: GameServer objects are not deleted parent: Troubleshooting nav_order: 1 --- -# Deleting namespace thundernetes-system stuck in terminating state +# GameServer objects are not deleted -Thundernetes creates finalizers for the GameServer custom resource. So, if you delete the thundernetes controller and you try to remove the GameServer Pods and/or the namespace they are in, the namespace might be stuck in terminating state. To fix this, you can run the following commands: +Thundernetes creates [finalizers](https://kubernetes.io/docs/concepts/overview/working-with-objects/finalizers/) for every GameServer custom resource. If you delete the Thundernetes controller and you try to remove the GameServer Pods and/or the namespace they are in, the namespace might be stuck in terminating state since there is no controller to handle the finalizer notification. To fix this and have the resources cleaned up, you can run the following commands: ```bash kubectl get namespace thundernetes-system -o json > tmp.json @@ -44,5 +44,3 @@ kubectl proxy # this command is blocking, so you can either run it on background curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/thundernetes-system/finalize kubectl get ns # verify that the namespace is gone ``` - -For more information about deleting namespaces stuck in terminating state check the [link](https://www.ibm.com/docs/en/cloud-private/3.2.0?topic=console-namespace-is-stuck-in-terminating-state). \ No newline at end of file diff --git a/docs/usingwrapper.md b/docs/usingwrapper.md index 605339be..ea680811 100644 --- a/docs/usingwrapper.md +++ b/docs/usingwrapper.md @@ -1,27 +1,27 @@ --- layout: default title: Using a wrapper utility -nav_order: 3 +nav_order: 4 --- # Using a wrapper utility ## Mpswrapper -Your game server needs to be integrated with the Game Server SDK ([GSDK](https://github.com/PlayFab/gsdk)) in order to work with thundernetes. However, you can use a wrapper application which: +Your game server needs to be integrated with the Game Server SDK ([GSDK](https://github.com/PlayFab/gsdk)) in order to work with Thundernetes. However, if you want to try your game server withoug integrating with GSDK you can use a wrapper utility application which: - integrates with GSDK - starts and monitors ("wraps") your game server executable -We have built such a utility (called 'mpswrapper') [here](https://github.com/PlayFab/MpsSamples/tree/master/wrappingGsdk). Mpswrapper is also published as container image on GitHub Container Registry [here](https://github.com/PlayFab/MpsSamples/pkgs/container/mpswrapper) which should be used as the container image in your GameServerBuild definition. +We have built such a utility (called 'mpswrapper') [here](https://github.com/PlayFab/MpsSamples/tree/master/wrappingGsdk). Mpswrapper is also published as container image on GitHub Container Registry [here](https://github.com/PlayFab/MpsSamples/pkgs/container/mpswrapper) which should be used as the container image in the GameServerBuild definition. -Your game server needs to be built and packaged in a place where your Kubernetes cluster can access it. Kubernetes uses [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) to access data external to the Pod. For this sample, you will use [Azure Files](https://azure.microsoft.com/en-us/services/storage/files/) storage to place your game server files but you are free to use the storage service of your choice. +The game server needs to be built and packaged in a place where your Kubernetes cluster can access it. Kubernetes workloads can use [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) to access data external to the Pod. For this sample, we will demonstrate the use of [Azure Files](https://azure.microsoft.com/en-us/services/storage/files/) storage to place the game server files but you are free to use the storage service of your choice. > Azure Files **Premium** is recommended for optimal performance. -> Your game server should *NOT* be integrated with GSDK, since the mpswrapper application is already integrated. +Your game server should *NOT* be integrated with GSDK, since the mpswrapper application is already integrated. It should be compiled as a Linux executable. -As soon as you create an Azure Files account, create a Files share and put your files there, you should make sure that Kubernetes knows how to authenticate to your Azure Storage account. One way to do that is create a Kubernetes secret using the below script: +As soon as you create an Azure Files account, create a Files share and copy your game server there. You should make sure that Kubernetes knows how to authenticate to your Azure Storage account. One way to do that is create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) using the below script: ```bash SECRET_NAME=azure-secret # name of the Kubernetes secret object @@ -30,15 +30,15 @@ ACCOUNT_KEY=YOUR_ACCOUNT_KEY # key for your Azure Storage account kubectl create secret generic $SECRET_NAME --from-literal=azurestorageaccountname=$SHARE_NAME --from-literal=azurestorageaccountkey=$ACCOUNT_KEY ``` -As soon as the secret is created, you are ready to create your thundernetes GameServerBuild. The `containers' part of the YAML should look like this: +When the secret is created, you are ready to create your Thundernetes GameServerBuild. The `containers' part of the YAML should look like this: -```YAML +```yaml containers: - image: ghcr.io/playfab/mpswrapper:0.1.0 name: mpswrapper command: ["/bin/bash", "-c", "chmod +x /assets/fakegame && ./wrapper -g /assets/fakegame"] # we use /assets since this is the folder specified on volumeMounts.mountPath below ports: - - containerPort: 80 # your game server port + - containerPort: 80 # your game server port, change it to the appropriate one protocol: TCP # your game server port protocol name: gameport # required field volumeMounts: @@ -56,4 +56,4 @@ volumes: - Mpswrapper code is [here](https://github.com/PlayFab/MpsSamples/tree/master/wrappingGsdk) and the latest container image version can be found [here](https://github.com/PlayFab/MpsSamples/pkgs/container/mpswrapper) - Instructions on how to mount an Azure Files share on your Kubernetes Pods can be found [here](https://docs.microsoft.com/en-us/azure/aks/azure-files-volume) -- You can find a sample YAML file [here](../samples/fileshare/sample.yaml) \ No newline at end of file +- You can find a sample YAML file [here](https://github.com/playfab/thundernetes/samples/fileshare/sample.yaml) \ No newline at end of file diff --git a/docs/yourgameserver.md b/docs/yourgameserver.md index a30a9307..1de11788 100644 --- a/docs/yourgameserver.md +++ b/docs/yourgameserver.md @@ -1,12 +1,12 @@ --- layout: default title: Running your own game server -nav_order: 4 +nav_order: 5 --- -# How to run your game server on thundernetes? +# How to run your game server on Thundernetes? -You can use thundernetes to host your game servers. This guide will walk you through on using thundernetes i) locally using kind and ii) on Azure Kubernetes Service. +You can use Thundernetes to host your game servers. This guide will walk you through on using Thundernetes i) locally using kind or ii) on Azure Kubernetes Service. ## Creating a local cluster with kind