Skip to content

Commit

Permalink
HPCC-27610 Update Container Placements Documentation
Browse files Browse the repository at this point in the history 
Signed-off-by: g-pan <[email protected]>
  • Loading branch information
@g-pan
g-pan committed Oct 25, 2023
1 parent 531d3c5 commit 1cfd609
Showing 1 changed file with 46 additions and 45 deletions.
91 changes: 46 additions & 45 deletions docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1222,22 +1222,22 @@ thor:
<sect2 id="YAML_FileStruct_Placement">
<title>Placements</title>

<para>The Placements is a term used by HPCC Systems, which in Kubernetes
is referred to as the scheduler or scheduling/assigning. In order to
avoid confusion with the HPCC Systems and ECL component scheduler terms.
Placements is a value in an HPCC Systems configuration which is at a
level above items, such as the nodeSelector, Toleration, Affinity and
Anti-Affinity, and TopologySpreadConstratints</para>
<para>The Placements is a term used by HPCC Systems, which Kubernetes
refers to as the scheduler or scheduling/assigning. In order to avoid
confusion within the HPCC Systems and ECL specific scheduler terms,
refer to Kubernetes scheduling as placements. Placements is a value in
an HPCC Systems configuration which is at a level above items, such as
the nodeSelector, Toleration, Affinity and Anti-Affinity, and
TopologySpreadConstratints.</para>

<para>The placement is responsible for finding the best node for a pod.
Most often placement is handled automatically by Kubernetes. You can
Most often placement are handled automatically by Kubernetes. You can
constrain a Pod so that it can only run on particular set of
Nodes.</para>

<para>The Placement concept allows you to spread your pods across types
of nodes with particular characteristics. Placements would be used to
ensure that pods or jobs that want nodes with specific characteristics
are placed on those nodes.</para>
<para>Placements would then be used to ensure that pods or jobs that
want nodes with specific characteristics are placed on those
nodes.</para>

<para>For instance a Thor cluster could be targeted for machine learning
using nodes with a GPU. Another job may want nodes with a good amount
Expand All @@ -1256,10 +1256,9 @@ thor:
<sect3 id="PlacementScope">
<title>Placement Scope</title>

<para>The concept here is to use pod patterns to apply the scope for
the placement.</para>
<para>Use pod patterns to apply the scope for the placements.</para>

<para>The pods: [list] item can contain one of the following.</para>
<para>The pods: [list] item can contain one of the following:</para>

<para><orderedlist>
<listitem>
Expand Down Expand Up @@ -1303,13 +1302,14 @@ thor:

<para>Regardless of the order the placements appear in the
configuration, they will be processed in the following order: "all",
"type", "target" and "pod" or "job".</para>
"type", "target", and then "pod"/"job".</para>

<sect4>
<title>Mixed combinations</title>

<para>NodeSelector and tolerations etc can be placed on the same
pods: [list] both per zone and per node on Azure <programlisting>placements:
<para>NodeSelector, taints and tolerations, and other values can all
be placed on the same pods: [list] both per zone and per node on
Azure <programlisting>placements:
- pods: ["eclwatch","roxie-workunit","^compile-.*$","mydali"]
placement:
nodeSelector:
Expand All @@ -1325,11 +1325,11 @@ thor:
schedule your nodes. The recommended approaches all use label selectors
to facilitate the selection. Generally, you may not need to set such
constraints; as the scheduler usually does reasonably acceptable
placement automatically. However some deployments you may want more
control for specific pods.</para>
placement automatically. However, with some deployments you may want
more control over specific pods.</para>

<para>Kubernetes uses the following methods to choose where to schedule
pods.</para>
pods:</para>

<para><itemizedlist>
<listitem>
Expand Down Expand Up @@ -1358,8 +1358,8 @@ thor:

<para>Kubernetes nodes have labels. Kubernetes has a standard set of
labels used for nodes in a cluster. You can also manually attach
labels which is recommended as the value of these labels is cloud
provider specific and not guaranteed to be reliable.</para>
labels which is recommended as the value of these labels is
cloud-provider specific and not guaranteed to be reliable.</para>

<para>Adding labels to nodes allows you to schedule pods to nodes or
groups of nodes. You can then use this functionality to ensure that
Expand All @@ -1375,7 +1375,7 @@ thor:
node selection constraint. It selects nodes based on the labels, but
it has some limitations. It only supports one label key and one label
value. If you wanted to match multiple labels or use more complex
expressions, you need to use node affinity.</para>
expressions, you need to use node Affinity.</para>

<para>Add the nodeSelector field to your pod specification and specify
the node labels you want the target node to have. You must have the
Expand All @@ -1385,16 +1385,16 @@ thor:

<para>The following example shows the nodeSelector placed in the pods
list. This example schedules "all" HPCC components to use the node
pool with the label group: "hpcc". </para>
pool with the label group: "hpcc".</para>

<para><programlisting> placements:
- pods: ["all"]
placement:
nodeSelector:
group: "hpcc"</programlisting></para>

<para>Note:the label: group:hpcc matches the node pool label: "hpcc".
</para>
<para>Note:the label: group:hpcc matches the node pool label:
"hpcc".</para>

<para>This next example shows how to configure a node pool to prevent
scheduling a Dali component onto this node pool labelled with the key
Expand All @@ -1412,8 +1412,6 @@ thor:

<para>When using nodeSelector, multiple nodeSelectors can be applied.
If duplicate keys are defined, only the last one prevails.</para>

<para>Examples:</para>
</sect3>

<sect3 id="CV-TAINTS_TOLERATIONS">
Expand Down Expand Up @@ -1452,7 +1450,7 @@ thor:
<para>Kubernetes can schedule a pod on to any node pool without a
taint. In the following examples Kubernetes can only schedule the
two components to the node pools with these exact labels, group and
gpu. </para>
gpu.</para>

<para><programlisting> placements:
- pods: ["all"]
Expand All @@ -1471,7 +1469,7 @@ thor:
effect: "NoSchedule" </programlisting>

<para>Multiple tolerations can also be used. The following example
has two tolerations, group and gpu. </para>
has two tolerations, group and gpu.</para>

<programlisting>#The settings will be applied to all thor pods/jobs and myeclccserver pod and job
- pods: ["thorworker-", "thor-thoragent", "thormanager-","thor-eclagent","hthor-"]
Expand All @@ -1495,37 +1493,40 @@ thor:
utilizing the nodeSelector, the taint will force the pod to deploy
only to the pods who match that node label. There are two
constraints then, in this example one from the node pool and the
other from the pod. </para>
other from the pod.</para>
</sect4>
</sect3>

<sect3 id="CV_TopologySpreadConstraints">
<title>Topology Spread Constraints</title>

<para>You can use topology spread constraints to control how Pods are
<para>You can use topology spread constraints to control how pods are
spread across your cluster among failure-domains such as regions,
zones, nodes, and other user-defined topology domains. This can help
to achieve high availability as well as efficient resource
utilization. You can set cluster-level constraints as a default, or
configure topology spread constraints for individual workloads. The
Topology Spread Constraints <emphasis
role="bold">topologySpreadConstraints</emphasis> Requires Kubernetes
role="bold">topologySpreadConstraints</emphasis> requires Kubernetes
v1.19+.or better.</para>

<para>Reference:</para>
<para>For more information see:</para>

<para><ulink
url="https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/">https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/</ulink>
and</para>

<para><ulink
url="https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/">https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/</ulink>
for additional information and options.</para>
</para>

<para>Using the "topologySpreadConstraints" example, there are two
node pools which have "hpcc=nodepool1" and "hpcc=nodepool2"
respectively. The Roxie pods will be evenly scheduled on the two node
pools. After deployment verify it with</para>
pools. </para>

<para>After deployment you can verify by issuing the following:
</para>

<programlisting>kubectl get pod -o wide | grep roxie</programlisting>

Expand Down Expand Up @@ -1554,7 +1555,7 @@ thor:
logic.</para>

<para>The are two main kinds of affinity, Node Affinity and Pod
Affinity</para>
Affinity.</para>

<sect4 id="CS-Node-Affiniy">
<title>Node Affinity</title>
Expand All @@ -1572,9 +1573,9 @@ thor:
<para>There is no schema check for the content of affinity. Only one
affinity can be applied to a pod or job. If a pod/job matches
multiple placement 'pods' lists, then only the last affinity
definition applies..</para>
definition applies.</para>

<para>Reference <ulink
<para>For more information, see <ulink
url="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/">https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/</ulink></para>

<para>There are two types of node affinity:</para>
Expand Down Expand Up @@ -1609,7 +1610,7 @@ thor:
<emphasis>requiredDuringSchedulingIgnoredDuringExecution</emphasis>
and
<emphasis>preferredDuringSchedulingIgnoredDuringExecution</emphasis>
fields. However these should be nested under the
fields. However, these should be nested under the
<emphasis>spec.affinity.podAffinity</emphasis> or
<emphasis>spec.affinity.podAntiAffinity</emphasis> fields depending
on whether you want to increase or reduce the Pod's affinity.</para>
Expand All @@ -1636,8 +1637,9 @@ thor:
<para>In the above "schedulerName" example, the "affinity" settings
can be included.</para>

<para>Caution as the "affinity" value in the "schedulerName" file is
only supported in Kubernetes 1.20.0 beta and later versions.</para>
<para><emphasis role="bold">Note:</emphasis> The "affinity" value in
the "schedulerName" field is only supported in Kubernetes 1.20.0
beta and later versions.</para>
</sect4>
</sect3>

Expand All @@ -1648,10 +1650,9 @@ thor:
specifies the name of the scheduler that is responsible for scheduling
a pod or a task. In Kubernetes, you can configure multiple schedulers
with different names and profiles to run simultaneously in the
cluster. Profile names "affinity" defined in the scheduler profile
requires Kubernetes 1.20.0 beta or later.</para>
cluster. </para>

<para>Only one "schedulerName" can be applied to any pod/job. </para>
<para>Only one "schedulerName" can be applied to any pod/job.</para>

<para>A SchedulerName example:</para>

Expand Down

0 comments on commit 1cfd609

Please sign in to comment.