app deployment using helm with full per-app customization

Signed-off-by: Pascal Sthamer <pascal+github@sthamer.xyz>
This commit is contained in:
Pascal Sthamer
2025-12-02 12:57:51 +01:00
parent 78f5e6c0b9
commit b6376cf967
7 changed files with 933 additions and 374 deletions
+93 -8
View File
@@ -135,7 +135,7 @@ To deploy apps using helm, set `apps.enabled=true`. By default, this deploys the
You can also deploy your own apps. See the following values file for an example. You can also deploy your own apps. See the following values file for an example.
```yaml ```yaml
app: app:
replicas: 1 # default to 1 replica per app replicaCount: 1 # default to 1 replica per app
resources: {} # default resources for apps resources: {} # default resources for apps
# ... configure default options for all apps here # ... configure default options for all apps here
@@ -150,7 +150,7 @@ apps:
http: http:
enabled: true # default enabled: true # default
# optionally override defaults from app values: # optionally override defaults from app values:
replicas: 1 replicaCount: 1
resources: {} resources: {}
# Deploy additional apps (e.g. opensearch) # Deploy additional apps (e.g. opensearch)
@@ -159,13 +159,14 @@ apps:
name: opensearch # required. The name and version must match the values of the `api.yaml` file of the app. name: opensearch # required. The name and version must match the values of the `api.yaml` file of the app.
version: 1.1.0 # required. version: 1.1.0 # required.
# optionally change app configuration: # optionally change app configuration:
replicas: 3 replicaCount: 3
resources: {} resources: {}
``` ```
The key of an app in the `apps` map does not matter, as long as it is unique. We are not using an array here, to allow overriding values in stage-specific value files or using the command line, e.g. The key of an app in the `apps` map does not matter, as long as it is unique. We are not using an array here, to allow overriding values in stage-specific value files or using the command line, e.g.
`helm upgrade ... --set apps.shuffleTools.replicas=3`. `helm upgrade ... --set apps.shuffleTools.replicas=3`.
See the "Parameters to deploy apps using helm" section below for a complete list of helm values, that can be used to customize app deployments. You can override any value set in `app.*` (e.g. `app.image`, `app.replicaCount`, `app.resources`, `app.podSecurityContext`) for each app
(e.g. for the `shuffle-tools` app using `apps.shuffleTools.image`, `apps.shuffleTools.replicaCount`, ...).
It is possible to use a hybrid approach - deploy some apps using helm, while still allowing Worker to create additional apps on-demand. It is possible to use a hybrid approach - deploy some apps using helm, while still allowing Worker to create additional apps on-demand.
@@ -651,12 +652,40 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia
| `worker.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` | | `worker.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` |
| `worker.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `worker.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` |
| `worker.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | | `worker.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` |
| `worker.manageAppDeployments` | Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see app.deployViaHelm). | `true` | | `worker.manageAppDeployments` | Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see apps.enabled). | `true` |
### app Parameters ### app Parameters
| Name | Description | Value | | Name | Description | Value |
| ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- |
| `app.image.registry` | app image registry (defaults to shuffle.appRegistry) | `""` |
| `app.image.repository` | app image repository (defaults to shuffle.appBaseImageName) | `""` |
| `app.image.tag` | app image tag (defaults to the apps version) | `""` |
| `app.image.pullPolicy` | default image pull policy for app deployments. Only effective for helm-deployed apps (see apps.enabled). | `IfNotPresent` |
| `app.image.pullSecrets` | default image pull secrets for app deployments. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.replicaCount` | Default number of replicas to deploy for each app. Only effective for helm-deployed apps (see apps.enabled). | `1` |
| `app.extraContainerPorts` | Optionally specify extra list of additional ports for app containers. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.livenessProbe.enabled` | Enable livenessProbe on app containers. Only effective for helm-deployed apps (see apps.enabled). | `false` |
| `app.livenessProbe.initialDelaySeconds` | Initial delay seconds for livenessProbe | `0` |
| `app.livenessProbe.periodSeconds` | Period seconds for livenessProbe | `15` |
| `app.livenessProbe.timeoutSeconds` | Timeout seconds for livenessProbe | `1` |
| `app.livenessProbe.failureThreshold` | Failure threshold for livenessProbe | `4` |
| `app.livenessProbe.successThreshold` | Success threshold for livenessProbe | `1` |
| `app.readinessProbe.enabled` | Enable readinessProbe on app containers. Only effective for helm-deployed apps (see apps.enabled). | `false` |
| `app.readinessProbe.initialDelaySeconds` | Initial delay seconds for readinessProbe | `0` |
| `app.readinessProbe.periodSeconds` | Period seconds for readinessProbe | `5` |
| `app.readinessProbe.timeoutSeconds` | Timeout seconds for readinessProbe | `1` |
| `app.readinessProbe.failureThreshold` | Failure threshold for readinessProbe | `3` |
| `app.readinessProbe.successThreshold` | Success threshold for readinessProbe | `1` |
| `app.startupProbe.enabled` | Enable startupProbe on app containers. Only effective for helm-deployed apps (see apps.enabled). | `false` |
| `app.startupProbe.initialDelaySeconds` | Initial delay seconds for startupProbe | `0` |
| `app.startupProbe.periodSeconds` | Period seconds for startupProbe | `1` |
| `app.startupProbe.timeoutSeconds` | Timeout seconds for startupProbe | `1` |
| `app.startupProbe.failureThreshold` | Failure threshold for startupProbe | `60` |
| `app.startupProbe.successThreshold` | Success threshold for startupProbe | `1` |
| `app.customLivenessProbe` | Custom livenessProbe that overrides the default one. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.customReadinessProbe` | Custom readinessProbe that overrides the default one. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.customStartupProbe` | Custom startupProbe that overrides the default one. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.resourcesPreset` | Set app container resources according to one common preset (allowed values: none, nano, small, medium, large, xlarge, 2xlarge). This is ignored if app.resources is set (app.resources is recommended for production). | `nano` | | `app.resourcesPreset` | Set app container resources according to one common preset (allowed values: none, nano, small, medium, large, xlarge, 2xlarge). This is ignored if app.resources is set (app.resources is recommended for production). | `nano` |
| `app.resources` | Set app container requests and limits for different resources like CPU or memory (essential for production workloads) | `{}` | | `app.resources` | Set app container requests and limits for different resources like CPU or memory (essential for production workloads) | `{}` |
| `app.podSecurityContext.enabled` | Enable app pods' Security Context | `true` | | `app.podSecurityContext.enabled` | Enable app pods' Security Context | `true` |
@@ -674,6 +703,49 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia
| `app.containerSecurityContext.allowPrivilegeEscalation` | Set allowPrivilegeEscalation in app container' Security Context | `false` | | `app.containerSecurityContext.allowPrivilegeEscalation` | Set allowPrivilegeEscalation in app container' Security Context | `false` |
| `app.containerSecurityContext.capabilities.drop` | List of capabilities to be dropped in app container | `["ALL"]` | | `app.containerSecurityContext.capabilities.drop` | List of capabilities to be dropped in app container | `["ALL"]` |
| `app.containerSecurityContext.seccompProfile.type` | Set seccomp profile in app container | `RuntimeDefault` | | `app.containerSecurityContext.seccompProfile.type` | Set seccomp profile in app container | `RuntimeDefault` |
| `app.command` | Override default app container command (useful when using custom images) | `[]` |
| `app.args` | Override default app container args (useful when using custom images) | `[]` |
| `app.automountServiceAccountToken` | Mount Service Account token in app pods. Only effective for helm-deployed apps (see apps.enabled). | `false` |
| `app.hostAliases` | app pods host aliases. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.deploymentAnnotations` | Annotations for app deployment. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.podLabels` | Extra labels for app pods. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.podAnnotations` | Annotations for app pods. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.podAffinityPreset` | Pod affinity preset. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.podAntiAffinityPreset` | Pod anti-affinity preset. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`. Only effective for helm-deployed apps (see apps.enabled). | `soft` |
| `app.nodeAffinityPreset.type` | Node affinity preset type. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.nodeAffinityPreset.key` | Node label key to match. Ignored if `app.affinity` is set | `""` |
| `app.nodeAffinityPreset.values` | Node label values to match. Ignored if `app.affinity` is set | `[]` |
| `app.affinity` | Affinity for app pods assignment. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.nodeSelector` | Node labels for app pods assignment. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.tolerations` | Tolerations for app pods assignment. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.updateStrategy.type` | app deployment strategy type. Only effective for helm-deployed apps (see apps.enabled). | `RollingUpdate` |
| `app.priorityClassName` | app pods' priorityClassName. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.topologySpreadConstraints` | Topology Spread Constraints for app pod assignment spread across your cluster among failure-domains. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.schedulerName` | Name of the k8s scheduler (other than default) for app pods. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.terminationGracePeriodSeconds` | Seconds app pods need to terminate gracefully. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.lifecycleHooks` | for app containers to automate configuration before or after startup. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.extraEnvVars` | Array with extra environment variables to add to app containers. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.extraEnvVarsCM` | Name of existing ConfigMap containing extra env vars for app containers. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.extraEnvVarsSecret` | Name of existing Secret containing extra env vars for app containers. Only effective for helm-deployed apps (see apps.enabled). | `""` |
| `app.extraVolumes` | Optionally specify extra list of additional volumes for the app pods. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.extraVolumeMounts` | Optionally specify extra list of additional volumeMounts for the app containers. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.sidecars` | Add additional sidecar containers to the app pods. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.initContainers` | Add additional init containers to the app pods. Only effective for helm-deployed apps (see apps.enabled). | `[]` |
| `app.pdb.create` | Enable/disable a Pod Disruption Budget creation. Only effective for helm-deployed apps (see apps.enabled). | `true` |
| `app.pdb.minAvailable` | Minimum number/percentage of pods that should remain scheduled | `""` |
| `app.pdb.maxUnavailable` | Maximum number/percentage of pods that may be made unavailable. Defaults to `1` if both `app.pdb.minAvailable` and `app.pdb.maxUnavailable` are empty. | `""` |
| `app.autoscaling.vpa.enabled` | Enable VPA for app pods. Only effective for helm-deployed apps (see apps.enabled). | `false` |
| `app.autoscaling.vpa.annotations` | Annotations for VPA resource | `{}` |
| `app.autoscaling.vpa.controlledResources` | VPA List of resources that the vertical pod autoscaler can control. Defaults to cpu and memory | `[]` |
| `app.autoscaling.vpa.maxAllowed` | VPA Max allowed resources for the pod | `{}` |
| `app.autoscaling.vpa.minAllowed` | VPA Min allowed resources for the pod | `{}` |
| `app.autoscaling.vpa.updatePolicy.updateMode` | Autoscaling update policy | `Auto` |
| `app.autoscaling.hpa.enabled` | Enable HPA for app pods. Only effective for helm-deployed apps (see apps.enabled). | `false` |
| `app.autoscaling.hpa.minReplicas` | Minimum number of replicas | `""` |
| `app.autoscaling.hpa.maxReplicas` | Maximum number of replicas | `""` |
| `app.autoscaling.hpa.targetCPU` | Target CPU utilization percentage | `""` |
| `app.autoscaling.hpa.targetMemory` | Target Memory utilization percentage | `""` |
| `app.service.labels` | Extra labels for app service. Only effective for helm-deployed apps (see apps.enabled). | `{}` |
| `app.serviceAccount.create` | Specifies whether a ServiceAccount should be created | `true` | | `app.serviceAccount.create` | Specifies whether a ServiceAccount should be created | `true` |
| `app.serviceAccount.name` | The name of the ServiceAccount to use. | `""` | | `app.serviceAccount.name` | The name of the ServiceAccount to use. | `""` |
| `app.serviceAccount.annotations` | Additional Service Account annotations (evaluated as a template) | `{}` | | `app.serviceAccount.annotations` | Additional Service Account annotations (evaluated as a template) | `{}` |
@@ -685,12 +757,24 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia
| `app.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` | | `app.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` |
| `app.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `app.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` |
| `app.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | | `app.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` |
| `app.mountTmpVolume` | Whether a writable /tmp emptyDir volume should be mounted to the app. | `false` |
| `app.exposedContainerPort` | The port that shuffle app containers will listen on for new requests. | `80` | | `app.exposedContainerPort` | The port that shuffle app containers will listen on for new requests. | `80` |
| `app.sdkTimeout` | The timeout in seconds for app actions. | `300` | | `app.sdkTimeout` | The timeout in seconds for app actions. | `300` |
| `app.disableLogs` | Do not capture app logs. By default, app logs are captured, so that they are visible in the frontend. | `false` | | `app.disableLogs` | Do not capture app logs. By default, app logs are captured, so that they are visible in the frontend. | `false` |
| `app.deployViaHelm` | A list of apps that should be deployed using helm. | `{}` |
| `app.deployViaHelm.MY_APP.app` | The name of the app (required, e.g. shuffle-tools) | | ### Parameters to deploy apps using helm
| `app.deployViaHelm.MY_APP.version` | The version of the app (required, e.g. 1.2.0) | |
| Name | Description | Value |
| ----------------------------- | -------------------------------------------------- | ------- |
| `apps.enabled` | Whether apps should be deployed using helm. | `false` |
| `apps.shuffleTools.enabled` | Whether the shuffle-tools app is enabled | `true` |
| `apps.shuffleTools.version` | The version of the shuffle-tools app to deploy. | `1.2.0` |
| `apps.shuffleSubflow.enabled` | Whether the shuffle-subflow app is enabled | `true` |
| `apps.shuffleSubflow.version` | The version of the shuffle-subflow app to deploy. | `1.1.0` |
| `apps.http.enabled` | Whether the http app is enabled | `true` |
| `apps.http.version` | The version of the http app to deploy. | `1.4.0` |
| `apps.MY_APP.app` | The name of the app (required, e.g. shuffle-tools) | |
| `apps.MY_APP.version` | The version of the app (required, e.g. 1.2.0) | |
### Traffic Exposure Parameters ### Traffic Exposure Parameters
@@ -793,3 +877,4 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia
@@ -87,9 +87,11 @@ app.kubernetes.io/managed-by: {{ .context.Release.Service }}
app.kubernetes.io/part-of: shuffle app.kubernetes.io/part-of: shuffle
app.shuffler.io/name: {{ include "shuffle.appInstance.name" .app }} app.shuffler.io/name: {{ include "shuffle.appInstance.name" .app }}
app.shuffler.io/version: {{ .app.version | quote }} app.shuffler.io/version: {{ .app.version | quote }}
{{- if .customValues }}
{{- range $key, $value := .customLabels }} {{- range $key, $value := .customLabels }}
{{ $key }}: {{ $value }} {{ $key }}: {{ $value }}
{{- end }} {{- end }}
{{- end }}
{{- end -}} {{- end -}}
{{/* {{/*
@@ -1,17 +0,0 @@
TODO:
- Deploy each app (configurable in values) here.
- Have defaults for all apps
- Allow overriding per app (serviceAccount, securityContext, volumes , env, resources, replicas, ...)
{{- range $index, $app := .Values.app.deployViaHelm }}
---
apiVersion: v1
kind: Service
metadata:
spec:
---
apiVersion: apps/v1
kind: Deployment
metadata:
spec:
{{- end }}
@@ -0,0 +1,306 @@
{{- if .Values.apps.enabled }}
{{- range $key, $app := .Values.apps }}
{{- $ignoredKeys := list "enabled" -}}
{{- if and (not ($ignoredKeys | has $key)) $app.enabled }}
{{/* Merge .Values.app into $app, giving precedence to the values defined on the app. */}}
{{- $appValues := mustMerge $app $.Values.app -}}
{{/* use shuffle.appRegistry as default image registry */}}
{{- $_ := set $appValues.image "registry" ($appValues.image.registry | default $.Values.shuffle.appRegistry) -}}
{{/* use shuffle.appBaseImageName as default image repository */}}
{{- $_ := set $appValues.image "repository" ($appValues.image.repository | default (printf "%s/%s" $.Values.shuffle.appBaseImageName $appValues.name)) -}}
{{/* use app version as default tag */}}
{{- $_ := set $appValues.image "tag" ($appValues.image.tag | default $appValues.version) -}}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "shuffle.appInstance.fullname" $app }}
namespace: {{ include "common.names.namespace" $ | quote }}
{{- $serviceLabels := include "common.tplvalues.merge" (dict "values" (list $appValues.service.labels $.Values.commonLabels) "context" $) }}
labels: {{- include "shuffle.appInstance.labels" ( dict "app" $app "customLabels" $serviceLabels "context" $ ) | nindent 4 }}
{{- if $.Values.commonAnnotations }}
annotations: {{- include "common.tplvalues.render" (dict "value" $.Values.commonAnnotations "context" $) | nindent 4 }}
{{- end }}
spec:
type: ClusterIP
ports:
- name: http
port: 80
targetPort: {{ $.Values.app.exposedContainerPort }}
protocol: TCP
appProtocol: http
---
apiVersion: {{ include "common.capabilities.deployment.apiVersion" $ }}
kind: Deployment
metadata:
name: {{ include "shuffle.appInstance.fullname" $app }}
namespace: {{ include "common.names.namespace" $ | quote }}
labels: {{- include "shuffle.appInstance.labels" ( dict "app" $app "customLabels" $.Values.commonLabels "context" $ ) | nindent 4 }}
{{- if or $appValues.deploymentAnnotations $.Values.commonAnnotations }}
{{- $annotations := include "common.tplvalues.merge" (dict "values" (list $appValues.deploymentAnnotations $.Values.commonAnnotations) "context" $) }}
annotations: {{- include "common.tplvalues.render" ( dict "value" $annotations "context" $ ) | nindent 4 }}
{{- end }}
spec:
{{- if not $appValues.autoscaling.hpa.enabled }}
replicas: {{ $appValues.replicaCount }}
{{- end }}
{{- if $appValues.updateStrategy }}
strategy: {{- toYaml $appValues.updateStrategy | nindent 4 }}
{{- end }}
{{- $podLabels := include "common.tplvalues.merge" (dict "values" (list $appValues.podLabels $.Values.commonLabels) "context" $) }}
selector:
matchLabels: {{- include "shuffle.appInstance.matchLabels" ( dict "app" $app "customLabels" $podLabels "context" $ ) | nindent 6 }}
template:
metadata:
{{- if $appValues.podAnnotations }}
annotations: {{- include "common.tplvalues.render" (dict "value" $appValues.podAnnotations "context" $) | nindent 8 }}
{{- end }}
labels: {{- include "shuffle.appInstance.labels" ( dict "app" $app "customLabels" $podLabels "context" $ ) | nindent 8 }}
spec:
{{- include "common.images.renderPullSecrets" (dict "images" (list $appValues.image) "context" $) | nindent 6 }}
{{- if $appValues.serviceAccount.create }}
serviceAccountName: {{ default (include "shuffle.app.name" $) $appValues.serviceAccount.name | trunc 63 | trimSuffix "-" }}
{{- else }}
serviceAccountName: {{ default "default" $appValues.serviceAccount.name }}
{{- end }}
automountServiceAccountToken: {{ $appValues.automountServiceAccountToken }}
{{- if $appValues.hostAliases }}
hostAliases: {{- include "common.tplvalues.render" (dict "value" $appValues.hostAliases "context" $) | nindent 8 }}
{{- end }}
{{- if $appValues.affinity }}
affinity: {{- include "common.tplvalues.render" ( dict "value" $appValues.affinity "context" $) | nindent 8 }}
{{- else }}
affinity:
podAffinity: {{- include "common.affinities.pods" (dict "type" $appValues.podAffinityPreset "customLabels" $podLabels "context" $) | nindent 10 }}
podAntiAffinity: {{- include "common.affinities.pods" (dict "type" $appValues.podAntiAffinityPreset "customLabels" $podLabels "context" $) | nindent 10 }}
nodeAffinity: {{- include "common.affinities.nodes" (dict "type" $appValues.nodeAffinityPreset.type "key" $appValues.nodeAffinityPreset.key "values" $appValues.nodeAffinityPreset.values) | nindent 10 }}
{{- end }}
{{- if $appValues.nodeSelector }}
nodeSelector: {{- include "common.tplvalues.render" ( dict "value" $appValues.nodeSelector "context" $) | nindent 8 }}
{{- end }}
{{- if $appValues.tolerations }}
tolerations: {{- include "common.tplvalues.render" (dict "value" $appValues.tolerations "context" $) | nindent 8 }}
{{- end }}
{{- if $appValues.priorityClassName }}
priorityClassName: {{ $appValues.priorityClassName | quote }}
{{- end }}
{{- if $appValues.schedulerName }}
schedulerName: {{ $appValues.schedulerName | quote }}
{{- end }}
{{- if $appValues.topologySpreadConstraints }}
topologySpreadConstraints: {{- include "common.tplvalues.render" (dict "value" $appValues.topologySpreadConstraints "context" $) | nindent 8 }}
{{- end }}
{{- if $appValues.podSecurityContext.enabled }}
securityContext: {{- omit $appValues.podSecurityContext "enabled" | toYaml | nindent 8 }}
{{- end }}
{{- if $appValues.terminationGracePeriodSeconds }}
terminationGracePeriodSeconds: {{ $appValues.terminationGracePeriodSeconds }}
{{- end }}
initContainers:
{{- if $appValues.initContainers }}
{{- include "common.tplvalues.render" (dict "value" $appValues.initContainers "context" $) | nindent 8 }}
{{- end }}
containers:
- name: {{ include "shuffle.appInstance.fullname" $app }}
image: {{ include "common.images.image" ( dict "imageRoot" $appValues.image "global" $.Values.global "chart" $.Chart ) }}
imagePullPolicy: {{ $appValues.image.pullPolicy }}
{{- if $appValues.containerSecurityContext.enabled }}
securityContext: {{- include "common.compatibility.renderSecurityContext" (dict "secContext" $appValues.containerSecurityContext "context" $) | nindent 12 }}
{{- end }}
{{- if $.Values.diagnosticMode.enabled }}
command: {{- include "common.tplvalues.render" (dict "value" $.Values.diagnosticMode.command "context" $) | nindent 12 }}
{{- else if $appValues.command }}
command: {{- include "common.tplvalues.render" (dict "value" $appValues.command "context" $) | nindent 12 }}
{{- end }}
{{- if $.Values.diagnosticMode.enabled }}
args: {{- include "common.tplvalues.render" (dict "value" $.Values.diagnosticMode.args "context" $) | nindent 12 }}
{{- else if $appValues.args }}
args: {{- include "common.tplvalues.render" (dict "value" $appValues.args "context" $) | nindent 12 }}
{{- end }}
env:
{{- $env := include "shuffle.appInstance.env" $ | fromYaml }}
{{- range $key, $val := $env }}
- name: {{ $key | quote }}
value: {{ $val | quote }}
{{- end }}
{{- if $appValues.extraEnvVars }}
{{- include "common.tplvalues.render" (dict "value" $appValues.extraEnvVars "context" $) | nindent 12 }}
{{- end }}
envFrom:
{{- if $appValues.extraEnvVarsCM }}
- configMapRef:
name: {{ include "common.tplvalues.render" (dict "value" $appValues.extraEnvVarsCM "context" $) }}
{{- end }}
{{- if $appValues.extraEnvVarsSecret }}
- secretRef:
name: {{ include "common.tplvalues.render" (dict "value" $appValues.extraEnvVarsSecret "context" $) }}
{{- end }}
{{- if $appValues.resources }}
resources: {{- toYaml $appValues.resources | nindent 12 }}
{{- else if ne $appValues.resourcesPreset "none" }}
resources: {{- include "common.resources.preset" (dict "type" $appValues.resourcesPreset) | nindent 12 }}
{{- end }}
ports:
- name: http
containerPort: {{ $appValues.exposedContainerPort }}
{{- if $appValues.extraContainerPorts }}
{{- include "common.tplvalues.render" (dict "value" $appValues.extraContainerPorts "context" $) | nindent 12 }}
{{- end }}
{{- if not $.Values.diagnosticMode.enabled }}
{{- if $appValues.customLivenessProbe }}
livenessProbe: {{- include "common.tplvalues.render" (dict "value" $appValues.customLivenessProbe "context" $) | nindent 12 }}
{{- else if $appValues.livenessProbe.enabled }}
livenessProbe: {{- include "common.tplvalues.render" (dict "value" (omit $appValues.livenessProbe "enabled") "context" $) | nindent 12 }}
httpGet:
path: /api/v1/health
port: {{ $appValues.containerPorts.http }}
{{- end }}
{{- if $appValues.customReadinessProbe }}
readinessProbe: {{- include "common.tplvalues.render" (dict "value" $appValues.customReadinessProbe "context" $) | nindent 12 }}
{{- else if $appValues.readinessProbe.enabled }}
readinessProbe: {{- include "common.tplvalues.render" (dict "value" (omit $appValues.readinessProbe "enabled") "context" $) | nindent 12 }}
httpGet:
path: /api/v1/health
port: {{ $appValues.containerPorts.http }}
{{- end }}
{{- if $appValues.customStartupProbe }}
startupProbe: {{- include "common.tplvalues.render" (dict "value" $appValues.customStartupProbe "context" $) | nindent 12 }}
{{- else if $appValues.startupProbe.enabled }}
startupProbe: {{- include "common.tplvalues.render" (dict "value" (omit $appValues.startupProbe "enabled") "context" $) | nindent 12 }}
httpGet:
path: /api/v1/health
port: {{ $appValues.containerPorts.http }}
{{- end }}
{{- end }}
{{- if $appValues.lifecycleHooks }}
lifecycle: {{- include "common.tplvalues.render" (dict "value" $appValues.lifecycleHooks "context" $) | nindent 12 }}
{{- end }}
volumeMounts:
{{- if $appValues.mountTmpVolume }}
- name: empty-dir
mountPath: /tmp
subPath: tmp-dir
{{- end }}
{{- if $appValues.extraVolumeMounts }}
{{- include "common.tplvalues.render" (dict "value" $appValues.extraVolumeMounts "context" $) | nindent 12 }}
{{- end }}
{{- if $appValues.sidecars }}
{{- include "common.tplvalues.render" ( dict "value" $appValues.sidecars "context" $) | nindent 8 }}
{{- end }}
volumes:
- name: empty-dir
emptyDir: {}
{{- if $appValues.extraVolumes }}
{{- include "common.tplvalues.render" (dict "value" $appValues.extraVolumes "context" $) | nindent 8 }}
{{- end }}
---
{{- if $appValues.pdb.create }}
apiVersion: {{ include "common.capabilities.policy.apiVersion" $ }}
kind: PodDisruptionBudget
metadata:
name: {{ include "shuffle.appInstance.fullname" $app }}
namespace: {{ include "common.names.namespace" $ | quote }}
labels: {{- include "shuffle.appInstance.labels" ( dict "app" $app "customLabels" $.Values.commonLabels "context" $ ) | nindent 4 }}
{{- if $.Values.commonAnnotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" $.Values.commonAnnotations "context" $ ) | nindent 4 }}
{{- end }}
spec:
{{- if $appValues.pdb.minAvailable }}
minAvailable: {{ $appValues.pdb.minAvailable }}
{{- end }}
{{- if or $appValues.pdb.maxUnavailable ( not $appValues.pdb.minAvailable ) }}
maxUnavailable: {{ $appValues.pdb.maxUnavailable | default 1 }}
{{- end }}
{{- $podLabels := include "common.tplvalues.merge" ( dict "values" ( list $appValues.podLabels $.Values.commonLabels ) "context" $ ) }}
selector:
matchLabels: {{- include "shuffle.appInstance.matchLabels" ( dict "app" $app "customLabels" $podLabels "context" $ ) | nindent 6 }}
{{- end }}
---
{{- if $appValues.autoscaling.hpa.enabled }}
apiVersion: {{ include "common.capabilities.hpa.apiVersion" $ }}
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "shuffle.appInstance.fullname" $app }}
namespace: {{ include "common.names.namespace" $ | quote }}
labels: {{- include "shuffle.appInstance.labels" ( dict "app" $app "customLabels" $.Values.commonLabels "context" $ ) | nindent 4 }}
{{- if $.Values.commonAnnotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" $.Values.commonAnnotations "context" $ ) | nindent 4 }}
{{- end }}
spec:
scaleTargetRef:
apiVersion: {{ include "common.capabilities.deployment.apiVersion" $ }}
kind: Deployment
name: {{ include "shuffle.appInstance.fullname" $app }}
minReplicas: {{ $appValues.autoscaling.hpa.minReplicas }}
maxReplicas: {{ $appValues.autoscaling.hpa.maxReplicas }}
metrics:
{{- if $appValues.autoscaling.hpa.targetMemory }}
- type: Resource
resource:
name: memory
{{- if semverCompare "<1.23-0" (include "common.capabilities.kubeVersion" $) }}
targetAverageUtilization: {{ $appValues.autoscaling.hpa.targetMemory }}
{{- else }}
target:
type: Utilization
averageUtilization: {{ $appValues.autoscaling.hpa.targetMemory }}
{{- end }}
{{- end }}
{{- if $appValues.autoscaling.hpa.targetCPU }}
- type: Resource
resource:
name: cpu
{{- if semverCompare "<1.23-0" (include "common.capabilities.kubeVersion" $) }}
targetAverageUtilization: {{ $appValues.autoscaling.hpa.targetCPU }}
{{- else }}
target:
type: Utilization
averageUtilization: {{ $appValues.autoscaling.hpa.targetCPU }}
{{- end }}
{{- end }}
{{- end }}
---
{{- if and ($.Capabilities.APIVersions.Has "autoscaling.k8s.io/v1/VerticalPodAutoscaler") $appValues.autoscaling.vpa.enabled }}
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: {{ include "shuffle.appInstance.fullname" $app }}
namespace: {{ include "common.names.namespace" $ | quote }}
labels: {{- include "shuffle.appInstance.labels" ( dict "app" $app "customLabels" $.Values.commonLabels "context" $ ) | nindent 4 }}
{{- if or $appValues.autoscaling.vpa.annotations $.Values.commonAnnotations }}
{{- $annotations := include "common.tplvalues.merge" ( dict "values" ( list $appValues.autoscaling.vpa.annotations $.Values.commonAnnotations ) "context" $ ) }}
annotations: {{- include "common.tplvalues.render" ( dict "value" $annotations "context" $) | nindent 4 }}
{{- end }}
spec:
resourcePolicy:
containerPolicies:
- containerName: TODO
{{- with $appValues.autoscaling.vpa.controlledResources }}
controlledResources:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $appValues.autoscaling.vpa.maxAllowed }}
maxAllowed:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $appValues.autoscaling.vpa.minAllowed }}
minAllowed:
{{- toYaml . | nindent 8 }}
{{- end }}
targetRef:
apiVersion: {{ include "common.capabilities.deployment.apiVersion" $ }}
kind: Deployment
name: {{ include "shuffle.appInstance.fullname" $app }}
{{- if $appValues.autoscaling.vpa.updatePolicy }}
updatePolicy:
{{- with $appValues.autoscaling.vpa.updatePolicy.updateMode }}
updateMode: {{ . }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
@@ -76,9 +76,11 @@ app.kubernetes.io/part-of: shuffle
{{- with .context.Chart.AppVersion }} {{- with .context.Chart.AppVersion }}
app.kubernetes.io/version: {{ . | replace "+" "_" | quote }} app.kubernetes.io/version: {{ . | replace "+" "_" | quote }}
{{- end -}} {{- end -}}
{{- if .customValues }}
{{- range $key, $value := .customLabels }} {{- range $key, $value := .customLabels }}
{{ $key }}: {{ $value }} {{ $key }}: {{ $value }}
{{- end }} {{- end }}
{{- end }}
{{- end -}} {{- end -}}
{{/* {{/*
@@ -113,6 +115,7 @@ SHUFFLE_BASE_IMAGE_REGISTRY: "{{ .Values.shuffle.appRegistry }}"
SHUFFLE_BASE_IMAGE_NAME: "{{ .Values.shuffle.appBaseImageName }}" SHUFFLE_BASE_IMAGE_NAME: "{{ .Values.shuffle.appBaseImageName }}"
# Shuffle app deployment configuration # Shuffle app deployment configuration
SHUFFLE_APP_MOUNT_TMP_VOLUME: {{ .Values.app.mountTmpVolume | quote }}
SHUFFLE_APP_SERVICE_ACCOUNT_NAME: {{ include "shuffle.app.serviceAccount.name" . | quote }} SHUFFLE_APP_SERVICE_ACCOUNT_NAME: {{ include "shuffle.app.serviceAccount.name" . | quote }}
{{- if .Values.app.podSecurityContext.enabled }} {{- if .Values.app.podSecurityContext.enabled }}
SHUFFLE_APP_POD_SECURITY_CONTEXT: {{ omit .Values.app.podSecurityContext "enabled" | mustToJson | quote }} SHUFFLE_APP_POD_SECURITY_CONTEXT: {{ omit .Values.app.podSecurityContext "enabled" | mustToJson | quote }}
@@ -2695,7 +2695,7 @@
}, },
"manageAppDeployments": { "manageAppDeployments": {
"type": "boolean", "type": "boolean",
"description": "Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see app.deployViaHelm).", "description": "Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see apps.enabled).",
"default": true "default": true
} }
} }
@@ -2703,6 +2703,168 @@
"app": { "app": {
"type": "object", "type": "object",
"properties": { "properties": {
"image": {
"type": "object",
"properties": {
"registry": {
"type": "string",
"description": "app image registry (defaults to shuffle.appRegistry)",
"default": ""
},
"repository": {
"type": "string",
"description": "app image repository (defaults to shuffle.appBaseImageName)",
"default": ""
},
"tag": {
"type": "string",
"description": "app image tag (defaults to the apps version)",
"default": ""
},
"pullPolicy": {
"type": "string",
"description": "default image pull policy for app deployments. Only effective for helm-deployed apps (see apps.enabled).",
"default": "IfNotPresent"
},
"pullSecrets": {
"type": "array",
"description": "default image pull secrets for app deployments. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
}
}
},
"replicaCount": {
"type": "number",
"description": "Default number of replicas to deploy for each app. Only effective for helm-deployed apps (see apps.enabled).",
"default": 1
},
"extraContainerPorts": {
"type": "array",
"description": "Optionally specify extra list of additional ports for app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"livenessProbe": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Enable livenessProbe on app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": false
},
"initialDelaySeconds": {
"type": "number",
"description": "Initial delay seconds for livenessProbe",
"default": 0
},
"periodSeconds": {
"type": "number",
"description": "Period seconds for livenessProbe",
"default": 15
},
"timeoutSeconds": {
"type": "number",
"description": "Timeout seconds for livenessProbe",
"default": 1
},
"failureThreshold": {
"type": "number",
"description": "Failure threshold for livenessProbe",
"default": 4
},
"successThreshold": {
"type": "number",
"description": "Success threshold for livenessProbe",
"default": 1
}
}
},
"readinessProbe": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Enable readinessProbe on app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": false
},
"initialDelaySeconds": {
"type": "number",
"description": "Initial delay seconds for readinessProbe",
"default": 0
},
"periodSeconds": {
"type": "number",
"description": "Period seconds for readinessProbe",
"default": 5
},
"timeoutSeconds": {
"type": "number",
"description": "Timeout seconds for readinessProbe",
"default": 1
},
"failureThreshold": {
"type": "number",
"description": "Failure threshold for readinessProbe",
"default": 3
},
"successThreshold": {
"type": "number",
"description": "Success threshold for readinessProbe",
"default": 1
}
}
},
"startupProbe": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Enable startupProbe on app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": false
},
"initialDelaySeconds": {
"type": "number",
"description": "Initial delay seconds for startupProbe",
"default": 0
},
"periodSeconds": {
"type": "number",
"description": "Period seconds for startupProbe",
"default": 1
},
"timeoutSeconds": {
"type": "number",
"description": "Timeout seconds for startupProbe",
"default": 1
},
"failureThreshold": {
"type": "number",
"description": "Failure threshold for startupProbe",
"default": 60
},
"successThreshold": {
"type": "number",
"description": "Success threshold for startupProbe",
"default": 1
}
}
},
"customLivenessProbe": {
"type": "object",
"description": "Custom livenessProbe that overrides the default one. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"customReadinessProbe": {
"type": "object",
"description": "Custom readinessProbe that overrides the default one. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"customStartupProbe": {
"type": "object",
"description": "Custom startupProbe that overrides the default one. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"resourcesPreset": { "resourcesPreset": {
"type": "string", "type": "string",
"description": "Set app container resources according to one common preset (allowed values: none, nano, small, medium, large, xlarge, 2xlarge). This is ignored if app.resources is set (app.resources is recommended for production).", "description": "Set app container resources according to one common preset (allowed values: none, nano, small, medium, large, xlarge, 2xlarge). This is ignored if app.resources is set (app.resources is recommended for production).",
@@ -2810,6 +2972,273 @@
} }
} }
}, },
"command": {
"type": "array",
"description": "Override default app container command (useful when using custom images)",
"default": [],
"items": {}
},
"args": {
"type": "array",
"description": "Override default app container args (useful when using custom images)",
"default": [],
"items": {}
},
"automountServiceAccountToken": {
"type": "boolean",
"description": "Mount Service Account token in app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": false
},
"hostAliases": {
"type": "array",
"description": "app pods host aliases. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"deploymentAnnotations": {
"type": "object",
"description": "Annotations for app deployment. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"podLabels": {
"type": "object",
"description": "Extra labels for app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"podAnnotations": {
"type": "object",
"description": "Annotations for app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"podAffinityPreset": {
"type": "string",
"description": "Pod affinity preset. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"podAntiAffinityPreset": {
"type": "string",
"description": "Pod anti-affinity preset. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`. Only effective for helm-deployed apps (see apps.enabled).",
"default": "soft"
},
"nodeAffinityPreset": {
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "Node affinity preset type. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"key": {
"type": "string",
"description": "Node label key to match. Ignored if `app.affinity` is set",
"default": ""
},
"values": {
"type": "array",
"description": "Node label values to match. Ignored if `app.affinity` is set",
"default": [],
"items": {}
}
}
},
"affinity": {
"type": "object",
"description": "Affinity for app pods assignment. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"nodeSelector": {
"type": "object",
"description": "Node labels for app pods assignment. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"tolerations": {
"type": "array",
"description": "Tolerations for app pods assignment. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"updateStrategy": {
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "app deployment strategy type. Only effective for helm-deployed apps (see apps.enabled).",
"default": "RollingUpdate"
}
}
},
"priorityClassName": {
"type": "string",
"description": "app pods' priorityClassName. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"topologySpreadConstraints": {
"type": "array",
"description": "Topology Spread Constraints for app pod assignment spread across your cluster among failure-domains. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"schedulerName": {
"type": "string",
"description": "Name of the k8s scheduler (other than default) for app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"terminationGracePeriodSeconds": {
"type": "string",
"description": "Seconds app pods need to terminate gracefully. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"lifecycleHooks": {
"type": "object",
"description": "for app containers to automate configuration before or after startup. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
},
"extraEnvVars": {
"type": "array",
"description": "Array with extra environment variables to add to app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"extraEnvVarsCM": {
"type": "string",
"description": "Name of existing ConfigMap containing extra env vars for app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"extraEnvVarsSecret": {
"type": "string",
"description": "Name of existing Secret containing extra env vars for app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": ""
},
"extraVolumes": {
"type": "array",
"description": "Optionally specify extra list of additional volumes for the app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"extraVolumeMounts": {
"type": "array",
"description": "Optionally specify extra list of additional volumeMounts for the app containers. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"sidecars": {
"type": "array",
"description": "Add additional sidecar containers to the app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"initContainers": {
"type": "array",
"description": "Add additional init containers to the app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": [],
"items": {}
},
"pdb": {
"type": "object",
"properties": {
"create": {
"type": "boolean",
"description": "Enable/disable a Pod Disruption Budget creation. Only effective for helm-deployed apps (see apps.enabled).",
"default": true
},
"minAvailable": {
"type": "string",
"description": "Minimum number/percentage of pods that should remain scheduled",
"default": ""
},
"maxUnavailable": {
"type": "string",
"description": "Maximum number/percentage of pods that may be made unavailable. Defaults to `1` if both `app.pdb.minAvailable` and `app.pdb.maxUnavailable` are empty.",
"default": ""
}
}
},
"autoscaling": {
"type": "object",
"properties": {
"vpa": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Enable VPA for app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": false
},
"annotations": {
"type": "object",
"description": "Annotations for VPA resource",
"default": {}
},
"controlledResources": {
"type": "array",
"description": "VPA List of resources that the vertical pod autoscaler can control. Defaults to cpu and memory",
"default": [],
"items": {}
},
"maxAllowed": {
"type": "object",
"description": "VPA Max allowed resources for the pod",
"default": {}
},
"minAllowed": {
"type": "object",
"description": "VPA Min allowed resources for the pod",
"default": {}
},
"updatePolicy": {
"type": "object",
"properties": {
"updateMode": {
"type": "string",
"description": "Autoscaling update policy",
"default": "Auto"
}
}
}
}
},
"hpa": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Enable HPA for app pods. Only effective for helm-deployed apps (see apps.enabled).",
"default": false
},
"minReplicas": {
"type": "string",
"description": "Minimum number of replicas",
"default": ""
},
"maxReplicas": {
"type": "string",
"description": "Maximum number of replicas",
"default": ""
},
"targetCPU": {
"type": "string",
"description": "Target CPU utilization percentage",
"default": ""
},
"targetMemory": {
"type": "string",
"description": "Target Memory utilization percentage",
"default": ""
}
}
}
}
},
"service": {
"type": "object",
"properties": {
"labels": {
"type": "object",
"description": "Extra labels for app service. Only effective for helm-deployed apps (see apps.enabled).",
"default": {}
}
}
},
"serviceAccount": { "serviceAccount": {
"type": "object", "type": "object",
"properties": { "properties": {
@@ -2883,6 +3312,11 @@
} }
} }
}, },
"mountTmpVolume": {
"type": "boolean",
"description": "Whether a writable /tmp emptyDir volume should be mounted to the app.",
"default": false
},
"exposedContainerPort": { "exposedContainerPort": {
"type": "number", "type": "number",
"description": "The port that shuffle app containers will listen on for new requests. ", "description": "The port that shuffle app containers will listen on for new requests. ",
@@ -2900,6 +3334,61 @@
} }
} }
}, },
"apps": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Whether apps should be deployed using helm.",
"default": false
},
"shuffleTools": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Whether the shuffle-tools app is enabled",
"default": true
},
"version": {
"type": "string",
"description": "The version of the shuffle-tools app to deploy.",
"default": "1.2.0"
}
}
},
"shuffleSubflow": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Whether the shuffle-subflow app is enabled",
"default": true
},
"version": {
"type": "string",
"description": "The version of the shuffle-subflow app to deploy.",
"default": "1.1.0"
}
}
},
"http": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"description": "Whether the http app is enabled",
"default": true
},
"version": {
"type": "string",
"description": "The version of the http app to deploy.",
"default": "1.4.0"
}
}
}
}
},
"ingress": { "ingress": {
"type": "object", "type": "object",
"properties": { "properties": {
+39 -348
View File
@@ -1743,10 +1743,16 @@ worker:
## @section app Parameters ## @section app Parameters
## ##
app: app:
## @param app.image.registry app image registry (defaults to shuffle.appRegistry)
## @param app.image.repository app image repository (defaults to shuffle.appBaseImageName)
## @param app.image.tag app image tag (defaults to the apps version)
## @param app.image.pullPolicy default image pull policy for app deployments. Only effective for helm-deployed apps (see apps.enabled). ## @param app.image.pullPolicy default image pull policy for app deployments. Only effective for helm-deployed apps (see apps.enabled).
## @param app.image.pullSecrets default image pull secrets for app deployments. Only effective for helm-deployed apps (see apps.enabled). ## @param app.image.pullSecrets default image pull secrets for app deployments. Only effective for helm-deployed apps (see apps.enabled).
## ##
image: image:
registry: ""
repository: ""
tag: ""
## Specify a imagePullPolicy ## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent' ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images ## ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images
@@ -1882,7 +1888,12 @@ app:
drop: ["ALL"] drop: ["ALL"]
seccompProfile: seccompProfile:
type: "RuntimeDefault" type: "RuntimeDefault"
## @param app.command Override default app container command (useful when using custom images)
##
command: []
## @param app.args Override default app container args (useful when using custom images)
##
args: []
## @param app.automountServiceAccountToken Mount Service Account token in app pods. Only effective for helm-deployed apps (see apps.enabled). ## @param app.automountServiceAccountToken Mount Service Account token in app pods. Only effective for helm-deployed apps (see apps.enabled).
## ##
automountServiceAccountToken: false automountServiceAccountToken: false
@@ -2109,6 +2120,9 @@ app:
## ##
extraEgress: [] extraEgress: []
## @param app.mountTmpVolume Whether a writable /tmp emptyDir volume should be mounted to the app.
##
mountTmpVolume: false
## @param app.exposedContainerPort The port that shuffle app containers will listen on for new requests. ## @param app.exposedContainerPort The port that shuffle app containers will listen on for new requests.
## ##
exposedContainerPort: 80 exposedContainerPort: 80
@@ -2134,373 +2148,50 @@ apps:
enabled: false enabled: false
shuffleTools: shuffleTools:
## @param apps.shuffleTools.enabled Whether the app is enabled ## @param apps.shuffleTools.enabled Whether the shuffle-tools app is enabled
## ##
enabled: true enabled: true
## app image ## @skip apps.shuffleTools.name
## @param apps.shuffleTools.image.registry app image registry
## @param apps.shuffleTools.image.repository app image repository
## @param apps.shuffleTools.image.tag app image tag (immutable tags are recommended, defaults to appVersion)
## @param apps.shuffleTools.image.digest app image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag image tag (immutable tags are recommended)
## @param apps.shuffleTools.image.pullPolicy app image pull policy
## @param apps.shuffleTools.image.pullSecrets app image pull secrets
## ##
image: name: shuffle-tools
registry: ghcr.io ## @param apps.shuffleTools.version The version of the shuffle-tools app to deploy.
repository: shuffle/shuffle-app
tag: ""
digest: ""
## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: https://kubernetes.io/docs/concepts/containers/images/#pre-pulled-images
##
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
## e.g:
## pullSecrets:
## - myRegistryKeySecretName
##
pullSecrets: []
## @param apps.shuffleTools.replicaCount [number, nullable] Number of app replicas to deploy
## ##
replicaCount: null version: 1.2.0
## @param apps.shuffleTools.extraContainerPorts Optionally specify extra list of additional ports for app containers
## e.g:
## extraContainerPorts:
## - name: myservice
## containerPort: 9090
##
extraContainerPorts: []
## Configure extra options for app containers' liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes
## @param apps.shuffleTools.livenessProbe.enabled Enable livenessProbe on app containers
## @param apps.shuffleTools.livenessProbe.initialDelaySeconds Initial delay seconds for livenessProbe
## @param apps.shuffleTools.livenessProbe.periodSeconds Period seconds for livenessProbe
## @param apps.shuffleTools.livenessProbe.timeoutSeconds Timeout seconds for livenessProbe
## @param apps.shuffleTools.livenessProbe.failureThreshold Failure threshold for livenessProbe
## @param apps.shuffleTools.livenessProbe.successThreshold Success threshold for livenessProbe
##
livenessProbe:
enabled: false
initialDelaySeconds: 0
periodSeconds: 15
timeoutSeconds: 1
failureThreshold: 4
successThreshold: 1
## @param apps.shuffleTools.readinessProbe.enabled Enable readinessProbe on app containers
## @param apps.shuffleTools.readinessProbe.initialDelaySeconds Initial delay seconds for readinessProbe
## @param apps.shuffleTools.readinessProbe.periodSeconds Period seconds for readinessProbe
## @param apps.shuffleTools.readinessProbe.timeoutSeconds Timeout seconds for readinessProbe
## @param apps.shuffleTools.readinessProbe.failureThreshold Failure threshold for readinessProbe
## @param apps.shuffleTools.readinessProbe.successThreshold Success threshold for readinessProbe
##
readinessProbe:
enabled: false
initialDelaySeconds: 0
periodSeconds: 5
timeoutSeconds: 1
failureThreshold: 3
successThreshold: 1
## @param apps.shuffleTools.startupProbe.enabled Enable startupProbe on app containers
## @param apps.shuffleTools.startupProbe.initialDelaySeconds Initial delay seconds for startupProbe
## @param apps.shuffleTools.startupProbe.periodSeconds Period seconds for startupProbe
## @param apps.shuffleTools.startupProbe.timeoutSeconds Timeout seconds for startupProbe
## @param apps.shuffleTools.startupProbe.failureThreshold Failure threshold for startupProbe
## @param apps.shuffleTools.startupProbe.successThreshold Success threshold for startupProbe
##
startupProbe:
enabled: false
initialDelaySeconds: 0
periodSeconds: 1
timeoutSeconds: 1
failureThreshold: 60
successThreshold: 1
## @param apps.shuffleTools.customLivenessProbe Custom livenessProbe that overrides the default one
##
customLivenessProbe: {}
## @param apps.shuffleTools.customReadinessProbe Custom readinessProbe that overrides the default one
##
customReadinessProbe: {}
## @param apps.shuffleTools.customStartupProbe Custom startupProbe that overrides the default one
##
customStartupProbe: {}
## app resource requests and limits
## ref: http://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
## @param apps.shuffleTools.resourcesPreset Set app container resources according to one common preset (allowed values: none, nano, small, medium, large, xlarge, 2xlarge). This is ignored if app.resources is set (app.resources is recommended for production).
## More information: https://github.com/bitnami/charts/blob/main/bitnami/common/templates/_resources.tpl#L15
## Shuffle gets OOM killed with 256M memory during startup. Up to 360MiB of memory usage were observed during testing.
## The small preset grants 512M.
##
resourcesPreset: "small"
## @param apps.shuffleTools.resources Set app container requests and limits for different resources like CPU or memory (essential for production workloads)
## Example:
## resources:
## requests:
## cpu: 2
## memory: 512Mi
## limits:
## cpu: 3
## memory: 1024Mi
##
resources: {}
## Configure Pods Security Context
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod
## @param apps.shuffleTools.podSecurityContext.enabled Enable app pods' Security Context
## @param apps.shuffleTools.podSecurityContext.fsGroupChangePolicy Set filesystem group change policy for app pods
## @param apps.shuffleTools.podSecurityContext.sysctls Set kernel settings using the sysctl interface for app pods
## @param apps.shuffleTools.podSecurityContext.supplementalGroups Set filesystem extra groups for app pods
## @param apps.shuffleTools.podSecurityContext.fsGroup Set fsGroup in app pods' Security Context
##
podSecurityContext:
enabled: true
fsGroupChangePolicy: Always
sysctls: []
supplementalGroups: []
fsGroup: 1001
## Configure Container Security Context
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container
## @param apps.shuffleTools.containerSecurityContext.enabled Enabled app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.seLinuxOptions [object,nullable] Set SELinux options in app container
## @param apps.shuffleTools.containerSecurityContext.runAsUser Set runAsUser in app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.runAsGroup Set runAsGroup in app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.runAsNonRoot Set runAsNonRoot in app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.readOnlyRootFilesystem Set readOnlyRootFilesystem in app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.privileged Set privileged in app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.allowPrivilegeEscalation Set allowPrivilegeEscalation in app container' Security Context
## @param apps.shuffleTools.containerSecurityContext.capabilities.drop List of capabilities to be dropped in app container
## @param apps.shuffleTools.containerSecurityContext.seccompProfile.type Set seccomp profile in app container
##
containerSecurityContext:
enabled: true
seLinuxOptions: {}
runAsUser: 1001
runAsGroup: 1001
runAsNonRoot: true
readOnlyRootFilesystem: true
privileged: false
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
seccompProfile:
type: "RuntimeDefault"
## @param apps.shuffleTools.command Override default app container command (useful when using custom images)
##
command: []
## @param apps.shuffleTools.args Override default app container args (useful when using custom images)
##
args: []
## @param apps.shuffleTools.automountServiceAccountToken Mount Service Account token in app pods
## NOTE: app requires the service account credentials to be mounted
##
automountServiceAccountToken: true
## @param apps.shuffleTools.hostAliases app pods host aliases
## https://kubernetes.io/docs/concepts/services-networking/add-entries-to-pod-etc-hosts-with-host-aliases/
##
hostAliases: []
## @param apps.shuffleTools.deploymentAnnotations Annotations for app deployment
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
##
deploymentAnnotations: {}
## @param apps.shuffleTools.podLabels Extra labels for app pods
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
##
podLabels: {}
## @param apps.shuffleTools.podAnnotations Annotations for app pods
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
##
podAnnotations: {}
## @param apps.shuffleTools.podAffinityPreset Pod affinity preset. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`
## ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
##
podAffinityPreset: ""
## @param apps.shuffleTools.podAntiAffinityPreset Pod anti-affinity preset. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`
## ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
##
podAntiAffinityPreset: soft
## Node app.affinity preset
## ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity
##
nodeAffinityPreset:
## @param apps.shuffleTools.nodeAffinityPreset.type Node affinity preset type. Ignored if `app.affinity` is set. Allowed values: `soft` or `hard`
##
type: ""
## @param apps.shuffleTools.nodeAffinityPreset.key Node label key to match. Ignored if `app.affinity` is set
##
key: ""
## @param apps.shuffleTools.nodeAffinityPreset.values Node label values to match. Ignored if `app.affinity` is set
## E.g.
## values:
## - e2e-az1
## - e2e-az2
##
values: []
## @param apps.shuffleTools.affinity Affinity for app pods assignment
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
## NOTE: `app.podAffinityPreset`, `app.podAntiAffinityPreset`, and `app.nodeAffinityPreset` will be ignored when it's set
##
affinity: {}
## @param apps.shuffleTools.nodeSelector Node labels for app pods assignment
## ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
##
nodeSelector: {}
## @param apps.shuffleTools.tolerations Tolerations for app pods assignment
## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
tolerations: []
## @param apps.shuffleTools.updateStrategy.type app deployment strategy type
## ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy
##
updateStrategy:
## Can be set to RollingUpdate or Recreate
## app uses ReadWriteOnce volumes by default, which is incompatible with RollingUpdate
##
type: Recreate
## @param apps.shuffleTools.priorityClassName app pods' priorityClassName
##
priorityClassName: ""
## @param apps.shuffleTools.topologySpreadConstraints Topology Spread Constraints for app pod assignment spread across your cluster among failure-domains
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/#spread-constraints-for-pods
##
topologySpreadConstraints: []
## @param apps.shuffleTools.schedulerName Name of the k8s scheduler (other than default) for app pods
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
schedulerName: ""
## @param apps.shuffleTools.terminationGracePeriodSeconds Seconds app pods need to terminate gracefully
## ref: https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods
##
terminationGracePeriodSeconds: ""
## @param apps.shuffleTools.lifecycleHooks for app containers to automate configuration before or after startup
##
lifecycleHooks: {}
## @param apps.shuffleTools.extraEnvVars Array with extra environment variables to add to app containers
## e.g:
## extraEnvVars:
## - name: FOO
## value: "bar"
##
extraEnvVars: []
## @param apps.shuffleTools.extraEnvVarsCM Name of existing ConfigMap containing extra env vars for app containers
##
extraEnvVarsCM: ""
## @param apps.shuffleTools.extraEnvVarsSecret Name of existing Secret containing extra env vars for app containers
##
extraEnvVarsSecret: ""
## @param apps.shuffleTools.extraVolumes Optionally specify extra list of additional volumes for the app pods
##
extraVolumes: []
## @param apps.shuffleTools.extraVolumeMounts Optionally specify extra list of additional volumeMounts for the app containers
##
extraVolumeMounts: []
## @param apps.shuffleTools.sidecars Add additional sidecar containers to the app pods
## e.g:
## sidecars:
## - name: your-image-name
## image: your-image
## imagePullPolicy: Always
## ports:
## - name: portname
## containerPort: 1234
##
sidecars: []
## @param apps.shuffleTools.initContainers Add additional init containers to the app pods
## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
## e.g:
## initContainers:
## - name: your-image-name
## image: your-image
## imagePullPolicy: Always
## command: ['sh', '-c', 'echo "hello world"']
##
initContainers: []
## Pod Disruption Budget configuration
## ref: https://kubernetes.io/docs/tasks/run-application/configure-pdb
## @param apps.shuffleTools.pdb.create Enable/disable a Pod Disruption Budget creation
## @param apps.shuffleTools.pdb.minAvailable Minimum number/percentage of pods that should remain scheduled
## @param apps.shuffleTools.pdb.maxUnavailable Maximum number/percentage of pods that may be made unavailable. Defaults to `1` if both `app.pdb.minAvailable` and `app.pdb.maxUnavailable` are empty.
##
pdb:
create: true
minAvailable: ""
maxUnavailable: ""
## Autoscaling configuration
## ref: https://kubernetes.io/docs/concepts/workloads/autoscaling/
##
autoscaling:
## @param apps.shuffleTools.autoscaling.vpa.enabled Enable VPA for app pods
## @param apps.shuffleTools.autoscaling.vpa.annotations Annotations for VPA resource
## @param apps.shuffleTools.autoscaling.vpa.controlledResources VPA List of resources that the vertical pod autoscaler can control. Defaults to cpu and memory
## @param apps.shuffleTools.autoscaling.vpa.maxAllowed VPA Max allowed resources for the pod
## @param apps.shuffleTools.autoscaling.vpa.minAllowed VPA Min allowed resources for the pod
##
vpa:
enabled: false
annotations: {}
controlledResources: []
maxAllowed: {}
minAllowed: {}
## @param apps.shuffleTools.autoscaling.vpa.updatePolicy.updateMode Autoscaling update policy
## Specifies whether recommended updates are applied when a Pod is started and whether recommended updates are applied during the life of a Pod
## Possible values are "Off", "Initial", "Recreate", and "Auto".
##
updatePolicy:
updateMode: Auto
## @param apps.shuffleTools.autoscaling.hpa.enabled Enable HPA for app pods
## @param apps.shuffleTools.autoscaling.hpa.minReplicas Minimum number of replicas
## @param apps.shuffleTools.autoscaling.hpa.maxReplicas Maximum number of replicas
## @param apps.shuffleTools.autoscaling.hpa.targetCPU Target CPU utilization percentage
## @param apps.shuffleTools.autoscaling.hpa.targetMemory Target Memory utilization percentage
##
hpa:
enabled: false
minReplicas: ""
maxReplicas: ""
targetCPU: ""
targetMemory: ""
## Service configuration
##
service:
## @param apps.shuffleTools.service.labels Extra labels for app service
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
##
labels: {}
shuffleSubflow: shuffleSubflow:
## @param apps.shuffleSubflow.enabled Whether the shuffle-subflow app is enabled
##
enabled: true enabled: true
## @skip apps.shuffleSubflow.name
##
name: shuffle-subflow
## @param apps.shuffleSubflow.version The version of the shuffle-subflow app to deploy.
##
version: 1.1.0
http: http:
## @param apps.http.enabled Whether the http app is enabled
##
enabled: true enabled: true
## @skip apps.http.name
##
name: http
## @param apps.http.version The version of the http app to deploy.
##
version: 1.4.0
# replicas, image, etc. # replicas, image, etc.
## @extra apps.MY_APP.app [string] The name of the app (required, e.g. shuffle-tools) ## @extra apps.MY_APP.app [string] The name of the app (required, e.g. shuffle-tools)
## @extra apps.MY_APP.version [string] The version of the app (required, e.g. 1.2.0) ## @extra apps.MY_APP.version [string] The version of the app (required, e.g. 1.2.0)
## TODO: Add more extras
## Add your own apps here. The key of the app does not matter, as long as it is unique. ## Add your own apps here. The key of the app does not matter, as long as it is unique.
## myApp: ## myApp:
## enabled: true ## enabled: true
## name: my-app ## name: my-app
## version: 1.0.0 ## version: 1.0.0
## ... ## ... Overwrite .app.* values here, e.g.:
## replicaCount: 3
# TODO: Convert sample value to @extra. ## resources: {}
## Sample value:
## - app: my-app # required
## version: 1.0.0 # required
## replicas: 1 # optional, default to 1
## image: "" # optional, default to appregistry/baseimage/appname:appversion
## imagePullSecrets: null # optional, defaults to global.imagePullSecrets
## resources: null # optional, defaults to app.resources (or app.resourcePreset)
## extraVolumes: [] # optional
## extraVolumeMounts: [] # optional
## extraEnv: [] # optional
## podSecurityContext: null # optional, defaults to app.podSecurityContext
## containerSecurityContext: null # optional, defaults to app.containerSecurityContext
## TODO: allow to override resources, serviceAccount, mounts, env, security Contexts. Fall back to app defaults as defined above.
##
## @section Traffic Exposure Parameters ## @section Traffic Exposure Parameters
## ##