diff --git a/functions/kubernetes/charts/shuffle/README.md b/functions/kubernetes/charts/shuffle/README.md index 7c1315b9..ee8f8e06 100644 --- a/functions/kubernetes/charts/shuffle/README.md +++ b/functions/kubernetes/charts/shuffle/README.md @@ -215,7 +215,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia ## Parameters -##### Global parameters +#### Global parameters | Name | Description | Value | | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | @@ -225,7 +225,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `global.compatibility.openshift.adaptSecurityContext` | Adapt the securityContext sections of the deployment to make them compatible with Openshift restricted-v2 SCC: remove runAsUser, runAsGroup and fsGroup and let the platform use their allowed default IDs. Possible values: auto (apply if the detected running cluster is Openshift), force (perform the adaptation always), disabled (do not perform adaptation) | `auto` | | `global.compatibility.omitEmptySeLinuxOptions` | If set to true, removes the seLinuxOptions from the securityContexts when it is set to an empty object | `false` | -##### Common parameters +#### Common parameters | Name | Description | Value | | ------------------------ | --------------------------------------------------------------------------------------- | --------------- | @@ -241,7 +241,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `diagnosticMode.command` | Command to override all containers in the chart release | `["sleep"]` | | `diagnosticMode.args` | Args to override all containers in the chart release | `["infinity"]` | -##### Shared Shuffle Parameters +#### Shared Shuffle Parameters | Name | Description | Value | | -------------------------- | -------------------------------------------------------------------------------------------------------------------------- | --------------- | @@ -251,7 +251,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `shuffle.appBaseImageName` | The base image used for shuffle apps. The final image for an app is //: | `frikky` | | `shuffle.timezone` | The timezone used by Shuffle | `Europe/Berlin` | -##### backend Parameters +#### backend Parameters | Name | Description | Value | | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- | @@ -356,6 +356,8 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `backend.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` | | `backend.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `backend.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | +| `backend.debug` | Enable debug mode for backend, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments. | `false` | +| `backend.autoGOMEMLIMIT` | Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit. | `true` | | `backend.cleanupSchedule` | The interval in seconds at which the cleanup job runs | `300` | | `backend.openSearch.url` | The URL at which OpenSearch is available | `http://{{ .Release.Name }}-opensearch:9200` | | `backend.openSearch.username` | The username that is used for authenticating with OpenSearch | `admin` | @@ -366,7 +368,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `backend.apps.downloadBranch` | The branch from which apps should be downloaded on startup. | `master` | | `backend.apps.forceUpdate` | Force an update of apps on startup. | `false` | -##### frontend Parameters +#### frontend Parameters | Name | Description | Value | | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | @@ -472,7 +474,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `frontend.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `frontend.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | -##### orborus Parameters +#### orborus Parameters | Name | Description | Value | | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | @@ -575,10 +577,12 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `orborus.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` | | `orborus.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `orborus.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | +| `orborus.debug` | Enable debug mode for orborus, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments. | `false` | +| `orborus.autoGOMEMLIMIT` | Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit. | `true` | | `orborus.executionConcurrency` | The maximum amount of concurrent workflow executions per worker | `25` | | `orborus.manageWorkerDeployments` | Whether workers are deployed and managed by orborus. When disabled, every worker is expected to be already deployed (see worker.enableHelmDeployment). | `true` | -##### worker Parameters +#### worker Parameters | Name | Description | Value | | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | @@ -684,9 +688,11 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `worker.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` | | `worker.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `worker.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | +| `worker.debug` | Enable debug mode for worker, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments. | `false` | +| `worker.autoGOMEMLIMIT` | Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit. | `true` | | `worker.manageAppDeployments` | Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see apps.enabled). | `true` | -##### app Parameters +#### app Parameters | Name | Description | Value | | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | @@ -789,120 +795,13 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `app.networkPolicy.allowExternalEgress` | Allow the pod to access any range of port and all destinations. | `true` | | `app.networkPolicy.extraIngress` | Add extra ingress rules to the NetworkPolicy | `[]` | | `app.networkPolicy.extraEgress` | Add extra ingress rules to the NetworkPolicy (ignored if allowExternalEgress=true) | `[]` | +| `app.debug` | Enable debug mode for app, which enables extra logging | `false` | | `app.mountTmpVolume` | Whether a writable /tmp emptyDir volume should be mounted to the app. | `true` | | `app.exposedContainerPort` | The port that shuffle app containers will listen on for new requests. | `80` | | `app.sdkTimeout` | The timeout in seconds for app actions. | `300` | | `app.disableLogs` | Do not capture app logs. By default, app logs are captured, so that they are visible in the frontend. | `false` | -##### Parameters to deploy apps using helm - -| Name | Description | Value | -| ----------------------------- | -------------------------------------------------- | ------- | -| `apps.enabled` | Whether apps should be deployed using helm. | `false` | -| `apps.shuffleTools.enabled` | Whether the shuffle-tools app is enabled | `true` | -| `apps.shuffleTools.version` | The version of the shuffle-tools app to deploy. | `1.2.0` | -| `apps.shuffleSubflow.enabled` | Whether the shuffle-subflow app is enabled | `true` | -| `apps.shuffleSubflow.version` | The version of the shuffle-subflow app to deploy. | `1.1.0` | -| `apps.http.enabled` | Whether the http app is enabled | `true` | -| `apps.http.version` | The version of the http app to deploy. | `1.4.0` | -| `apps.MY_APP.app` | The name of the app (required, e.g. shuffle-tools) | | -| `apps.MY_APP.version` | The version of the app (required, e.g. 1.2.0) | | - -##### Traffic Exposure Parameters - -| Name | Description | Value | -| -------------------------- | ----------------------------------------------------------------------------------------------------- | --------------- | -| `ingress.enabled` | Enable ingress record generation for frontend and backend | `false` | -| `ingress.pathType` | Ingress path type for the frontend path | `Prefix` | -| `ingress.backendPathType` | Ingress path type for the backend path | `Prefix` | -| `ingress.apiVersion` | Force Ingress API version (automatically detected if not set) | `""` | -| `ingress.hostname` | Default host for the ingress record | `shuffle.local` | -| `ingress.ingressClassName` | IngressClass that will be be used to implement the Ingress (Kubernetes 1.18+) | `nginx` | -| `ingress.path` | Ingress path for Shuffle frontend | `"/"` | -| `ingress.backendPath` | Ingress path for Shuffle backend | `"/api/"` | -| `ingress.annotations` | Additional annotations for the Ingress resource. | `{}` | -| `ingress.tls` | Enable TLS configuration for the host defined at `ingress.hostname` parameter | `false` | -| `ingress.selfSigned` | Create a TLS secret for this ingress record using self-signed certificates generated by Helm | `false` | -| `ingress.extraHosts` | An array with additional hostname(s) to be covered with the ingress record | `[]` | -| `ingress.extraPaths` | An array with additional arbitrary paths that may need to be added to the ingress under the main host | `[]` | -| `ingress.extraTls` | TLS configuration for additional hostname(s) to be covered with this ingress record | `[]` | -| `ingress.secrets` | Custom TLS certificates as secrets | `[]` | -| `ingress.extraRules` | Additional rules to be covered with this ingress record | `[]` | - -##### Istio Parameters - -| Name | Description | Value | -| --------------------------------------- | ------------------------------------------------------------------------------- | ------------------------ | -| `istio.enabled` | Enable creation of an Istio Gateway and VirtualService for frontend and backend | `false` | -| `istio.apiVersion` | The istio apiVersion to use for Gateway and VirtualService resources | `networking.istio.io/v1` | -| `istio.hosts` | One or more hosts exposed by Istio | `[]` | -| `istio.gateway.annotations` | Additional annotations for the Gateway resource | `{}` | -| `istio.gateway.selector` | The selector matches the ingress gateway pod labels | `{ istio: ingress }` | -| `istio.gateway.http.enabled` | Enable HTTP server port 80 | `true` | -| `istio.gateway.http.httpsRedirect` | If set to true, a 301 redirect is send for all HTTP connections | `false` | -| `istio.gateway.https.enabled` | Enable HTTPS server on port 443 | `false` | -| `istio.gateway.https.tlsCredentialName` | The name of the secret that holds the TLS certs including the CA certificates. | `""` | -| `istio.gateway.https.tlsCipherSuites` | If specified, only support the specified cipher list. | `[]` | -| `istio.gateway.extraServers` | Additional servers for the Gateway resource | `[]` | -| `istio.virtualService.annotations` | Additional annotations for the VirtualService resource. | `{}` | -| `istio.virtualService.backendHeaders` | Header manipulation rules for backend traffic | `{}` | -| `istio.virtualService.frontendHeaders` | Header manipulation rules for frontend traffic | `{}` | - -##### Persistence Parameters - -| Name | Description | Value | -| ------------------------------------- | ------------------------------------------------- | ------------------- | -| `persistence.enabled` | Enable persistence using Persistent Volume Claims | `true` | -| `persistence.apps.existingClaim` | Name of an existing PVC to use | `""` | -| `persistence.apps.storageClass` | PVC Storage Class for shuffle-apps volume | `""` | -| `persistence.apps.subPath` | The sub path used in the volume | `""` | -| `persistence.apps.accessModes` | The access mode of the volume | `["ReadWriteOnce"]` | -| `persistence.apps.size` | The size of the volume | `5Gi` | -| `persistence.apps.annotations` | Annotations for the PVC | `{}` | -| `persistence.apps.selector` | Selector to match an existing Persistent Volume | `{}` | -| `persistence.appBuilder.storageClass` | PVC Storage Class for backend-apps-claim volume | `""` | -| `persistence.appBuilder.accessModes` | The access mode of the volume | `["ReadWriteOnce"]` | -| `persistence.appBuilder.size` | The size of the volume | `5Gi` | -| `persistence.appBuilder.annotations` | Annotations for the PVC | `{}` | -| `persistence.appBuilder.selector` | Selector to match an existing Persistent Volume | `{}` | -| `persistence.files.existingClaim` | Name of an existing PVC to use | `""` | -| `persistence.files.storageClass` | PVC Storage Class for shuffle-files volume | `""` | -| `persistence.files.subPath` | The sub path used in the volume | `""` | -| `persistence.files.accessModes` | The access mode of the volume | `["ReadWriteOnce"]` | -| `persistence.files.size` | The size of the volume | `5Gi` | -| `persistence.files.annotations` | Annotations for the PVC | `{}` | -| `persistence.files.selector` | Selector to match an existing Persistent Volume | `{}` | - -##### Init Container Parameters - -| Name | Description | Value | -| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | -| `volumePermissions.enabled` | Enable init container that changes the owner/group of the PV mount point to `runAsUser:fsGroup` | `false` | -| `volumePermissions.image.registry` | OS Shell + Utility image registry | `docker.io` | -| `volumePermissions.image.repository` | OS Shell + Utility image repository | `bitnamilegacy/os-shell` | -| `volumePermissions.image.tag` | OS Shell + Utility image tag (immutable tags are recommended) | `12-debian-12-r30` | -| `volumePermissions.image.pullPolicy` | OS Shell + Utility image pull policy | `IfNotPresent` | -| `volumePermissions.image.pullSecrets` | OS Shell + Utility image pull secrets | `[]` | -| `volumePermissions.resourcesPreset` | Set init container resources according to one common preset (allowed values: none, nano, small, medium, large, xlarge, 2xlarge). This is ignored if volumePermissions.resources is set (volumePermissions.resources is recommended for production). | `nano` | -| `volumePermissions.resources` | Set init container requests and limits for different resources like CPU or memory (essential for production workloads) | `{}` | -| `volumePermissions.containerSecurityContext.enabled` | Enabled init container' Security Context | `true` | -| `volumePermissions.containerSecurityContext.seLinuxOptions` | Set SELinux options in init container | `{}` | -| `volumePermissions.containerSecurityContext.runAsUser` | Set init container's Security Context runAsUser | `0` | - -##### OpenSearch Parameters - -| Name | Description | Value | -| -------------------- | ----------------------------------------------------- | ------ | -| `opensearch.enabled` | Switch to enable or disable the opensearch helm chart | `true` | - -##### Vault Parameters - -| Name | Description | Value | -| --------------- | -------------------------------------------------------------------------- | ----- | -| `vault.role` | Specify the Vault role, which should be used to get the secret from Vault. | `""` | -| `vault.secrets` | A list of VaultSecrets to create | `[]` | - -##### Other Parameters +#### Parameters to deploy apps using helm | Name | Description | Value | | ----------------------------- | -------------------------------------------------- | ------- | @@ -1012,6 +911,7 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia #### Other Parameters + | Name | Description | Value | | ----------------------------- | -------------------------------------------------- | ------- | | `apps.enabled` | Whether apps should be deployed using helm. | `false` | @@ -1119,3 +1019,4 @@ The password should be provided with the `SHUFFLE_OPENSEARCH_PASSWORD` env varia | `vault.secrets` | A list of VaultSecrets to create | `[]` | ### Other Parameters + diff --git a/functions/kubernetes/charts/shuffle/templates/_helpers.tpl b/functions/kubernetes/charts/shuffle/templates/_helpers.tpl index b78602bb..c3fae7b2 100644 --- a/functions/kubernetes/charts/shuffle/templates/_helpers.tpl +++ b/functions/kubernetes/charts/shuffle/templates/_helpers.tpl @@ -4,3 +4,46 @@ Return the proper image name (for the init container volume-permissions image) {{- define "shuffle.volumePermissions.image" -}} {{- include "common.images.image" ( dict "imageRoot" .Values.volumePermissions.image "global" .Values.global "chart" .Chart ) -}} {{- end -}} + +{{/* +Return a value for the GOMEMLIMIT env variable based on a given kubernetes resource memory limit. +Usage: +{{ include "shuffle.k8sMemoryLimitToGOMEMLIMIT" (dict "k8sMemoryLimit" $.Values.resources.limits.memory "context" $) }} +*/}} +{{- define "shuffle.k8sMemoryLimitToGOMEMLIMIT" -}} +{{- $memoryLimit := .k8sMemoryLimit | default "" -}} +{{- $result := "" -}} +{{- if and $memoryLimit (gt (len $memoryLimit) 0) -}} +{{- $bytes := 0 -}} +{{- if hasSuffix "Ki" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "Ki" | int) 1024 -}} +{{- else if hasSuffix "Mi" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "Mi" | int) 1048576 -}} +{{- else if hasSuffix "Gi" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "Gi" | int) 1073741824 -}} +{{- else if hasSuffix "Ti" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "Ti" | int) 1099511627776 -}} +{{- else if hasSuffix "Pi" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "Pi" | int) 1125899906842624 -}} +{{- else if hasSuffix "Ei" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "Ei" | int) 1152921504606846976 -}} +{{- else if hasSuffix "K" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "K" | int) 1000 -}} +{{- else if hasSuffix "M" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "M" | int) 1000000 -}} +{{- else if hasSuffix "G" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "G" | int) 1000000000 -}} +{{- else if hasSuffix "T" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "T" | int) 1000000000000 -}} +{{- else if hasSuffix "P" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "P" | int) 1000000000000000 -}} +{{- else if hasSuffix "E" $memoryLimit -}} +{{- $bytes = mul ($memoryLimit | trimSuffix "E" | int) 1000000000000000000 -}} +{{- else -}} +{{- $bytes = $memoryLimit | int -}} +{{- end -}} +{{- $gomaxmem := div (mul $bytes 9) 10 -}} +{{- $result = printf "%d" $gomaxmem -}} +{{- end -}} +{{- $result -}} +{{- end -}} \ No newline at end of file diff --git a/functions/kubernetes/charts/shuffle/templates/backend/_helpers.tpl b/functions/kubernetes/charts/shuffle/templates/backend/_helpers.tpl index 48bd6543..13cb0ce7 100644 --- a/functions/kubernetes/charts/shuffle/templates/backend/_helpers.tpl +++ b/functions/kubernetes/charts/shuffle/templates/backend/_helpers.tpl @@ -113,4 +113,13 @@ SHUFFLE_OPENSEARCH_INDEX_PREFIX: "{{ .Values.backend.openSearch.indexPrefix }}" SHUFFLE_RERUN_SCHEDULE: "{{ .Values.backend.cleanupSchedule }}" TZ: "{{ .Values.shuffle.timezone }}" REGISTRY_URL: "{{ .Values.shuffle.appRegistry }}" # Used by app builder +{{- if .Values.backend.debug }} +DEBUG: "true" +{{- end }} +{{- if .Values.backend.autoGOMEMLIMIT }} +{{- $backendResources := (.Values.backend.resources | default (include "common.resources.preset" (dict "type" .Values.backend.resourcesPreset) | fromYaml)) -}} +{{- if and $backendResources $backendResources.limits $backendResources.limits.memory }} +GOMEMLIMIT: {{ include "shuffle.k8sMemoryLimitToGOMEMLIMIT" (dict "k8sMemoryLimit" $backendResources.limits.memory) | quote }} +{{- end }} +{{- end }} {{- end -}} \ No newline at end of file diff --git a/functions/kubernetes/charts/shuffle/templates/orborus/_helpers.tpl b/functions/kubernetes/charts/shuffle/templates/orborus/_helpers.tpl index 3ced6821..0af574a1 100644 --- a/functions/kubernetes/charts/shuffle/templates/orborus/_helpers.tpl +++ b/functions/kubernetes/charts/shuffle/templates/orborus/_helpers.tpl @@ -89,6 +89,15 @@ TZ: "{{ .Values.shuffle.timezone }}" BASE_URL: {{ include "shuffle.backend.baseUrl" . | quote }} KUBERNETES_NAMESPACE: "{{ .Release.Namespace }}" SHUFFLE_ORBORUS_EXECUTION_CONCURRENCY: {{ .Values.orborus.executionConcurrency | quote }} +{{- if .Values.orborus.debug }} +DEBUG: "true" +{{- end }} +{{- if .Values.orborus.autoGOMEMLIMIT }} +{{- $orborusResources := (.Values.orborus.resources | default (include "common.resources.preset" (dict "type" .Values.orborus.resourcesPreset) | fromYaml)) -}} +{{- if and $orborusResources $orborusResources.limits $orborusResources.limits.memory }} +GOMEMLIMIT: {{ include "shuffle.k8sMemoryLimitToGOMEMLIMIT" (dict "k8sMemoryLimit" $orborusResources.limits.memory) | quote }} +{{- end }} +{{- end }} {{- if .Values.orborus.manageWorkerDeployments }} # Shuffle worker configuration diff --git a/functions/kubernetes/charts/shuffle/templates/shuffle-app/_helpers.tpl b/functions/kubernetes/charts/shuffle/templates/shuffle-app/_helpers.tpl index 5963f9c1..fd6cb6c9 100644 --- a/functions/kubernetes/charts/shuffle/templates/shuffle-app/_helpers.tpl +++ b/functions/kubernetes/charts/shuffle/templates/shuffle-app/_helpers.tpl @@ -121,9 +121,15 @@ Usage: {{/* Return the environment variables of shuffle apps in the format KEY: VALUE +Usage: +{{- include "shuffle.appInstance.env" (dict "app" $appValues "context" $) -}} + +WARNING: Do NOT add environment variables here that would conflict with shuffle.worker.env or shuffle.orborus.env. +Worker also sets all env variables that are defined here, because they will be passed down to apps when worker.manageAppDeployments is set. +Instead, add them directly to the deployment template (shuffle-apps.yaml). */}} {{- define "shuffle.appInstance.env" -}} -SHUFFLE_APP_SDK_TIMEOUT: {{ .Values.app.sdkTimeout | quote }} -SHUFFLE_APP_EXPOSED_PORT: {{ .Values.app.exposedContainerPort | quote }} -SHUFFLE_LOGS_DISABLED: {{ .Values.app.disableLogs | quote }} +SHUFFLE_APP_SDK_TIMEOUT: {{ .app.sdkTimeout | quote }} +SHUFFLE_APP_EXPOSED_PORT: {{ .app.exposedContainerPort | quote }} +SHUFFLE_LOGS_DISABLED: {{ .app.disableLogs | quote }} {{- end -}} diff --git a/functions/kubernetes/charts/shuffle/templates/shuffle-app/shuffle-apps.yaml b/functions/kubernetes/charts/shuffle/templates/shuffle-app/shuffle-apps.yaml index af04851f..521e7fe8 100644 --- a/functions/kubernetes/charts/shuffle/templates/shuffle-app/shuffle-apps.yaml +++ b/functions/kubernetes/charts/shuffle/templates/shuffle-app/shuffle-apps.yaml @@ -168,7 +168,11 @@ spec: value: {{ include "shuffle.backend.baseUrl" $ | quote }} - name: SHUFFLE_SWARM_CONFIG value: run # Shuffle Worker requires this to be set even when using K8s instead of swarm - {{- $env := include "shuffle.appInstance.env" $ | fromYaml }} + {{- if $appValues.debug }} + - name: DEBUG + value: "true" + {{- end }} + {{- $env := include "shuffle.appInstance.env" (dict "app" $appValues "context" $) | fromYaml }} {{- range $key, $val := $env }} - name: {{ $key | quote }} value: {{ $val | quote }} diff --git a/functions/kubernetes/charts/shuffle/templates/shuffle-worker/_helpers.tpl b/functions/kubernetes/charts/shuffle/templates/shuffle-worker/_helpers.tpl index 8feafa76..c0e75c72 100644 --- a/functions/kubernetes/charts/shuffle/templates/shuffle-worker/_helpers.tpl +++ b/functions/kubernetes/charts/shuffle/templates/shuffle-worker/_helpers.tpl @@ -131,12 +131,13 @@ http://shuffle-workers.{{ .Release.Namespace }}.svc.cluster.local {{/* Return the environment variables of shuffle-worker in the format KEY: VALUE + +WARNING: Do NOT add environment variables here that would conflict with shuffle.orborus.env. +Orborus also sets all env variables that are defined here, because they will be passed down to worker when orborus.manageWorkerDeployments is set. +Instead, add them directly to the deployment template (shuffle-worker-dpl.yaml). */}} {{- define "shuffle.workerInstance.env" -}} -IS_KUBERNETES: "true" -KUBERNETES_NAMESPACE: "{{ .Release.Namespace }}" SHUFFLE_SWARM_CONFIG: "run" # Shuffle Worker requires this to be set even when using K8s instead of swarm -BASE_URL: {{ include "shuffle.backend.baseUrl" . | quote }} SHUFFLE_APP_EXPOSED_PORT: {{ .Values.app.exposedContainerPort | quote }} WORKER_HOSTNAME: {{ include "shuffle.worker.hostname" . }} @@ -177,6 +178,6 @@ SHUFFLE_APP_EPHEMERAL_STORAGE_LIMIT: {{ (index $appResources.limits "ephemeral-s {{- end }} # Include shuffle app environment variables. Worker passes them down to apps, when creating their deployment. -{{ include "shuffle.appInstance.env" . }} +{{ include "shuffle.appInstance.env" (dict "app" .Values.app "context" $) -}} {{- end }} {{- end -}} diff --git a/functions/kubernetes/charts/shuffle/templates/shuffle-worker/shuffle-worker-dpl.yaml b/functions/kubernetes/charts/shuffle/templates/shuffle-worker/shuffle-worker-dpl.yaml index 1e724ab3..eb1c1682 100644 --- a/functions/kubernetes/charts/shuffle/templates/shuffle-worker/shuffle-worker-dpl.yaml +++ b/functions/kubernetes/charts/shuffle/templates/shuffle-worker/shuffle-worker-dpl.yaml @@ -85,6 +85,23 @@ spec: env: - name: CLEANUP value: "false" # Do not remove resources when restarting worker + - name: IS_KUBERNETES + value: "true" + - name: KUBERNETES_NAMESPACE + value: {{ .Release.Namespace | quote }} + - name: BASE_URL + value: {{ include "shuffle.backend.baseUrl" . | quote }} + {{- if .Values.worker.debug }} + - name: DEBUG + value: "true" + {{- end }} + {{- if .Values.worker.autoGOMEMLIMIT }} + {{- $workerResources := (.Values.worker.resources | default (include "common.resources.preset" (dict "type" .Values.worker.resourcesPreset) | fromYaml)) -}} + {{- if and $workerResources $workerResources.limits $workerResources.limits.memory }} + - name: GOMEMLIMIT + value: {{ include "shuffle.k8sMemoryLimitToGOMEMLIMIT" (dict "k8sMemoryLimit" $workerResources.limits.memory) | quote }} + {{- end }} + {{- end }} {{- $env := include "shuffle.workerInstance.env" . | fromYaml }} {{- range $key, $val := $env }} - name: {{ $key | quote }} diff --git a/functions/kubernetes/charts/shuffle/values.schema.json b/functions/kubernetes/charts/shuffle/values.schema.json index ca262866..10ae8ca9 100644 --- a/functions/kubernetes/charts/shuffle/values.schema.json +++ b/functions/kubernetes/charts/shuffle/values.schema.json @@ -771,6 +771,16 @@ } } }, + "debug": { + "type": "boolean", + "description": "Enable debug mode for backend, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments.", + "default": false + }, + "autoGOMEMLIMIT": { + "type": "boolean", + "description": "Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit.", + "default": true + }, "cleanupSchedule": { "type": "number", "description": "The interval in seconds at which the cleanup job runs", @@ -2059,6 +2069,16 @@ } } }, + "debug": { + "type": "boolean", + "description": "Enable debug mode for orborus, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments.", + "default": false + }, + "autoGOMEMLIMIT": { + "type": "boolean", + "description": "Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit.", + "default": true + }, "executionConcurrency": { "type": "number", "description": "The maximum amount of concurrent workflow executions per worker", @@ -2703,6 +2723,16 @@ } } }, + "debug": { + "type": "boolean", + "description": "Enable debug mode for worker, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments.", + "default": false + }, + "autoGOMEMLIMIT": { + "type": "boolean", + "description": "Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit.", + "default": true + }, "manageAppDeployments": { "type": "boolean", "description": "Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see apps.enabled).", @@ -3322,6 +3352,11 @@ } } }, + "debug": { + "type": "boolean", + "description": "Enable debug mode for app, which enables extra logging", + "default": false + }, "mountTmpVolume": { "type": "boolean", "description": "Whether a writable /tmp emptyDir volume should be mounted to the app.", diff --git a/functions/kubernetes/charts/shuffle/values.yaml b/functions/kubernetes/charts/shuffle/values.yaml index 590d2c10..87fd3bb0 100644 --- a/functions/kubernetes/charts/shuffle/values.yaml +++ b/functions/kubernetes/charts/shuffle/values.yaml @@ -516,6 +516,16 @@ backend: ## extraEgress: [] + ## @param backend.debug Enable debug mode for backend, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments. + ## + debug: false + + ## @param backend.autoGOMEMLIMIT Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit. + ## This helps prevent Go runtime memory limits from exceeding container limits, which can cause OOM kills. + ## Only effective when backend.resources.limits.memory is set or a resourcesPreset is used that defines memory limits. + ## + autoGOMEMLIMIT: true + ## @param backend.cleanupSchedule The interval in seconds at which the cleanup job runs ## cleanupSchedule: 300 @@ -1329,6 +1339,17 @@ orborus: ## extraEgress: [] + ## @param orborus.debug Enable debug mode for orborus, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments. + ## + debug: false + + ## @param orborus.autoGOMEMLIMIT Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit. + ## This helps prevent Go runtime memory limits from exceeding container limits, which can cause OOM kills. + ## Only effective when orborus.resources.limits.memory is set or a resourcesPreset is used that defines memory limits. + ## Only effective when orborus is deployed via helm. + ## + autoGOMEMLIMIT: true + ## @param orborus.executionConcurrency The maximum amount of concurrent workflow executions per worker ## executionConcurrency: 25 @@ -1739,6 +1760,17 @@ worker: ## extraEgress: [] + ## @param worker.debug Enable debug mode for worker, which enables extra logging. orborus.debug, worker.debug, and app.debug should be set to identical values, if shuffle manages these deployments. + ## + debug: false + + ## @param worker.autoGOMEMLIMIT Automatically set GOMEMLIMIT environment variable to 90% of the container memory limit. + ## This helps prevent Go runtime memory limits from exceeding container limits, which can cause OOM kills. + ## Only effective when worker.resources.limits.memory is set or a resourcesPreset is used that defines memory limits. + ## Only effective when worker is deployed via helm (see worker.enableHelmDeployment). + ## + autoGOMEMLIMIT: true + ## @param worker.manageAppDeployments Whether apps are deployed and managed by worker. When disabled, every used app is expected to to be already deployed (see apps.enabled). ## This effectively removes required RBAC permissions from the shuffle-worker service account to create deployments and services. ## The worker might still attempt to create kubernetes objects, resulting in an error. There is currently no way to tell the worker, that it should not manage k8s resources. @@ -2124,6 +2156,9 @@ app: ## extraEgress: [] + ## @param app.debug Enable debug mode for app, which enables extra logging + ## + debug: false ## @param app.mountTmpVolume Whether a writable /tmp emptyDir volume should be mounted to the app. ## mountTmpVolume: true