Execution memory settings
There are three places to impact the memory of an execution run pod. Execution pods contain three separate containers, Run, Executor & Nginx. The Kubernetes Memory Requests can be resized in the following ways.
This is set from the Hardware Tier definition page, Admin>>Advanced>>Hardware Tiers. Memory Requests set here will size the Run container memory where user executions occur.
The memory request size for this container where the domino executor runs can be set by the central config setting,
default value: 1500Mib
We have seen failures related to this setting when importing large projects. Imported projects in the 10s of GB range can cause run execution failures during the preparing phase of startup. If you are seeing this failure mode you can add additional memory here. Update this setting cautiously as it will use the additional memory resources on every execution.
Memory errors can express in a variety of ways, but an example of the execution logging we have seen related to this failure looks like this...
2021-02-11T08:18:48.718Z: Preparing working directory. 2021-02-11T08:19:18.904Z: Preparing working directory. 2021-02-11T08:19:44.888Z: Caught termination signal! 2021-02-11T08:19:44.888Z: -- killed by pod termination -- 2021-02-11T08:19:44.964Z: Trapped error signal in start-run, exiting
The memory request for the nginx container in the execution can be similarly set in Central Config,
default value: 50Mib
This setting is the total memory for nginx and should be set higher than client_max_body_size. The client_max_body_size controls the maximum size of individual files that can be uploaded to the execution.
default value: 25mb
If you need to increase this setting then you will need to ensure the memory request setting for nginx is increased as well.
More information on updating values in central config is here.