Runtime Environment Variables
Runtime Environment Variables¶
The following table describes runtime flags that are set in the environment to change the behavior as well as enable or disable some features.
Flag |
Default |
Description |
Consumer |
---|---|---|---|
|
1 |
Controls Execution mode:
|
Intel Gaudi PyTorch Bridge |
|
False |
Creates of graph visualization files. The output dump graphs are in ./.graph_dumps folder |
Intel Gaudi software |
|
Unset |
Holds the configuration of recipe cache.
Configuration is encoded as a comma separated list in the following format:
‘
Note: If a recipe cache is shared among a few processes (scale up), it must be stored on a local physical disk. Avoid using remote drives (such as NFS) where file locks are not supported, as it it may lead to instability and unpredictable behavior. |
Intel Gaudi PyTorch Bridge |
|
INT64_MAX |
Limits internal graph size to specified number of opsReduces the lazy mode memory overhead. This will be improved in future releases. Note: This may affect performance. |
Intel Gaudi PyTorch Bridge |
|
False |
The dynamic shapes feature is disabled by default. If a model experiences excessive recompilations due to Dynamic Data or Ops,
this variable can be set to enable the Intel Gaudi PyTorch bridge and graph compiler to automatically manage dynamic shapes in model scripts. The graphs will be automatically bucketed and padded into
ranges to achieve a common size, reducing recompilations and and improving performance when working with dynamic workloads. To run with dynamic shapes handling enabled,
set |
Intel Gaudi PyTorch Bridge |
|
30000 |
If cache evictions cause performance degradation, increasing the cache size will increase performance. The default value is 30000. Note: Boost in performance may cause an increase in host memory consumption. |
Intel Gaudi PyTorch Bridge |
|
1 |
This flag turns on host time optimization of lazy ops accumulation. It offloads ops accumulation to a separate thread, thus reducing computation time of the main thread. |
Intel Gaudi PyTorch Bridge |
|
Unset |
Path (file), where the collected metrics are stored.
Metrics are stored in a file only when |
Intel Gaudi PyTorch Bridge |
|
process_exit |
Once Supported values:
Multiple triggers can be enabled together by separating them with
a comma, for example:
|
Intel Gaudi PyTorch Bridge |
|
json |
Metrics file format. Both JSON and TEXT formats are supported:
|
Intel Gaudi PyTorch Bridge |
|
True |
Enables generic stream[2] which allows a user to submit different types operations in same user stream[1]. For a usage example of this flag, see Wav2Vec2 inference script.
[1] User stream: A queue of device work. The host (user) places the work in this queue and continues immediately. The device schedules the work in this queue when the resources are free.
[2] Generic stream: A user stream where all operations can be pushed to a stream irrespective of the type of operation (copy, compute, or collective operations).
|
Intel Gaudi PyTorch Bridge |
|
False |
Temporary performance improvement option for torch.compile mode only. Due to early adoption phase of torch.compile, there are still many Operations executed eagerly outside of graphs. Eager execution is less performant, and this option reduces the negative performance impact of it. |
Intel Gaudi PyTorch Bridge |
|
False |
Accelerate Eager Mode by enabling multithreaded pipeline in operations processing:
|
Intel Gaudi PyTorch Bridge |
|
False |
Enables native support for tensors with INT64 datatype:
Important: Not all ops support INT64. If an Op does not support INT64, implicit casts can be added, otherwise, CPU fallback might occur (with performance impact) or runtime error is thrown. Limitation: This flag is supported for Gaudi 2 only with |
Intel Gaudi PyTorch Bridge |
The following table describes runtime flags that are set in the environment to obtain Intel Gaudi software and Intel Gaudi PyTorch bridge level logs.
Flag |
Default |
Description |
Consumer |
---|---|---|---|
|
0 |
A Bitmask specifying components inside Intel Gaudi PyTorch Bridge module that are allowed to use profilers. Note that certain profilers may require additional environment variables to be set.
|
Intel Gaudi PyTorch Bridge |
|
False |
If set to |
Intel Gaudi software and Intel Gaudi PyTorch Bridge |
|
5 |
Logging level from Intel Gaudi software, perf_lib and Intel Gaudi PyTorch Bridge.
By default, logs are placed either in the console
(if |
Intel Gaudi software and Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for Intel Gaudi PyTorch Bridge.
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
0 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |
|
5 |
Logging level for
|
Intel Gaudi PyTorch Bridge |