Deploy Options
Pixie offers the following deploy options:
To see the full set of deploy options, install the Pixie CLI and run px deploy --help
.
Annotating Pixie's resources
When deploying Pixie, you have the option of adding one or more custom annotations to the Kubernetes objects deployed by Pixie.
To annotate Pixie's resources when deploying with the Pixie CLI, use the --annotations
flag:
To annotate Pixie's resources when deploying with Helm, use the annotations
field:
Labeling Pixie's resources
When deploying Pixie, you have the option of adding one or more custom labels to the Kubernetes objects deployed by Pixie. Certain labels are reserved for internal use by Pixie. The following are reserved label strings that may not be used:
"vizier-bootstrap"
“component"
“vizier-updater-dep"
“app"
To label Pixie's resources when deploying with the Pixie CLI, use the --labels
flag:
To label Pixie's resources when deploying with Helm, use the labels
field:
Deploy Pixie to a subset of nodes
When deploying Pixie, you have the option of selecting which nodes Pixie will be deployed to in your cluster. For example, Pixie runs on Linux nodes only, so you would use this option to deploy Pixie to clusters with mixed node types.
To deploy Pixie to a subset of the nodes in your cluster:
Label the target nodes with pixie=allowed
.
Add a nodeSelector
to Pixie's vizier-pem
datacollector pods during deployment:
When deploying with the Pixie CLI, use the --patches
flag:
When deploying with Helm, use the patches.vizier-pem
field:
Providing a custom cluster name
When deploying Pixie, you have the option of providing a custom cluster name. If you do not provide a cluster name, the name will be taken from the current kubeconfig.
To provide a custom cluster name when deploying with the Pixie CLI, use the --cluster_name
flag:
To provide a custom cluster name when deploying with Helm, use the clusterName
field:
Configure Pixie memory usage
When deploying Pixie, you have three options for configuring PEM memory usage:
For more information about these options, please refer to the Tuning Memory Usage page.
Setting the memory limit
The default memory limit is 2Gi per PEM. The lowest recommended value is 1Gi per PEM. 1Gi is not a suitable limit for a cluster with high throughput, but it is suitable for a small cluster with limited resources.
To set Pixie's memory limit when deploying with the Pixie CLI, use the --pem_memory_limit
flag:
To set Pixie's memory limit when deploying with Helm, use the pemMemoryLimit
field:
Setting the memory request
By default, the PEM's memory request will be the same as the limit. When deploying Pixie, you can specify a different memory request than limit. This flag is useful for clusters in which the PEM pods are pending because there is not enough memory for them to be scheduled on the node.
Please note that PEM memory request must be less than or equal to the PEM memory limit.
To set Pixie's memory request when deploying with the Pixie CLI, use the --pem_memory_request
flag:
To set Pixie's memory request when deploying with Helm, use the pemMemoryRequest
field:
Setting the data table storage memory limit
This is an advanced option that most developers shouldn't need. For more discussion see the Tuning Memory Usage page.
To set Pixie's data store memory limit when deploying with the Pixie CLI, use the --pem_flags
flag:
To set Pixie's data store memory limit when deploying with Helm, use the dataCollectorParams.customPEMFlags.PL_TABLE_STORE_DATA_LIMIT_MB
field:
Setting the data access mode
When deploying Pixie, you have the option of setting the data access mode. This mode controls what data can be displayed when executing a PxL script to query Pixie platform telemetry data. Pixie offers two data access modes:
Full
(default): The user has full data access, and Pixie does not redact any collected data from the user during script execution.
Restricted
: The user has restricted data access. Pixie will redact all columns that may potentially contain sensitive data (e.g. request/response bodies and headers). All rows in the column will be redacted, regardless of whether they do or do not actually contain PII. Metadata about these columns, such as length, will still be queryable.
If no data access mode is specified when deploying Pixie, the deploy assumes Full
as default.
To set the data access mode when deploying with the Pixie CLI, use the --data_access
flag:
To set the data access mode when deploying with Helm, use the dataAccess
field:
You may also directly update the dataAccess
field in your values.yaml
file.
Select metadata storage option
By default, Pixie uses a persistent volume to store 24 hours' worth of Kubernetes metadata updates.
For clusters that don't support persistent volumes, we have an alternative mode that uses the etcd operator.
To deploy using the etcd operator using px deploy
, use the --use_etcd_operator
flag.
To deploy with Helm using the etcd operator, use the --useEtcdOperator
flag.
Custom Image Registry
By default, Pixie uses images hosted on gcr.io
. Pixie allows you to specify a custom image registry for clusters which may not have access to gcr.io
or for users who simply want to host their own images.
Prerequisites
- Install the crane cli utility. This will be used to copy Pixie's source images to your registry while keeping the the image digest intact. Note that
docker tag
and other tools that change the image digest will fail to install during the final deploy step.
Collect the Vizier images
- Download the Vizier artifacts:
- Determine whether you'd like to deploy Pixie with or without etcd. We recommend installing Pixie without etcd as long as your cluster supports Pixie creating and using PVs.
To list the images required to deploy Pixie without etcd (Recommended):
To list the images required to deploy Pixie with etcd:
- Collect and publish the images listed in Step 2 to your custom registry.
Note that Pixie expects hosted images to adhere to the following format: ${custom_registry}/${defaultImagePath | sed 's/\//-/g'}
. In other words, Pixie will expect your images to be hosted on your registry, where the image name in your registry is Pixie's full image path with any /
replaced with -
. For example: gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image:latest
should pushed to $registry/gcr.io-pixie-oss-pixie-dev-vizier-metadata_server_image:latest
and contain the same image digest.
This can be accomplished with the crane cli tool and an example of it's copy command is shown below:
Collect the OLM images
Pixie depends on OLM to deploy its operator. The required OLM images are listed below:
You will need to build your own OLM bundle.
Download opm:
Find the current operator version listed in the downloaded/pixie-operator/package.yaml
file.
Locate the operator's csv in the downloaded/pixie-operator/<version>/csv.yaml
file. Remove the replaces
line and update the image tag for gcr.io/pixie-oss/pixie-prod/operator/operator_image:<version>
to your hosted image.
Next, build your bundle by running the following:
Deploy Pixie using your custom registry
Deploy Pixie Vizier with the registry
flag. You can deploy Pixie Vizier in one of three ways.
Using the CLI:
Using Helm:
Using Yamls:
To deploy Pixie without etcd, use the following yamls:
To deploy Pixie with etcd, use the following yamls:
Set a PEM flag
PEM flags are used to configure the Pixie Edge Module, Pixie's data collector component. PEM flags are often used to enable beta features.
For example, to disable tracing for specific protocols (e.g. to reduce memory usage) when deploying with the Pixie CLI, use the --pem_flags
flag.
Flags follow the pattern PX_STIRLING_ENABLE_[PROTOCOL]_TRACING
. Available protocols are listed on the data sources page and tracing may be disabled by passing 0
or enabled with 1
.
To disable a protocol when deploying with Helm, use the dataCollectorParams.customPEMFlags
field: