Multiple Operators and Selective Upgrades

It is possible to set up multiple Camel K operators on a cluster to watch resources on namespaces. It’s not possible to configure Camel K this way using OLM (Operator Hub), since OLM prevents two operators from watching the same namespaces, but it’s technically possible to achieve this setup manually.

A typical example is when you need to install multiple global operators in different namespaces in order to have multiple tenants on the cluster working with Camel K. In this multi operator situation Camel K needs to avoid that the same resources on the cluster are managed by more than one operator at the same time. Operators must not contend the integration and the reconciliation because this will most probably result in an error (actual behavior is undefined).

To avoid contention, Camel K uses an operator id for each operator. The operator id must be unique on the cluster and any custom resource (CR) is assigned to a specific operator using an annotation. The assigned operator will be responsible for the reconciliation of the annotated CR and explicitly manages the resource no matter where it lives.

In detail, the Camel K operator supports the environment variable OPERATOR_ID. The value is an identifier that can be equal to any string (e.g. OPERATOR_ID=operator-1). Once the operator is assigned with an identifier, it will only reconcile Camel K custom resources that are assigned to that ID (unannotated resources will be ignored as well).

By default, the Camel K operator is using the id camel-k. When installing many operators the instances must use a different operator id. (e.g. kamel install --global --olm=false -n camel-ns-1 --operator-id=operator-2).

To assign a resource to a specific operator, the user can annotate it with camel.apache.org/operator.id. For example:

kind: Integration
apiVersion: camel.apache.org/v1
metadata:
  annotations:
    camel.apache.org/operator.id: operator-2
# ...

By default, Camel K custom resources use the default operator id camel-k as a value in this annotation. And more precisely the default operator with id camel-k and only this specific operator is allowed to also reconcile resources that are missing the operator id annotation.

The annotation can be put on any resource belonging to the "camel.apache.org" group.

When a resource creates additional resources in order to proceed with the reconciliation (for example an Integration may create an IntegrationKit, which in turn creates a Build resource), the annotation will be propagated to all resources created in the process, so they’ll be all reconciled by the same operator.

By using the camel.apache.org/operator.id annotation, it’s possible to move integrations between two or more operators running different versions of the Camel K platform, i.e. selectively upgrading or downgrading them. Just change the annotation on that particula resource to point to a new operator id:

kubectl annotate integration timer-to-log camel.apache.org/operator.id=operator-2 --overwrite

Configuring Multiple Integration Platforms

Any running Camel K integration is associated to a shared IntegrationPlatform resource that contains general configuration options. The integration platform is located in the integration namespace (or also in the operator namespace, in case of global installation) and typically only one ("primary", see later) integration platform is allowed to obtain a "Ready" state in a namespace, while others get the "Duplicate" state (i.e. IntegrationPlatform resources are somewhat "singleton" in a namespace).

There’s a way to allow two or more integration platforms to get a "Ready" state in a namespace and for them to be used by integrations: platforms can be marked with the annotation camel.apache.org/secondary.platform=true. That annotation marks the platform as secondary so that it will never be used as default platform during the reconciliation of an integration, unless explicitly selected (any resource belonging to the "camel.apache.org" group can select a particular integration platform). Secondary platforms are also allowed to reach the "Ready" state without becoming "Duplicate".

To specify which integration platform should be used to reconcile a specific CR, the CR can be annotated like in the following example:

kind: Integration
apiVersion: camel.apache.org/v1
metadata:
  annotations:
    camel.apache.org/platform.id: my-platform-name
# ...

The value of the camel.apache.org/platform.id annotation must match the name of an IntegrationPlatform custom resource, in the annotated resource namespace or also in the operator namespace.

The selection of a secondary IntegrationPlatform enables new configuration scenarios, for example, sharing global configuration options for groups of integrations, or also providing per-operator specific configuration options e.g. when you install multiple global operators in the same namespace.

Back to top