Skip to main content
Version: 3.0 (beta)

Virtual Cluster Instance

A virtual cluster is a fully functional Kubernetes cluster that runs inside the namespace of another Kubernetes cluster (host cluster). Virtual clusters are very useful if you are hitting the limits of namespaces and do not want to make special exceptions to the multi-tenancy configuration of the underlying cluster, e.g. a user needs their own CRD or user needs pods from 2 namespaces to communicate with each other but your standard NetworkPolicy does not allow this, then a virtual cluster may be perfect for this user.

Example Virtual Cluster​

An example Virtual Cluster:

apiVersion: management.loft.sh/v1
kind: VirtualClusterInstance
metadata:
  creationTimestamp: null
  name: my-virtual-cluster
  namespace: loft-p-my-project
spec:
  clusterRef: {}
  displayName: my-display-name
  owner:
    user: my-user
  parameters: 'my-parameter: my-value'
  templateRef:
    name: my-virtual-cluster-template
status: {}

Virtual Cluster Reference​

kind required string ​

Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

apiVersion required string ​

APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

metadata required object ​

name required string ​

Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names

generateName required string ​

GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.

If this field is specified and the generated name exists, the server will return a 409.

Applied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#idempotency

namespace required string ​

Namespace defines the space within which each name must be unique. An empty namespace is equivalent to the "default" namespace, but "default" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.

Must be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces

Deprecated: selfLink is a legacy read-only field that is no longer populated by the system.

uid required string ​

UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.

Populated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids

resourceVersion required string ​

An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.

Populated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency

generation required integer ​

A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.

creationTimestamp required object ​

CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC.

Populated by the system. Read-only. Null for lists. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

deletionTimestamp required object ​

DeletionTimestamp is RFC 3339 date and time at which this resource will be deleted. This field is set by the server when a graceful deletion is requested by the user, and is not directly settable by a client. The resource is expected to be deleted (no longer visible from resource lists, and not reachable by name) after the time in this field, once the finalizers list is empty. As long as the finalizers list contains items, deletion is blocked. Once the deletionTimestamp is set, this value may not be unset or be set further into the future, although it may be shortened or the resource may be deleted prior to this time. For example, a user may request that a pod is deleted in 30 seconds. The Kubelet will react by sending a graceful termination signal to the containers in the pod. After that 30 seconds, the Kubelet will send a hard termination signal (SIGKILL) to the container and after cleanup, remove the pod from the API. In the presence of network partitions, this object may still exist after this timestamp, until an administrator or automated process can determine the resource is fully terminated. If not set, graceful deletion of the object has not been requested.

Populated by the system when a graceful deletion is requested. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

deletionGracePeriodSeconds required integer ​

Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.

labels required <label_name>:string ​

Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels

annotations required <annotation_name>:string ​

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations

ownerReferences required object[] ​

List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.

apiVersion required string ​

API version of the referent.

kind required string ​

Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

name required string ​

Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names

uid required string ​

UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids

controller required boolean false ​

If true, this reference points to the managing controller.

blockOwnerDeletion required boolean false ​

If true, AND if the owner has the "foregroundDeletion" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. See https://kubernetes.io/docs/concepts/architecture/garbage-collection/#foreground-deletion for how the garbage collector interacts with this field and enforces the foreground deletion. Defaults to false. To set this field, a user needs "delete" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.

finalizers required string[] ​

Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed. Finalizers may be processed and removed in any order. Order is NOT enforced because it introduces significant risk of stuck finalizers. finalizers is a shared field, any actor with permission can reorder it. If the finalizer list is processed in order, then this can lead to a situation in which the component responsible for the first finalizer in the list is waiting for a signal (field value, external system, or other) produced by a component responsible for a finalizer later in the list, resulting in a deadlock. Without enforced ordering finalizers are free to order amongst themselves and are not vulnerable to ordering changes in the list.

managedFields required object[] ​

ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like "ci-cd". The set of fields is always in the version that the workflow used when modifying the object.

manager required string ​

Manager is an identifier of the workflow managing these fields.

operation required string ​

Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.

apiVersion required string ​

APIVersion defines the version of this resource that this field set applies to. The format is "group/version" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.

time required object ​

Time is the timestamp of when the ManagedFields entry was added. The timestamp will also be updated if a field is added, the manager changes any of the owned fields value or removes a field. The timestamp does not update when a field is removed from the entry because another manager took it over.

fieldsType required string ​

FieldsType is the discriminator for the different fields format and version. There is currently only one possible value: "FieldsV1"

fieldsV1 required object ​

FieldsV1 holds the first JSON version format as described in the "FieldsV1" type.

subresource required string ​

Subresource is the name of the subresource used to update that object, or empty string if the object was updated through the main resource. The value of this field is used to distinguish between managers, even if they share the same name. For example, a status update will be distinct from a regular update using the same manager name. Note that the APIVersion field is not related to the Subresource field and it always corresponds to the version of the main resource.

spec required object ​

displayName required string ​

DisplayName is the name that should be displayed in the UI

description required string ​

Description describes a virtual cluster instance

owner required object ​

Owner holds the owner of this object

user required string ​

User specifies a Loft user.

team required string ​

Team specifies a Loft team.

templateRef required object ​

TemplateRef holds the virtual cluster template reference

name required string ​

Name holds the name of the template to reference.

version required string ​

Version holds the template version to use. Version is expected to be in semantic versioning format. Alternatively, you can also exchange major, minor or patch with an 'x' to tell Loft to automatically select the latest major, minor or patch version.

syncOnce required boolean false ​

SyncOnce tells the controller to sync the instance once with the template. This is useful if you want to sync an instance after a template was changed. To automatically sync an instance with a template, use 'x.x.x' as version instead.

template required object ​

Template is the inline template to use for virtual cluster creation. This is mutually exclusive with templateRef.

metadata required object ​

labels required <label_name>:string ​

Labels are labels on the object

annotations required <annotation_name>:string ​

Annotations are annotations on the object

apps required object[] ​

Apps specifies the apps that should get deployed by this template

name required string ​

Name of the target app

namespace required string ​

Namespace specifies in which target namespace the app should get deployed in

releaseName required string ​

ReleaseName is the name of the app release

version required string ​

Version of the app

parameters required string ​

Parameters to use for the app

charts required object[] ​

Charts are helm charts that should get deployed

name required string ​

Name is the chart name in the repository

version required string ​

Version is the chart version in the repository

repoURL required string ​

RepoURL is the repo url where the chart can be found

username required string ​

The username that is required for this repository

password required string ​

The password that is required for this repository

insecureSkipTlsVerify required boolean false ​

If tls certificate checks for the chart download should be skipped

releaseName required string ​

ReleaseName is the preferred release name of the app

releaseNamespace required string ​

ReleaseNamespace is the preferred release namespace of the app

values required string ​

Values are the values that should get passed to the chart

wait required boolean false ​

Wait determines if Loft should wait during deploy for the app to become ready

timeout required string ​

Timeout is the time to wait for any individual Kubernetes operation (like Jobs for hooks) (default 5m0s)

objects required string ​

Objects are Kubernetes style yamls that should get deployed into the virtual cluster

access required object ​

Access defines the access of users and teams to the virtual cluster.

defaultClusterRole required string ​

Specifies which cluster role should get applied to users or teams that do not match a rule below.

rules required object[] ​

Rules defines which users and teams should have which access to the virtual cluster. If no rule matches an authenticated incoming user, the user will get cluster admin access.

users required string[] ​

Users this rule matches. * means all users.

teams required string[] ​

Teams that this rule matches.

clusterRole required string ​

ClusterRole is the cluster role that should be assigned to the

helmRelease required object ​

HelmRelease is the helm release configuration for the virtual cluster.

chart required object ​

infos about what chart to deploy

name required string ​

the name of the helm chart

repo required string ​

the repo of the helm chart

version required string ​

the version of the helm chart to use

values required string ​

the values for the given chart

accessPoint required object ​

AccessPoint defines settings to expose the virtual cluster directly via an ingress rather than through the (default) Loft proxy

ingress required object ​

Ingress defines virtual cluster access via ingress

enabled required boolean true ​

Enabled defines if the virtual cluster access point (via ingress) is enabled or not; requires the connected cluster to have the loft.sh/ingress-suffix annotation set to define the domain name suffix used for the ingress.

spaceTemplate required object ​

SpaceTemplate holds the space template

metadata required object ​
labels required <label_name>:string ​

Labels are labels on the object

annotations required <annotation_name>:string ​

Annotations are annotations on the object

objects required string ​

Objects are Kubernetes style yamls that should get deployed into the virtual cluster namespace

charts required object[] ​

Charts are helm charts that should get deployed

name required string ​

Name is the chart name in the repository

version required string ​

Version is the chart version in the repository

repoURL required string ​

RepoURL is the repo url where the chart can be found

username required string ​

The username that is required for this repository

password required string ​

The password that is required for this repository

insecureSkipTlsVerify required boolean false ​

If tls certificate checks for the chart download should be skipped

releaseName required string ​

ReleaseName is the preferred release name of the app

releaseNamespace required string ​

ReleaseNamespace is the preferred release namespace of the app

values required string ​

Values are the values that should get passed to the chart

wait required boolean false ​

Wait determines if Loft should wait during deploy for the app to become ready

timeout required string ​

Timeout is the time to wait for any individual Kubernetes operation (like Jobs for hooks) (default 5m0s)

apps required object[] ​

Apps specifies the apps that should get deployed by this template

name required string ​

Name of the target app

namespace required string ​

Namespace specifies in which target namespace the app should get deployed in

releaseName required string ​

ReleaseName is the name of the app release

version required string ​

Version of the app

parameters required string ​

Parameters to use for the app

clusterRef required object ​

ClusterRef is the reference to the connected cluster holding this virtual cluster

cluster required string ​

Cluster is the connected cluster the space will be created in

namespace required string ​

Namespace is the namespace inside the connected cluster holding the space

virtualCluster required string ​

VirtualCluster is the name of the virtual cluster inside the namespace

parameters required string ​

Parameters are values to pass to the template

extraAccessRules required object[] ​

ExtraAccessRules defines extra rules which users and teams should have which access to the virtual cluster.

users required string[] ​

Users this rule matches. * means all users.

teams required string[] ​

Teams that this rule matches.

clusterRole required string ​

ClusterRole is the cluster role that should be assigned to the

access required object[] ​

Access to the virtual cluster object itself

name required string ​

Name is an optional name that is used for this access rule

verbs required string[] ​

Verbs is a list of Verbs that apply to ALL the ResourceKinds and AttributeRestrictions contained in this rule. VerbAll represents all kinds.

subresources required string[] ​

Subresources defines the sub resources that are allowed by this access rule

users required string[] ​

Users specifies which users should be able to access this secret with the aforementioned verbs

teams required string[] ​

Teams specifies which teams should be able to access this secret with the aforementioned verbs

status required object ​

phase required string ​

Phase describes the current phase the virtual cluster instance is in

reason required string ​

Reason describes the reason in machine-readable form why the cluster is in the current phase

message required string ​

Message describes the reason in human-readable form why the cluster is in the current phase

conditions required object ​

Conditions holds several conditions the virtual cluster might be in

virtualClusterObjects required object ​

VirtualClusterObjects are the objects that were applied within the virtual cluster itself

lastAppliedObjects required string ​

LastAppliedObjects holds the status for the objects that were applied

charts required object[] ​

Charts are the charts that were applied

name required string ​

Name of the chart that was applied

namespace required string ​

Namespace of the chart that was applied

lastAppliedChartConfigHash required string ​

LastAppliedChartConfigHash is the last applied configuration

apps required object[] ​

Apps are the apps that were applied

name required string ​

Name of the target app

namespace required string ​

Namespace specifies in which target namespace the app should get deployed in

releaseName required string ​

ReleaseName is the name of the app release

version required string ​

Version of the app

parameters required string ​

Parameters to use for the app

spaceObjects required object ​

SpaceObjects are the objects that were applied within the virtual cluster space

lastAppliedObjects required string ​

LastAppliedObjects holds the status for the objects that were applied

charts required object[] ​

Charts are the charts that were applied

name required string ​

Name of the chart that was applied

namespace required string ​

Namespace of the chart that was applied

lastAppliedChartConfigHash required string ​

LastAppliedChartConfigHash is the last applied configuration

apps required object[] ​

Apps are the apps that were applied

name required string ​

Name of the target app

namespace required string ​

Namespace specifies in which target namespace the app should get deployed in

releaseName required string ​

ReleaseName is the name of the app release

version required string ​

Version of the app

parameters required string ​

Parameters to use for the app

virtualCluster required object ​

VirtualCluster is the template rendered with all the parameters

metadata required object ​

labels required <label_name>:string ​

Labels are labels on the object

annotations required <annotation_name>:string ​

Annotations are annotations on the object

apps required object[] ​

Apps specifies the apps that should get deployed by this template

name required string ​

Name of the target app

namespace required string ​

Namespace specifies in which target namespace the app should get deployed in

releaseName required string ​

ReleaseName is the name of the app release

version required string ​

Version of the app

parameters required string ​

Parameters to use for the app

charts required object[] ​

Charts are helm charts that should get deployed

name required string ​

Name is the chart name in the repository

version required string ​

Version is the chart version in the repository

repoURL required string ​

RepoURL is the repo url where the chart can be found

username required string ​

The username that is required for this repository

password required string ​

The password that is required for this repository

insecureSkipTlsVerify required boolean false ​

If tls certificate checks for the chart download should be skipped

releaseName required string ​

ReleaseName is the preferred release name of the app

releaseNamespace required string ​

ReleaseNamespace is the preferred release namespace of the app

values required string ​

Values are the values that should get passed to the chart

wait required boolean false ​

Wait determines if Loft should wait during deploy for the app to become ready

timeout required string ​

Timeout is the time to wait for any individual Kubernetes operation (like Jobs for hooks) (default 5m0s)

objects required string ​

Objects are Kubernetes style yamls that should get deployed into the virtual cluster

access required object ​

Access defines the access of users and teams to the virtual cluster.

defaultClusterRole required string ​

Specifies which cluster role should get applied to users or teams that do not match a rule below.

rules required object[] ​

Rules defines which users and teams should have which access to the virtual cluster. If no rule matches an authenticated incoming user, the user will get cluster admin access.

users required string[] ​

Users this rule matches. * means all users.

teams required string[] ​

Teams that this rule matches.

clusterRole required string ​

ClusterRole is the cluster role that should be assigned to the

helmRelease required object ​

HelmRelease is the helm release configuration for the virtual cluster.

chart required object ​

infos about what chart to deploy

name required string ​

the name of the helm chart

repo required string ​

the repo of the helm chart

version required string ​

the version of the helm chart to use

values required string ​

the values for the given chart

accessPoint required object ​

AccessPoint defines settings to expose the virtual cluster directly via an ingress rather than through the (default) Loft proxy

ingress required object ​

Ingress defines virtual cluster access via ingress

enabled required boolean true ​

Enabled defines if the virtual cluster access point (via ingress) is enabled or not; requires the connected cluster to have the loft.sh/ingress-suffix annotation set to define the domain name suffix used for the ingress.

spaceTemplate required object ​

SpaceTemplate holds the space template

metadata required object ​
labels required <label_name>:string ​

Labels are labels on the object

annotations required <annotation_name>:string ​

Annotations are annotations on the object

objects required string ​

Objects are Kubernetes style yamls that should get deployed into the virtual cluster namespace

charts required object[] ​

Charts are helm charts that should get deployed

name required string ​

Name is the chart name in the repository

version required string ​

Version is the chart version in the repository

repoURL required string ​

RepoURL is the repo url where the chart can be found

username required string ​

The username that is required for this repository

password required string ​

The password that is required for this repository

insecureSkipTlsVerify required boolean false ​

If tls certificate checks for the chart download should be skipped

releaseName required string ​

ReleaseName is the preferred release name of the app

releaseNamespace required string ​

ReleaseNamespace is the preferred release namespace of the app

values required string ​

Values are the values that should get passed to the chart

wait required boolean false ​

Wait determines if Loft should wait during deploy for the app to become ready

timeout required string ​

Timeout is the time to wait for any individual Kubernetes operation (like Jobs for hooks) (default 5m0s)

apps required object[] ​

Apps specifies the apps that should get deployed by this template

name required string ​

Name of the target app

namespace required string ​

Namespace specifies in which target namespace the app should get deployed in

releaseName required string ​

ReleaseName is the name of the app release

version required string ​

Version of the app

parameters required string ​

Parameters to use for the app

ignoreReconciliation required boolean true ​

IgnoreReconciliation tells the controller to ignore reconciliation for this instance -- this is primarily used when migrating virtual cluster instances from project to project; this prevents a situation where there are two virtual cluster instances representing the same virtual cluster which could cause issues with concurrent reconciliations of the same object. Once the virtual cluster instance has been cloned and placed into the new project, this (the "old") virtual cluster instance can safely be deleted.

Retrieve: Virtual Clusters​

You can either use curl or kubectl to retrieve Virtual Clusters.

Retrieve a list of Virtual Clusters​

Run the following command to list all Virtual Clusters in project my-project:

kubectl get virtualclusterinstances.management.loft.sh -n loft-p-my-project -o yaml

Retrieve a single Virtual Cluster by name​

Run the following kubectl command to get Virtual Cluster my-virtual-cluster in project my-project:

kubectl get virtualclusterinstances.management.loft.sh my-virtual-cluster -n loft-p-my-project -o yaml

Create: Virtual Cluster​

You can either use curl or kubectl to create a new Virtual Cluster. Make sure to set the project in the metadata.namespace field you want to create the Virtual Cluster in. If your project has the id my-project, the corresponding namespace would be loft-p-my-project.

Create a file object.yaml with the following contents:

apiVersion: management.loft.sh/v1
kind: VirtualClusterInstance
metadata:
  creationTimestamp: null
  name: my-virtual-cluster
  namespace: loft-p-my-project
spec:
  clusterRef: {}
  displayName: my-display-name
  owner:
    user: my-user
  parameters: 'my-parameter: my-value'
  templateRef:
    name: my-virtual-cluster-template
status: {}

Then create the Virtual Cluster my-virtual-cluster in project my-project with:

kubectl create -f object.yaml -n loft-p-my-project

Update: Virtual Cluster​

You can either use curl or kubectl to update Virtual Clusters.

Update Virtual Cluster​

Run the following command to update Virtual Cluster my-virtual-cluster in project my-project:

kubectl edit virtualclusterinstances.management.loft.sh my-virtual-cluster -n loft-p-my-project

Then edit the object and upon save, kubectl will update the resource.

Patch Virtual Cluster​

Patching a resource is useful if you want to generically exchange only a small portion of the object instead of retrieving the whole object first and then modifying it. To learn more about patches in Kubernetes, please take a look at the official docs.

Run the following kubectl command to add a new annotation my-annotation: my-value to the Virtual Cluster my-virtual-cluster in project my-project via a patch:

kubectl patch virtualclusterinstances.management.loft.sh my-virtual-cluster -n loft-p-my-project \
        --type json \
        -p '[{"op": "add", "path": "/metadata/annotations/my-annotation", "value": "my-value"}]'

Delete: Virtual Cluster​

You can either use curl or kubectl to delete Virtual Clusters.

Run the following command to delete Virtual Cluster my-virtual-cluster in project my-project:

kubectl delete virtualclusterinstances.management.loft.sh my-virtual-cluster -n loft-p-my-project