Graph Lakehouse™ - operator image

Description

anzograph-operator

By Altair Engineering Inc.

Supported tags

• latest
   

Project Status: stable
Operator Version: v2

Prerequisites

• Red Hat Openshift Container Platform on Kubernetes, version >= 4.8
• Kubectl, versions {1.33-1.25}

Graph Lakehouse Docker Images Used for Deployment

When you deploy Graph Lakehouse cluster using operator, following are the set of images used for actual cluster deployments. We have given reference docker commands to download the latest releases for each of them below.

Graph Lakehouse Operator

• To download latest release, please use:

   docker pull registry.connect.redhat.com/cambridgesemantics/anzograph-operator:latest

Graph Lakehouse Database

• To download latest release, please use:

   docker pull registry.connect.redhat.com/cambridgesemantics/anzograph-db:latest

Graph Lakehouse Frontend

• If you want to enable UI for the Graph Lakehouse database, following is the image used.

   docker pull registry.connect.redhat.com/cambridgesemantics/anzograph-frontend:latest

Graph Lakehouse / AnzoGraph AllInOne Image(DB + Frontend)

• To download latest release, please use:

   docker pull registry.connect.redhat.com/cambridgesemantics/anzograph:latest

Steps to deploy anzograph-operator

Deployment of Graph Lakehouse cluster can be achieved using following steps.

# Create Namespace, mention name of your namespace in metadata.name $ kubectl create -f deploy/v1_namespace_default.yaml # Setup Service Account $ kubectl create -f deploy/default_v1_serviceaccount_anzograph-operator.yaml --namespace <namespace> # Setup RBAC $ kubectl create -f deploy/default_rbac.authorization.k8s.io_v1_role_anzograph-operator.yaml --namespace <namespace> $ kubectl create -f deploy/default_rbac.authorization.k8s.io_v1_rolebinding_anzograph-operator.yaml --namespace <namespace> $ kubectl create -f deploy/rbac.authorization.k8s.io_v1_clusterrole_anzograph-operator.yaml $ kubectl create -f deploy/rbac.authorization.k8s.io_v1_clusterrolebinding_anzograph-operator.yaml # Setup the CRD $ kubectl create -f deploy/crds/apiextensions.k8s.io_v1_customresourcedefinition_anzographs.anzograph.clusters.cambridgesemantics.com.yaml # Deploy anzograph-operator $ kubectl create -f deploy/default_apps_v1_deployment_anzograph-operator.yaml --namespace <namespace> # Deploy AnzoGraph Custom Resource(CR), i.e. AnzoGraph cluster deployment $ kubectl apply -f deploy/default_anzograph.clusters.cambridgesemantics.com_v2_anzograph_azg01.yaml --namespace <namespace>

NOTE One needs to edit operator deployment, CR deployment with right docker image details.

Steps to delete Graph Lakehouse CR and anzograph-operator

# Delete AnzoGraph CR $ kubectl delete -f deploy/default_anzograph.clusters.cambridgesemantics.com_v2_anzograph_azg01.yaml --namespace <namespace> # Delete anzograph-operator $ kubectl delete -f deploy/default_apps_v1_deployment_anzograph-operator.yaml --namespace <namespace> # Delete RBAC $ kubectl delete -f deploy/default_rbac.authorization.k8s.io_v1_role_anzograph-operator.yaml --namespace <namespace> $ kubectl delete -f deploy/default_rbac.authorization.k8s.io_v1_rolebinding_anzograph-operator.yaml --namespace <namespace> $ kubectl delete -f deploy/rbac.authorization.k8s.io_v1_clusterrole_anzograph-operator.yaml $ kubectl delete -f deploy/rbac.authorization.k8s.io_v1_clusterrolebinding_anzograph-operator.yaml # Delete Service Account $ kubectl delete -f deploy/default_v1_serviceaccount_anzograph-operator.yaml --namespace <namespace> # Delete CRD $ kubectl delete -f deploy/crds/apiextensions.k8s.io_v1_customresourcedefinition_anzographs.anzograph.clusters.cambridgesemantics.com.yaml

## Persistence
The Graph Lakehouse custom resource utilizes PVs (disks) to save the Graph Lakehouse config and, if enabled, persisted data. The default policy(no annotations are specified) will delete these PVCs/PVs when the Graph Lakehouse custom resource is deleted. It is often preferable to change this default behavior using annotations specified in next section.

## Graph Lakehouse CustomResource(CR) annotations
 Annotation keys and values can only be strings. All types must be string-encoded.

You can opt for config, spill and data persistence by setting `cambridgesemantics/retainConfigStorage`, `cambridgesemantics/retainSpillStorage` and `cambridgesemantics/retainDataStorage` to `true`.

Please note that data persistence does not support, if

* you change the "shape" of the Graph Lakehouse cluster (different slice/vCPU, different count of AZG database nodes)
* you change the database container of the CR.

Please make sure you export your (in memory) data of the Graph Lakehouse instance before you redeploy and re-load the data again.

## Upgrading Graph Lakehouse Custom Resource
Please note that Graph Lakehouse dynamic deployments(embedded from within anzo) do not currently support custom resource upgrades, from Graph Studio / Anzo user interface. In case of standalone installs, there are a few manual steps needed to be performed before `kubectl apply`. We intend to introduce the complete support for upgrade in future releases.

1. Persistence should be enabled in the original CR deployment. This can be achieved by setting
```yaml
annotations:
 cambridgesemantics/configStorageInGB: 5Gi
 cambridgesemantics/retainConfigStorage: "true"
 cambridgesemantics/dataStorageInGB: "30"
 cambridgesemantics/retainDataStorage: "true"
 cambridgesemantics/spillStorageInGB: 32Gi
 cambridgesemantics/retainSpillStorage: "false"
```

2. Stop Graph Lakehouse Database
```sh
   export AZG_NS=default
   export AZG_CR=azg01
   kubectl --namespace="${AZG_NS}" get pod --selector="app_mgmt=anzograph-mgmt-grpc,cluster_name=${AZG_CR},apps.kubernetes.io/pod-index=0" --output=name \
   | xargs -I{} kubectl --namespace="${AZG_NS}" exec {} -- bash -c 'azgctl -stop'
```

3. Prepare the database for new UDXs
   ```sh
   export AZG_NS=default
   export AZG_CR=azg01
   kubectl --namespace="${AZG_NS}" get pod --selector="app_mgmt=anzograph-mgmt-grpc,cluster_name=${AZG_CR}" --output=name \
   | xargs -I{} kubectl --namespace="${AZG_NS}" exec {} -- bash -c 'rm -rf lib/udx/{.activated,*}'
   ```

4. Remove spool files in /opt/anzograph/spill/

   ```sh
   export AZG_NS=default
   export AZG_CR=azg01
   kubectl --namespace="${AZG_NS}" get pod --selector="app_mgmt=anzograph-mgmt-grpc,cluster_name=${AZG_CR}" --output=name \
   | xargs -I{} kubectl --namespace="${AZG_NS}" exec {} -- bash -c 'rm -f spill/*'
   ```

5. Delete persisted data (you have a backup, right?)

   ```sh
   export AZG_NS=default
   export AZG_CR=azg01
   kubectl --namespace="${AZG_NS}" get pod --selector="app_mgmt=anzograph-mgmt-grpc,cluster_name=${AZG_CR}" --output=name \
   | xargs -I{} kubectl --namespace="${AZG_NS}" exec {} -- bash -c 'rm -fr persistence/*'
   ```

6. Uninstall current Graph Lakehouse CR

   ```sh
   export AZG_NS=default
   export AZG_CR=azg01
   kubectl --namespace="${AZG_NS}" delete anzograph/${AZG_CR}
   ```

7. Install CR with same name and updated CR yaml file

   ```sh
   export AZG_NS=default
   export AZG_CR=azg01
   kubectl --namespace="${AZG_NS}" apply -f <cr-yaml-file>
   ```

Graph Lakehouse CustomResource(CR) Specification

The following table lists the configurable parameters for Graph Lakehouse and their default values.(CR API Version: v2)

References

https://docs.cambridgesemantics.com/