Description
Getting Started
Quick start:
At this time Operator Lifecycle Manager does not support the creation of Secret, ValidatingWebhookConfiguration, or Service resources as part of installation. When installing the DataStax Kubernetes Operator for Apache Cassandra these must be created externally before the operator will fully come online.
---
apiVersion: v1
data:
tls.crt: ""
tls.key: ""
kind: Secret
metadata:
name: cass-operator-webhook-config
namespace: test-operator
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
name: cassandradatacenter-webhook-registration
webhooks:
- admissionReviewVersions:
- v1beta1
clientConfig:
service:
name: cassandradatacenter-webhook-service
namespace: test-operator
path: /validate-cassandra-datastax-com-v1beta1-cassandradatacenter
failurePolicy: Ignore
matchPolicy: Equivalent
name: cassandradatacenter-webhook.cassandra.datastax.com
rules:
- apiGroups:
- cassandra.datastax.com
apiVersions:
- v1beta1
operations:
- CREATE
- UPDATE
resources:
- cassandradatacenters
scope: '*'
sideEffects: None
timeoutSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
labels:
name: cass-operator-webhook
name: cassandradatacenter-webhook-service
namespace: test-operator
spec:
ports:
- port: 443
targetPort: 8443
selector:
name: cass-operator
Note the namespace above must be adjusted to match the namespace where cass-operator is being deployed.
With these resources in place the operator will install and start monitoring for CassandraDatacenter custom resources. Consider using the OpenShift console for creation of these resources or submitting a YAML file with oc.
Storage
When running on OpenShift an appropriate StorageClass must be used. If your cluster currently does not provide a storage class consider creating your own with the NoProvisioner provisioner. This will prevent the dynamic creation of volumes and support manually created PersistentVolumes. For example:
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: no-provisioner
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
From here you can create a persistent volume with the following syntax:
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: cass-operator-test-pv
spec:
capacity:
storage: 100Gi
hostPath:
path: /mnt/pv-data/cass-op-pv
type: ''
accessModes:
- ReadWriteOnce
- ReadWriteMany
- ReadOnlyMany
persistentVolumeReclaimPolicy: Recycle
volumeMode: Filesystem
storageClassName: no-provisioner
Note the storageClassName matches the previously defined StorageClass.
Creating a CassandraDatacenter
The following resource defines a Cassandra 3.11.6 datacenter with 3 nodes on one rack:
---
apiVersion: cassandra.datastax.com/v1beta1
kind: CassandraDatacenter
metadata:
name: dc1
spec:
clusterName: cluster1
serverType: cassandra
serverVersion: 3.11.6
managementApiAuth:
insecure: {}
size: 3
storageConfig:
cassandraDataVolumeClaimSpec:
storageClassName: no-provisioner
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
config:
cassandra-yaml:
authenticator: org.apache.cassandra.auth.PasswordAuthenticator
authorizer: org.apache.cassandra.auth.CassandraAuthorizer
role_manager: org.apache.cassandra.auth.CassandraRoleManager
jvm-options:
initial_heap_size: 800M
max_heap_size: 800M
Apply the above as follows:
oc apply -f cassdc.yaml
You can check the status of pods in the Cassandra cluster as follows:
$ kubectl -n cass-operator get pods --selector cassandra.datastax.com/cluster=cluster1
NAME READY STATUS RESTARTS AGE
cluster1-dc1-default-sts-0 2/2 Running 0 26h
cluster1-dc1-default-sts-1 2/2 Running 0 26h
cluster1-dc1-default-sts-2 2/2 Running 0 26h
You can check to see the current progress of bringing the Cassandra datacenter online by checking the cassandraOperatorProgress field of the CassandraDatacenter's status sub-resource as follows:
$ kubectl -n cass-operator get cassdc/dc1 -o "jsonpath={.status.cassandraOperatorProgress}"
Ready
(cassdc and cassdcs are supported short forms of CassandraDatacenter.)
A value of "Ready", as above, means the operator has finished setting up the Cassandra datacenter.
You can also check the Cassandra cluster status using nodetool by invoking it on one of the pods in the Cluster as follows:
$ oc exec -it -c cassandra cluster1-dc1-default-sts-0 -- nodetool status
Datacenter: dc1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving/Stopped
-- Address Load Tokens Owns (effective) Host ID Rack
UN 10.233.105.125 224.82 KiB 1 65.4% 5e29b4c9-aa69-4d53-97f9-a3e26115e625 r1
UN 10.233.92.96 186.48 KiB 1 61.6% b119eae5-2ff4-4b06-b20b-c492474e59a6 r1
UN 10.233.90.54 205.1 KiB 1 73.1% 0a96e814-dcf6-48b9-a2ca-663686c8a495 r1
Features
- Proper token ring initialization, with only one node bootstrapping at a time
- Seed node management - one per rack, or three per datacenter, whichever is more
- Server configuration integrated into the CassandraDatacenter CRD
- Rolling reboot nodes by changing the CRD
- Store data in a rack-safe way - one replica per cloud AZ
- Scale up racks evenly with new nodes
- Replace dead/unrecoverable nodes
- Multi DC clusters (limited to one Kubernetes namespace)
All features are documented in the User Documentation.
Containers
The operator is comprised of the following container images working in concert:
Requirements
- Kubernetes cluster, 1.12 or newer.
- Users who want to use a Kubernetes version from before 1.15 can use a manifest that supports x-preserve-unknown-fields on the CassandraDatacenter CRD - manifest
Contributing
As of version 1.0, Cass Operator is maintained by a team at DataStax and it is part of what powers DataStax Astra. We would love for open source users to contribute bug reports, documentation updates, tests, and features.
Developer setup
Almost every build, test, or development task requires the following pre-requisites...
- Golang 1.13
- Docker, either the docker.io packages on Ubuntu, Docker Desktop for Mac, or your preferred docker distribution.
- mage: There are some tips for using mage in docs/developer/mage.md
Building
The operator uses mage for its build process.
Build the Operator Container Image
This build task will create the operator container image, building or rebuilding the binary from golang sources if necessary:
mage operator:buildDocker
Build the Operator Binary
If you wish to perform ONLY to the golang build or rebuild, without creating a container image:
mage operator:buildGo
Testing
mage operator:testGo
End-to-end Automated Testing
Run fully automated end-to-end tests...
mage integ:run
Docs about testing are here. These work against any k8s cluster with six or more worker nodes.
Manual Local Testing
There are a number of ways to run the operator, see the following docs for more information:
- kind: Kubernetes in Docker is the recommended Kubernetes distribution for use by software engineers working on the operator. KIND can simulate a k8s cluster with multiple worker nodes on a single physical machine, though it's necessary to dial down the database memory requests.
The user documentation also contains information on spinning up your first operator instance that is useful regardless of what Kubernetes distribution you're using to do so.
Not (Yet) Supported Features
- Cassandra:
- Integrated data repair solution
- Integrated backup and restore solution
- DSE:
- Advanced Workloads: Analytics
License
Copyright DataStax, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
