5.3. Importing Data

There are several methods available to initialize a Tungsten Cluster with data upon creation.

If an init block is specified within the YAML, or the mysql-init-script.yaml template is used, the cluster is initially created with only MySQL running, without any Tungsten components. Once the MySQL instances are up and running, each instance is populated with the specified data. After the data import process is completed, the Tungsten components are started. The replicator should then begin operating as expected.

Warning

It's crucial to ensure that the SQL statements used for initializing the cluster are deterministic. This means that they should produce the same result each time they are executed, ensuring that each MySQL instance ends up with identical data. Avoid using non-deterministic statements, such as now(), which can yield different results each time.

5.3.1. Initializing using SQL or URL

First, configure the init script, the example below is taken from the mysql-init-script.yaml file, and the name can be changed to suit if you wish. Make a note of the name, as that will then be used in the next step:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-init-script
data:
  import.sql: |
    create database sample;
    use sample;
    CREATE TABLE test(data VARCHAR(255));
    insert into test values("testing");

The above example shows raw SQL, however the same can also be achieved by specifying a URL, as shown in the following example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-init-script
spec:
  init:
    fromExternalArtifact:
      url: https://test.com/sample.sql.gz

Then, create the cluster using init block with fromVolume, ensuring name refers to the name you set in the previous step. The example below is taken from the tungsten_v1alpha1_mysqlcluster_with_init.yaml file:

...
  name: sample-with-init
spec:
  configuration:
    customPasswordSecret: tungsten-passwords-sample-secret
  init:
    fromVolume:
      configMap:
       name: mysql-init-script
  dataServices:
  - name: alpha
...

If you combine all of the above in a single file, you can then simply issue the following:

copy
shell> kubectl apply -f <your-filename-here>

If you choose to keep two seperate files and use the same name as the templates:

copy
shell> kubectl apply -f mysql-init-script.yaml
shell> kubectl apply -f tungsten_v1alpha1_mysqlcluster_with_init.yaml

5.3.2. Initializing using existing TungstenBackup

To initialize a cluster using an existing TungstenBackup, you need to ensure that the TungstenBackup is present in the same namespace as the cluster being created.

First, verify the name of the TungstenBackup you wish to use. You can do this by listing all available backups in the current namespace with the following command:

copy
shell> kubectl tungsten backup list

Once you have confirmed the name of the TungstenBackup, you can proceed to create a new cluster. In the init block of your cluster configuration, specify the fromBackup field with the name of your TungstenBackup. Additionally, it's necessary to specify name of the configuration in configurationRef. Ensure that the configuration you specify is the same one used when the backup was created. Here is an example taken from the tungsten_v1alpha1_backupconfiguration.yaml which would have previously been applied with a name of backupconfiguration-sample:

...
  name: backupconfiguration-sample
spec:
  destination:
    bucket: "backups"
    directory: "test"
    backend: sample-rclone-backend
  policy:
    purgeRemoteBackupOnDeletion: true
    overwriteExistingPhysicalResource: false
...

Here is a sample taken from tungsten_v1alpha1_mysqlcluster_from_backup.yaml showing the init block that will use the backup configuration above, and the name of the available backup:

...
  name: sample-from-backup
spec:
  configuration:
    customPasswordSecret: tungsten-passwords-sample-secret
  backup:
    configurationRef:
      name: backupconfiguration-sample
  init:
    fromBackup:
      name: tungstenbackup-sample
  dataServices:
  - name: alpha
...

When the YAML files are ready, you would simply execute:

copy
shell> kubectl apply -f tungsten_v1alpha1_mysqlcluster_from_backup.yaml

After creating the cluster, it's important to verify that the initialization jobs have completed successfully and that the cluster is in a healthy state. Monitor the status of your cluster and its jobs to ensure everything is functioning as expected.