A.1. Tungsten Replicator 8.0.0 GA (Not Yet Set)

Version End of Life. Not Yet Set

Release 8.0.0 contains a number of key bug fixess and improvements.

Deprecation Notices

The following features/tools have been deprecated from this release and will be removed in future releases. Any self written scripts that utilise these features should be updated.

  • Command-line Tools

    • tungsten_health_check and tungsten_provision_thl have been deprecated and will be removed in a future release.

      Issues: CT-2509

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • All Tungsten components now use log4j version 2

      Issues: CT-1410, CT-1412

    • The rest-api-admin-user and rest-api-admin-password TPM properties are now now required during a default installation or when explicitly setting the replicator-rest-api=true or rest-apis=true property.

      Issues: CT-2409

  • Command-line Tools

    • Additional validation has been added to tprovision when parallel apply is in use.

      Issues: CT-2275

    • When provisioning a replica using mysqldump, the database will be set to offline mode until loading is complete to prevent accidental writes during the load.

      Issues: CT-2305

    • The tungsten_send_diag command has been updated with new security keys and the ability to accept security keys on the CLI.

      Issues: CT-2438

    • Removed sigtrap module dependency from tungsten_find_orphaned.

      Issues: CT-2451

    • The tpm command now allows hyphens in dataservice names.

      Issues: CT-2453

    • A new cli argument has been added (--apiver 8) to allow the tapi command to use APIv8. By default, tapi will continue to use APIv2 against v8 installations, using the --apiver 8 option is experimental and should be used with care. Full APIv8 functionality will be included in tapi in a future release.

      Issues: CT-2456

    • The tpm ask command now returns the appropriate values for keys: stagingdir, stagingpath, staginghost and staginguser.

      Additionally, the tpm ask staging command will return a JSON structure with all values. All values are the same as the output of the tpm query staging command, just presented differently for easier machine usage.

      Issues: CT-2510

    • The tpm cert info command now provides better error messages when a specified typeSpec file does not exist.

      Issues: CT-2530

  • Core Replicator

    • Replicator now gets higher CPU priority (-9 nice value) through systemd configuration file. When not using systemd, we advised to use the sudo renice <pid> command to avoid monitoring and processing pauses during high CPU usage periods. Note that this command has to be run as a privileged user, which is why the command cannot be part of the regular replicator scripts

      Issues: CT-2330

Improvements, new features and functionality

  • Command-line Tools

    • The tpm ask command now supports the -s|--service option for trrole and trstate keys. The option alias for --short is now -S (capitalized), was previously lower-case. Example: tpm ask trrole --service alpha

      Issues: CT-2400

    • The tpm cert command will now handle public key files. The command tpm cert gen public will create $CONTINUENT_HOME/generated/tungsten_public.jks from the $CONTINUENT_HOME/share/tungsten_keystore.jks file and the tpm cert list public command will show the contents, etc.

      Issues: CT-2434

    • A new -z option has been added to tprovision which will enable compression when using the rsync method.

      Issues: CT-2489

    • The tpm diag command now gathers the output of the tpm ini check -x command as the tpmINICheck call.

      Issues: CT-2499

    • The tpm ini command now handles all new tpm options.

      Issues: CT-2543

    • The tpm ask command will now convert values 0 and false to No and values 1 and true to Yes as human output when given --human or -n cli args.

      Issues: CT-2559

  • Core Replicator

    • Replicator now recognizes the new MySQL 8.4 show binary log status syntax (As well as show master status) and will run replacement configured request appropriately.

      Issues: CT-2291

    • A new applying field has been added to the output of trepctl status.

      ...
      applying : 3 / 0 (92.86%)
      ...

      In the above example, the applier is processing seqno 3 / fragno 0 and already applied 92.86% (this represents a percentage of rows and statements that were already applied).

      When parallel apply in enabled, there will be one applying field per channel configured, for example the following shows 2 channels:

      ...
      applying.0 : 
      applying.1 : 25 / 0 (92.86%)
      ...

      Issues: CT-2315

    • Introduced a new cloud-install tpm flag that will skip checks for hostname entry in /etc/hosts, allowing smoother installations against cloud based sources or targets such as Amazon Aurora or Google Cloud SQL.

      Issues: CT-2317

    • A new thl apply command is available which provides the ability to pull/push a range of transactions from source THL to a destination. The command can apply the THL either directly into the destination database (MySQL or PostgreSQL) or it can simply write the THL to the filesystem.

      The destination does not need to be running Tungsten Replicator services for this to be used and provides a way of applying data outside of normal Tungsten operations and can be used to extend backup/recovery operations in some scenarios.

      See Section 8.19.2, “thl apply Command” for more information.

      Issues: CT-2347

    • The trepctl perf output has been improved when parallel apply is enabled to show granular statistics per channel.

      Issues: CT-2517

  • Documentation

    • API documentation has been updated to reflect changes to many of the endpoints, including updated examples of expected inputs and outputs.

      Issues: CT-2339

Bug Fixes

  • Installation and Deployment

    • Hyphens are now allowed in service names by tpm.

      Issues: CT-2367

    • The tpm update command no longer displays the message "common::diffPasswordsDotStore: No custom source passwords.store file defined - SKIPPING"

      Issues: CT-2406

    • tpm now properly detects MariaDB default settings.

      Issues: CT-2424

    • tpm flags datasource-log-directory and repl-direct-datasource-log-directory will now use the value of each other as a default if only one is specified

      Issues: CT-2425

  • Command-line Tools

    • New shotcut options are availabe with tprovision

      • --primary and --create-primary as an additional alias for -c

      • -C, --mysqlclone as aliases for -m mysqlclone

      • -D, --mysqldump as aliases for -m mysqldump

      • -R, --rsync as aliases for -m rsync

      • -X, --xtrabackup as aliases for -m xtrabackup

      Issues: CT-2311

    • The tpm diag command now properly gathers the uptime command.

      Issues: CT-2319

    • tprovision was incorrectly setting to the MySQL datadir to tungsten group ownership..

      Issues: CT-2322

    • When using the -m rsync option with tprovision, the rsync command now includes the oStrictHostKeyChecking=no options.

      Issues: CT-2348

    • The tpm diag command no longer displays a NOTICE for missing optional files.

      Issues: CT-2355

    • The tpm check ports and tpm check ports -c commands are now able to handle replicator-only nodes. By default, it will automatically try to discover the node type, and you may manually specify the node type with -r|--replicator or -c|--cluster.

      Issues: CT-2356

    • The tpm update command now properly deletes services when run in a standalone Tungsten Replicator deployment.

      Issues: CT-2383

    • The tapi command now creates proper URL's for the standard endpoints when -R is specified along with one of the following strings: ping, tungstenVersion, createAdminUser, deleteAdminUser, listAdminUsers, rmiSecurity or user.

      Issues: CT-2391

    • The tpm diag command now gathers:

      • tungsten-replicator/log/replicator-api.log

      Issues: CT-2397

    • The tpm diag command now correctly connects to an off-host database when run under standalone Tungsten Replicator.

      Issues: CT-2402

    • The tapi command now accepts non-standard ports on the command line, and also properly accepts a user/password pair on the CLI.

      Issues: CT-2420

    • The tpm update command now properly identifies and removes services with mixed-case service names.

      Issues: CT-2426

    • The tpm uninstall command now properly calls the tpm keep command for standalone Tungsten Replicator deployments.

      Issues: CT-2436

    • The tpm keep command is now properly called by tpm uninstall, and a regression in a call to the mysql cli has been fixed as well.

      Issues: CT-2462

    • executable-prefix is now applied to deployall and undeployall script names. For example, when executable-prefix=mm, deployall would be named mm_deployall.

      Issues: CT-2467

    • Fixed an issue where the database could potentially get locked in certain cases when using the rsync method with tprovision

      Issues: CT-2470

    • tprovision will now correctly handle, and support, selinux if enabled.

      Issues: CT-2473

    • The tpm ask version and tpm ask versions commands now display values.

      Issues: CT-2475

    • During a tpm uninstall, when skipping data services not managed locally, tpm will now print a notice rather than a warning, since this is the expected behavior

      Issues: CT-2479

    • The tungsten_send_diag command now handles curl and AWS S3 errors separately and displays the proper message back.

      Issues: CT-2488

    • Fixes an issue where dsctl set with the -reset option could show warnings when run several times in a row, as shown:

      WARNING: Could not execute query DROP TABLE `tungsten_alpha`.`heartbeat` : Unknown table 'tungsten_alpha.heartbeat'

      Issues: CT-2502

    • The tpm cert aliases --running command now properly handles the tls_passwordstore and jmx_passwordstore typeSpecs.

      Issues: CT-2512

    • The tpm cert info command now properly handles all available typeSpecs.

      Issues: CT-2532

  • MongoDB Applier

    • The replicator now reads the TLS certificate for Mongo Atlas connectivity from the correct keystore (tungsten_truststore.ts)

      Issues: CT-2227

  • Core Replicator

    • Fixed an issue that would wrongly flag a SQL statement starting with a # comment as bidi_unsafe

      Issues: CT-2405

    • The replicator will now properly rollback an XA transaction in case of failure.

      Issues: CT-2410

  • API

    • Fixed an issue where a component stopped through through the API would wrongly indicate "unable to stop"

      Issues: CT-2515