Continuent Documentation

Manual Home
Tungsten Clustering (for MySQL) 6.1 Manual
Breadcrumbs
Chapters
Preface
1. Introduction
2. Deployment
3. Deployment: MySQL Topologies
4. Deployment: Advanced
5. Deployment: Security
6. Operations Guide
7. Tungsten Connector
8. Tungsten Manager
9. Command-line Tools
10. The tpm Deployment Command
11. Replication Filters
12. Performance, Tuning and Testing
A. Release Notes
B. Prerequisites
C. Troubleshooting
D. Files, Directories, and Environment
E. Terminology Reference
F. Internals
G. Frequently Asked Questions (FAQ)
Tungsten Clustering (for MySQL) 6.1 Manual
    Next

Tungsten Clustering (for MySQL) 6.1 Manual

Continuent Ltd

Copyright © 2025 Continuent Ltd

Abstract

This manual documents Tungsten Clustering (for MySQL), a high performance, High Availability and Disaster Recovery for MySQL clustering.

This manual includes information for 6.1, up to and including 6.1.25.

Build date: 2025-04-24 (14156408)

Up to date builds of this document: Tungsten Clustering (for MySQL) 6.1 Manual (Online), Tungsten Clustering (for MySQL) 6.1 Manual (PDF)

Table of Contents

Preface
1. Legal Notice
2. Conventions
3. Quickstart Guide
1. Introduction
1.1. Tungsten Replicator
1.1.1. Transaction History Log (THL)
1.2. Tungsten Manager
1.3. Tungsten Connector
2. Deployment
2.1. Host Types
2.1.1. Manager Hosts
2.1.2. Connector (Router) Hosts
2.1.3. Replicator Hosts
2.1.4. Active Witness Hosts
2.2. Deployment Sources
2.2.1. Using the TAR/GZipped files
2.2.2. Using the RPM package files
2.3. Common tpm Options During Deployment
2.4. Best Practices
2.4.1. Best Practices: Deployment
2.4.2. Best Practices: Upgrade
2.4.3. Best Practices: Operations
2.4.4. Best Practices: Maintenance
3. Deployment: MySQL Topologies
3.1. Deploying Standalone HA Clusters
3.1.1. Prepare: Standalone HA Cluster
3.1.2. Install: Standalone HA Cluster
3.2. Deploying Composite Active/Passive Clustering
3.2.1. Prepare: Composite Active/Passive Cluster
3.2.2. Install: Composite Active/Passive Cluster
3.2.3. Adding a remote Composite Cluster
3.3. Deploying Composite Active/Active Clusters
3.3.1. Prepare: Composite Active/Active Clusters
3.3.2. Install: Composite Active/Active Clusters
3.3.3. Best Practices: Composite Active/Active Clusters
3.3.4. Resetting a single dataservice
3.3.5. Resetting all dataservices
3.3.6. Dataserver maintenance
3.3.7. Adding a Cluster to a Composite Active/Active Topology
3.4. Deploying Multi-Site/Active-Active Clustering
3.4.1. Prepare: Multi-Site/Active-Active Clusters
3.4.2. Install: Multi-Site/Active-Active Clusters
3.4.3. Best Practices: Multi-Site/Active-Active Clusters
3.4.4. Resetting a single dataservice
3.4.5. Resetting all dataservices
3.4.6. Adding a new Cluster/Dataservice
3.4.7. Enabling SSL for Replicators Only
3.4.8. Dataserver maintenance
3.5. Deploying Tungsten Connector Only
3.6. Deploying Additional Datasources, Managers, or Connectors
3.6.1. Adding Datasources to an Existing Deployment
3.6.2. Adding Active Witnesses to an Existing Deployment
3.6.3. Replacing an Active Witness as a Full Cluster Node
3.6.4. Replacing a Full Cluster Node as an Active Witness
3.6.5. Adding Connectors to an Existing Deployment
3.6.6. Converting from a single cluster to a composite cluster
3.7. Replicating Data Into an Existing Dataservice
3.8. Replicating Data Out of a Cluster
3.8.1. Prepare: Replicating Data Out of a Cluster
3.8.2. Deploy: Replicating Data Out of a Cluster
3.9. Migrating and Seeding Data
3.9.1. Migrating from MySQL Native Replication 'In-Place'
3.9.2. Migrating from MySQL Native Replication Using a New Service
3.9.3. Seeding Data through MySQL
4. Deployment: Advanced
4.1. Deploying Parallel Replication
4.1.1. Application Prerequisites for Parallel Replication
4.1.2. Enabling Parallel Apply During Install
4.1.3. Channels
4.1.4. Parallel Replication and Offline Operation
4.1.5. Adjusting Parallel Replication After Installation
4.1.6. Monitoring Parallel Replication
4.1.7. Controlling Assignment of Shards to Channels
4.1.8. Disk vs. Memory Parallel Queues
4.2. Starting and Stopping Tungsten Components
4.2.1. Stopping/Starting Tungsten Replicator
4.2.2. Stopping/Starting Tungsten Connectors
4.2.3. Stopping/Starting Tungsten Managers
4.2.4. Restarting the Multi-Site/Active-Active Replicator Service
4.3. Configuring Startup on Boot
4.3.1. Configuring Multi-Site/Active-Active Replicator Startup on Boot
4.4. Removing Datasources, Managers or Connectors
4.4.1. Removing a Datasource from an Existing Deployment
4.4.2. Removing a Composite Datasource/Cluster from an Existing Deployment
4.4.3. Removing a Connector from an Existing Deployment
5. Deployment: Security
5.1. Enabling Security
5.1.1. Enabling Security using the Staging Method
5.1.2. Enabling Security using the INI Method
5.2. Disabling Security
5.3. Creating Suitable Certificates
5.3.1. Creating Tungsten Internal Certificates Manually
5.4. Installing from a Staging Host with Custom Certificates
5.4.1. Installing from a Staging Host with Manually-Generated Certificates
5.5. Installing via INI File with Custom Certificates
5.5.1. Installing via INI File with Manually-Generated Certificates
5.6. Installing via INI File with CA-Signed Certificates
5.7. Replacing the JGroups Certificate from a Staging Directory
5.8. Replacing the TLS Certificate from a Staging Directory
5.9. Removing JGroups Encryption from a Staging Directory
5.10. Removing JGroups Encryption via INI File
5.11. Removing TLS Encryption from a Staging Directory
5.12. Removing TLS Encryption via INI File
5.13. Enabling Tungsten<>Database Security
5.13.1. Enabling Database SSL
5.13.2. Configure Tungsten<>Database Secure Communication
5.13.3. Configuring Connector SSL
6. Operations Guide
6.1. The Home Directory
6.2. Establishing the Shell Environment
6.3. Checking Dataservice Status
6.3.1. Latency or Relative Latency Display
6.3.2. Getting Detailed Information
6.3.3. Understanding Datasource Roles
6.3.4. Understanding Datasource States
6.3.5. Understanding Replicator Roles
6.3.6. Checking Replication Status
6.3.7. Changing Datasource States
6.3.8. Datasource Statuses
6.3.9. Datasource States and Policy Mode Interactions
6.4. Policy Modes
6.4.1. Setting Policy Modes
6.4.2. Setting Policy Modes in a Composite cluster
6.5. Switching Primary Hosts
6.5.1. Automatic Primary Failover
6.5.2. Manual Primary Switch
6.6. Datasource Recovery Steps
6.6.1. Recover a failed Replica
6.6.2. Recover a failed Primary
6.7. Composite Cluster Switching, Failover and Recovery
6.7.1. Composite Cluster Site Switch
6.7.2. Composite Cluster Site Failover (Forced Switch)
6.7.3. Composite Cluster Site Recovery
6.7.4. Composite Cluster Relay Recovery
6.8. Composite Active/Active Recovery
6.9. Managing Transaction Failures
6.9.1. Identifying a Transaction Mismatch
6.9.2. Skipping Transactions
6.10. Creating a Backup
6.10.1. Using a Different Backup Tool
6.10.2. Automating Backups
6.10.3. Using a Different Directory Location
6.10.4. Creating an External Backup
6.11. Restoring a Backup
6.11.1. Restoring a Specific Backup
6.11.2. Restoring an External Backup
6.11.3. Restoring from Another Replica
6.11.4. Manually Recovering from Another Replica
6.11.5. Reprovision a MySQL Replica using rsync
6.11.6. Rebuilding a Lost Datasource
6.11.7. Resetting an Entire Dataservice from Filesystem Snapshots
6.12. Migrating and Seeding Data
6.12.1. Migrating from MySQL Native Replication 'In-Place'
6.12.2. Migrating from MySQL Native Replication Using a New Service
6.13. Resetting a Tungsten Cluster Dataservice
6.13.1. Reset a Single Site in a Multi-Site/Active-Active Topology
6.13.2. Reset All Sites in a Multi-Site/Active-Active topology
6.14. Replicator Fencing
6.14.1. Fencing a Replica Node Due to a Replication Fault
6.14.2. Fencing Primary Replicators
6.15. Upgrading Tungsten Cluster
6.15.1. Upgrading using the Staging Method
6.15.2. Upgrading when using INI-based configuration
6.15.3. Upgrade/Convert: From Multi-Site/Active-Active (MSAA) to Composite Active/Passive (CAP)
6.15.4. Upgrade/Convert: From Multi-Site/Active-Active (MSAA) to Composite Active/Active (CAA)
6.15.5. Installing an Upgraded JAR Patch
6.16. Performing Database or OS Maintenance
6.16.1. Performing Maintenance on a Single Replica
6.16.2. Performing Maintenance on a Primary
6.16.3. Performing Maintenance on an Entire Dataservice
6.16.4. Upgrading MySQL
6.16.5. Upgrading or Updating your JVM
6.16.6. Making Schema Changes
6.16.7. Updating Database User Passwords
6.17. Monitoring Tungsten Cluster
6.17.1. Managing Log Files with logrotate
6.17.2. Monitoring Status Using cacti
6.17.3. Monitoring Status Using nagios
6.18. Managing THL Files
6.18.1. Moving the THL File Location
6.18.2. Changing the THL Retention Times
6.18.3. Rebuilding THL on the Primary
7. Tungsten Connector
7.1. Connector/Application Basics
7.1.1. Connector Control Flow
7.2. Basic Connector Configuration
7.3. Clients and Deployment
7.3.1. Connection Pools
7.4. Routing Methods
7.4.1. Connector Routing Methods
7.4.2. Primary/Replica Selection
7.4.3. Connector Load Balancers
7.4.4. Specifying Required Latency
7.4.5. Setting Read Affinity and Direct Reads
7.4.6. Setting Read/Write Affinity in Composite Active/Active Environments
7.4.7. Setting Negative Affinity
7.4.8. Routing using embedded syntax in connect string
7.4.9. Connector Datasource Selection in Composite Clusters
7.4.10. Smartscale Routing
7.4.11. Direct Routing
7.4.12. SQL Routing
7.4.13. Host-based Routing
7.4.14. Port-based Routing
7.4.15. Read-only Routing
7.5. Using Bridge Mode
7.5.1. Enabling Bridge Mode
7.6. User Authentication
7.6.1. user.map File Format
7.6.2. Dual-Paswords and MySQL 8
7.6.3. user.map Direct Routing
7.6.4. user.map Host Options
7.6.5. user.map Updates
7.6.6. Generating user.map Entries from a Script
7.6.7. Encrypting user.map Data
7.6.8. Synchronizing user.map Data
7.6.9. user.map Limitations
7.6.10. Host-based Authentication
7.7. Testing Connectivity Via the Connector
7.7.1. Testing Connectivity in Bridge Mode
7.7.2. Testing Connectivity in Proxy Mode with No R/W Splitting Enabled
7.7.3. Testing Connectivity in Proxy Mode with R/W Splitting Enabled (SmartScale or @direct)
7.8. Connector Operational States
7.8.1. Connections During Automatic Failure/Failover
7.8.2. Connections During Manual Switch
7.8.3. Connections During Connection Failures
7.8.4. Other Errors
7.8.5. Connector Keepalive
7.8.6. Connector Change User as Ping
7.9. Connector/Manager Interface
7.10. Connector Command-Line Interface
7.11. Connector Inline Command Interface
7.11.1. Connector tungsten cluster status Command
7.11.2. Connector tungsten connection count Command
7.11.3. Connector tungsten connection status Command
7.11.4. Connector tungsten flush privileges Command
7.11.5. Connector tungsten gc Command
7.11.6. Connector tungsten help Command
7.11.7. Connector tungsten mem info Command
7.11.8. Connector tungsten show [full] processlist Command
7.11.9. Connector show slave status Command
7.11.10. Connector tungsten show variables Command
7.12. Advanced Configuration
7.12.1. Working with Proxy Protocol v1
7.12.2. Using Multiple Dataservices
7.12.3. Connector Automatic Reconnect
7.12.4. Using the Connector with HAProxy
7.12.5. Using Fall-Back Bridge Mode
7.12.6. Using the Max Connections Feature
7.12.7. Adjusting the Client Disconnect Delay During Manual Switch
7.12.8. Adjusting the Bridge Mode Forced Client Disconnect Timeout
7.12.9. Adjusting the Connnector Response to Resource Losses
7.12.10. Connector Logging Configuration
7.12.11. Connector Audit Logging
7.12.12. Connector SSL Advertisement Configuration
7.12.13. Connector IP Address Configuration
7.13. Connector General Limitations
8. Tungsten Manager
8.1. Tungsten Manager Introduction
8.1.1. How Does Failover Work?
8.1.2. Best Practices for Proper Cluster Failovers
8.1.3. Command-Line Monitoring Tools
8.2. Tungsten Manager Failover Behavior
8.2.1. Failover Replication State Scenarios
8.2.2. Recovery Behavior After Failover
8.2.3. Failover Response when MySQL Server Fails
8.2.4. Failover Response when Replica Applier is Latent
8.2.5. Understanding the FAILSAFE-SHUN state.
8.3. Tungsten Manager Failover Tuning
8.4. Tungsten Manager Failover Internals
8.4.1. Manual Switch Versus Automatic Failover
8.4.2. Switch and Failover Steps for Local Clusters
8.4.3. Switch and Failover Steps for Composite Services
8.5. Tungsten Manager Fault Detection, Fencing and Recovery
8.5.1. Tungsten Manager Definitions
8.5.2. Cluster Monitoring and Notification Events
8.5.3. Rule Organization - Detection, Investigation, Fencing, Recovery
8.6. Cluster State Savepoints
9. Command-line Tools
9.1. The cctrl Command
9.1.1. cctrl Command-line Options
9.1.2. cctrl Modes
9.1.3. cctrl Commands
9.2. The check_tungsten_latency Command
9.3. The check_tungsten_online Command
9.4. The check_tungsten_policy Command
9.5. The check_tungsten_progress Command
9.6. The check_tungsten_services Command
9.7. The clean_release_directory Command
9.8. The cluster_backup Command
9.9. The connector Command
9.10. The deployall Command
9.11. The dsctl Command
9.11.1. dsctl get Command
9.11.2. dsctl set Command
9.11.3. dsctl reset Command
9.11.4. dsctl help Command
9.12. env.sh Script
9.13. The manager Command
9.14. The multi_trepctl Command
9.14.1. multi_trepctl Options
9.14.2. multi_trepctl Commands
9.15. The query Command
9.16. The replicator Command
9.17. The startall Command
9.18. The stopall Command
9.19. The thl Command
9.19.1. thl Position Commands
9.19.2. thl index Command
9.19.3. thl info Command
9.19.4. thl list Command
9.19.5. thl purge Command
9.19.6. thl help Command
9.20. The tmonitor Command
9.21. The tpasswd Command
9.22. The trepctl Command
9.22.1. trepctl Options
9.22.2. trepctl Global Commands
9.22.3. trepctl Service Commands
9.23. The tungsten_find_orphaned Command
9.24. The tungsten_find_position Command
9.25. The tungsten_find_seqno Command
9.26. The tungsten_get_mysql_datadir Script
9.27. The tungsten_get_status Script
9.28. The tungsten_get_ports Script
9.29. The tungsten_merge_logs Script
9.30. The tungsten_monitor Script
9.31. The tungsten_newrelic_event Command
9.32. The tungsten_post_process Command
9.33. The tungsten_prep_upgrade Script
9.34. The tungsten_provision_slave Script
9.35. The tungsten_reset_manager Command
9.36. The tungsten_send_diag Script
9.36.1. Updating tungsten_send_diag
9.37. The tungsten_show_processlist Script
9.38. The tungsten_skip_all Command
9.39. The tungsten_skip_seqno Script
9.40. The undeployall Command
9.41. The zabbix_tungsten_latency Command
9.42. The zabbix_tungsten_online Command
9.43. The zabbix_tungsten_progress Command
9.44. The zabbix_tungsten_services Command
10. The tpm Deployment Command
10.1. Processing Installs and Upgrades
10.2. tpm INI File Configuration
10.2.1. Creating an INI file
10.2.2. Installation with INI File
10.2.3. Upgrades with an INI File
10.2.4. Configuration Changes with an INI file
10.3. Converting from Staging to INI
10.3.1. Using the translatetoini.pl Script
10.4. tpm Staging Configuration
10.4.1. Configuring default options for all services
10.4.2. Configuring a single service
10.4.3. Configuring a single host
10.4.4. Reviewing the current configuration
10.4.5. Installation
10.4.6. Upgrades from a Staging Directory
10.4.7. Configuration Changes from a Staging Directory
10.5. tpm Commands
10.5.1. tpm ask Command
10.5.2. tpm check Command
10.5.3. tpm configure Command
10.5.4. tpm connector Command
10.5.5. tpm delete-service Command
10.5.6. tpm diag Command
10.5.7. tpm fetch Command
10.5.8. tpm firewall Command
10.5.9. tpm find-seqno Command
10.5.10. tpm generate-haproxy-for-api Command
10.5.11. tpm help Command
10.5.12. tpm install Command
10.5.13. tpm mysql Command
10.5.14. tpm policy Command
10.5.15. tpm post-process Command
10.5.16. tpm promote-connector Command
10.5.17. tpm query Command
10.5.18. tpm reverse Command
10.5.19. tpm uninstall Command
10.5.20. tpm update Command
10.5.21. tpm validate Command
10.5.22. tpm validate-update Command
10.6. tpm Common Options
10.7. tpm Validation Checks
10.8. tpm Configuration Options
10.8.1. A tpm Options
10.8.2. B tpm Options
10.8.3. C tpm Options
10.8.4. D tpm Options
10.8.5. E tpm Options
10.8.6. F tpm Options
10.8.7. H tpm Options
10.8.8. I tpm Options
10.8.9. J tpm Options
10.8.10. L tpm Options
10.8.11. M tpm Options
10.8.12. N tpm Options
10.8.13. O tpm Options
10.8.14. P tpm Options
10.8.15. R tpm Options
10.8.16. S tpm Options
10.8.17. T tpm Options
10.8.18. U tpm Options
10.8.19. W tpm Options
11. Replication Filters
11.1. Enabling/Disabling Filters
11.2. Enabling Additional Filters
11.3. Filter Reference
11.3.1. ansiquotes.js Filter
11.3.2. BidiRemoteSlave (BidiSlave) Filter
11.3.3. breadcrumbs.js Filter
11.3.4. CaseTransform Filter
11.3.5. ColumnName Filter
11.3.6. ConvertStringFromMySQL Filter
11.3.7. DatabaseTransform (dbtransform) Filter
11.3.8. dbrename.js Filter
11.3.9. dbselector.js Filter
11.3.10. dbupper.js Filter
11.3.11. dropcolumn.js Filter
11.3.12. dropcomments.js Filter
11.3.13. dropddl.js Filter
11.3.14. dropmetadata.js Filter
11.3.15. droprow.js Filter
11.3.16. dropstatementdata.js Filter
11.3.17. dropsqlmode.js Filter
11.3.18. dropxa.js Filter
11.3.19. Dummy Filter
11.3.20. EnumToString Filter
11.3.21. EventMetadata Filter
11.3.22. foreignkeychecks.js Filter
11.3.23. Heartbeat Filter
11.3.24. insertsonly.js Filter
11.3.25. Logging Filter
11.3.26. MySQLSessionSupport (mysqlsessions) Filter
11.3.27. mapcharset Filter
11.3.28. NetworkClient Filter
11.3.29. nocreatedbifnotexists.js Filter
11.3.30. OptimizeUpdates Filter
11.3.31. PrimaryKey Filter
11.3.32. PrintEvent Filter
11.3.33. Rename Filter
11.3.34. Replicate Filter
11.3.35. ReplicateColumns Filter
11.3.36. Row Add Database Name Filter
11.3.37. Row Add Transaction Info Filter
11.3.38. SetToString Filter
11.3.39. Shard Filter
11.3.40. shardbyseqno.js Filter
11.3.41. shardbytable.js Filter
11.3.42. SkipEventByType Filter
11.3.43. TimeDelay (delay) Filter
11.3.44. TimeDelayMsFilter (delayInMS) Filter
11.3.45. tosingledb.js Filter
11.3.46. truncatetext.js Filter
11.3.47. zerodate2null.js Filter
11.4. Standard JSON Filter Configuration
11.4.1. Rule Handling and Processing
11.4.2. Schema, Table, and Column Selection
11.5. JavaScript Filters
11.5.1. Writing JavaScript Filters
11.5.2. Installing Custom JavaScript Filters
12. Performance, Tuning and Testing
12.1. Block Commit
12.1.1. Monitoring Block Commit Status
12.2. Improving Network Performance
12.3. Tungsten Replicator Block Commit and Memory Usage
12.4. Connector Memory Management
12.5. Functional Testing
12.5.1. Manual and Automatic Failover
12.5.2. Backup and Restore
12.5.3. Connectivity
12.5.4. Performance and Other Tests
A. Release Notes
A.1. Tungsten Clustering 6.1.25 GA (7 July 2024)
A.2. Tungsten Clustering 6.1.24 GA (11 Dec 2023)
A.3. Tungsten Clustering 6.1.23 GA (31 Aug 2023)
A.4. Tungsten Clustering 6.1.22 GA (6 July 2023)
A.5. Tungsten Clustering 6.1.21 GA (18 Apr 2023)
A.6. Tungsten Clustering 6.1.20 GA (19 Dec 2022)
A.7. Tungsten Clustering 6.1.19 GA (17 Oct 2022)
A.8. Tungsten Clustering 6.1.18 GA (7 Sept 2022)
A.9. Tungsten Clustering 6.1.17 GA (16 May 2022)
A.10. Tungsten Clustering 6.1.16 GA (20 Dec 2021)
A.11. Tungsten Clustering 6.1.15 GA (19 Oct 2021)
A.12. Tungsten Clustering 6.1.14 GA (17 Aug 2021)
A.13. Tungsten Clustering 6.1.13 GA (27 May 2021)
A.14. Tungsten Clustering 6.1.12 GA (8 Mar 2021)
A.15. Tungsten Clustering 6.1.11 GA (21 Jan 2021)
A.16. Tungsten Clustering 6.1.10 GA (15 Dec 2020)
A.17. Tungsten Clustering 6.1.9 GA (23 Nov 2020)
A.18. Tungsten Clustering 6.1.8 GA (2 Nov 2020)
A.19. Tungsten Clustering 6.1.7 GA (5 Oct 2020)
A.20. Tungsten Clustering 6.1.6 GA (20 Aug 2020)
A.21. Tungsten Clustering 6.1.5 GA (5 Aug 2020)
A.22. Tungsten Clustering 6.1.4 GA (4 June 2020)
A.23. Tungsten Clustering 6.1.3 GA (17 February 2020)
A.24. Tungsten Clustering 6.1.2 GA (20 January 2020)
A.25. Tungsten Clustering 6.1.1 GA (28 October 2019)
A.26. Tungsten Clustering 6.1.0 GA (31 July 2019)
B. Prerequisites
B.1. Requirements
B.1.1. Operating Systems Support
B.1.2. Database Support
B.1.3. RAM Requirements
B.1.4. Disk Requirements
B.1.5. Java Requirements
B.1.6. Cloud Deployment Requirements
B.1.7. Docker Support Policy
B.2. Staging Host Configuration
B.3. Host Configuration
B.3.1. Operating System Version Support
B.3.2. Creating the User Environment
B.3.3. Configuring Network and SSH Environment
B.3.4. Directory Locations and Configuration
B.3.5. Configure Software
B.3.6. sudo Configuration
B.3.7. SELinux Configuration
B.4. MySQL Database Setup
B.4.1. MySQL Version Support
B.4.2. MySQL Configuration
B.4.3. MySQL Configuration for Active/Active Deployments
B.4.4. MySQL User Configuration
B.4.5. MySQL Unprivileged Users
B.5. Prerequisite Checklist
C. Troubleshooting
C.1. Contacting Support
C.1.1. Support Request Procedure
C.1.2. Creating a Support Account
C.1.3. Open a Support Ticket
C.1.4. Open a Support Ticket via Email
C.1.5. Getting Updates for all Company Support Tickets
C.1.6. Support Severity Level Definitions
C.2. Support Tools
C.2.1. Generating Diagnostic Information
C.2.2. Generating Advanced Diagnostic Information
C.2.3. Using tungsten_upgrade_manager
C.3. Error/Cause/Solution
C.3.1. MySQL is incorrectly configured
C.3.2. WARNING: An illegal reflective access operation has occurred
C.3.3. element 'mysql_readonly' not found in path
C.3.4. Latency is high: master:ONLINE, progress=41331580333, THL latency=78849.733
C.3.5. Backup/Restore is not bringing my host back to normal
C.3.6. Attempt to write new log record with equal or lower fragno: seqno=3 previous stored fragno=32767 attempted new fragno=-32768
C.3.7. Triggers not firing correctly on Replica
C.3.8. ERROR backup.BackupTask Backup operation failed: null
C.3.9. cctrl reports MANAGER(state=STOPPED)
C.3.10. Starting replication after performing a restore because of an invalid restart sequence number
C.3.11. WARN [KeepAliveTimerTask] - Error while sending a KEEP_ALIVE query to connection.
C.3.12. [S1000][unixODBC][MySQL][ODBC 5.3(w) Driver]SSL connection error: unknown error number [ISQL]ERROR: Could not SQLConnect
C.3.13. MySQL 8.0+, User Roles and Smartscale
C.3.14. Connector shows errors with "java.net.SocketException: Broken pipe"
C.3.15. Services requires a reset
C.3.16. trepctl status hangs
C.3.17. Replicator reports an Out of Memory error
C.3.18. Backup agent name not found: xtrabackup-full
C.3.19. ERROR 1010 (HY000) at line 5094506: Error dropping database (can't rmdir './mysql-bin/', errno: 17)
C.3.20. ERROR >> host1 >> can't alloc thread
C.3.21. Too many open processes or files
C.3.22. Replicator fails to connect after updating password
C.3.23. Cluster remains in MAINTENANCE mode after tpm update
C.3.24. Replication latency very high
C.3.25. ERROR 2013 (HY000) at line 583: Lost connection to MySQL server during query
C.3.26. pendingExceptionMessage": "Unable to update last commit seqno: Incorrect datetime value: '2025-03-13 02:02:26' for column 'update_timestamp' at row 1
C.3.27. Lots of entries added to replicator log
C.3.28. Error: could not settle on encryption_client algorithm
C.3.29. Unable to update the configuration of an installed directory
C.3.30. ERROR 1580 (HY000) at line 5093787: You cannot 'DROP' a log table if logging is enabled
C.3.31. The Primary replicator stopped with a JDBC error.
C.3.32. Replicator runs out of memory
C.4. Known Issues
C.4.1. Triggers
C.5. Troubleshooting Timeouts
C.6. Troubleshooting Backups
C.7. Running Out of Diskspace
C.8. Troubleshooting SSH and tpm
C.9. Troubleshooting Data Differences
C.9.1. Identify Structural Differences
C.9.2. Identify Data Differences
C.10. Comparing Table Data
C.11. Troubleshooting Memory Usage
D. Files, Directories, and Environment
D.1. The Tungsten Install Directory
D.1.1. The backups Directory
D.1.2. The releases Directory
D.1.3. The service_logs Directory
D.1.4. The share Directory
D.1.5. The thl Directory
D.1.6. The tungsten Directory
D.2. Log Files
D.3. Environment Variables
E. Terminology Reference
E.1. Transaction History Log (THL)
E.1.1. THL Format
E.2. Generated Field Reference
E.2.1. Terminology: Fields masterConnectUri
E.2.2. Terminology: Fields masterListenUri
E.2.3. Terminology: Fields accessFailures
E.2.4. Terminology: Fields active
E.2.5. Terminology: Fields activeSeqno
E.2.6. Terminology: Fields appliedLastEventId
E.2.7. Terminology: Fields appliedLastGoodLatency
E.2.8. Terminology: Fields appliedLastGoodSeqno
E.2.9. Terminology: Fields appliedLastSeqno
E.2.10. Terminology: Fields appliedLatency
E.2.11. Terminology: Fields applier.class
E.2.12. Terminology: Fields applier.name
E.2.13. Terminology: Fields applyTime
E.2.14. Terminology: Fields autoRecoveryEnabled
E.2.15. Terminology: Fields autoRecoveryTotal
E.2.16. Terminology: Fields averageBlockSize
E.2.17. Terminology: Fields blockCommitRowCount
E.2.18. Terminology: Fields cancelled
E.2.19. Terminology: Fields channel
E.2.20. Terminology: Fields channels
E.2.21. Terminology: Fields clusterName
E.2.22. Terminology: Fields commits
E.2.23. Terminology: Fields committedMinSeqno
E.2.24. Terminology: Fields criticalPartition
E.2.25. Terminology: Fields currentBlockSize
E.2.26. Terminology: Fields currentEventId
E.2.27. Terminology: Fields currentLastEventId
E.2.28. Terminology: Fields currentLastFragno
E.2.29. Terminology: Fields currentLastSeqno
E.2.30. Terminology: Fields currentTimeMillis
E.2.31. Terminology: Fields dataServerHost
E.2.32. Terminology: Fields discardCount
E.2.33. Terminology: Fields doChecksum
E.2.34. Terminology: Fields estimatedOfflineInterval
E.2.35. Terminology: Fields eventCount
E.2.36. Terminology: Fields extensions
E.2.37. Terminology: Fields extractTime
E.2.38. Terminology: Fields extractor.class
E.2.39. Terminology: Fields extractor.name
E.2.40. Terminology: Fields filter.#.class
E.2.41. Terminology: Fields filter.#.name
E.2.42. Terminology: Fields filterTime
E.2.43. Terminology: Fields flushIntervalMillis
E.2.44. Terminology: Fields fsyncOnFlush
E.2.45. Terminology: Fields headSeqno
E.2.46. Terminology: Fields intervalGuard
E.2.47. Terminology: Fields lastCommittedBlockSize
E.2.48. Terminology: Fields lastCommittedBlockTime
E.2.49. Terminology: Fields latestEpochNumber
E.2.50. Terminology: Fields logConnectionTimeout
E.2.51. Terminology: Fields logDir
E.2.52. Terminology: Fields logFileRetainMillis
E.2.53. Terminology: Fields logFileSize
E.2.54. Terminology: Fields maxChannel
E.2.55. Terminology: Fields maxDelayInterval
E.2.56. Terminology: Fields maxOfflineInterval
E.2.57. Terminology: Fields maxSize
E.2.58. Terminology: Fields maximumStoredSeqNo
E.2.59. Terminology: Fields minimumStoredSeqNo
E.2.60. Terminology: Fields name
E.2.61. Terminology: Fields offlineRequests
E.2.62. Terminology: Fields otherTime
E.2.63. Terminology: Fields pendingError
E.2.64. Terminology: Fields pendingErrorCode
E.2.65. Terminology: Fields pendingErrorEventId
E.2.66. Terminology: Fields pendingErrorSeqno
E.2.67. Terminology: Fields pendingExceptionMessage
E.2.68. Terminology: Fields pipelineSource
E.2.69. Terminology: Fields processedMinSeqno
E.2.70. Terminology: Fields queues
E.2.71. Terminology: Fields readOnly
E.2.72. Terminology: Fields relativeLatency
E.2.73. Terminology: Fields resourcePrecedence
E.2.74. Terminology: Fields rmiPort
E.2.75. Terminology: Fields role
E.2.76. Terminology: Fields seqnoType
E.2.77. Terminology: Fields serializationCount
E.2.78. Terminology: Fields serialized
E.2.79. Terminology: Fields serviceName
E.2.80. Terminology: Fields serviceType
E.2.81. Terminology: Fields shard_id
E.2.82. Terminology: Fields simpleServiceName
E.2.83. Terminology: Fields siteName
E.2.84. Terminology: Fields sourceId
E.2.85. Terminology: Fields stage
E.2.86. Terminology: Fields started
E.2.87. Terminology: Fields state
E.2.88. Terminology: Fields stopRequested
E.2.89. Terminology: Fields store.#
E.2.90. Terminology: Fields storeClass
E.2.91. Terminology: Fields syncInterval
E.2.92. Terminology: Fields taskCount
E.2.93. Terminology: Fields taskId
E.2.94. Terminology: Fields timeInCurrentEvent
E.2.95. Terminology: Fields timeInStateSeconds
E.2.96. Terminology: Fields timeoutMillis
E.2.97. Terminology: Fields totalAssignments
E.2.98. Terminology: Fields transitioningTo
E.2.99. Terminology: Fields uptimeSeconds
E.2.100. Terminology: Fields version
F. Internals
F.1. Extending Backup and Restore Behavior
F.1.1. Backup Behavior
F.1.2. Restore Behavior
F.1.3. Writing a Custom Backup/Restore Script
F.1.4. Enabling a Custom Backup Script
F.2. Character Sets in Database and Tungsten Cluster
F.3. Understanding Replication of Date/Time Values
F.3. Best Practices
F.4. Tungsten Clustering Schemas
G. Frequently Asked Questions (FAQ)
G.1. General Questions
G.2. Cloud Deployment and Management

List of Figures

3.1. Topologies: Standalone HA Cluster
3.2. Topologies: Composite Active/Passive Cluster
3.3. Topologies: Composite Active/Active Clusters
3.4. Topologies: Multi-Site/Active-Active Clusters
3.5. Topologies: Replicating into a Dataservice
3.6. Topologies: Replicating Data Out of a Cluster
3.7. Migration: Migrating Native Replication using a New Service
5.1. Security Internals: Cluster Communication Channels
6.1. Migration: Migrating Native Replication using a New Service
6.2. Cacti Monitoring: Example Graphs
7.1. Tungsten Connector Basic Architecture
7.2. Basic MySQL/Application Connectivity
7.3. Advanced MySQL/Application Connectivity
7.4. Using Tungsten Connector for MySQL/Application Connectivity
7.5. Tungsten Connector during a failed datasource
7.6. Tungsten Connector routing architecture
7.7. Tungsten Connector Bridge Mode Architecture
7.8. Tungsten Connector Authentication
8.1. Failover Scenario 1
8.2. Failover Scenario 2
8.3. Failover Scenario 3
11.1. Filters: Pipeline Stages on Extractors
11.2. Filters: Pipeline Stages on Appliers
B.1. Tungsten Deployment

List of Tables

1.1. Key Terminology
2.1. Key Terminology
7.1. Routing Method Selection
7.2. Connector Command Line Sub-Commands
7.3. Inline Interface Commands
9.1. cctrl Command-line Options
9.2. cctrl Command-line Options
9.3. cctrl Command-line Options
9.4. cctrl Command-line Options
9.5. cctrl Command-line Options
9.6. cctrl Command-line Options
9.7. cctrl Command-line Options
9.8. cctrl Command-line Options
9.9. cctrl Command-line Options
9.10. cctrl Command-line Options
9.11. cctrl Command-line Options
9.12. cctrl Commands
9.13. cctrl datasource Commands
9.14. cctrl replicator Commands
9.15. check_tungsten_latency Options
9.16. check_tungsten_online Options
9.17. check_tungsten_policy Options
9.18. check_tungsten_progress Options
9.19. check_tungsten_services Options
9.20. cluster_backup Command-line Options
9.21. connector Commands
9.22. dsctl Commands
9.23. dsctl Command-line Options
9.24. dsctl Command-line Options
9.25. dsctl Command-line Options
9.26. manager Commands
9.27. multi_trepctl Command-line Options
9.28. multi_trepctl--output Option
9.29. multi_trepctl Commands
9.30. query Common Options
9.31. replicator Commands
9.32. replicator Commands Options for condrestart
9.33. replicator Commands Options for console
9.34. replicator Commands Options for restart
9.35. replicator Commands Options for start
9.36. thl Options
9.37. tmonitor Common Options
9.38. tpasswd Common Options
9.39. trepctl Command-line Options
9.40. trepctl Replicator Wide Commands
9.41. trepctl Service Commands
9.42. trepctl backup Command Options
9.43. trepctl clients Command Options
9.44. trepctl offline-deferred Command Options
9.45. trepctl online Command Options
9.46. trepctl pause Command Options
9.47. trepctl purge Command Options
9.48. trepctl reset Command Options
9.49. trepctl resume Command Options
9.50. trepctl setdynamic Command Options
9.51. trepctl setrole Command Options
9.52. trepctl shard Command Options
9.53. trepctl status Command Options
9.54. trepctl wait Command Options
9.55. tungsten_find_orphaned Options
9.56. tungsten_find_position Options
9.57. tungsten_get_mysql_datadir Command-line Options
9.58. tungsten_get_ports Options
9.59. tungsten_merge_logs Command-line Options
9.60. tungsten_monitor Command-line Options
9.61. tungsten_newrelic_event Command-line Options
9.62. tungsten_prep_upgrade Command-line Options
9.63. tungsten_provision_slave Command-line Options
9.64. tungsten_reset_manager Options
9.65. tungsten_send_diag Command-line Options
9.66. tungsten_show_processlist Command-line Options
9.67. tungsten_skip_seqno Command-line Options
9.68. zabbix_tungsten_latency Options
9.69. zabbix_tungsten_online Options
9.70. zabbix_tungsten_progress Options
9.71. check_tungsten_services Options
10.1. tpm Core Options
10.2. tpm Commands
10.3. tpm delete-service Common Options
10.4. tpm diag Common Options
10.5. tungsten_find_seqno Options
10.6. tpm generate-haproxy-for-api Common Options
10.7. tpm policy Options
10.8. tpm-post-process Options
10.9. tpm Common Options
10.10. tpm Validation Checks
10.11. tpm Configuration Options
B.1. Tungsten OS Support
B.2. MySQL/Tungsten Version Support
D.1. Continuent Tungsten Directory Structure
D.2. Continuent Tungsten tungsten Sub-Directory Structure
E.1. THL Event Format
   Next
   Preface
Copyright © 2025 Continuent, Ltd.1 (415) 529-4901Terms of UsePrivacy & CookiesLegal NoticeExhibits