The issue outlined below only affects connectors running in Proxy Mode, and more imporantly the behavior all depends on when connectors are restarted as part of the process.
When the connectors are running in Proxy Mode, they advertise a cached, static, MySQL version that is retrieved at connector startup from the database, and is assumed by the connectors to be the global version of all database nodes.
In versions prior to 7.2.0, this request was made on a replica (to avoid unneeded load on the primary). With versions 7.2.0 onwards, the request is now done on the primary by default, however this behavior can be changed to the pre-7.2.0 behavior by adding the following property to your configuration:
property=readAutodetectConfigFromPrimary=false
In situations where a rolling upgrade is being performed, and you have upgraded the replica node to a higher version of MySQL, for example MySQL 8.0, the connectors will still report MySQL 5.7 since this version reflects the primary node.
If your application relies on reading this version to ensure it behaves accordingly, this advertised version may cause the application to behave in a way that would fail if the advertised version does not match the actual version of the database that is handling a specific request, for example the advertised version could be 5.7 reflecting the primary however the request is directed to a replica which has been upgraded and is in fact running 8.0.
This is most commonly an issue if you are using hibernate.
To workaround this, there are a number of options you could employ:
Temporarily switch the connectors to run in Bridge Mode. In this mode the connectors do not advertise the version of MySQL and therefore will always be correct and mirror the version of the database that the connection is established too.
There is a very small window of downtime in this approach at the point at which a switch is issued to move the primary node. This will only be a matter of seconds.
On all connector nodes, edit your /etc/tungsten/tungsten.ini and add (or modify) the
connector-bridge-mode property, setting the value
to true, followed by tpm update
Do NOT add the --replace-release switch to the update to ensure
the connectors do not get restarted
If you have a Composite Active/Passive or Composite Active/Active cluster, and have the ability to redirect traffic easily, simply point all traffic to one cluster whilst upgrading the other, then switch between clusters and redirect traffic. Once both clusters have been upgraded, rebalance traffic as normal. This could also be combined with the above procedure to switch to Bridge Mode if desired.
High level of this procedure would be as follows:
Direct all traffic to SiteA (Only required for Composite Active/Active clusters)
Upgrade all nodes in SiteB to the new version of MySQL (Including the connectors)
If this is a Composite Active/Passive cluster, issue a switch to promote SiteB to be active and SiteA to be passive.
Direct all traffic to SiteB (Only required for Composite Active/Active clusters)
Upgrade all nodes in SiteA to the new version of MySQL (Including the connectors)
If this is a Composite Active/Passive cluster, and you wish to return SiteA to be active, then issue the switch to promote SiteA to be active and SiteB to be passive.
Rebalance traffic across both clusters (Only required for Composite Active/Active clusters)
Add the connector-db-version property to your /etc/tungsten/tungsten.ini
on the connectors. This will force the connectors to advertise a specific mysql version.
This option could still result in the wrong version being advertised to the applications, and therefore should only be used when you are confident that the application will not misbehave when the version does not match. It is not recommended for hibernate users.