OKD/OpenShift >4.14 - Is it still possible to run cluster/status on the Open vSwitch DB?

[glowing-axolotl]

TL;DR is the ovn-appctl -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound command still supported in OKD/OCP?

---

As per https://access.redhat.com/articles/6963671 , it may be sometimes necessary to rebuild the OVN databases for some nodes in specific conditions.

When rebuilding OVN on v4.8 - v4.13 you could run the following commands to get an overview of the OVN status:

oc -n openshift-ovn-kubernetes get pods
oc -n openshift-ovn-kubernetes exec -it -c northd $OVNMASTER ovn-appctl -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
oc -n openshift-ovn-kubernetes exec -it -c northd $OVNMASTER ovn-appctl -t /var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound

Which would give an output in the line of:

# oc exec -n openshift-ovn-kubernetes ovnkube-master-7c42x -- /usr/bin/ovn-appctl -t /var/run/ovn/ovnn_db_ctl --timeout=3 cluster/status OVN_Northbound
Defaulted container "northd" out of: northd, nbdb, kube-rbac-proxy, sbdb, ovnkube-master, ovn-dbchecker
Name: OVN Northbound
Cluster ID: 7603 (76038aa-bf55-46c2-8eec-35100b72e640)
Server ID: b288 (b288e90e-ca8f-4428-8101-2d6d88eb12f7)
Address: ssl:172.27.128.102:9643
Status: cluster member
Role: follower
Term: 2912
Leader: acad
Vote: acad

Last Election started 55726924 ms ago, reason: leadership_transfer
Last Election won: 55726914 ms ago
Election timer: 10000
Log: [7530155, 7535276]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: <-acad <-acad <-24b9 ->24b9
Disconnections: 1
Servers:
    acad (acad at ssl:172.27.128.109:9643) last msg 253 ms ago
    b288 (b288 at ssl:172.27.128.102:9643) (self)
    24b9 (24b9 at ssl:172.27.128.101:9643) last msg 46469106 ms ago

When rebuilding OVN on 4.14 and later, you are told to verify that the pods are up and running, but the command cluster/status isn't used anymore.

We can clearly see this removal as it is present in the doc for 4.13:
https://docs.okd.io/4.13/networking/ovn_kubernetes_network_provider/ovn-kubernetes-architecture-assembly.html#nw-ovn-kubernetes-list-database-contents_ovn-kubernetes-architecture

But not in 4.14:
https://docs.okd.io/4.14/networking/ovn_kubernetes_network_provider/ovn-kubernetes-architecture-assembly.html#nw-ovn-kubernetes-list-database-contents_ovn-kubernetes-architecture

It was removed from https://github.com/openshift/openshift-docs/ in commit 02c14ce669f70b55635bdfc6c65caa6736b53933 :

commit 02c14ce669f70b55635bdfc6c65caa6736b53933
Author: JoeAldinger <97132922+JoeAldinger@users.noreply.github.com>
Date:   Fri Jul 7 12:39:44 2023 -0400

    OSDOCS-6269: ovn-k interconnect 4.14
...
diff --git a/modules/nw-ovn-kubernetes-list-database-contents.adoc b/modules/nw-ovn-kubernetes-list-database-contents.adoc
index a2f7f08f5c..44021f9937 100644
--- a/modules/nw-ovn-kubernetes-list-database-contents.adoc
+++ b/modules/nw-ovn-kubernetes-list-database-contents.adoc
...
@@ -33,109 +30,71 @@ $ oc get po -n openshift-ovn-kubernetes
...
 [source,terminal]
 ----
-$ oc exec -n openshift-ovn-kubernetes ovnkube-master-7j97q \
--- /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl \
---timeout=3 cluster/status OVN_Northbound
+$ oc get pods -n openshift-ovn-kubernetes -owide
...

The command isn't explicitly deprecated anywhere, for example it is still used in the upstream documentation of OVN Kubernetes here https://ovn-kubernetes.io/installation/ha/#master2-master3-initialization .

At one point in time, it was even used in https://github.com/openshift/cluster-network-operator for the pod's probes (relevant bug is https://bugzilla.redhat.com/show_bug.cgi?id=1838343 ):

git show 8ea98a00a
...
commit 8ea98a00aa06e6ffef373a82093ae58773ebce99
Author: Anil Vishnoi <avishnoi@redhat.com>
Date:   Fri May 29 16:34:13 2020 -0700

    Bug 1838343 - Improve the sb-db and nb-db readiness check to ensure it fails when cluster is not stable.

    Raft cluster can get into state where they are not able
    to form concensus for raft leader. Cluster in this state
    is not ready to accept any transaction and the requent
    can have unexpected results.

    Current version of readiness probe check for the cluster/status
    and check if the node ip is set. This establishes that the
    cluster node is up but it doesn't mean that the cluster is
    in healty state to process the transaction request. This check
    will only fail if cluster/status command itself fails.

    This patch looks at the state of leader and if it's in unknown
    state for the given node for 3 continuous checks, it will mark
    the container unhealthy. This gives a better check on the state
    of individual nodes.
    Note: Overall convergence of the cluster still depends on
    how these nodes individually will restart and what is the state
    of other nodes in the cluster. Using just local readiness probes,
    it hard to determine the overall cluster health.

    Signed-off-by: Anil Vishnoi <avishnoi@redhat.com>
...
diff --git a/bindata/network/ovn-kubernetes/ovnkube-master.yaml b/bindata/network/ovn-kubernetes/ovnkube-master.yaml
index 3067babb8..6f08608b3 100644
--- a/bindata/network/ovn-kubernetes/ovnkube-master.yaml
+++ b/bindata/network/ovn-kubernetes/ovnkube-master.yaml
@@ -388,15 +388,20 @@ spec:
                 - /var/run/ovn/ovnnb_db.ctl
                 - exit
         readinessProbe:
-          initialDelaySeconds: 60
+          initialDelaySeconds: 90
           timeoutSeconds: 5
           exec:
             command:
             - /bin/bash
             - -c
             - |
-              set -xe
-              /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status OVN_Northbound  2>/dev/null | grep ${K8S_NODE_IP} | grep -v Address -q
+              set -xeo pipefail
+              leader_status=$(/usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status OVN_Northbound  2>/dev/null | { grep "Leader: unknown" || true; })
+              if [[ ! -z "${leader_status}" ]]; then
+                echo "NB DB Raft leader is unknown to the cluster node."
+                exit 1
+              fi
+
...

If we try running it on a 4.14 or, in this case, a 4.19 cluster we see that it doesn't work on any of the pods, both "control-plane" as well as ovnkube-node pods on masters:

# oc -n openshift-ovn-kubernetes get pods -o wide
NAME                                     READY   STATUS    RESTARTS   AGE   IP             NODE                        NOMINATED NODE   READINESS GATES
ovnkube-control-plane-75c4598f86-dw98l   2/2     Running   2          19d   10.114.80.64   okd-bf46r-master-1          <none>           <none>
ovnkube-control-plane-75c4598f86-q784w   2/2     Running   2          19d   10.114.80.58   okd-bf46r-master-0          <none>           <none>
ovnkube-node-92r8d                       8/8     Running   8          19d   10.114.80.61   okd-bf46r-worker-0-m65fg    <none>           <none>
ovnkube-node-96tbk                       8/8     Running   8          9d    10.114.80.54   okd-bf46r-infra-0-22t8h     <none>           <none>
ovnkube-node-cspb6                       8/8     Running   8          9d    10.114.80.59   okd-bf46r-storage-0-xkw59   <none>           <none>
ovnkube-node-gjmz9                       8/8     Running   8          9d    10.114.80.60   okd-bf46r-storage-0-gxgz2   <none>           <none>
ovnkube-node-m2psn                       8/8     Running   8          19d   10.114.80.55   okd-bf46r-worker-0-v5d6j    <none>           <none>
ovnkube-node-ndsmr                       8/8     Running   8          19d   10.114.80.57   okd-bf46r-master-2          <none>           <none>
ovnkube-node-nqpk9                       8/8     Running   8          9d    10.114.80.62   okd-bf46r-infra-0-84n7s     <none>           <none>
ovnkube-node-tcdm8                       8/8     Running   8          9d    10.114.80.56   okd-bf46r-storage-0-lkb9f   <none>           <none>
ovnkube-node-vqscx                       8/8     Running   8          19d   10.114.80.64   okd-bf46r-master-1          <none>           <none>
ovnkube-node-wk95k                       8/8     Running   8          19d   10.114.80.58   okd-bf46r-master-0          <none>           <none>

# oc exec -n openshift-ovn-kubernetes ovnkube-control-plane-75c4598f86-dw98l -c ovnkube-cluster-manager -- /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status OVN_Northbound
2025-12-29T14:00:32Z|00001|unixctl|WARN|failed to connect to /var/run/ovn/ovnnb_db.ctl
ovn-appctl: cannot connect to "/var/run/ovn/ovnnb_db.ctl" (No such file or directory)
command terminated with exit code 1
# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -- /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl--timeout=3 cluster/status OVN_Northbound
Defaulted container "ovn-controller" out of: ovn-controller, ovn-acl-logging, kube-rbac-proxy-node, kube-rbac-proxy-ovn-metrics, northd, nbdb, sbdb, ovnkube-controller, kubecfg-setup (init)
"cluster/status" is not a valid command (use "list-commands" to see a list of valid commands)
ovn-appctl: /var/run/ovn/ovnnb_db.ctl: server returned an error
command terminated with exit code 2
# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c northd -- /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status OVN_Northbound
"cluster/status" is not a valid command (use "list-commands" to see a list of valid commands)
ovn-appctl: /var/run/ovn/ovnnb_db.ctl: server returned an error
command terminated with exit code 2
# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status OVN_Northbound
"cluster/status" is not a valid command (use "list-commands" to see a list of valid commands)
ovn-appctl: /var/run/ovn/ovnnb_db.ctl: server returned an error
command terminated with exit code 2

Taking a look at the ovnkube-node-wk95k pod's probe:

# oc -n openshift-ovn-kubernetes get pods ovnkube-node-wk95k -o yaml
...
    readinessProbe:
      exec:
        command:
        - /bin/bash
        - -c
        - |
          set -xeo pipefail
          . /ovnkube-lib/ovnkube-lib.sh || exit 1
          ovndb-readiness-probe "nb"
...

It is a function that under the hood uses ovn-appctl ... ovsdb-server/sync-status:

# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; type ovndb-readiness-probe "
+ . /ovnkube-lib/ovnkube-lib.sh
++ set -x
++ K8S_NODE=okd-bf46r-master-0
++ [[ -n okd-bf46r-master-0 ]]
++ [[ -f /env/okd-bf46r-master-0 ]]
++ northd_pidfile=/var/run/ovn/ovn-northd.pid
++ controller_pidfile=/var/run/ovn/ovn-controller.pid
++ controller_logfile=/var/log/ovn/acl-audit-log.log
++ vswitch_dbsock=/var/run/openvswitch/db.sock
++ nbdb_pidfile=/var/run/ovn/ovnnb_db.pid
++ nbdb_sock=/var/run/ovn/ovnnb_db.sock
++ nbdb_ctl=/var/run/ovn/ovnnb_db.ctl
++ sbdb_pidfile=/var/run/ovn/ovnsb_db.pid
++ sbdb_sock=/var/run/ovn/ovnsb_db.sock
++ sbdb_ctl=/var/run/ovn/ovnsb_db.ctl
+ type ovndb-readiness-probe
ovndb-readiness-probe is a function
ovndb-readiness-probe ()
{
    local dbname=$1;
    if [[ $# -ne 1 ]]; then
        echo "Expected one argument but got $#";
        exit 1;
    fi;
    local ctlfile;
    if [[ "${dbname}" = "nb" ]]; then
        ctlfile=${nbdb_ctl};
    else
        if [[ "${dbname}" = "sb" ]]; then
            ctlfile=${sbdb_ctl};
        else
            echo "unknown DB name ${dbname}";
            exit 1;
        fi;
    fi;
    status=$(/usr/bin/ovn-appctl -t ${ctlfile} --timeout=3 ovsdb-server/sync-status  2>/dev/null | { grep "state: active" || false; });
    if [[ -z "${status}" ]]; then
        echo "${dbname} DB is not running or active.";
        exit 1;
    fi
}

Running it manually:

# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; ovndb-readiness-probe "nb""
+ . /ovnkube-lib/ovnkube-lib.sh
++ set -x
++ K8S_NODE=okd-bf46r-master-0
++ [[ -n okd-bf46r-master-0 ]]
++ [[ -f /env/okd-bf46r-master-0 ]]
++ northd_pidfile=/var/run/ovn/ovn-northd.pid
++ controller_pidfile=/var/run/ovn/ovn-controller.pid
++ controller_logfile=/var/log/ovn/acl-audit-log.log
++ vswitch_dbsock=/var/run/openvswitch/db.sock
++ nbdb_pidfile=/var/run/ovn/ovnnb_db.pid
++ nbdb_sock=/var/run/ovn/ovnnb_db.sock
++ nbdb_ctl=/var/run/ovn/ovnnb_db.ctl
++ sbdb_pidfile=/var/run/ovn/ovnsb_db.pid
++ sbdb_sock=/var/run/ovn/ovnsb_db.sock
++ sbdb_ctl=/var/run/ovn/ovnsb_db.ctl
+ echo /usr/bin/ovn-appctl -t --timeout=3 ovsdb-server/sync-status
/usr/bin/ovn-appctl -t --timeout=3 ovsdb-server/sync-status
# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; type ovndb-readiness-probe "
+ . /ovnkube-lib/ovnkube-lib.sh
++ set -x
++ K8S_NODE=okd-bf46r-master-0
++ [[ -n okd-bf46r-master-0 ]]
++ [[ -f /env/okd-bf46r-master-0 ]]
++ northd_pidfile=/var/run/ovn/ovn-northd.pid
++ controller_pidfile=/var/run/ovn/ovn-controller.pid
++ controller_logfile=/var/log/ovn/acl-audit-log.log
++ vswitch_dbsock=/var/run/openvswitch/db.sock
ovndb-readiness-probe is a function
ovndb-readiness-probe ()
{
    local dbname=$1;
    if [[ $# -ne 1 ]]; then
        echo "Expected one argument but got $#";
        exit 1;
    fi;
    local ctlfile;
    if [[ "${dbname}" = "nb" ]]; then
        ctlfile=${nbdb_ctl};
    else
        if [[ "${dbname}" = "sb" ]]; then
            ctlfile=${sbdb_ctl};
        else
            echo "unknown DB name ${dbname}";
            exit 1;
        fi;
    fi;
    status=$(/usr/bin/ovn-appctl -t ${ctlfile} --timeout=3 ovsdb-server/sync-status  2>/dev/null | { grep "state: active" || false; });
    if [[ -z "${status}" ]]; then
        echo "${dbname} DB is not running or active.";
        exit 1;
    fi
}
++ nbdb_pidfile=/var/run/ovn/ovnnb_db.pid
++ nbdb_sock=/var/run/ovn/ovnnb_db.sock
++ nbdb_ctl=/var/run/ovn/ovnnb_db.ctl
++ sbdb_pidfile=/var/run/ovn/ovnsb_db.pid
++ sbdb_sock=/var/run/ovn/ovnsb_db.sock
++ sbdb_ctl=/var/run/ovn/ovnsb_db.ctl
+ type ovndb-readiness-probe
# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; ovndb-readiness-probe "nb""
+ . /ovnkube-lib/ovnkube-lib.sh
++ set -x
++ K8S_NODE=okd-bf46r-master-0
++ [[ -n okd-bf46r-master-0 ]]
++ [[ -f /env/okd-bf46r-master-0 ]]
++ northd_pidfile=/var/run/ovn/ovn-northd.pid
++ controller_pidfile=/var/run/ovn/ovn-controller.pid
++ controller_logfile=/var/log/ovn/acl-audit-log.log
++ vswitch_dbsock=/var/run/openvswitch/db.sock
++ nbdb_pidfile=/var/run/ovn/ovnnb_db.pid
++ nbdb_sock=/var/run/ovn/ovnnb_db.sock
++ nbdb_ctl=/var/run/ovn/ovnnb_db.ctl
++ sbdb_pidfile=/var/run/ovn/ovnsb_db.pid
++ sbdb_sock=/var/run/ovn/ovnsb_db.sock
++ sbdb_ctl=/var/run/ovn/ovnsb_db.ctl
+ ovndb-readiness-probe nb
+ local dbname=nb
+ [[ 1 -ne 1 ]]
+ local ctlfile
+ [[ nb = \n\b ]]
+ ctlfile=/var/run/ovn/ovnnb_db.ctl
++ /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 ovsdb-server/sync-status
++ grep 'state: active'
+ status='state: active'
+ [[ -z state: active ]]
# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 ovsdb-server/sync-status "
...
state: active

And, running it with our cluster/status gives:

# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status "
+ . /ovnkube-lib/ovnkube-lib.sh
++ set -x
++ K8S_NODE=okd-bf46r-master-0
++ [[ -n okd-bf46r-master-0 ]]
++ [[ -f /env/okd-bf46r-master-0 ]]
++ northd_pidfile=/var/run/ovn/ovn-northd.pid
++ controller_pidfile=/var/run/ovn/ovn-controller.pid
++ controller_logfile=/var/log/ovn/acl-audit-log.log
++ vswitch_dbsock=/var/run/openvswitch/db.sock
++ nbdb_pidfile=/var/run/ovn/ovnnb_db.pid
++ nbdb_sock=/var/run/ovn/ovnnb_db.sock
++ nbdb_ctl=/var/run/ovn/ovnnb_db.ctl
++ sbdb_pidfile=/var/run/ovn/ovnsb_db.pid
++ sbdb_sock=/var/run/ovn/ovnsb_db.sock
++ sbdb_ctl=/var/run/ovn/ovnsb_db.ctl
+ /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 cluster/status
"cluster/status" is not a valid command (use "list-commands" to see a list of valid commands)
ovn-appctl: /var/run/ovn/ovnnb_db.ctl: server returned an error
command terminated with exit code 2

From the above we can understand that the syntax of the command is correct. The problem is that the DB has no "cluster/status" command being recognized:

# oc exec -n openshift-ovn-kubernetes ovnkube-node-wk95k -c nbdb -- bash -c "set -xeo pipefail ; . /ovnkube-lib/ovnkube-lib.sh || exit 1 ; /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 list-commands "
+ . /ovnkube-lib/ovnkube-lib.sh
++ set -x
++ K8S_NODE=okd-bf46r-master-0
++ [[ -n okd-bf46r-master-0 ]]
++ [[ -f /env/okd-bf46r-master-0 ]]
++ northd_pidfile=/var/run/ovn/ovn-northd.pid
++ controller_pidfile=/var/run/ovn/ovn-controller.pid
++ controller_logfile=/var/log/ovn/acl-audit-log.log
++ vswitch_dbsock=/var/run/openvswitch/db.sock
++ nbdb_pidfile=/var/run/ovn/ovnnb_db.pid
++ nbdb_sock=/var/run/ovn/ovnnb_db.sock
++ nbdb_ctl=/var/run/ovn/ovnnb_db.ctl
++ sbdb_pidfile=/var/run/ovn/ovnsb_db.pid
++ sbdb_sock=/var/run/ovn/ovnsb_db.sock
++ sbdb_ctl=/var/run/ovn/ovnsb_db.ctl
+ /usr/bin/ovn-appctl -t /var/run/ovn/ovnnb_db.ctl --timeout=3 list-commands
The available commands are:
  coverage/read-counter   COUNTER
  coverage/show
  exit
  list-commands
  memory/show
  ovsdb-server/add-db     DB
  ovsdb-server/add-remote REMOTE
  ovsdb-server/compact
  ovsdb-server/connect-active-ovsdb-server
  ovsdb-server/disable-monitor-cond
  ovsdb-server/disconnect-active-ovsdb-server
  ovsdb-server/get-active-ovsdb-server
  ovsdb-server/get-db-storage-status DB
  ovsdb-server/get-sync-exclude-tables
  ovsdb-server/list-dbs
  ovsdb-server/list-remotes
  ovsdb-server/memory-trim-on-compaction on|off
  ovsdb-server/perf-counters-clear
  ovsdb-server/perf-counters-show
  ovsdb-server/reconnect
  ovsdb-server/reload
  ovsdb-server/remove-db  DB
  ovsdb-server/remove-remote REMOTE
  ovsdb-server/set-active-ovsdb-server
  ovsdb-server/set-active-ovsdb-server-probe-interval
  ovsdb-server/set-relay-source-probe-interval
  ovsdb-server/set-sync-exclude-tables
  ovsdb-server/sync-status
  ovsdb-server/tlog-list
  ovsdb-server/tlog-set   DB:TABLE on|off
  set-options             [--format text|json]
  version
  vlog/close
  vlog/disable-rate-limit [module]...
  vlog/enable-rate-limit  [module]...
  vlog/list
  vlog/list-pattern
  vlog/reopen
  vlog/set                {spec | PATTERN:destination:pattern}

But this makes no sense, as this command is used in upstream OVN Kube and was used just not that long ago.

It is also used in many other resources:

• https://cozystack.io/docs/operations/troubleshooting/kube-ovn/
• https://gist.github.com/velp/21f05ed57400b077efd4cefb171e5d27
• https://gist.github.com/odivlad/302d30e333ac2296850827bf4917913a
• OCP doc for 4.13 https://docs.redhat.com/en/documentation/openshift_container_platform/4.13/html/networking/ovn-kubernetes-network-plugin#ovn-kubernetes-architecture-assembly
• Upstream ovs project OVN Cluster Health openvswitch/ovs-issues#154
• RH conversations as of 2024 https://bugzilla.redhat.com/show_bug.cgi?id=1810730 https://bugzilla.redhat.com/show_bug.cgi?id=1953688 , ending with this 2024 commit improving the readability of the cluster/status output openvswitch/ovs@e8451e1 .
• Another RH discussion about cluster/status https://bugzilla.redhat.com/show_bug.cgi?id=1929690 from https://access.redhat.com/errata/RHBA-2021:2083
• OpenStack docs https://docs.openstack.org/charm-guide/zed/admin/networking/ovn/queries.html
• man page for ovsdb-server https://man.archlinux.org/man/ovsdb-server.1 https://www.openvswitch.org/support/dist-docs/ovsdb-server.1.txt

Additionally, the Prometheus metric "ovn_db_cluster_id" was also removed from newer clusters (it was previously used to detect OVN DB partitionings between nodes with a PrometheusAlert using the query sum(ovn_db_cluster_id{db_name="OVN_Northbound"}) by (cluster_id)).

Is there any reason in particular for this change? Is there a way to use the command in modern releases of OKD or is it explicitly disabled? And if yes, is there a specific reason?

Kudos to anyone who can solve this (and if nobody can solve it, I'll just leave it open for reference since I spent some good hours on this and hopefully no one else gets lost in this rabbit hole).