Dell VxRail: How to Run the VxVerify Tool

Introduction to VxRail VxVerify:

• VxVerify is designed to proactively detect issues which could cause complications or failures during VxRail upgrades.
• The VxVerify tool unpacks a Python program called the 'minion', which is sent to run on each ESXi node. This 'minion' reviews a list of host checks, such as checking iDRAC information and reviewing the vSAN health-check for each node.
• Given the frequent updates to VxVerify tool itself, the life cycle of the 'minion' is designed to last 21 days. Always download the latest version (see additional information section for links to download) before running VxVerify.

Install VxVerify:

• To install VxVerify, download the latest vxverify_x-x-xxx.zip file (see additional information section below for links to download) and copy it to VxRail Manager using SSH.
• On VxRail Manager, extract the vxverify_x-x-xxx.zip file and run VxVerify.
  • The VxVerify pyc filename includes the file version, whereas the examples below are simplified to vxverify.pyc.
  • If the pyc is renamed, VxVerify can still be run from Python, as listed below, but the menu script method of starting VxVerify (vxverify.sh) would not work.
• VxVerify runs by default from the following folder, which would be the best place to copy the file to (create it if necessary): 

/tmp/vxv

• VxVerify and VxRail use the same IP ports. VxVerify results are generated using SSH to the nodes and API from VxRM. Other ports are used to check specific services, such as port 9090 to the nodes.
  • A full list of ports can be found in the VxRail Customer Firewall Configuration  

Run VxVerify (using vxverify.sh):

• The mystic user does not have sufficient permissions to VxVerify, so either the sudo must be used or use su to log in as root.
• It is recommended to use the menu system ((vxverify.sh)), which prompts the user for what type of health check is required based on the inputs provided.  

sudo ./vxverify.sh

VxVerify Best Practice: 

• VxVerify.sh prompts the user to provide the vCenter root and administrator SSO users and passwords to run the tests.
  • If using a VxRail-managed vCenter, VxRail Engineering recommends providing these credentials to allow all tests to run fully.
  • If using a customer-managed vCenter, these tests may report settings or conditions on the vCenter that do not apply to the VxRail
• Providing VxVerify.sh root credentials, allows additional VxRM tests to run but logging in as a standard user, such as 'mystic', is sufficient for all the host tests and most of the VxRM tests.

Troubleshooting and Log Collection:

| vCenter_root           | Warning  21527 | warning: VCSA tests not run: no root password supplied                           .|

| vCenter_admin          | Warning  21527 | warning: VCSA admin tests not run: no SSO credentials supplied                   .|

Watch the following video "Dell VxRail: How to run the VxRail Verify Tool."

Run VxVerify (using Python):

python ./vxverify.pyc -r root -u administrator@vsphere.local -g 7.0.401

• sudo may have to be added in front of the command above, if not running it from root (the same applies to VxVerify.sh).
• The Python pyc file is compiled for a specific Python release family, such as Python 3.6 (vxverify3). Running the wrong VxVerify pyc file for the VxRail release results in Python responding with 'magic number' errors.
• The example above prompts for the passwords that match the specified users.
• If the root user is not included, the VCSA-root tests do not run, but ESXi and VxRM tests do run with or without the root user.
• If the SSO user is not included, the VCSA-admin tests do not run and some VxRM tests would have reduced functionality.
• Without the additional credentials listed above, VxVerify still runs, but does not include tests that require those credentials and so some issues are not tested for, which could potentially cause an upgrade to fail.

Alternative Python Versions

Some VxRail releases contain more than one version of Python, which means that it may be possible to run more than one version of VxVerify on that release. However, VxVerify requires several additional Python libraries to be present in the various 'site-packages' folders, which may not be present for some Python levels. 

The default Python level (which runs from the command 'python'), may not have the necessary packages and so the correct Python level may have to be specified in the Python command. For example,

python3.11 ./vxverify4.pyc -r root -u administrator@vsphere.local -g 8.0.210

• sudo may have to be added in front of the command above, if not running it from root (the same applies to VxVerify.sh).

VxVerify Release Train:

• VxVerify 2.xx.xxx is for VxRail 4.5, 4.7, and 7.0.000 (which uses Python 2.7)
• VxVerify 3.xx.xxx is for VxRail 7.0.010 and later (which use Python 3.6)
• VxVerify 4.xx.xxx is for VxRail 8.0.210 and later (which use Python 3.11)

The VxVerify .zip bundles numbered x.40 and later, contain packages for all supported VxRail releases (as shown above). Running VxVerify using VxVerify.sh automatically selects the correct pyc file, to match the Python level running on VxRM.

Download the latest VxVerify:

Related Resources

Here are some recommended resources related to this topic that might be of interest:

• How to Check the ESXi Maintenance Mode Status from the vSAN Perspective
• Dell VxRail Troubleshooting Guide for a VMware ESXi hosts in "Not Responding" State