---
# source: blog-content: posts/en/tips-and-tricks-for-vsan-troubleshooting.md
# route:  /en/insights/tips-and-tricks-for-vsan-troubleshooting/
title: Tips and Tricks for vSAN troubleshooting
date: 2017-07-27
author: dario-doerflinger
locale: en
summary: Today I attended a Web-Session from VMware with the topic vSAN troubleshooting tips and tricks and I wanted to share those tips and tricks with you.
capabilities: [storage]
vendors: [vmware]
series: lessons-learned
legacySlug: tips-and-tricks-for-vsan-troubleshooting
migrated: 2026-08-24
draft: false
---

Today I attended a Web-Session from VMware with the topic vSAN troubleshooting tips and tricks and I wanted to share those tips and tricks with you.

## Common sense

Ensure that your hardware is on the VMware HCL  
– Disks / Controllers / Firmare / Drivers

Ensure that you have up-to-date backups (and test the restore process fully)

## vSAN Overview

Cluster: 2-64 physical hosts  
Host: 1-5 disk groups  
Disk Group: 1 flash for cache, 1-7 flash or HDD devices for capacity

vSAN Objects:  
– VM Home, VM Swap, VMDK, Delta Disk, Memory Delta

Storage Policies:  
– Applied at per VM level or VMDK level  
– Define protection level & performance

Each object is made up of one or more components (depending on your storage policy)

![Screen Shot 2017-07-27 at 13.12.56](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/01.webp)

C1 & C2 = Components, W = Witness

Component states:

-   Active – component accessible
-   Absent – Inaccessible, but no explicit error codes sensed
    -   host outage or maint mode with "ensure accessibility"
    -   rebuild begins after 60 minute timeout
-   Degraded – Inaccessible with error codes sensed
    -   device failure
    -   rebuild begins immediately
-   Active – stale
    -   In queue of objects to rebuild

## vSAN Tools

-   vRealize Ops / Log Insight
-   ESXCLI
-   RVC
-   Health Check
-   [vSAN Observer](https://kb.vmware.com/kb/2064240) (for performance issues)

### ESXCLI

**esxcli vsan** – gives the available namespaces  
*New in 6.6: debug & health*

**esxcli vsan helath cluster list** – Gives you an overview with the traffic light system (green, yellow, red) where you see all tests  
**esxcli vsan health cluster get -t "vSAN Disk Balance"**  – get results of a test from above command  
**esxcli vsan health cluster get -t "vSAN object health"** – vSAN object health could mean serious problems if status is red. (Get UUID of object to track the problem with those objects)

**esxcli vsan debug** – gives available namespaces  
**esxcli vsan debug resync summary** – give information of current resync process  
**esxcli vsan debug object health summary get** – gives you an overview of your health  
**esxcli vsan debug object list |more** – Gives back all objects back including component states  
**esxcli vsan debug disk list** – gives you information about your disk and if they can keep up  
**esxcli vsan debug controller list** – gives information about your disk controllers (HCL information, Queue depth)

### RVC (Ruby vSphere Console)

The RVC is preinstalled on all vCenter Server variants.

vsan.check\_state 0  
![Screen Shot 2017-07-27 at 13.28.44](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/02.webp)

vsan.disks\_stats 0  
![Screen Shot 2017-07-27 at 13.29.08.png](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/03.webp)

vsan.cluster\_info 0  
![Screen Shot 2017-07-27 at 13.29.53](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/04.webp)

### Health UI

![Screen Shot 2017-07-27 at 13.30.33](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/05.webp)

vSAN Disk balance  
![Screen Shot 2017-07-27 at 13.32.00](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/06.webp)

## vSAN Health

**python /usr/lib/vmware-vpx/vsan-health/vsan-vc-health-status.py > /tmp/vsan\_status.txt**

![Screen Shot 2017-07-27 at 13.34.56](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/07.webp)

Running this on a individual node:  
Location: /usr/lib/vmware/vsan/bin/vsan-health-status.pyc

> python /usr/lib/vmware/vsan/bin/vsan-health-status.pyc

[KB 2107705](http://kb.vmware.com/kb/2107705) has more information on that.

When to use those: Health Service on vCenter not available

## Use Cases

Cluster with 6 hosts. Three nodes were added. No capacity added after adding the hosts.

> esxcli vsan storage list

![Screen Shot 2017-07-27 at 13.40.13](/blog-assets/tips-and-tricks-for-vsan-troubleshooting/08.webp)**In CMMDS: false – take note of the naa… ID**

-   Check vobd.log (search for naa… ID)
-   Check boot.gz (use zcat; search for naa… ID)
    -   Disk naa… ID detected to be a snapshot

Resolution:  
– Disk cannot be added because it already has a filesystem on it  
– Disk was given UUID, it was in a cluster and used at some point  
– Verified to delete data  
– Used partedutil to kill partitions  
– Delete disk groups and recreate disk groups

### Useful Logs:

> vobd.log

Search for "problem" and "permanent"