[EN] How to Safely Shut Down an HPE SimpliVity Cluster
Posted inHPE Simplivity

[EN] How to Safely Shut Down an HPE SimpliVity Cluster

HPE SimpliVity combines virtualization, data deduplication, compression, backup, and data protection capabilities within a single hyperconverged infrastructure platform. For this reason, shutting down a SimpliVity cluster must be handled differently from shutting down a standard VMware ESXi cluster.

Directly powering off the ESXi hosts in a SimpliVity environment may cause the OmniStack Virtual Controllers to stop unexpectedly. This can create risks related to data accessibility, Storage High Availability, and overall cluster integrity.

Before planned maintenance, a power outage, data center work, or hardware intervention, the cluster should be shut down according to the following basic sequence:

Virtual machines → SimpliVity OVCs → ESXi hosts

This article explains in detail how to safely shut down an HPE SimpliVity cluster, the key points that must be considered, and the recommended startup sequence after maintenance.

1. Checks to Perform Before the Shutdown

Before starting the shutdown process, verify the overall health of the environment. Shutting down a cluster that is already unhealthy, has connectivity issues, or is still synchronizing data is not recommended.

The following points should be checked:

  • All SimpliVity nodes must be reachable.
  • OmniStack Virtual Controller virtual machines must be running.
  • All SimpliVity Federation members must appear connected.
  • SimpliVity datastores must be accessible.
  • No backup or restore task should be running.
  • VM migration and Storage vMotion tasks must be completed.
  • Virtual machines should be compliant with Storage HA requirements.
  • Access to vCenter Server, ESXi Host Client, and iLO should be verified.

Connect to one of the OVCs through SSH and check the Federation status:

svt-federation-show

To identify Storage HA or policy violations affecting virtual machines, use:

svt-vm-show --violations

If the command output shows a disconnected node, a Storage HA violation, or an unexpected error, resolve the issue before proceeding with the shutdown.

In particular, OVCs should not be shut down while backup, restore, cloning, migration, or data synchronization tasks are still running. Wait until all tasks have completed and the environment has returned to a stable state.

2. Verify Management Access in Advance

If vCenter Server is running as a virtual machine inside the SimpliVity cluster, access to vCenter will be lost during the shutdown procedure.

For this reason, direct access to every ESXi host should be tested before starting:

https://ESXI-HOST-IP/ui

Access to the physical servers through HPE iLO should also be verified.

iLO access is particularly important in the following situations:

  • The ESXi shutdown process does not complete.
  • A host becomes unresponsive.
  • Servers must be powered on remotely.
  • The physical power state must be verified.
  • Data center personnel must be given remote instructions.

The ESXi root credentials, iLO credentials, and management IP addresses should be available before the shutdown begins.

3. Complete All Running Tasks

Before shutting down the cluster, allow all active SimpliVity data operations to complete.

Tasks that should be reviewed include:

  • SimpliVity backup operations
  • Backup restore operations
  • Virtual machine cloning
  • Datastore migration
  • Storage vMotion
  • Replication tasks
  • Virtual machine migration
  • Policy changes
  • Long-running snapshot operations

For a planned shutdown, backup schedules may be temporarily paused to prevent new backup jobs from starting during the maintenance window.

However, active backup jobs should not be canceled without first checking their current status. The decision should be made in accordance with the organization’s data protection procedures.

4. Shut Down Virtual Machines in the Correct Order

Once cluster health has been verified, application virtual machines should be shut down in a controlled manner.

Virtual machines should not be turned off using the hard Power Off option unless absolutely necessary. Whenever possible, perform a normal operating system shutdown.

A general shutdown order may be:

  1. User-facing applications
  2. Web and application servers
  3. Integration services
  4. Database servers
  5. File servers and supporting services
  6. Monitoring and management servers
  7. Domain Controllers and DNS servers
  8. vCenter Server
  9. Arbiter components, when required

Application dependencies must be considered. For example, if an application server depends on a database server, the application server should be shut down first, followed by the database server.

Domain Controllers and DNS Servers

If multiple Domain Controllers are available, keeping at least one Domain Controller and DNS server running until management operations are completed can be beneficial.

The following systems may depend on DNS or Active Directory authentication:

  • vCenter Server
  • ESXi host management
  • Backup software
  • Monitoring platforms
  • Administrator logins

Domain Controllers and DNS servers are generally shut down after application workloads but before the final management components.

5. Important Considerations When vCenter Runs Inside the Cluster

If the vCenter Server Appliance is hosted on the SimpliVity cluster being shut down, the ESXi host running vCenter should be shut down last.

For example, in a three-node environment where vCenter Server is running on the third ESXi host, the shutdown order may be:

ESXi-01 → ESXi-02 → ESXi-03

In this scenario, ESXi-03 remains available as the final management host.

The first two hosts can be managed through vCenter. After vCenter Server is shut down, the final host must be managed directly through the ESXi Host Client.

Before shutting down vCenter, complete the following checks:

  • Confirm the IP address of the final ESXi host.
  • Test direct access to the ESXi Host Client.
  • Verify the ESXi root credentials.
  • Identify all virtual machines running on the final host.
  • Confirm which OVC is running on the final host.
  • Test iLO access.

After vCenter is powered off, DRS, centralized cluster management, and vCenter-based SimpliVity menus will no longer be available.

6. What Is the OmniStack Virtual Controller?

Each SimpliVity node contains an OmniStack Virtual Controller, commonly referred to as an OVC.

The OVC performs critical tasks such as:

  • Providing SimpliVity datastore services
  • Data deduplication
  • Data compression
  • Backup processing
  • Replication
  • Metadata management
  • SimpliVity Federation communication
  • Storage HA operations

For this reason, the OVC should not be treated as an ordinary virtual machine.

The OVC should not be shut down by selecting the standard VMware Power Off option. The supported HPE SimpliVity shutdown procedure should be used instead.

7. Safely Shutting Down the OVC

After the application virtual machines on the relevant ESXi host have been shut down or migrated, the OVC can be shut down safely.

In the vSphere Client, select the relevant ESXi host and open the SimpliVity actions menu:

ESXi Host
  → Actions
  → All HPE SimpliVity Actions
  → Shut Down Virtual Controller

The exact menu names may differ slightly depending on the installed SimpliVity and vCenter versions.

When the Shut Down Virtual Controller operation is started, the OVC stops datastore and Federation services in a controlled manner. The ESXi host must not be shut down before this process has completed.

During the OVC shutdown, verify the following:

  • Monitor the task status.
  • Confirm that the OVC has completely shut down.
  • Verify that the OVC appears as Powered Off in the ESXi host inventory.
  • Do not shut down the ESXi host if the OVC shutdown fails.
  • Record any error messages and share them with HPE Support if required.

Safe Shutdown Through SSH

In some versions or maintenance scenarios, the OVC can be shut down through SSH using:

svt-shutdown-safe

The shutdown status can be checked with:

svt-shutdown-status

Command availability may vary depending on the SimpliVity OmniStack software version. If the command is unavailable or returns an unexpected error, do not force the shutdown.

8. Put the ESXi Host into Maintenance Mode

After the OVC has fully shut down, the relevant ESXi host can be placed into Maintenance Mode.

From the vSphere Client:

ESXi Host
  → Actions
  → Maintenance Mode
  → Enter Maintenance Mode

Ensure that no other virtual machines are still running on the host.

If virtual machines are running on other hosts in the cluster, DRS may attempt to migrate workloads. In a full planned cluster shutdown, however, most virtual machines should already have been shut down.

Some SimpliVity datastore warnings may appear because the OVC is no longer running. The shutdown sequence must therefore be followed carefully.

Once the host has successfully entered Maintenance Mode, proceed with the ESXi shutdown.

9. Shut Down the ESXi Host

After the host enters Maintenance Mode, shut down ESXi gracefully.

From the vSphere Client:

ESXi Host
  → Actions
  → Power
  → Shut Down

A maintenance reason can be entered in the description field:

Planned SimpliVity cluster shutdown for data center maintenance

After issuing the shutdown command, use iLO to verify the physical server’s power state.

Once the server is completely powered off, continue with the next SimpliVity node.

All nodes should not be shut down at the same time. Complete the following sequence for each node:

Safe OVC shutdown
→ Enter Maintenance Mode
→ ESXi shutdown
→ Verify physical power state

Completing the process on one node before moving to the next makes the overall procedure easier to control and troubleshoot.

10. Example Shutdown Sequence for a Three-Node Cluster

The following example assumes a three-node SimpliVity cluster and that vCenter Server is running on the third node.

First Node

  1. Shut down or migrate the application VMs running on ESXi-01.
  2. Safely shut down the OVC on ESXi-01.
  3. Confirm that the OVC is completely powered off.
  4. Put ESXi-01 into Maintenance Mode.
  5. Shut down ESXi-01.
  6. Verify its power state through iLO.

Second Node

  1. Shut down or migrate the application VMs running on ESXi-02.
  2. Safely shut down the OVC on ESXi-02.
  3. Confirm that the OVC is completely powered off.
  4. Put ESXi-02 into Maintenance Mode.
  5. Shut down ESXi-02.
  6. Verify its power state through iLO.

Third and Final Node

  1. Shut down the remaining application virtual machines.
  2. Review the shutdown timing for Domain Controller and DNS services.
  3. Start the safe shutdown procedure for the final OVC.
  4. Shut down the vCenter Server Appliance gracefully.
  5. Shut down the SimpliVity Arbiter system if required.
  6. Connect directly to the final ESXi host through the ESXi Host Client.
  7. Confirm that no virtual machines remain powered on.
  8. Put the ESXi host into Maintenance Mode.
  9. Shut down the ESXi host.
  10. Verify the physical power state through iLO.

At this point, the SimpliVity cluster is completely shut down.

11. When Should the SimpliVity Arbiter Be Shut Down?

The SimpliVity Arbiter supports decision-making and quorum-related operations, particularly in two-node environments or specific Federation architectures.

The Arbiter usually runs on a Windows server outside the SimpliVity cluster. Therefore, shutting down the Arbiter is not always required.

If the Arbiter server provides other services, the entire Windows server should not be shut down only because the SimpliVity cluster is being powered off.

However, if the entire data center is being shut down or the Arbiter is connected to the same power infrastructure, the following sequence can be used:

  1. Shut down all SimpliVity nodes in a controlled manner.
  2. Confirm that the cluster shutdown has completed.
  3. Shut down the Arbiter service or Windows server last.

When powering the environment back on, it is preferable for the Arbiter to be accessible before the SimpliVity services fully start.

12. Why Force Shutdown Should Not Be Used

In some cases, the safe OVC shutdown may fail. Administrators may then encounter force shutdown commands such as:

svt-shutdown-force

A force shutdown should only be used under the direction of HPE Support.

Forcefully shutting down an OVC may cause:

  • Data integrity problems
  • Datastore accessibility issues
  • Federation communication failures
  • Metadata inconsistencies
  • Storage HA violations
  • OVC services failing to start
  • Virtual machines losing datastore access
  • Recovery procedures requiring HPE Support assistance

If the safe shutdown procedure fails, forcing the ESXi host to power off should not be the first response.

Collect the following information first:

  • OVC shutdown task error messages
  • OVC system status
  • Federation status output
  • ESXi host logs
  • vCenter task and event logs
  • Time when the issue occurred
  • Affected node and datastore information

Opening an HPE Support case with this information is the safest approach.

13. Additional Precautions Before a Power Outage

For a planned power outage or data center electrical maintenance, shutting down only the SimpliVity cluster may not be sufficient.

The following infrastructure components should also be included in the shutdown plan:

  • SAN switches
  • Ethernet switches
  • Firewalls
  • Load balancers
  • Backup servers
  • Domain Controllers
  • DNS servers
  • vCenter Server
  • HPE SimpliVity Arbiter
  • Monitoring servers
  • UPS systems
  • External storage or archive platforms

Network devices should not be shut down before the ESXi hosts and OVCs are completely powered off. Otherwise, cluster communication and management access may be lost.

A general data center shutdown sequence may be:

Application services
→ Database services
→ Virtual machines
→ SimpliVity OVCs
→ ESXi hosts
→ Management servers
→ Network devices
→ Storage systems
→ UPS and power infrastructure

This order may vary depending on the architecture of the environment.

14. How to Power On the SimpliVity Cluster Again

The startup process generally follows the reverse order of the shutdown procedure.

The recommended high-level startup sequence is:

Network infrastructure
→ Arbiter
→ ESXi hosts
→ OVCs
→ SimpliVity datastore and Federation checks
→ vCenter Server
→ Domain Controllers and DNS
→ Application virtual machines

14.1 Start the Network Infrastructure

First, verify that the following systems are operational:

  • Core switches
  • Management switches
  • SimpliVity Federation network
  • Storage or 10/25 GbE connections
  • Firewalls
  • DNS and NTP connectivity

ESXi hosts should not be started until the network infrastructure is fully ready.

14.2 Start the Arbiter

If an Arbiter is used, start the Windows server and verify that the Arbiter service is accessible.

Even if the service appears to be running, the SimpliVity nodes may be unable to communicate with it if DNS or network connectivity is unavailable.

14.3 Start the ESXi Hosts

Power on the ESXi hosts in a controlled manner through iLO.

Instead of starting all hosts simultaneously, it may be preferable to wait for the first node to stabilize before starting the remaining nodes.

14.4 Wait for the OVCs to Start

When an ESXi host starts, its OVC virtual machine should start automatically.

During OVC startup, verify:

  • Is the OVC powered on?
  • Is VMware Tools running?
  • Is the OVC network reachable?
  • Has Federation communication been established?
  • Are the SimpliVity datastores accessible?
  • Is datastore status healthy?

Application virtual machines should not be started immediately after the OVC powers on. Wait until the Federation and datastore services are fully available.

14.5 Check Federation Status

After connecting to an OVC through SSH, use:

svt-federation-show

Confirm that all nodes are connected and healthy.

Storage HA violations can also be checked with:

svt-vm-show --violations

If no problems are detected, management and application virtual machines can be started.

14.6 Start Management Services

A general startup sequence may be:

  1. Domain Controllers
  2. DNS
  3. NTP
  4. vCenter Server
  5. Backup servers
  6. Monitoring servers
  7. Database servers
  8. Application servers
  9. Web servers
  10. User-facing services

After the applications have started, test service dependencies and user access.

15. Post-Shutdown Checklist

After the cluster has been completely shut down, verify the following:

  • Are all application virtual machines powered off?
  • Were all OVCs shut down safely?
  • Are all ESXi hosts powered off?
  • Do the servers appear as powered off in iLO?
  • Is vCenter Server shut down?
  • Was the Arbiter shut down if required?
  • Was the correct network device shutdown order followed?
  • Were any backup or restore tasks interrupted?
  • Did any errors occur during the shutdown?
  • Was the procedure documented?

A separate checklist should also be used when the environment is powered back on.

16. Summary Shutdown Procedure

The recommended general shutdown workflow for an HPE SimpliVity cluster is:

1. Check Federation and cluster health.
2. Wait for backup, restore, and migration tasks to complete.
3. Verify ESXi Host Client and iLO access.
4. Shut down application virtual machines according to their dependencies.
5. Leave the ESXi host running vCenter until last.
6. Safely shut down the OVC on each node.
7. Confirm that the OVC has completely powered off.
8. Put the related ESXi host into Maintenance Mode.
9. Shut down the ESXi host gracefully.
10. Repeat the same procedure for the remaining nodes.
11. Shut down vCenter and the remaining management VMs on the final node.
12. Shut down the Arbiter and other infrastructure components if required.

Shutting down an HPE SimpliVity cluster involves more than simply powering off virtual machines and ESXi hosts. Because the OmniStack Virtual Controllers are central to SimpliVity datastore and data services, they must be stopped in a controlled manner.

The fundamental principle of a safe shutdown is:

First shut down the workloads, then the data services, and finally the physical infrastructure.

Following the correct order protects data integrity, simplifies the recovery of Federation services, and reduces the risk of datastore or OVC issues after planned maintenance.

If an OVC does not shut down safely, the Federation appears unhealthy, or a datastore accessibility issue is detected, do not use a force shutdown. Stop the procedure and contact HPE Support for assistance.

Tags:

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *