«

»

Aug 14 2014

Successfully delete a Nutanix Container

I received two questions on the same topic last week so i figure i’ll put together a blog post.
The subject was “Cannot delete a Nutanix Container for my vSphere environment” !!

A Nutanix Container is the logical unit, created from the Nutanix Storage Pool, where you store your virtual machines (VMs) and or files. A Nutanix Container is the unit presented as a NFS datastore in the vSphere environment.

The standard procedure for deleting a Nutanix Container is:

  • Remove all VMs and files from the Nutanix Container via the vSphere layer.
  • Unmount the Nutanix Container from the ESXi hosts where it is mounted using the Nutanix Prism Interface (UI).
    • Prism -> Storage -> Table -> Mark the Nutanix Container -> Click Update
      Screen Shot 2014-08-01 at 12.48.03
    • Select “Unmount on all hosts” -> Click Save -> Click Yes
      Screen Shot 2014-08-01 at 12.50.46
      Screen Shot 2014-08-01 at 12.53.35
  • Delete the Nutanix Container via the Nutanix Prism Interface.
    • Prism -> Storage -> Table -> Mark the Nutanix Container -> Click Delete -> Click Yes
      Screen Shot 2014-08-01 at 12.58.48
      Screen Shot 2014-08-01 at 12.58.13

So what can possible go wrong:) ??

Well, the last step failed with the error “Container ctr75-compression2 contains VDisk(s) NFS:111 not marked for removal”

Screen Shot 2014-08-01 at 12.13.35

A Nutanix VDisk is ANY file on the Nutanix Distributed File System (NDFS) with a size larger than 512 KB meaning not only VM related files.

So now we have to:

  • Mount the Nutanix Container to the ESXi hosts again.
    • Prism -> Storage -> Table -> Mark the Nutanix Container -> Click Update
      Screen Shot 2014-08-01 at 12.48.03
    • Select “Mount on all hosts” -> Click Save
      Screen Shot 2014-08-01 at 13.01.48
  • Log in to the vSphere client and verify if there are any files on the datastore. As you can see in the below print screen the NFS datastore doesn’t contain any VMs but it still contains VDisks since the vSphere-HA directory exists on the NFS Datastore.
    Screen Shot 2014-07-29 at 11.28.17
  • Remove the vSphere HA related files.
    Screen Shot 2014-07-29 at 11.28.55

When done, follow the unmount and delete steps outlined earlier in the blog post and you can successfully remove the Nutanix Container.

The main take away from this blog post is to keep in mind that you need to remove all files from the NFS Datastore/Nutanix Container including VM files, ISO files and other potential files e.g. vSphere HA once before you can successfully delete the Nutanix Container.

8 comments

Skip to comment form

  1. sumatra surf camp

    I am not really superb with English but I get hold this very easygoing to translate.

  2. Nikolay

    In my case, there was a file named protectedlist.
    I was able to delete it only from ESXi cli using root account.

    rm /vmfs/volumes/container_to_remove/.vSphere-HA/FDM-*/protedtedlist

  3. Netman

    Hello,

    Sorry to necro-post, but I haven’t found very much online regarding this problem.

    We are a good sized Nutanix/ESXi 6.0 U2 shop, and our problem is that, even after ensuring via Datastore Browser and cli that the datastore is empty, and disabling SIOC, the datastores will not even unmount.

    The error we receive is (from vSphere client and web client)
    Sysinfo error on operation returned status : Busy

    According to google and VMWare Support, this means there are open locks or handles against this datastore. All files on this datastore have been removed, and working with VMWare support they were unable to see any locks or handles on it. Their recommendation was to reboot all of our hosts (a reboot DOES seem to fix this issue), but that’s difficult to do in a prod environment.

    Is this by chance anything you’ve seen?

  4. magander3

    I have seen this when vSphere HA has been enabled since it almost immediately creates new files and a folder on the datastore. Are you able to unmount the datastore from any host or getting the error immediately?

    thanks

    //Magnus

  5. Netman

    Magnus,

    Thanks a million for the reply. You’re correct; I have noticed the .vSphereHA folder/files get created on the otherwise empty datastore.

    In an attempt to alleviate this, I have removed that datastore from HA heartbeating, deleted the files again, and I haven’t seen the files come back. It’s been months now and they’re still not there, so fortunately I don’t *think* that’s contributing to the issue.

    However, it would be quite interesting to disable HA in the cluster, ensure the datastores are still empty, and try another unmount.

    Interestingly, (I have this issue with 5 datastores) I have been able to unmount some of my datastores from some of my hosts, and not others. I’m curious what the difference may be, but yes that has happened.

    Thanks.

  6. magander3

    Unfortunately just removing datastore heartbeat didn’t help I actually had to disable HA entirely. Can you do ls -la via cli just to make sure there are no hidden files on the datastore?

    Thanks

  7. Netman

    Just tried disabling HA in my test cluster to unmount a stuck datastore. Still says “Busy” unfortunately.

    ls-lah shows nothing in the directory. Prism says the datastore is using 1.5GB, so I don’t know what that’s about, lol.

    In Prod, the datastores i’m trying to unmount that are stuck all have 0GB used in Prism, and an ls -lah shows nothing in them.

  8. magander3

    Hi,
    ok if HA is disabled the I would suggest you to list the files on the container via the CVM instead. try

    nfs_ls -lah /container and see if you can find any files

    Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">