UPDATE: It's been identified by Oracle as to: Bug 15351266 : SUNBT6472202 'zfs rollback' and 'zfs rename' require that clones be unmounted Fix is due for Solaris 11.3 though. ============================================================================================================= Today, I tried to rename a ZFS dataset who has cloned childs and found that while doing this on the parent, the child's NFS Share went down for a while, causing all of the VMs relying on that parent dataset to hardly crash. Here is how to reproduce: First, create the VM template: # zfs create tank/aavm/vms/test-template # zfs create tank/aavm/vms/test-template/os # dd if=/dev/uramdom of=/tank/aavm/vms/test-template/os/test-template-os.raw # I broke this one after ~300M # chown -R vmcloud:vmcloud /tank/aavm/vms/test-template/os # zfs snapshot -r tank/aavm/vms/test-template@01 Then, create a VM depending on that OS template: # zfs create tank/aavm/vms/test-vm01 # fs clone tank/aavm/vms/test-template/os@01 tank/aavm/vms/test-vm01/os Share it over NFS: # zfs set share=path=/tank/aavm/vms/test-vm01,prot=nfs,sec=sys,root=@10.2.0.0/24,rw=@10.2.0.0/24 tank/aavm/vms/test-vm01 # zfs set share=path=/tank/aavm/vms/test-vm01/os,prot=nfs,sec=sys,root=@10.2.0.0/24,rw=@10.2.0.0/24 tank/aavm/vms/test-vm01/os Then, on the client server, mount this filesystem: # mount -F nfs -o vers=4,hard tank-server:/tank/aavm/vms/test-vm01/os /mnt/test Finally, use (https://espix.net/~wildcat//tmp/genfileio.c) to generate IOs on this file: # gcc -o /var/tmp/genfileio genfileio.c # /var/tmp/genfileio -f /mnt/test/test-template-os.raw [-] Going to use /mnt/test/test-template-os.raw.. [-] File opened succesfully, going to read/write randomly forever now... [-] File size: 272917504 bytes [-READ] 70 MB in 1 seconds [WRITE] 70 MB in 1 seconds [-READ] 94 MB in 1 seconds [WRITE] 94 MB in 1 seconds [-READ] 97 MB in 1 seconds [WRITE] 97 MB in 1 seconds [-READ] 99 MB in 1 seconds [WRITE] 99 MB in 1 seconds [-READ] 95 MB in 1 seconds [WRITE] 95 MB in 1 seconds [-READ] 96 MB in 1 seconds [WRITE] 96 MB in 1 seconds [-READ] 95 MB in 1 seconds [WRITE] 95 MB in 1 seconds [-READ] 94 MB in 1 seconds [WRITE] 94 MB in 1 seconds [-READ] 99 MB in 1 seconds [WRITE] 99 MB in 1 seconds [-READ] 93 MB in 1 seconds [WRITE] 93 MB in 1 seconds --- Meanwhile, launch the "zfs rename tank/aavm/vms/test-template tank/vms/tpl/test-template" command --- NOTE: This will get the OS template renamed without being supposed to touch any of child's cloned dataset, so I wasn't expecting this to break NFS shares of the childs. The program execution will be failling: [-READ] 94 MB in 1 seconds [WRITE] 94 MB in 1 seconds [-READ] 99 MB in 1 seconds [WRITE] 99 MB in 1 seconds [-READ] 93 MB in 1 seconds [WRITE] 93 MB in 1 seconds [-READ] 98 MB in 1 seconds [WRITE] 98 MB in 1 seconds [!] Cannot read 2295 bytes in file NOTE: the errno is ESTALE (stale NFS file handle). ----- Local IOs ------ I have also been testing ZFS rename while having IOs on a file in a cloned dataset and zfs rename just break with the "cannot umount" error on the cloned dataset. This made me think that another bug here is that the zfs rename should break whatever is causing the dataset it tries to umount to be used would be be NFS or local IOs. Here it only breaks with the later. ----- Workaround ------ To be able to still rename the parent dataset without having a downtime on the cloned childs, I've been using the following process, it worked great so far: zfs create tank/aavm/vms/test-template zfs create tank/aavm/vms/test-template/os dd if=/dev/urandom of=/tank/aavm/vms/test-template/os/test-template-os.raw bs=128k count=$((1024*100/128)) chown -R vmcloud:vmcloud /tank/aavm/vms/test-template/os zfs snapshot -r tank/aavm/vms/test-template@01 zfs create tank/aavm/vms/test-vm01 zfs clone tank/aavm/vms/test-template/os@01 tank/aavm/vms/test-vm01/os zfs create tank/aavm/vms/test-vm02 zfs clone tank/aavm/vms/test-template/os@01 tank/aavm/vms/test-vm02/os zfs promote tank/aavm/vms/test-vm01/os zfs promote tank/aavm/vms/test-vm02/os zfs rename tank/aavm/vms/test-template tank/vms/tpl/test-template zfs promote tank/vms/tpl/test-template/os zfs get origin tank/aavm/vms/test-vm01/os zfs get origin tank/aavm/vms/test-vm02/os