Migrate your ESXI VMs to proxmox ZFS

Lately, I migrated my personal lab from ESXI to proxmox hypervisor.
Many reasons are behind this move :
– using vmware esxi free did not allow me proper HA or replication
– each update was painful, and got some CPU no longer supported warnings
– not free open source etc.
– saw proxmox running at other place and was looking good
– the need to learn something new, and to have HA at my home lab as I migrated my workloads from VPS to home (due to OVH suspending an old offer and not allowing to migrate, but rather requiring full re install, in short delays…. tired of not controlling anything, I wanted to move my stack home, and have some redundancy).

Basically, this operation was from the following :
ESXI 6.7 latest update available in 2020 April (aside of the new major 7) to Proxmox PVE 6.1.8

This assume you have an ESXI and proxmox up and running that can reach each other and that you have NO SNAPSHOT on vmware esxi.
I also assume you have a ZFS volume mounted in your proxmox.

On the proxmox node (pve), from the shell, I install SSHFS :

#install SSHFS to mount esxi volume
apt install sshfs

#creating mount point
mkdir /mnt/ssh

#Mount root directory of esxi on /mnt/ssh of pve node
sshfs root@esxi:/ /mnt/ssh

#Convert flat VMDK drive to raw image (reading on esxi via SSHFS and output on your local proxmox)
#The below command must run from a drive where you have enough space to store the full size VMDK (usually /volume-you-created on your proxmox)
qemu-img convert /mnt/ssh/vmfs/volumes/<yourdatastore>/<yourvmname>/<yourvmname>-flat.vmkd -O raw <yourvmname>.raw

#Create the target VM in proxmox with the same specs as ESXI, and making sure you pick ZFS volume as storage.
#identify target disk of the created VM in /dev/zvol/<volumeName>/<diskName>

#Once you know where is you new VM (usually a number associated with the VM) volume is you can dump the raw image to the new virtual disk
dd bs=1M if=<yourVMname>.raw of=/dev/zvol/<volumeName>/<diskName>

If your machine is loaded, the command above may stale your ZFS and overload the server ( load can be way above amount of availalbe threads, due to default ZFS settings no limiting amount of threads).

A workaround found here actually helped getting around the issue using the option oflag=direct
dd bs=1M oflag=direct if=<yourVMname>.raw of=/dev/zvol/<volumeName>/<diskName>

Hope this helps, when migrating big VMDKs over 1TB, I had ZFS crashes due to this, command above allows to import them properly.

#When done, start the new VM and see if it boots. The main issue I had either on CentOs or Debian, was that network interface name changed. So at first boot, VM has no network.
#Not a big issue, edit the interface name in /etc/network/interface (debian) or /etc/sysconfig/network-scripts/ifcfg-ethx  (CentOs)
#The MAC address changed, unless you forced it while creating the VM.

#As soon as your VM is up, delete the .raw image from your drive to free up space.

This is how I migrated 8 VMs, from ESXI to Proxmox without any issues.

Additional notes following this migration (2020 April 27) :

After migrating my last VM, I had a weird error stating I did not have enough space to replicate, as the system could not take any snapshot.

This is due to a default setting in ZFS, in Proxmox, where by default, a space reservation on the volume is made for snapshots etc, with the same size of the actual disk size.

While fine for most of the case, when you have a VM with 1.66TB drive, it starts to be an issue.

There is more explanation here about ZFS refreservation : http://www.mceith.com/blog/?p=153

Basically, in my case, the refreservation was as big at 1.66TB, not allowing snapshots to be taken, and therefore, not allowing VM replication.

You can see the information for a volume with the following command :

zfs get refquota,reservation,refreservation yourvolume/your-vm-disk

So basically, for my VM that had a refreservation default value of 1.66TB, I actually did set it to 500G, since my drive only had 578G of unassigned space.

This allowed the replication process to go on :

refreservation in proxmox

Hopefully this will help some, facing the disk full issue where disk is not actually full but reserved.


samedi, avril 25th, 2020 proxmox, Technologie

2 Commentaires to Migrate your ESXI VMs to proxmox ZFS

  • tresronours dit :

    If you want to run VMs in VMs, like a virtual proxmox, it is called « nested virtualization » and is doable, check proxmox wiki :

  • liu dit :

    qm importdisk VMID esxi.vmdk local-zfs -format raw

  • Ajouter un commentaire

    Not f'd — you won't find me on Facebook
    avril 2020
    L M M J V S D

    Suivez moi sur twitter - follow me on twitter
    Follow on LinkedIn
    [FSF Associate Member]
    Free Software, Free Society
    Compacter une image virtualbox VDI
    Bon petit tutoriel esxi
    Marche d'appliances vmware
    Installer ESXi sur un disque IDE
    Installer ESXi 3.5 sur un disque USB
    Installer proxmox avec DRBD et migration / réplication à chaud
    Installer OSSEC avec VMware
    Information sur le VDI
    Ouvrir des ports dynamiquement iptables - knockd
    Autre tres bon tuto knockd
    Docs Arp poisoning - Anglais
    Metasploit test de pénétration
    Zone H - sites piratés en temps réel
    Blog invisible things
    Tips protection sécurité wordpress
    Pfsense - distribution firewall opensource - adsl internet failover
    Iproute 2 mini how to - linux advanced routing
    ClearOS - la passerelle sécuritaire lan - wan
    CDN - Accélération de la distribution de données
    drbd iscsi ocfs2 dm multipath tutoriel
    Load balancing LVS
    Load balancing opensource list
    HA-Proxy :
    HAproxy - http load balancer
    Simple tutoriel HAproxy
    HAproxy - debian tutoriel
    Centos - Ip failover
    Configuratoin DM-Multipath Redhat
    VMware Doubletake - continuité
    Quelques liens sur la réplication MySQL : Manuel MySQL, chapitre sur la réplication
    Manuel MySQL, Tutoriel clair sur la mise en place
    Autre tuto sur la mise en place de la réplication MySQL
    Références pour optimisation du serveur MySQL
    Utilisation de EXPLAIN mysql pour optimiser vos bases
    optimiser vos bases - requetes et index
    Un outil de clonage disque en reseau
    Internet NAS 250Go 250 accès VPN
    Server ISCSI avec Ubuntu tuto
    ISCSI centos redhat tutoriel
    Gérer et étendre un LVM
    Créer sa piratebox ! trop cool
    Deaddrops, les clés USB dans les murs, aussi cool !
    Télécharger Xenu
    Comment utiliser Xenu
    optimisation hébergement wordpress
    Super howto wordpress (En)
    Test de charge serveur web - Load impact
    Zeroshell - le mini-routeur wifi tout en un
    Retroshare, votre réseau d'échange crypté!
    Openvpn sur centos redhat
    Intégrer Linux dans active directory
    Routage inter-vlan avec Linux
    Routage avec OSPF
    Network Weathermap
    Boutons twitter
    Analyser les tendances des recherches Google
    Protocole sitemap - robots.txt
    Creer des animations CSS3
    Code php pour interagir avec twitter
    E reputation
    Tuxmachines.org - Actus et tips linux
    Configurer GRUB2 et grub2 ici
    Panoet - en anglais - tips & tricks
    Readylines tips and trick pertinents
    Squid Clamav - proxy antivirus
    Apprendre Unix en 10 minutes
    13 tips sur les expressions régulières
    IE Sous linux IES
    LDAP 2.4 Quickstart guide
    Tutoriel LDAP
    Installation annuaire LDAP
    Serveur Mail Postfix - Dovecot - LDAP - MDS
    Créer un linux personnalisé en ligne - custom linux
    Super site sur linux - en
    Capistrano - déploiement automatisé
    Nagios tutoriel et doc
    Nagios plugin NRPE tuto
    Nagios plugin NRPE autre tuto
    Nagios plugin NRPE officiel
    Zabbix - fonctionnalités
    Zabbix - installation
    Guide MRTGsys - grapher la charge locale
    MRTGsys - ajouter des graphs
    MRTGsys - interpréter les données
    Shinken - Monitoring
    Thruk Monitoring webinterface
    Shinken - Tutoriel
    Shinken - Référence chez Nicolargo
    RemixJobs IT jobs
    USB Multiboot
    Reset mot de passe windows
    Java python et autres tips, intéressant !
    Forum inforeseau
    Open Clipart
    Excellent comic en ligne