Blog Inforeseau.com – GNU Linux, technologie et autres geekeries

Restrict access to proxmox web admin interface

When you install proxmox, you have by default access to your admin interface, usually using the following URL :

https://IP-OF-YOUR-INSTANCE:8006

This is totally fine when using an internal instance, but quite another story if you rent an online server with proxmox on it. We don’t want to let this access wide open.

While proxmox comes with a nice firewall, it doesn’t allow you to restrict access to port 8006 and 22 (obviously not to lock you out), because it won’t do this on your « local » network, but when you are on a publicly connected machine, the « local » network is internet.

Therefore, a way to restrict this access is by creating the following file /etc/default/pveproxy with the following content :

ALLOW_FROM="127.0.0.1,172.20.0.1"
DENY_FROM="all"
POLICY="allow"

Be sure to use lower case. Then restart the pveproxy service :

service pveproxy restart

And from now on, only the listed IPs have access to the admin interface.

Reference : https://vdillenschneider.fr/architecture-de-services-avec-proxmox-sur-un-serveur-kimsufi

Migrate your ESXI VMs to proxmox ZFS

Lately, I migrated my personal lab from ESXI to proxmox hypervisor.
Many reasons are behind this move :
– using vmware esxi free did not allow me proper HA or replication
– each update was painful, and got some CPU no longer supported warnings
– not free open source etc.
– saw proxmox running at other place and was looking good
– the need to learn something new, and to have HA at my home lab as I migrated my workloads from VPS to home (due to OVH suspending an old offer and not allowing to migrate, but rather requiring full re install, in short delays…. tired of not controlling anything, I wanted to move my stack home, and have some redundancy).

Basically, this operation was from the following :
ESXI 6.7 latest update available in 2020 April (aside of the new major 7) to Proxmox PVE 6.1.8

This assume you have an ESXI and proxmox up and running that can reach each other and that you have NO SNAPSHOT on vmware esxi.
I also assume you have a ZFS volume mounted in your proxmox.

On the proxmox node (pve), from the shell, I install SSHFS :

#install SSHFS to mount esxi volume
apt install sshfs

#creating mount point
mkdir /mnt/ssh

#Mount root directory of esxi on /mnt/ssh of pve node
sshfs root@esxi:/ /mnt/ssh

#Convert flat VMDK drive to raw image (reading on esxi via SSHFS and output on your local proxmox)
#The below command must run from a drive where you have enough space to store the full size VMDK (usually /volume-you-created on your proxmox)
qemu-img convert /mnt/ssh/vmfs/volumes/<yourdatastore>/<yourvmname>/<yourvmname>-flat.vmkd -O raw <yourvmname>.raw

#Create the target VM in proxmox with the same specs as ESXI, and making sure you pick ZFS volume as storage.
#identify target disk of the created VM in /dev/zvol/<volumeName>/<diskName>

#Once you know where is you new VM (usually a number associated with the VM) volume is you can dump the raw image to the new virtual disk
dd bs=1M if=<yourVMname>.raw of=/dev/zvol/<volumeName>/<diskName>

If your machine is loaded, the command above may stale your ZFS and overload the server ( load can be way above amount of availalbe threads, due to default ZFS settings no limiting amount of threads).

A workaround found here actually helped getting around the issue using the option oflag=direct
dd bs=1M oflag=direct if=<yourVMname>.raw of=/dev/zvol/<volumeName>/<diskName>

Hope this helps, when migrating big VMDKs over 1TB, I had ZFS crashes due to this, command above allows to import them properly.

 

#When done, start the new VM and see if it boots. The main issue I had either on CentOs or Debian, was that network interface name changed. So at first boot, VM has no network.
#Not a big issue, edit the interface name in /etc/network/interface (debian) or /etc/sysconfig/network-scripts/ifcfg-ethx  (CentOs)
#The MAC address changed, unless you forced it while creating the VM.

#As soon as your VM is up, delete the .raw image from your drive to free up space.

This is how I migrated 8 VMs, from ESXI to Proxmox without any issues.

Additional notes following this migration (2020 April 27) :

After migrating my last VM, I had a weird error stating I did not have enough space to replicate, as the system could not take any snapshot.

This is due to a default setting in ZFS, in Proxmox, where by default, a space reservation on the volume is made for snapshots etc, with the same size of the actual disk size.

While fine for most of the case, when you have a VM with 1.66TB drive, it starts to be an issue.

There is more explanation here about ZFS refreservation : http://www.mceith.com/blog/?p=153

Basically, in my case, the refreservation was as big at 1.66TB, not allowing snapshots to be taken, and therefore, not allowing VM replication.

You can see the information for a volume with the following command :

zfs get refquota,reservation,refreservation yourvolume/your-vm-disk

So basically, for my VM that had a refreservation default value of 1.66TB, I actually did set it to 500G, since my drive only had 578G of unassigned space.

This allowed the replication process to go on :

Hopefully this will help some, facing the disk full issue where disk is not actually full but reserved.

 

Install your VPN server with pi-hole on OVH VPS in 30 min

Prereqs : You must have an SSH client (native Linux SSH command, or putty on windows… poor you), you must know how to edit a file in command line (using vim or nano, as you like), you must know how to use terminal menu (tab to switch options, space to select, enter to validate)
Port : This tutorial expect to run Openvpn on port 443 (SSL port), if you want to use another one, just replace 443 by the port you wish in the whole tutorial below.

Note : This is done using an OVH VPS, with Debian 9, other cloud VPS might differ if not default debian 9.

Note2 : This will allow all devices on VPN to communicate with each other (client to client options enabled by default, and firewall rules allows this as well) – If you want to block client to client communication, check this pagehttps://blog.inforeseau.com/openvpn-and-pihole-on-ovh-vps-server-faq

Order OVH VPS with latest Debian (VPS SSD, cheapest at CAD $4.49/month ! )
SSH to it. (credentials for SSH root in the email you receive after OS installation)

Update and Install tools we want (editors, firewall, auto updates) :

apt-get update
apt upgrade
apt install aptitude iptables-persistent vim curl unattended-upgrades
 

It should look like this :

(YES to save current IPv4 rules and YES to save current IPv6 fules when install iptables)

Replace /etc/resolv.conf file content with the following

#Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 9.9.9.9

Configure iptables

We add the following content in /etc/iptables/rules.v4 (replace all the content using vim or another tool or remotely with filezilla)

#Generated by ourselves for pivpn and pi-hole
 *filter
 :INPUT ACCEPT [663907:41179802]
 :FORWARD ACCEPT [57036644:76048011107]
 :OUTPUT ACCEPT [5009068:528571183]
 -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
#Don t change the port 443 in the 3 lines below, this is for pihole performance
 -A INPUT -i tun0 -p tcp -m tcp --dport 443 -j REJECT --reject-with tcp-reset
 -A INPUT -i tun0 -p udp -m udp --dport 80 -j REJECT --reject-with icmp-port-unreachable
 -A INPUT -i tun0 -p udp -m udp --dport 443 -j REJECT --reject-with icmp-port-unreachable
#End of don t change these
#Port allowed for VPN, change if you decide to use another port for your VPN server (replace 443 below by whatever you chose)
 -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
 -A INPUT -p udp -m udp --dport 443 -j ACCEPT
#End of port to change if you get your VPN server to run on another port
 -A INPUT -i lo -p udp -m udp --dport 53 -j ACCEPT
 -A INPUT -i lo -p tcp -m tcp --dport 53 -j ACCEPT
 -A INPUT -i tun0 -p udp -m udp --dport 53 -j ACCEPT
 -A INPUT -i tun0 -p tcp -m tcp --dport 53 -j ACCEPT
 -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
 -A INPUT -p icmp -j ACCEPT
 -A INPUT -j DROP
 COMMIT
 #Completed on Sat Apr 27 16:18:39 2019
 #Generated by iptables-save v1.6.0 on Sat Apr 27 16:18:39 2019
 *nat
 :PREROUTING ACCEPT [496992:46323248]
 :INPUT ACCEPT [335723:24234770]
 :OUTPUT ACCEPT [486171:32444728]
 :POSTROUTING ACCEPT [491702:32816107]
 #if you need to use 1 or more other port(s), just add as many line as below, whitout the # so it is actually loaded, ens3 is the NIC of the VPS, adjust if yours differ
 #Below will allow to connect to port 8080 aside of 443. 
 #- A PREROUTING -i ens3 -p udp -m udp --dport 8080 -j REDIRECT --to-port 443
 COMMIT
 #Completed certainly after 2019 Apr 27t

Load the new rules as follow :

 iptables-restore < /etc/iptables/rules.v4

Install pi-vpn
adduser vpn (set a password of your choice)

 curl -L https://install.pivpn.io | bash

(if you have a name resolution issue, type the command again until it moves on)

The installer will transform your Raspberry Pi into an openvpn server (will do for your debian VPS too :P ) OK

the PiVPN need a static address, fine we are on VPS with assigned IP, OK

Since we think you are not using raspbian… correct, fine, OK

Choose user OK, select « vpn » that we just created and OK

It is recommended that you enable « unattended upgrade », OK, we already did this at the beginning

Do you want to enable unattended upgrades of security : NO (it is already done, no need to redo)

We want to use UDP protocol, Select UDP and OK

We modify the port to 443 as this will most likely will be allowed on most networks and OK

Are these settings correct : YES

YES to the next screen about Elliptic curves for key exchanges

Next, i’d recommend at least 384-bit encryption, safer, not crazy overhead. OK

Client can use the public IP, OK, this is totally fine as VPS has static IP (DNS is more useful in case of dynamic DNS use) – It should be the IP of YOUR VPS

Select DNS provider, this is a key point, we want the benefit of pi-hole on our VPN, so we’ll go down the list, and select « Custom » (go down with arrows on keyboard to select custom), then OK

Fill the info with 10.8.0.1 and OK (this will be the IP of the openvpn server on tap0 that we allowed in firewall)

Are these settings ok ? YES

validate the end of installation OK

reboot the VPS – YES and confirm OK

After reboot, SSH again on it

We’ll generate a client certifacte
as root :
pivpn -a
provide a client name (like device name such as myphone1) and a password.
As shown, the client ovpn file will be placed in /home/vpn/ovpns/ (since we use the user « vpn » to handle VPN).

Copy this file using filezilla via SFTP(SSH) or cat on ovpn (cat /home/vpn/ovpns/user-you-chose.ovpn) and copy the content to a local ovpn file.

Open this file with openvpn application on your phone and connect :D
If on linux,

openvpn --config user-you-chose.ovpn

and it will connect.
You should be connected after providing the chosen password used when creating VPN user, and surf internet (when pi-hole below is installed too :P ) from your VPS with pi-hole filtering enabled !
go to http://monip.org to check the IP seen by websites (it should be the IP of your VPS).

Create as many users as you want ! Yes it’s free !
To delete a user : pivpn -r user
To see more options : pivpn

Done on VPN, but now we miss the pi-hole DNS filtering service, here we go (otherwise DNS resolution and filtering won’t work)

Install pi-hole, use the following command :

 curl -sSL https://install.pi-hole.net | bash

(will install needed packages)

Enter on OK to start.

Enter on OK to pass donation page (donate if you want to support pihole project, it is amazing)

Select interface TUN0 (DNS server must listen on VPN interface)
We have a static IP, ok

Pick the DNS server you want above pihole (i pick Quad9, but anything is good as you like) and press ok

Pi-hole third party list, select all and press OK

Select Protocols, leave both IPv4 and IPv6 checked, OK

Do you want to use your current network settings, YES ! ( This should show the IP of YOUR VPS)

IP Conflict, OK…fine fine move on :P

Do you wish to install the web admin interface : OFF and OK (we don’t need it, we won’t connect to it, won’t open it)

Do you wish to install the web server (lighttpd) ? OFF and OK (not needed)

Do you want to log queries : OFF (no we don’t want to leave tracks…)

Select a privacy mode on FTP : 4 Disabled statistics (we are all about privacy here)

Firewall Detected, Install Pi-hole default firewall rules? NO, they are already in the iptables rules we copied

Configure your devices to use the pi-hole…. OK, we’ll do this in VPN config after.

We then want pi-hole to update its blacklist automatically everyday.
Create the file /etc/cron.daily/piholeupdate with the following content :

#!/bin/bash
/usr/local/bin/pihole -g

Save and make it executable :

chmod +x /etc/cron.daily/piholeupdate

Done for pihole.

Enjoy !

Use the command « pivpn » to generate as many users as devices you have, install openvpn client on it (android, iOS, linux or windows or tunnelblick on MAC), and import the .ovpn file that you generated using the password you entered when creating user.

If one of your device is stolen or lost, revoke the certificate.

You can now browse internet, seen as your VPS, and no one can snoop on your traffic, without adds, without tracking :P (still take a canvas randomizer plugin for your browser to avoid device signature).

Security note : By default the OVH VPS is installed with a ROOT account and a password provided to you. Online servers are under attack on a permanent basis.
I recommend that you log on your VPS with an SSH private/public key instead of the password.
To do this, you need to generate a private/public key on your client machine (on linux, usually located in /home/your_user/.ssh/id_rsa.pub). If you have one, you can use it otherwise, generate one with the command « ssh-keygen -t rsa », then you copy the content of your local file id_rsa.pub to your VPS in the file /root/.ssh/authorized_keys)
Then try to ssh to the machine, ssh root@your_server, accepting he signature, you should be logged in without password
Now that we can SSH without password, we’ll disable the password authentication in SSH server. Edit the file /etc/ssh/sshd_config, and add the following line (if not already set) :
PasswordAuthentication no
Save and quit, restart the sshd server : service sshd restart
From this point, you can no longer login with a password, only the SSH key is accepted.
No more dictionary attack or password guessing. If ever you lose your local machine/private key, you can always go in the OVH console,through web management interface, and open the vKVM, to connect on the machine through the console with the root password. Enjoy.