Proxmox VE Cluster - Chapter 002 - Plan for Architecture Level 1.0

Proxmox VE Cluster - Chapter 002 - Plan for Architecture Level 1.0

A voyage of adventure, moving a diverse workload running on OpenStack, Harvester, and RKE2 K8S clusters over to a Proxmox VE cluster.

Here's my detailed plan for Architecture Level 1.0:

In summary, Level 1.0 means drop everything from multiple old clusters into a single large, simple-as-possible Proxmox VE cluster via several conversion phases, while making absolute minimum changes to design and workload.

A list of Goals for Architecture Level 1.0: 

1. All production storage on the giant TrueNAS over NFS.  Cluster-wide filesystems implemented later.
2. VMs manually provisioned, like the older VMWare era.  I like Terraform and Ansible as tools to provision IaaS, but I will implement that later.
3. Generally document and make minimal changes in Ansible with respect to the workload VMs and containers.

Some of the eternal ongoing projects such as re-IP addressing will continue as part of Arch 1.0, which is ambitious.  In a way it makes the conversion from OpenStack and Harvester to Proxmox VE simpler, if the new VM has a new IP address.  As usual I will polish and refine my Netbox information, clean up runbooks stored in Redmine, but the theme will be making minimum-possible changes rather than implementing ambitious new ideas at the same time as the cluster conversion.

The question of Docker Containers...

https://pve.proxmox.com/wiki/Linux_Container

"If you want to run application containers, for example, Docker images, it is recommended that you run them inside a Proxmox QEMU VM. This will give you all the advantages of application containerization, while also providing the benefits that VMs offer, such as strong isolation from the host and the ability to live-migrate, which otherwise isn’t possible with containers."

OpenStack Zen containers were cool, and K8S obviously runs container workloads very well, however the above Proxmox information implies I will have to go back to the VMware era of setting up "container containers" to hold my Docker containers.  No big deal, but it will take some time to roll everything.  Generally I do "container containers" by installing a simple Ubuntu server, then running Docker off a NFS mount so there is no local state stored on the Ubuntu server (making it trivial to rebuild, and also making backups very simple as all state is just files on the NFS server).

In the next post, I will discuss the complicated order of operations due to various dependencies and operations requirements.