About Deployments
A Deployment is an instantiation of a virtual data center consisting of two VMs (servers). Here's a diagram to help you visualize what's being created:
Each deployment is composed of two isolated VMs connected via an OVN-based network bridge for inter-VM communication. Each deployment has two identities on the network underlay: one for www and one for btcpayserver. Each of these VMs MUST have a reservation created in your DHCP server AS WELL AS host entries in your INTERNAL DNS server.
Servers are VMs, logical networks allow VMs to communicate, and projects isolate instances and resources from each other. All this is so you can run multiple Deployments on the same hardware.
About Projects
Incus projects allow you to isolate deployments. Projects enable you to run regtest, testnet, and mainnet deployments all on the same hardware. For example, dev-regtest, staging-testnet, and production-mainnet, all running on the same hardware, yet isolated from each other.
The above quality means you can perform reliable and repeatable system upgrade/migrations on the same hardware with reasonably low downtime. This approach is called Immutable Infrastructure.
Project Resources
When defining a project, you get to specify which resources should be shared or dedicated to the project. In the case of Sovereign Stack, each new project is able to access the base base image from the default project. In each Sovereign Stack deployment, profiles, networks, and storage volumes are specific to each project.
The objective of this arrangement is to have one base image that is shared among the various projects. Then when a new project (deployment) is specified by the System Owner, the project will get created and the incus client will switch to that project. Any new resources spawned under that project (e.g., VMs, networks, profiles) will exist in an isolated context. But because Sovereign Stack uses incus images, disk utilization should be minimal due to the copy-on-write strategy of ZFS.
The Data Plane
Incoming traffic originating from the firewall eventually makes its way to the www server that attaches to the network underlay using MACVLAN. Incoming requests are first delivered to a nginx container running on the host. nginx does all the routing of the host headers to the appropriate container and provides all TLS termination. In addition, the nginx instance is configured to cache websites where possible. This makes client requests complete quickly and helps take load off back-end containers (e.g., ghost).
Networking
On the networking side, an OVN network is created for each deployment that allows the www vm and the btcpayserver VM within a project to communicate. The OVN parent interface is incusrbr1,a non-natting, non-routable layer 2 bridge residing in the default project.
Base Image
Each VM is based on a base image created in the default project. This base image ss-docker-jammy is created by the create_lxc_base.sh script.
The diagram above depicts how things are basically running once a deployment is fully instantiated. When the VMs start and dockerd is running, various docker containers get spawned; they are all connected to each other by various docker overlay networks so they can communicate where needed.
If the base image doesn't yet exist, one will be created. The base VM is called ss-docker-jammy and is available to all deployments. The base image is initialized using a cloud-init declaration embedded in the generated incus profile.
Scalability
If you have a deployment with a lot of websites or a website with heavily utilization, you may consider vertically scaling each VM by updating the CPU_COUNT and MEMORY_MB variables in the project.conf file for the deployment in question.
Future iterations of Sovereign Stack will allow you to scale-out by adding 1) additional cluster hosts at a particular data center and 2) adding additional data centers.
Think Sovereign Stack brings value to the Bitcoin Network? If so, donate to the monthly crowdfund, or you can leave an arbitrary tip with button below.
Now that you know a little about Sovereign Stack, go check out some of the example websites you can create!