Skip to main content
tiCrypt 2.17.11

Network

A tiCrypt deployment runs three guest networks, one per guest class, plus an untagged management network for the host operating systems. Each guest network is an 802.1Q VLAN trunked over a shared bond, bridged on every hypervisor by OpenVSwitch, and terminated on a sub-interface of the backend.

Two properties follow, and the platform's network controls depend on both: each guest class occupies a single broadcast domain spanning every hypervisor, and the backend is the only host that forwards, filters or translates guest traffic.

Select anything below for detail, or trace a path from the list on the left to follow an operation end to end.

  • Secure
  • Service
  • Data-in
  • Management
  • same Layer 2 segment
VM / Slurm host 2VM / Slurm host 3Backendgateway · DHCP · single exitVM / Slurm host 1one of many, one Layer 2Secure /isolated switchtrunk 1081 · 1082 · 1083Managementswitcheth0externaleth2VLAN trunketh1managementenp1bond0.1081enp2bond0.1082enp3bond0.1083VLAN 1081Secure · DHCPVLAN 1082Service · DHCPVLAN 1083Data-in · DHCPtiCrypt backendgateway · DHCP · firewalleth2VLAN trunketh1managementenp1bond0.1081enp2bond0.1082enp3bond0.1083br-secureVLAN 1081br-serviceVLAN 1082br-datainVLAN 1083Secure VMeth0Service VMeth0Data-in VMeth0World

The three networks​

VLAN IDs and address ranges are per deployment. The values below are the documented example set; the topology they describe is fixed.

SecureServiceData-in
VLAN108110821083
Bridgebr-securebr-servicebr-datain
Interfaceenp1 / bond0.1081enp2 / bond0.1082enp3 / bond0.1083
Range192.168.128.0/17192.168.122.0/24192.168.123.0/24
Gateway192.168.128.1192.168.122.1192.168.123.1
DHCP192.168.129.1–192.168.255.254192.168.122.3–192.168.122.254192.168.123.3–192.168.123.254
External accessPer-VM allow lists onlyGeneralGeneral

The secure range is a /17 because it holds every secure VM in the deployment, not every secure VM on one host. The service and data-in networks are deliberately unrestricted, because neither is a place research data is held: service VMs prepare boot images, and data-in VMs pull a dataset onto a drive that is then detached and attached to a secure VM.

Why the networks are flat​

Running each VM network as one Layer 2 segment spanning all hosts is what makes several things possible at once:

Guests are Layer 2 adjacent across hypervisors. Two secure guests on different physical hosts are on-subnet to each other. ARP resolves across the trunk and frames are switched, with no gateway, no NAT and no MTU change in the path. Clustering and Slurm workloads require that adjacency.

One DHCP authority per broadcast domain. dnsmasq on the backend leases addresses and serves DNS for every guest of a class, on any hypervisor. There is no per-host subnet to allocate, and no inter-host routing to maintain as guests are added.

Inbound access is DNAT, not a userspace relay. Because the guest subnet terminates on the backend at Layer 2, inbound connections are translated in netfilter rather than relayed by a proxy process. A routed, per-hypervisor design would force software proxying instead.

Policy lives in one nftables table. All tiCrypt rules are held in a dedicated ticrypt table maintained by ticrypt-firewall, so they neither depend on nor collide with other rules on the host, and the hypervisors carry no guest filtering at all.

No inter-VLAN routing

Forwarding between the three VLANs is deliberately absent. They are separate broadcast domains whose only common point is the backend, where egress is source-NATed and inbound is DNAT'd. Adding routes between them collapses the isolation the separate domains exist to provide.

What the VM hosts deliberately cannot do​

A hypervisor has no external interface and no off-site default route. It terminates the VLAN trunk and the management network and nothing else, so it performs no forwarding for guest traffic:

  • Hypervisors need no external connectivity, so they can be firewalled to the management network and the trunk alone.
  • Every guest packet leaving the deployment transits the backend, giving one enforcement point and one flow-logging point rather than one per host.
  • Egress is default-deny and ticrypt-allowedlist opens specific destination address and port pairs per guest, which is only enforceable because every flow passes the same host.