tiCrypt Secure VM networking
Secure virtual machines in tiCrypt run in almost complete isolation from each-other and the environment. The main motivation for this isolation is providing a high degree of security.
Setup
In this blog we will refer to a number of tiCrypt components that participate in the activities described here. They are:
ticrypt-connect
runs the application on user's deviceticrypt-frontend
runs in the browser on user's device and is served byticrypt-connect
ticrypt-backend
is the set of tiCrypt services running on the Backend serverticrypt-rest
is the entry point inticrypt-backend
and uses REST protocol.ticrypt-proxy
mediates communication between components by replicating traffic between two other connections.ticrypt-allowedlist
mediates access to external licensing serversticrypt-vmc
controls the secure VM and is responsible for all VM related security mechanisms
Overview
The VM security is provided through the following mechanisms:
- Reverse mechanism to talk to tiCrypt frontend using
ticrypt-proxy
component - The only opened ports for a VM are port
22
for traffic tunneling from tiCrypt Connect - Special mechanism to reach port
22
of VMs but no general mechanism to reach any other port from tiCrypt-backend - Strictly controlled outgoing traffic through
ticrypt-allowedlist
component
We take each mechanism in turn end provide details on how it works.
ticrypt-proxy
mediated VM communication
Due to the fact that all ports are blocked except 22
and that ticrypt-vmc
(VM controller) control this port for port forwarding, there is no mechanism to directly contact a secure VM using direct access. Instead, a mechanism fully controlled by the ticrytp-vmc
is employed. The mechanism relies on ticrypt-proxy
(component of ticrytp-backend
) to mediate communication with the user in the following manner:
- The
ticrypt-vmc
creates a web-socket connection withticrypt-proxy
at all times. - When a user wants to connect to a VM, the
ticrypt-frotnend
creates a web-socket to match the existing VM one inticrypt-proxy
. All traffic between the two matching web-sockets is replicaed byticrypt-proxy
. Imediatelyticrypt-vmc
creates a new web-socket for future communication. - The first message sent by both the
ticrypt-frontend
andticrypt-vmc
are digital signature proofs of identity and Diffie-Helman secret key negotiation. - If the digital signature fails or if the user is not authorized (as determined by
ticrypt-vmc
) the connection is immediately closed. Similarly, if the VM validation fails,ticrypt-frontend
will close the connection - All subsequent messages (in both directions) are encrypted using hte negotiated key. Except the setup messages, all traffic is hidden from
ticrypt-proxy
and any other part of the infrastructure. - All commands, terminal info, keys, etc are send through this encrypted channel.
The only message sent in the clear to the ticrypt-vmc
contains both authentication (digital signature) and key negotiation. Any other message or functionality is immediately rejected.
The communication does not rely on listening on regular ports and can only be mediated by ticrypt-proxy
.
The public key of the VM owner cannot change after learned by the ticrypt-vmc
thus hijacking the communication requires a compromise of user's private key.
Most of the VM functionality only requires this communication method and can only be performed using this mechanism. The only exception is application port tunneling.
Application port tunneling
In order to allow reach application deployment in VMs, a generic mechanism that tunnels TCP traffic on specific ports from the VM to user's device is provided. The mechanism is highly controlled (as explained below) but, in principle, can make any network application access possible.
The application tunneling is similar to the SSH tunneling mechanism used for reverse port forwarding but uses TLS
The communication pathway relies on an number of independent segments, all of which can prevent the mechanism from functioning. All traffic on these segments is proxied/forwarded and overall is secured with TLS encryption and authenticated with digital signature (not passwords). At no point any intermediate component can intercept the communication without breaking TLS.
Access to VMs is limited to port 22, not SSH as a service. ticrypt-connect
mediates port forwarding of any desired port using the mechanisim described with ticrypt-vmc
listening on port 22. Specifically, port 3389 for RDP can be forwarded. Usually, multiple such ports are forwarded to allow richer functionality.
Communication setup
Initiating the forwarding tunnel setup requires the following steps:
- The
ticrytp-frontend
using authenticated session, tellsticrypt-rest
(entry point ofticrypt-backend
) to create the pathway to a specific VM - The request is validated and
ticrypt-proxy
is informed of the operation - The
ticrypt-proxy
sets up a listening endpoint on one of the designated ports (usually 6000-6100). The endpoint is strictly set up and only allows access from the IP address used to make the request. - The
ticrypt-frontend
is informed of the allocated port. ticrypt-frontend
asksticrypt-connect
to generate a TLS certificate for authentication and connection encryption.ticrypt-frontend
tellsticrypt-vmc
to accept the application forwarding and provides the authentication TLS certificateticrypt-vmc
replies with a list of ports that need to be tunneledticrypt-frontend
tellsticrypt-connect
to start the connection and use the previous certificate.ticrypt-vmc
, upon connection creation, checks that the digital signature is correct and initiates the TLS mediated tunneling.- The traffic to/from local port on user device is tunneled and re-created within the VM to allow application access
A special case is provided for SFTP over port 2022 if the feature is enabled in ticrypt-vmc
.
It can be used to move large amount of data from user's device to the VM.
Communication pathway
The communication pathway for port tunneling is:
- The
ticrypt-connect
accesses the allocated port controlled byticrypt-proxy
. The connection is pinned to the IP address the setup request comes from. ticrypt-proxy
forwards the request to the VM Host endpoint (explained below)- The VM Host endpoint forwards the traffic to port
22
of the correct VM ticrypt-vmc
listens on port22
and runs the TLS protocol with port tunneling.
Port 22
is specifically selected to ensure no SSH server is deployed instead to allow normal access to the VM. The traffic on 22
is the TLS protocol with ticrypt-vmc
control and not SSH protocol.
Secure VM isolation
In order to ensure VM security, only port 22
is opened inbound. Furthermore, all outbound traffic is limited to ticrytp-rest
traffic unless an exception is provided by ticrypt-allowedlist
(next section)
The specific mechanisms deployed for VM isolation are:
- All traffic to ports other than
22
is blocked internally by the VM thus isolating any other access. Note that application access is provided by the port tunneling above. - All traffic to the VM ip on ports other than
22
is blocked using firewall rules on the VM host. - External traffic to VMs is not routed by the VM host (with the exception of port
22
) - Traffic from VMs to outside is blocked unless
ticrypt-allowedlist
mechanism is used
The specific mechanism to allow access on port 22
but no other traffic is to block any traffic routing but provide a forwarding mechanism from a range on the VM host to port 22
of the IP ranges dedicated to the secure VMs. There is simply no way from a server outside the specific VM host to access any other VM port.
The outbound traffic from secure VMs is blocked since it can be used to exfiltrate data thus resulting in data breaches.
Licensing server access
In order to use most commercial software, access to external licensing servers needs to be provided. Most of the time, the licensing servers reside within the organization, but occasionally, the servers are external. Recognizing this, tiCrypt provides a strictly controlled mechanism to allow access to licensing servers.
The ticrypt-allowlist
component mediates setting up the mechanism that allows access to licensing servers. It does so by manipulating:
- Firewall and port forwarding rules on the server running
ticrypt-backend
- DNS replies to requests that VMs make
The mechanism is strict and very specific. Unless a mapping is provided using ipsets
and firewall rules, any outgoing traffic from VMs is blocked.
The ticrypt-allowlist
component works in conjunction with ticrypt-frontend
to provide a convenient way to enable/disable access. This mechanism requires SuperAdmin
access.