Skip to main content

Adding customer proxy

Background​

In some cases customers use in their automation workflows some tools that are self-hosted in the customer's environment and are reachable from the customer's VNET but not from the public internet. Customers would still be able to run guided automations of use cases that use those tools, but we would be unable to reach those tools from our backend for pollers and workers for worker investigations.

To overcome this we create a dedicated EC2 instance for each such customer, have that instance connect to the customer's VNET using whatever VPN the customer uses and make that instance a proxy server that our poller and worker will use to reach those internal tools (both for API calls and for browser navigation).

Instructions to add a proxy instance for a customer​

1. Add and deploy proxy EC2 instance​

  1. In the backend infra Pulumi configuration file update the proxy_instances variable for the customer's region and add a new item there with the customer's internal org id as the instance name (to allow easy correlation from instance to customer for debugging)
  2. Deploy the infra to the prod region by following the instructions in the infra repo readme file

2. Connect to the EC2 instance​

  1. In AWS portal navigate to EC2 > Instances under our AWS production account and under the customer's region, and select the customer's EC2 instance by its name (which is the customer org id from step 1)
  2. Click 'Connect', switch to 'Session Manager' tab and click 'Connect' to connect to the instance
  3. (Optional but recommended: run bash in the opened terminal to use bash terminal instead of default and more annoying to use ubuntu shell)
  4. Run sudo apt-get update to make sure packages installed will be up to date
  5. Run ip a to list the instance's network interfaces and note down the name of the interface connected to the internet (usually ens5)

3. Connect the EC2 instance to the customer's VPN​

  1. When connected to the EC2 instance, follow the instructions of the VPN tool used by the customer to install it and connect it to the customer's network.
  2. After the VPN is connected, note down the name of the service running the VPN. The list of all running services on the machine can be seen by running systemctl list-units --type=service and finding the service matching the installed VPN client there

For example, for ZeroTier VPN client:

  1. Install VPN client: curl -s https://install.zerotier.com | sudo bash
  2. Connect to customer's network: sudo zerotier-cli join <network-id>
  3. If needed, extract the node id with sudo zerotier-cli status and send to customer so they approve joining the network
  4. After customer approved the request, verify connection to customer VPN by running sudo zerotier-cli listnetworks and verify status is OK

4. Set up proxy server​

  1. After VPN is installed and connected it adds a new network interface to the EC2 instance which we need when configuring the proxy. Run ip a and note down the new network interface the VPN added (that wasn't there when running the command during step 1)

  2. Install Dante proxy server: sudo apt-get install dante-server

  3. Edit the proxy config by running sudo nano /etc/danted.conf (can first run sudo mv /etc/danted.conf /etc/danted.conf.bak to quickly clear the default config file) and paste the following contents:
    (make sure to replace <internet-interface-name> and <vpn-interface-name> with the names extracted previously)

    logoutput: stdout

    internal: <internet-interface-name> port = 1380
    # Optional for debugging:
    # internal: 127.0.0.1 port = 1380
    external: <vpn-interface-name>

    clientmethod: none
    socksmethod: none
    user.privileged: root
    user.notprivileged: nobody

    client pass {
            from: 0.0.0.0/0 to: 0.0.0.0/0
            log: error connect disconnect
    }

    socks pass {
            from: 0.0.0.0/0 to: 0.0.0.0/0
            log: error connect disconnect
    }

    Configuration explanation:

    • We set logging to console output to be able to easily view status and error by running proxy status command (in next steps)
    • We use 'resolveprotocol' to have the proxy do DNS resolution in the VNET instead of by the caller (poller/worker), which won't have access to VNET DNS servers
    • We set up input from the outward network interface (from poller and worker) and output to the VPN interface, so all traffic will be forwarded to the VPN as-is
    • We allow all requests to be forwarded with no authentication (since we're in our private subnet) or limiting by source/target IP range

  4. Restart the proxy server for the configuration changes to take effect: sudo systemctl restart danted

  5. Check the proxy server status by running: sudo systemctl status danted. If everything is setup correctly the status should be 'active (running)' written in green, otherwise check the error messages in output to find and fix the config issue and restart the server again until ready

5. Set up proxy server persistency​

We need to update dante service file to make it only start running after VPN service is running (otherwise dante will load in error mode due to missing network interface until manually restarted from terminal).

  1. Run sudo systemctl edit --full danted.service to edit the service file
  2. Append the VPN service name (found in step 3.2) to 'After' field under '[Unit]' section. For example: After=network.target zerotier-one.service
  3. Add a new field Requires=<vpn-service-name> under '[Unit]' section. For example: Requires=zerotier-one.service
  4. Add the following lines at the end of the '[Service]' section:
Restart=always
RestartSec=5s
  1. Save and close the file
  2. Apply the changes by running the following commands
sudo systemctl daemon-reload
sudo systemctl restart danted
sudo systemctl enable danted

6. (Optional) Verify proxy correctly routes traffic to customer network​

  1. While connected to the proxy instance (following the instructions in step 2), run the following command to verify an IP address in the customer VNet is reachable by the proxy: curl --socks5 127.0.0.1:1380 <internal-ip-address>. For example: curl --socks5 127.0.0.1:1380 192.168.0.15/thehive/cases

7. Configure Legion to use the new EC2 proxy instance​

  1. Note down the 'Private IP DNS name' for the proxy EC2 instance
  2. Connect your local backend environment to production DB in the relevant region by following the Connecting to production DB guide
  3. In backend repo run the add_proxy dev tool to save the customer's proxy configuration to the DB which will make the poller and worker start using it. Make sure to add the mappings as <internal-tool-name>:"socks5://<ec2-private-dns-name>:1380" (note the 'socks5' and port which must be exactly these values, and the tool name must exactly match how it's written in the skill definition). For example: "TheHive": "socks5://ip-172-31-20-121.il-central-1.compute.internal:1380"