NeverSawUs

Using ProxyCommand to SSH to Cloud Managed Servers

Problem Statement

Given:

  • You have a series of server instances: something like an AWS Autoscaling Group (or "ASG").
  • These groups are named in a regular pattern: environment-service.
  • You don't necessarily have a stable set of IP addresses to SSH into.
  • Your instances are attached to a private network.
  • You have a well-known bastion server with SSH access.
  • You don't want to use AWS Secure Session Manager, or their equivalent (for whatever reason!)
  • You have jq installed.

You wish to be able to SSH to an ASG-managed instance without knowing the IP address ahead of time.

Solution

These steps enable you to run ssh [email protected].

  • Use .ssh/config host aliases to capture an autoscaling group name.
  • Point that host alias at a ProxyCommand.
  • The ProxyCommand script receives two arguments: the user-specified host and the port (See Fig. 1)
  • The script removes the host alias prefix (myhosts.), extracts the environment, service name, and (optional) instance index into variables (See Fig. 2, section 1)
  • The script uses a command line interface with your cloud provider to list instances for that autoscaling group and select an instance by the requested index (See Fig. 2, section 2.)
  • Then the script retrieves private IP information for that instance. (See Fig. 2, section 3.)
  • Finally, we connect to the target instance through a jumpbox.

Figures

Figure 1: An SSH config with ProxyCommand inserted:

Host myhosts.*
    ProxyCommand /path/to/proxy-command-script %h %p
    ForwardAgent yes

Figure 2: An example proxy command:

#!/bin/bash

# if necessary!
source path/to/some/credentials

# Section 1
target=${1/myhosts\./}
environment=$(echo $target | cut -d'.' -f1)
service=$(echo $target | cut -d'.' -f2)
num=$(echo $target | cut -d'.' -f3)
num=${num:-0}

# Section 2
autoScalingGroup=$(
  aws autoscaling describe-auto-scaling-groups --output json | \
  jq -r '.AutoScalingGroups[] |
  select( .AutoScalingGroupName == "'"${environment}-${service}"'" ) |
  .Instances[].InstanceId' | \
  sort
)

if [ ${#autoScalingGroup} -lt 1 ]; then
  >&2 echo "Autoscaling group \"$1\" not found or empty"
  exit 1
fi

instance=$(echo ${autoScalingGroup} | head -n$((num + 1)) | tail -n1)

if [ -z "$instance" ]; then
  >&2 echo "Could not find server by index \"$num\""
  exit 1
fi

# Section 3
ip=$(
  aws ec2 describe-instances --instance-id "$instance" --output json | \
  jq -r '.Reservations[].Instances[] | .PrivateIpAddress'
)

if [ -z "$ip" ]; then
  >&2 echo "Could not find ip of instance \"$instance\""
  exit 1
fi

# Section 4
ssh -q [email protected].$environment.example.com -W $ip:$2