## AWS EC2 Instance Connection Manager
### Background
As I recovered from doing cartwheels having secured my very own Amazon
EC2 instance a while back - overjoyed as I was that I could leave all the
conventional worries of localized data loss and machine outages behind me -
I quickly realized that I would probably be needing a utility script to help me
connect to my instance.
I've never been a huge GUI fan. If it's not design-related, anything I can
point and click at I'd much rather do from the command line, thank you very
much. Not that there's anything wrong with that - it's just my personal
preference (I repeat - design/creative-wise, it's another story altogether).
I hasten to point out that in this respect Amazon have done very well in that
it appears they've bent over backwards to be as user friendly as possible
in their instance management dashboards. No fault there, whatsoever.
However, using the dashboard the first few times was all fine and dandy until
I realized that I would be quite regularly starting/connecting/stopping my
instance(s) (yes, even during the free trial period I was mindful of how
I would eventually save on both costs and reduce wasted energy, if only
marginally).
The idea was that I'd likely have an instance or two I wouldn't need to be
*serving* at all times (so not a production machine, for example) and for
which I could exercise control over both costs and energy much more efficiently
from my fingertips at the command line.
All this meant that for long-term productivity, the GUI was a no-go.
So I decided to cobble together a utility script to take care of this for me,
instead. Originally, I had a simple bash one-liner I ran from the command
line which essentially wrapped the underlying Amazon connection protocols
(as detailed, by the way, in their *very* handy reference [manuals](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html)).
However, as I wanted to add more features like describing the instance state
and having the flexibility to attach to different instances and, perhaps most
importantly, being able to do it all with as few keystrokes as possible for
the sake of convenience, I found I had to get serious about what this little
utility script was supposed to do.
The following is what I decided to cobble together for this task.
### Organization
First, I decided that rather than trying to fold everything into bash one-
liners, I would be better served at writing a proper underlying script (one
which doesn't launch dozens of processes as bash does for every task it would
need to execute) and then using bash to call the underlying with supplied
arguments. As a result, I settled upon a bash + Perl solution to achieve this
with Perl as the underlying script.
Second, I settled upon an approach wherein I would use a dedicated module to
store all my account-specific and instance-relevant configurations. This way,
all AWS details could be kept in one place and accessed when needed for
ssh connection and authentication protocols.
The module is organized as follows - note the especially handy attribute of
storing multiple instance configurations in one place (named, in this case,
'dev' and 'prod'):
<div><pre class="code">
#====================================================================
#
# Filename: CfgAws.pm
# Author: Ozan Akcin
# Created: 02-12-2016
# Purpose: key parameter-argument couplings for AWS servers connectivity
# References: http://docs.aws.amazon.com/cli/latest/reference/ec2/start-instances.html
# http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html
# http://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html
#====================================================================
package FsData::CfgAws; #---modify to match target lib path in your env
use strict;
use warnings;
use Carp;
my %awsCfg =
(
'awsGlobalVars' => {
accountNum => ""
, configFile => "$ENV{HOME}/.aws/config"
, credFile => "$ENV{HOME}/.aws/credentials"
, ipFile => "$ENV{HOME}/.aws/pubDNS_INSTANCE"
},
'awsReturnValues' => {
0 => "pending"
, 16 => "running"
, 32 => "shutting-down"
, 48 => "terminated"
, 64 => "stopping"
, 80 => "stopped"
},
'instances' => {
'dev' => {
ami => "" #---amazon machine image
, description => ""
, instanceId => ""
, instanceType => "" #---e.g. 't2.micro', etc.
, keyPairFile => "" #---path to AWS *.pem file (generally under ~/.ssh)
, keyPairName => "" #---file name of key pair given during ssh-keygen process
, secGroupId => "" #---security group id (from AWS instance description)
, vpcId => "" #---virtual private cloud
},
'prod' => {
},
}
);
#===============================================================================
sub new
{
my( $class )=@_;
my $self = {};
bless $self, $class;
$self->_init;
return $self;
}
#===============================================================================
#===============================================================================
sub _init
{
my ( $self ) = @_;
$self->{awsCfgVars} = \%awsCfg;
}
#===============================================================================
</pre></div>
The connection manager script follows below. I like to standardize certain
facets of my working environment/infrastructure which means keeping key
environment-relevant parameter-argument couplings in one place and logging
task outputs in a consistent manner. These appear in the code below in the
'FsData' modules, which may be ignored/edit/removed to suit your own needs.
Both the script and module maybe [forked](https://github.com/builderLabs/awsConn) from my GitHub profile.
```
#====================================================================
#
# Filename: awsConn.pl
# Author: Ozan Akcin
# Created: 20161202
# Purpose: manages AWS server connections
# Last Modified:
#
#====================================================================
#!/usr/bin/perl
use strict;
use warnings;
use POSIX;
use File::Path;
use File::Basename;
use Time::Local;
use FsData::FsSysVars; #---env-specific key-value pairings (replace/ignore)
use FsData::FsLog; #---env-specific logging utility (replace/ignore)
use Util::CfgAws;
my ( $fsVars, $fsLog, $svcTag, $appName );
my ( $defInst, $defCmd, $defUser, $startSleep, $stopSleep, @chkParam );
#---general settings-------------------------------------------------
#---job definition:
$svcTag = 'AWS';
$appName = 'AWS_CONNECTION_MANAGER';
#---task-specific:
$defInst = 'dev';
$defCmd = 'connect';
$defUser = $ENV{USER};
$startSleep = 20;
$stopSleep = 60;
@chkParam = (
'OwnerId'
, 'PublicIpAddress'
, 'VpcId'
, 'ImageId'
, 'KeyName'
, 'GroupId'
, 'InstanceType'
);
#--------------------------------------------------------------------
#==============================================================================
sub setTaskDef
{
my ( $program, $ymd, $HHMMSS, $dtStampLocal );
my ($logRoot, $logFile, $taskArgs, $fsTaskDef );
$dtStampLocal = strftime("%Y%m%d %H%M%S", localtime);
$program = basename($0);
$program =~ s/\.\w*//;
$ymd = substr($dtStampLocal,0,8);
$HHMMSS = substr($dtStampLocal,-6);
$logRoot = "$program.".$HHMMSS;
$taskArgs = {
appName => $appName
, processName => $appName
, logRoot => $logRoot
};
$fsTaskDef = {
SERVICE_TAG => "${svcTag}"
};
$fsVars = new FsData::FsSysVars($taskArgs, $fsTaskDef);
$logFile = defined($fsVars->getArgv('logFile'))
? $fsVars->getArgv('logFile')
: $fsVars->getFsVar('FSDATA_LOG_DIR')
. '/' . $ymd
. '/' . $fsVars->getFsVar('SERVICE_TAG')
. '/' . $fsVars->getArgv('logRoot')
;
$fsLog = new FsData::FsLog( $fsVars->getFsVars, $logFile );
$fsVars->setArgv('YYYYMMDD',$ymd);
$fsVars->setArgv('HHMMSS',$HHMMSS);
#---print invocation to logFile:
$fsLog->print("***START***");
$fsLog->print(`ps -o args -C perl | grep $0 | tr -d '\n'`)
}
#==============================================================================
#==============================================================================
sub initArgs
{
#---future instances need to be coded for here, including if not 'current'
my ( $fsLog, $fsVars ) = @_;
$fsLog->print("Initializing arguments...");
my ( $workDir, $inst, $cmd, $user, $awsCfg );
$workDir = defined($fsVars->getArgv('workDir'))
? $fsVars->getArgv('workDir')
: $fsVars->getFsVar('FSDATA_OUTPUT_DIR')
. '/' . $svcTag
. '/' . 'tmp_' . $$
;
$inst = defined($fsVars->getArgv('inst'))
? $fsVars->getArgv('inst')
: $defInst
;
$cmd = defined($fsVars->getArgv('cmd'))
? $fsVars->getArgv('cmd')
: $defCmd
;
$user = defined($fsVars->getArgv('user'))
? $fsVars->getArgv('user')
: $defUser
;
$user = 'oakcin' if ( $user eq 'akcinoz' );
$awsCfg = new FsData::CfgAws;
mkpath($workDir) if ( ! -d $workDir && $cmd eq 'start' );
$fsVars->setArgv('workDir',$workDir);
$fsVars->setArgv('inst',$inst);
$fsVars->setArgv('cmd',$cmd);
$fsVars->setArgv('user',$user);
$fsVars->setArgv('awsCfg', $awsCfg);
}
#==============================================================================
#==============================================================================
sub awsConnect
{
my ( $fsLog, $fsVars ) = @_;
my ( %awsCfg, $user, $inst, $instanceId, $keyPairFile, $ipFile, $invFile, $awsIp, $cmdString );
$fsLog->print("Reading configurations for connection to AWS server...");
$user = $fsVars->getArgv('user');
$inst = $fsVars->getArgv('inst');
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$ipFile = $awsCfg{awsGlobalVars}->{ipFile};
$ipFile =~ s/INSTANCE/$inst/g;
($invFile = $ipFile) =~ s/pubDNS/invalid/g;
$fsLog->die("ERROR: invalid IP/log-on control file: $invFile detected...") if ( -f $invFile );
$fsLog->die("ERROR: no IP file detected - try starting remote AWS server for: $inst first.") if ( ! -f $ipFile );
$keyPairFile = $awsCfg{instances}->{$inst}->{keyPairFile};
$instanceId = $awsCfg{instances}->{$inst}->{instanceId};
open ( my $fh, $ipFile ) or $fsLog->die("Cannot open: $ipFile");
{ $awsIp = <$fh>; }
close($fh);
chomp $awsIp;
$cmdString = "ssh -q -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no ";
$cmdString .= "-i KEYPAIRFILE USER\@AWSIP";
$cmdString =~ s/KEYPAIRFILE/$keyPairFile/g;
$cmdString =~ s/USER/$user/g;
$cmdString =~ s/AWSIP/$awsIp/g;
$fsLog->print("Connecting to AWS instance id: $instanceId...bye!");
$fsLog->print("$cmdString");
exec ("$cmdString");
#---> check existence of valid file in ~/.aws/ first, raiseerror if not there.
}
#==============================================================================
#==============================================================================
sub getLocParam
{
my ( $fsLog, $fsVars ) = @_;
my ( $inst, %awsCfg, %locParam );
$inst = $fsVars->getArgv('inst');
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$fsLog->print("Collating local connection parameters for server: $inst...");
$locParam{'InstanceType'} = $awsCfg{instances}->{$inst}->{instanceType};
$locParam{'VpcId'} = $awsCfg{instances}->{$inst}->{vpcId};
$locParam{'OwnerId'} = $awsCfg{awsGlobalVars}->{accountNum};
$locParam{'KeyName'} = $awsCfg{instances}->{$inst}->{keyPairName};
$locParam{'ImageId'} = $awsCfg{instances}->{$inst}->{ami};
$locParam{'GroupId'} = $awsCfg{instances}->{$inst}->{secGroupId};
$fsVars->setArgv('locParam',\%locParam);
}
#==============================================================================
#==============================================================================
sub awsAuthenticate
{
my ( $fsLog, $fsVars ) = @_;
getRemParam($fsLog, $fsVars);
getLocParam($fsLog, $fsVars);
my ( %awsCfg, %locParam, %remParam, $valid, $inst, $instanceId, $ipFile, $invFile );
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$ipFile = $awsCfg{awsGlobalVars}->{ipFile};
$inst = $fsVars->getArgv('inst');
$instanceId = $awsCfg{instances}->{$inst}->{instanceId};
$ipFile =~ s/INSTANCE/$inst/g;
($invFile = $ipFile) =~ s/pubDNS/invalid/g;
$valid = 0;
$fsLog->print("Comparing remote and local AWS parameter settings...");
%locParam = %{$fsVars->getArgv('locParam')};
%remParam = %{$fsVars->getArgv('remParam')};
#---compare parameters and log differences
foreach my $skey ( keys %remParam ) {
next if ( $skey eq 'PublicIpAddress' );
if ( $locParam{$skey} ne $remParam{$skey} ) {
$fsLog->print("Parameter mismatch - $skey: $locParam{$skey} (loc) v. $remParam{$skey} (rem)");
$valid++;
} else {
$fsLog->print("Validated $skey: $locParam{$skey} (loc) and $remParam{$skey} (rem)");
}
}
if ( $valid > 0 ) {
$fsLog->print("Generating invalid ip-logon file...");
$fsLog->print("System(bash -c \"touch $invFile\")");
my $sysMsg = `bash -c \"touch $invFile\"`;
$fsLog->die($sysMsg) if ($sysMsg);
$fsLog->die("ERROR: One or more settings do not match..see $fsLog->{_logFile} for details...");
}
$fsLog->print("Expected remote and local settings verified - updating ip file for inst: $inst...");
unlink $ipFile if ( -f $ipFile );
open my $out, '>', $ipFile;
print $out $remParam{PublicIpAddress};
close($out);
}
#==============================================================================
#==============================================================================
sub awsDescribe
{
my ( $fsLog, $fsVars ) = @_;
my ( %awsCfg, $inst, $instanceId );
my ( $cmdStringRoot, $cmdStringCode, $cmdStringState, $cmdStringIP, $remState, $remCode, $awsIp);
$inst = $fsVars->getArgv('inst');
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$instanceId = $awsCfg{instances}->{$inst}->{instanceId};
$cmdStringRoot = "aws ec2 describe-instances --instance-ids $instanceId";
$cmdStringCode = $cmdStringRoot." --query 'Reservations[0].Instances[0].State.Code'";
$cmdStringState = $cmdStringRoot." --query 'Reservations[0].Instances[0].State.Name'";
$fsLog->print("Checking AWS status code...");
$fsLog->print("System('$cmdStringCode')");
$remCode = `bash -c \"$cmdStringCode\"`;
$fsLog->print("Checking AWS status name...");
$fsLog->print("System('$cmdStringState')");
$remState = `bash -c \"$cmdStringState\"`;
chomp $remCode;
chomp $remState;
$remState =~ s/\"//g;
$remState =~ s/\'//g;
if ( ! defined($remCode) || ! defined($remState) ){
$fsLog->die("Error in ascertaining one or more state descriptive variables: 'Code', 'Name'");
}
if ( ! defined($awsCfg{awsReturnValues}->{$remCode}) ) {
$fsLog->print("Unable to verify returned status code: $remCode");
$fsLog->die("No such code recorded in module/library.");
}
if ( $awsCfg{awsReturnValues}->{$remCode} ne $remState ) {
$fsLog->print("Unable to verify returned status: $remState for code: $remCode.");
$fsLog->die("Regsitered status for code: $remCode is: ".$awsCfg{awsReturnValues}->{$remCode});
}
my $status = "Verified AWS server instance: $inst ($instanceId) "
." returned status: $remState ($remCode) ";
if ( $remCode == 16 ) {
$cmdStringIP = $cmdStringRoot." --query 'Reservations[0].Instances[0].PublicIpAddress'";
$fsLog->print("Checking AWS IP address...");
$fsLog->print("System('$cmdStringIP')");
$awsIp = `bash -c \"$cmdStringIP\"`;
$status .= "on ip address: $awsIp";
}
$fsLog->print($status);
print $status."\n";
}
#==============================================================================
#==============================================================================
sub getRemParam
{
my ( $fsLog, $fsVars ) = @_;
my ( $workDir, $outFile, %awsCfg, $inst, $instanceId, $cmdString, $response );
my ( %remParam, $remState, $remCode, $awsIp );
$workDir = $fsVars->getArgv('workDir');
$inst = $fsVars->getArgv('inst');
$outFile = $workDir.'/aws_response';
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$instanceId = $awsCfg{instances}->{$inst}->{instanceId};
$fsLog->print("Checking AWS server response...");
$cmdString = "aws ec2 describe-instances --instance-ids $instanceId ";
$response = `bash -c \"$cmdString\"`;
$fsLog->print("$response");
$response =~ s/\,/\n/g;
$response =~ s/\"//g;
open my $out, '>', $outFile;
print $out $response;
close($out);
open (my $fh, $outFile);
while(<$fh>){
chomp;
$_ =~ s/\s//g;
my ( $param, $arg ) = split /\:/, $_;
next if ( ! defined($param) );
if ( $param eq 'PublicIpAddress' ) {
$awsIp = $arg;
$fsVars->setArgv('awsIp',$awsIp);
}
if ( grep { $chkParam[$_] eq $param } 0 ..$#chkParam ) {
$remParam{$param} = $arg;
}
}
$fsVars->setArgv('remParam',\%remParam);
}
#==============================================================================
#==============================================================================
sub awsStart
{
my ( $fsLog, $fsVars ) = @_;
my ( %awsCfg, $inst, $instance, $instanceId, $cmdString, $response );
$inst = $fsVars->getArgv('inst');
$fsLog->print("Reading configurations to start AWS server...");
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$instanceId = $awsCfg{instances}->{$inst}->{instanceId};
$cmdString = "aws ec2 start-instances --instance-ids $instanceId";
$fsLog->print("Sending start command to remote AWS server instance: $instanceId...");
$fsLog->print("$cmdString");
$response = `bash -c \"$cmdString\"`;
$fsLog->print("AWS Response: $response...");
sleep($startSleep);
awsDescribe( $fsLog, $fsVars );
awsAuthenticate( $fsLog, $fsVars );
}
#==============================================================================
#==============================================================================
sub awsStop
{
my ( $fsLog, $fsVars ) = @_;
my ( %awsCfg, $inst, $instanceId, $cmdString, $response, $ipFile );
$inst = $fsVars->getArgv('inst');
$fsLog->print("Reading configurations to stop AWS server...");
%awsCfg = %{$fsVars->getArgv('awsCfg')};
%awsCfg = %{$awsCfg{awsCfgVars}};
$ipFile = $awsCfg{awsGlobalVars}->{ipFile};
$inst = $fsVars->getArgv('inst');
$instanceId = $awsCfg{instances}->{$inst}->{instanceId};
$ipFile =~ s/INSTANCE/$inst/g;
$cmdString = "aws ec2 stop-instances --instance-ids $instanceId";
$fsLog->print("Sending stop command to remote AWS server instance: $instanceId...");
$fsLog->print("$cmdString");
$response = `bash -c \"$cmdString\"`;
$fsLog->print("AWS Response: $response...");
sleep($stopSleep);
awsDescribe( $fsLog, $fsVars );
$fsLog->print("Removing ip file: $ipFile...");
unlink $ipFile if ( -f $ipFile );
}
#==============================================================================
#==============================================================================
sub awsConnManager
{
my ( $fsLog, $fsVars) = @_;
my $cmd = $fsVars->getArgv('cmd');
if ( $cmd eq 'connect' ) {
awsConnect($fsLog, $fsVars);
} elsif ( $cmd eq 'start' ) {
awsStart($fsLog, $fsVars);
} elsif ( $cmd eq 'describe' ) {
awsDescribe( $fsLog , $fsVars);
} elsif ( $cmd eq 'stop' ) {
awsStop($fsLog, $fsVars);
} else {
$fsLog->die("Unrecognized command: $cmd...");
}
}
#==============================================================================
#==============================================================================
sub cleanWork
{
my ($fsLog, $fsVars) = @_;
$fsLog->print("Cleaning up...");
my $workDir = $fsVars->getArgv('workDir');
if ( -d $workDir ) {
$fsLog->print("Removing work directory: $workDir");
$fsLog->print("System('rm -r $workDir')");
my $sysMsg = `bash -c \"rm -r $workDir\"`;
if ( $sysMsg ) {
$fsLog->print("Unexpected return encountered trying to remove: $workDir...");
$fsLog->die("SYSMSG: $sysMsg");
}
}
1;
}
#==============================================================================
exit(main());
#==============================================================================
sub main
{
setTaskDef;
initArgs($fsLog, $fsVars);
awsConnManager($fsLog,$fsVars);
cleanWork($fsLog, $fsVars);
$fsLog->print("***DONE***");
}
#==============================================================================
```
One of the most useful aspects is tucking the connectivity functions into
utilities in my bash profile.
For instance, sourcing a utility function such as the following:
```
#====================================================================
function sshaws(){
perl -w ${HOME}/fsbase/releases/fsprod/src/util/awsConn.pl --argv inst=dev --argv cmd=connect
}
#====================================================================
```
in my .bashrc means it will always be ready upon my logging into my local
machine and that by simply typing:
```
sshaws
```
at the command line gets me connected to my running AWS EC2 instance.
Hope you find the scripts to be of use.
