Getting Started with AWS Systems Manager Session Manager: Secure Shell Access Without SSH Keys

In the realm of cloud infrastructure, securely accessing your Amazon EC2 instances is paramount. Traditionally, this has been achieved using SSH (Secure Shell) keys, which involve managing .pem files, configuring security groups to open Port 22, and potentially setting up bastion hosts for instances in private subnets. While effective, this approach introduces operational overhead, potential security risks, and can be cumbersome, especially in large environments or for teams.

Enter AWS Systems Manager Session Manager. This powerful capability of AWS Systems Manager provides a fully managed, secure, and auditable way to access your EC2 instances (and even on-premises servers or virtual machines) without the need for SSH keys, open inbound ports, or bastion hosts. It significantly simplifies remote instance management, enhances your security posture, and offers a more centralized control plane.

This article will guide you through the process of getting started with AWS Systems Manager Session Manager, covering its benefits, prerequisites, step-by-step setup, and how to initiate your first secure session.

The Challenges of Traditional SSH Access

Before we delve into Session Manager, let’s briefly review the inherent challenges that come with traditional SSH key-based access to EC2 instances:

1. Key Management Overhead:

• Distribution: Securely distributing and managing SSH keys across a team can be complex.
• Rotation: Keys rarely get rotated, increasing the risk if a key is compromised.
• Loss/Compromise: A lost or stolen private key can grant unauthorized access to your instances.
• Auditability: Tracking who used which key to access what instance is difficult without additional tooling.

2. Network Security Risks:

• Open Port 22: Exposing Port 22 (SSH) in security groups is a common attack vector, making your instances vulnerable to brute-force attacks or other exploits if not properly restricted.
• Public IPs: Instances often require public IP addresses to be directly accessible via SSH from the internet, which isn’t always desirable from a security perspective.
• Bastion Hosts Complexity: For instances in private subnets, setting up and maintaining bastion hosts adds another layer of complexity and cost.

3. Auditing and Compliance:

• Default SSH access provides limited native logging of commands executed within a session, making auditing for compliance or troubleshooting difficult.

4. Operational Inefficiency:

• Manual SSH key distribution, security group adjustments, and bastion host management consume valuable time for DevOps and security teams.

These challenges highlight the need for a modern, more secure, and operationally efficient way to interact with your cloud instances.

What is AWS Systems Manager Session Manager?

AWS Systems Manager (SSM) is a comprehensive service that allows you to gain operational insights and take action on your AWS resources. Session Manager is one of its core capabilities, specifically designed to provide secure and auditable shell access to your instances.

Instead of SSH, Session Manager uses the SSM Agent installed on your EC2 instance (or on-premises server) to establish a secure, HTTPS-based connection through the AWS Systems Manager service endpoint. This connection is initiated directly from the AWS Management Console or the AWS CLI, eliminating the need for SSH keys, open inbound ports, or direct internet access to the instance.

Key Benefits of AWS Session Manager over Traditional SSH:

Enhanced Security: No Open Inbound Ports

• Perhaps the most significant advantage is that Session Manager does not require opening Port 22 or any other inbound ports on your instance’s security group. All communication happens over Port 443 (HTTPS outbound), making your instances inherently more secure by reducing the attack surface.
• No SSH keys to manage, store, or rotate. Access is managed purely through IAM policies.
• Works seamlessly with instances in private subnets without needing a bastion host or VPN.

Centralized Access Control with IAM:

• Access to EC2 instances is governed by AWS Identity and Access Management (IAM) policies. You can define precise permissions, specifying which users or roles can connect to which instances, and even restrict command execution.
• This provides granular access control and aligns perfectly with the principle of least privilege.

Comprehensive Auditing and Logging:

• Session Manager can automatically log all commands executed during a session to Amazon CloudWatch Logs or Amazon S3, providing a complete audit trail for compliance and security monitoring.
• Session data can also be encrypted using AWS KMS for an additional layer of security.

Simplified Management and Scalability:

• Start and end sessions directly from the AWS Management Console or AWS CLI with a single click or command.
• Manage access to hundreds or thousands of instances from a single pane of glass.
• No need to install or configure SSH clients on local machines.

Cross-Platform Support:

• Session Manager supports both Linux and Windows instances, providing a unified access method across your diverse fleet.

Cost-Effective:

• Session Manager is free for interactive sessions, and you only pay for standard AWS services like CloudWatch Logs or S3 if you enable logging. This eliminates the cost and complexity of maintaining bastion hosts.

Prerequisites for Using AWS Session Manager

Before you can start a session, ensure the following prerequisites are met:

1. SSM Agent Installation:

• The AWS Systems Manager Agent (SSM Agent) must be installed and running on your EC2 instance.
• Good news: Many Amazon Machine Images (AMIs) come with the SSM Agent pre-installed, including:
  • Amazon Linux 2
  • Amazon Linux 2023 (AL2023)
  • Ubuntu Server 16.04, 18.04, 20.04, 22.04 LTS
  • Windows Server 2012 R2 and later
• For other operating systems or older AMIs, you might need to manually install the SSM Agent. Refer to the official AWS documentation for installation instructions for your specific OS.

2. IAM Instance Profile (Role for EC2):

• Your EC2 instance needs an IAM instance profile attached with permissions that allow the SSM Agent to communicate with the Systems Manager service.
• The simplest way to do this is to attach the AWS-managed policy AmazonSSMManagedInstanceCore to the IAM role associated with your EC2 instance. This policy grants the necessary permissions for SSM Agent to register and operate.

3. Network Connectivity:

• Your EC2 instance needs outbound connectivity to the AWS Systems Manager endpoints over Port 443 (HTTPS). This is typically handled by your VPC’s default route to an Internet Gateway (IGW) or a NAT Gateway/NAT instance for instances in private subnets.
• For enhanced security and to keep traffic within your AWS network, you can configure VPC Endpoints for Systems Manager (ssm), Session Manager (ssmmessages), and EC2 Messages (ec2messages). This eliminates the need for an Internet Gateway or NAT Gateway for SSM traffic.

4. IAM Permissions for Users/Roles:

• The IAM user or role attempting to initiate the session must have permissions to start and terminate sessions.
• A common approach is to grant the ssm:StartSession and ssm:TerminateSession actions. You can further refine this by specifying which instances (based on tags or instance IDs) the user can connect to.
• Example Policy Snippet (attach to the IAM user/role who will start sessions):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:StartSession",
                "ssm:TerminateSession",
                "ssm:ResumeSession"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:instance/i-0123456789abcdef0",
                "arn:aws:ssm:*:*:document/SSM-SessionManagerRunShell"
            ]
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:StartSession",
                "ssm:TerminateSession",
                "ssm:ResumeSession"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:instance/i-0123456789abcdef0",
                "arn:aws:ssm:*:*:document/SSM-SessionManagerRunShell"
            ]
        }
    ]
}

(Replace i-0123456789abcdef0 with specific instance IDs or use * for all instances, though * is generally not recommended for production environments – prefer tagging or specific IDs for least privilege.)

5. Session Manager Plugin for AWS CLI (Optional):

• If you plan to connect via the AWS CLI instead of the console, you’ll need to install the Session Manager plugin for the AWS CLI on your local machine. This is a simple one-time installation.

Step-by-Step Setup and Connection

Let’s walk through the process to enable AWS Systems Manager Session Manager for an EC2 instance and connect to it.

Step 1: Launch/Verify EC2 Instance and SSM Agent

Ensure you have an EC2 instance running. For this tutorial, we recommend using an AMI with the SSM Agent pre-installed (e.g., Amazon Linux 2, Ubuntu Server).

• If launching a new instance: Select an appropriate AMI. The SSM Agent will be installed by default.
• If using an existing instance:
  1. Verify the SSM Agent is installed and running. For Linux, you can SSH into it (if still configured for SSH) and run: sudo systemctl status amazon-ssm-agent (for systemd) or sudo service amazon-ssm-agent status (for sysvinit). For Windows, check services.
  2. If not installed, follow the AWS documentation to manually install the SSM Agent for your OS.

Step 2: Create and Attach an IAM Role (Instance Profile) to the EC2 Instance

This is the most critical step for allowing your instance to be “managed” by Systems Manager.

1. Create an IAM Role:
   • Open the AWS Management Console and navigate to IAM.
   • In the left navigation pane, choose Roles, then click “Create role.”
   • For “Trusted entity type,” select “AWS service.”
   • For “Use case,” select “EC2” from the dropdown, then click “Next.”
   • In the “Add permissions” section, search for AmazonSSMManagedInstanceCore.
   • Select the checkbox next to AmazonSSMManagedInstanceCore policy. This policy grants the necessary permissions for your EC2 instance to communicate with the Systems Manager service.
   • Click “Next.”
   • Give your role a descriptive name (e.g., EC2SessionManagerRole). Add an optional description.
   • Click “Create role.”
2. Attach the IAM Role to your EC2 Instance:
   • Go to the EC2 console.Select the EC2 instance you want to manage.Click on “Actions” -> “Security” -> “Modify IAM role.”From the “IAM role” dropdown, select the role you just created (e.g., EC2SessionManagerRole).Click “Update IAM role.”

Self-Correction/Troubleshooting: After attaching the role, it might take a few minutes for the SSM Agent on the instance to pick up the new permissions and register as a managed instance. You can check the “Fleet Manager” section under AWS Systems Manager to see if your instance appears as a “Managed node.” If not, sometimes restarting the SSM Agent (or the instance) can expedite this.

Infographic Concept: >> IAM Role Attachment Flow

Step 3: Configure IAM Permissions for Users/Roles (Who Will Connect)

Now, ensure that the IAM user or role you are using to access the AWS console (or CLI) has the necessary permissions to initiate sessions.

1. Attach Policy to User/Role:
   • Go to the IAM console.
   • Navigate to Users or Roles (depending on whether you’re using an IAM user or assuming an IAM role).
   • Select the specific user or role that will be starting Session Manager sessions.
   • Click “Add permissions” or “Attach policies directly”.
   • Search for AmazonSSMManagedInstanceCore (if you want the user to have broad SSM access) or create a custom policy.
   • For least privilege, it’s better to create a custom IAM policy with specific permissions like:
     • ssm:StartSession
     • ssm:TerminateSession
     • ssm:ResumeSession
     • ssm:DescribeInstances (to see available instances)
     • You can restrict Resource to specific instance ARNs or tags if needed.
   • Attach this policy to your user/role.

Step 4: (Optional) Install Session Manager Plugin for AWS CLI

If you prefer to connect via your local terminal using the AWS CLI, you’ll need the plugin.

1. Download and Install: Follow the official AWS documentation for installing the Session Manager plugin on your specific operating system (Windows, macOS, Linux). It’s typically a simple download and installation command.
   • Link to AWS Documentation: Install the Session Manager plugin for the AWS CLI (You should embed this or similar official links on your actual site).
2. Verify Installation: After installation, open your terminal and run: session-manager-plugin (or the equivalent command for your OS). If it shows usage information, the plugin is installed correctly.

Step 5: Start a Session!

You can now connect to your EC2 instance using Session Manager.

Method 1: AWS Management Console (Browser-based Shell)

1. Navigate to Systems Manager: Open the AWS Management Console and search for “Systems Manager.”
2. Session Manager: In the left navigation pane, under “Node Management” (or “Instance Management”), choose “Session Manager.”
3. Start Session: Click the orange “Start session” button.
4. Select Instance: You will see a list of your managed instances. Select the EC2 instance you configured in the previous steps.
5. Connect: Click “Start session” again.

A new browser tab will open, presenting you with a command-line shell directly connected to your EC2 instance. You can now execute commands as if you were SSHed in, all without any SSH keys or open Port 22.

Method 2: AWS CLI

1. Open Terminal: Ensure you have the AWS CLI configured with credentials that have the necessary Session Manager permissions.
2. Start Session: Use the following command, replacing <your-instance-id> with the actual ID of your EC2 instance:

aws ssm start-session --target <your-instance-id>

aws ssm start-session --target <your-instance-id>

3. Your local terminal will connect directly to the instance, providing an interactive shell.

Troubleshooting Tips if Your Instance Isn’t Visible/Connectable:

• SSM Agent Status: Is the SSM Agent running on the instance? Check its logs (/var/log/amazon/ssm/amazon-ssm-agent.log on Linux).
• IAM Role Attached? Double-check that the AmazonSSMManagedInstanceCore policy is attached to the IAM role of your EC2 instance.
• Outbound Connectivity: Does your instance’s security group and network ACLs allow outbound HTTPS (Port 443) traffic to the internet (or to VPC Endpoints if configured)?
• Managed Instances List: Check the “Fleet Manager” section in Systems Manager to see if your instance is listed as a “Managed node.” If it says “Connection Lost” or isn’t listed, it usually points to an SSM Agent or permissions issue.

Advanced Session Manager Features and Best Practices

Once you’re comfortable with basic connections, explore these advanced features for even greater security, control, and auditability:

1. Session Logging to S3 and CloudWatch Logs:

• Enable session logging in the Session Manager preferences. This records all commands executed and session output.
• This is invaluable for auditing, compliance, and troubleshooting.

2. KMS Encryption for Session Data:

• Encrypt session data at rest using AWS Key Management Service (KMS). This ensures that even if logged session data is accessed, it remains encrypted.

3. Run As Support:

• Specify which OS user (ssm-user by default, but you can configure others) the session should run as on Linux/macOS instances. This allows you to enforce least privilege at the operating system level.

4. Session Preferences:

• Configure global preferences like idle session timeout, maximum session duration, and shell profiles to standardize session behavior across your environment.

5. VPC Endpoints:

• For a production environment, strongly consider configuring VPC Endpoints for Systems Manager. This routes all Session Manager traffic through your AWS network rather than the public internet, further enhancing security and potentially reducing data transfer costs.

6. Granular IAM Policies:

• Beyond AmazonSSMManagedInstanceCore, create custom IAM policies to define very specific permissions. For example, allow a user to only connect to instances with a certain tag (e.g., Environment:Dev) or only to specific instance IDs.

7. Multi-Factor Authentication (MFA):

• Always enforce MFA for IAM users and roles that have access to initiate Session Manager sessions.

8. Automated Patching and Management:

• Remember that Session Manager is part of AWS Systems Manager. Leverage other SSM capabilities like Patch Manager for automated OS patching, Run Command for executing scripts across fleets, and Parameter Store for secure configuration management. These often build on the same SSM Agent and IAM setup.

Conclusion: Modernizing Your EC2 Access

AWS Systems Manager Session Manager is a game-changer for EC2 instance access. It addresses critical security concerns, simplifies operations, and provides robust auditing capabilities that traditional SSH struggles to match. By eliminating the need for SSH keys, open Port 22, and bastion hosts, Session Manager empowers you to establish a more secure, scalable, and manageable remote access solution for your cloud infrastructure.

Adopting Session Manager aligns with AWS security best practices and moves you towards a more automated and secure cloud environment. Start implementing it today to experience a streamlined and safer way to manage your AWS EC2 fleet.

🚀 Explore Popular Learning Tracks

Learn Python

Learn AWS

Learn Git & GitHub