SSH Config File Examples: How to Manage Multi-Host Connections

If you manage multiple remote servers, cloud instances, or virtual machines, you already know the frustration of repeatedly typing out complex SSH commands. Entering ssh [email protected] -p 2222 -i ~/.ssh/prod_rsa every time you need to check a production server is not only tedious but also prone to typos. Remembering custom ports, distinct identity keys, and exact IP addresses for dozens of staging, production, and backup instances significantly slows down your DevOps workflow.

Fortunately, OpenSSH provides a robust, built-in solution: the SSH config file. Rather than relying on bash history or memory, you can define standardized connection profiles in a single, local text file. This allows you to abstract away the complexity of network details, ports, and credentials.

With a properly configured SSH file, that long, complex command is reduced to a simple alias:

$ ssh prod-web
Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.8.0-1011-aws x86_64)

Last login: Mon Jun  1 10:23:45 2026 from 203.0.113.5
deploy-user@prod-web:~$ 

In this comprehensive guide, we'll dive deep into writing, optimizing, and organizing your SSH configuration file. From basic alias mapping to complex bastion proxy jumps and connection multiplexing, you'll learn how to master multi-host connections.

1. What is the SSH Config File and Where Does it Live?

The SSH client configuration is a plain-text file that lives within your user's home directory. Whenever you initiate an SSH connection, the SSH client reads this file to parse host-specific overrides and global defaults before attempting to connect.

Depending on your operating system, the configuration file is typically located here:

• macOS & Linux: ~/.ssh/config
• Windows (PowerShell/Command Prompt): C:\Users\YourUsername\.ssh\config (or simply $HOME\.ssh\config)

If the file does not exist, you can safely create an empty one.

The Importance of Strict Permissions

Because your SSH configuration can point directly to highly privileged identity files (private keys) and define network routing, the OpenSSH client enforces strict file permissions. If the permissions are too loose, the SSH client will aggressively refuse to parse the file, aborting the connection entirely.

In a Linux or macOS terminal, you must restrict read and write access so that only your user can interact with it. Anyone else on the system should be blocked.

$ ls -l ~/.ssh/config
-rw-rw-r-- 1 gbharat users 1230 Jun 1 10:00 /home/gbharat/.ssh/config

# Remove read/write access for groups and others
$ chmod 600 ~/.ssh/config

$ ls -l ~/.ssh/config
-rw------- 1 gbharat users 1230 Jun 1 10:01 /home/gbharat/.ssh/config

2. Core SSH Configuration Parameters

The SSH config file relies on a block-based structure. A configuration file consists of one or more Host declarations. The SSH client matches the alias you type against these `Host` blocks and applies the settings nested beneath them.

Here is a quick reference table of the most critical SSH parameters you will use daily:

3. Basic SSH Config Examples

Let's explore some foundational examples of how to map different types of servers in your config.

Example A: The Standard Web Server

Most web servers run SSH on a custom port to avoid automated bot scanning on port 22. Instead of supplying the port and key manually via command flags, you define it in the config:

Host web-staging
  HostName staging.mycompany.com
  User deploy
  Port 45022
  IdentityFile ~/.ssh/id_staging_ed25519

Example B: Managing Multiple GitHub Accounts

A frequent issue developers face is managing both a personal GitHub account and a corporate GitHub account on the same machine. By default, Git attempts to use the standard id_rsa or id_ed25519 key. You can use SSH config to alias GitHub into two different hosts:

# Personal GitHub Account
Host github.com-personal
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_personal

# Work GitHub Account
Host github.com-work
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_ed25519_work

When cloning a work repository, you modify the Git remote URL to match the alias: git clone [email protected]:mycompany/repo.git. The SSH client automatically applies the correct work identity key.

4. Advanced SSH Config Strategies

Once you understand the basics, you can leverage advanced OpenSSH features to dramatically optimize connection speed and security architecture.

Global Defaults with Wildcards

Instead of repeating the same options for every server (such as keep-alive intervals), you can define a wildcard block that applies to all connections. Note: SSH parses the file from top to bottom, applying the first matched parameter it sees. Always put specific hosts at the top and global wildcards at the bottom.

# Place this at the end of your ~/.ssh/config
Host *
  ServerAliveInterval 60
  ServerAliveCountMax 3
  AddKeysToAgent yes
  ForwardAgent no

Routing Through Bastion Hosts (ProxyJump)

In secure cloud environments (like AWS or GCP), internal application databases do not have public IP addresses. To access them, you must SSH into a hardened, public-facing "jump" or "bastion" server, and from there, SSH into the internal machine.

Historically, this required complex ProxyCommand netcat chains. Since OpenSSH 7.3, the ProxyJump directive handles this seamlessly.

# 1. Define the public Bastion host
Host secure-bastion
  HostName 203.0.113.15
  User ops-admin
  IdentityFile ~/.ssh/bastion_rsa

# 2. Define the internal Database host
Host internal-postgres
  HostName 10.0.3.40
  User dbadmin
  IdentityFile ~/.ssh/db_rsa
  ProxyJump secure-bastion

When you execute ssh internal-postgres, SSH automatically authenticates against `secure-bastion`, creates a secure TCP forwarding tunnel, and then authenticates against `internal-postgres`—all in one smooth step.

Speed Boost: SSH Multiplexing

If you run automation scripts or frequently open multiple terminal tabs to the same server, SSH multiplexing can reduce connection times from seconds to milliseconds. Multiplexing allows multiple SSH sessions to share a single, underlying TCP connection, avoiding the overhead of repeated cryptographic handshakes.

Host *
  ControlMaster auto
  ControlPath ~/.ssh/sockets/%r@%h-%p
  ControlPersist 10m

Make sure to create the ~/.ssh/sockets/ directory before enabling this, or SSH will fail to create the control sockets!

5. Troubleshooting Common SSH Errors

Even with a clean configuration file, network drops or permission errors can occur. Here are the most common errors you will encounter and how to debug them using the verbose flag (ssh -v).

Error: Bad owner or permissions

Bad owner or permissions on /home/user/.ssh/config
fatal: Bad owner or permissions on /home/user/.ssh/config

Cause: The configuration file or the directory itself has permissions that are too open.
Solution: Run chmod 600 ~/.ssh/config to lock down the file. Additionally, ensure the `.ssh` directory is secured with chmod 700 ~/.ssh.

Error: Permission denied (publickey)

[email protected]: Permission denied (publickey).

Cause: The server rejected your authentication attempt. This usually means the IdentityFile points to the wrong key, or the remote server does not have your public key inside its ~/.ssh/authorized_keys file.
Solution: Verify the `IdentityFile` path. Use ssh -v host to see exactly which keys are being offered to the server during the handshake. Ensure that ssh-add -l lists the required key if you are relying on an SSH agent.

Error: Too many authentication failures

Received disconnect from 198.51.100.24 port 22:2: Too many authentication failures

Cause: The SSH client sequentially tries every single key in your SSH agent or default directories. If the server only allows 3 attempts, and the correct key is 4th in the list, the server drops the connection before you can authenticate.
Solution: Add IdentitiesOnly yes to the specific `Host` block in your config. This forces SSH to only offer the exact key defined in `IdentityFile`, bypassing the agent's scattergun approach.

6. Best Practices for SSH Organization

• Use Include Statements: If your config file grows beyond 100 lines, break it apart. Use the Include config.d/* directive at the top of your config, and create separate files for different environments (e.g., ~/.ssh/config.d/production, ~/.ssh/config.d/personal).
• Hash Known Hosts: Set HashKnownHosts yes to obscure IP addresses and domains in your ~/.ssh/known_hosts file. This prevents an attacker who steals the file from easily seeing your entire infrastructure footprint.
• Avoid ForwardAgent when Possible: Agent forwarding (ForwardAgent yes) exposes your local SSH keys to the remote host. Only enable it on specific, trusted hosts, never globally. ProxyJump is almost always a safer alternative for bastion setups.

7. Integrating Local Configs with Ansible

The true power of a meticulously maintained local SSH configuration file reveals itself when integrating with infrastructure automation tools.

When you run an Ansible playbook, the underlying transport layer natively respects your ~/.ssh/config. This means your Ansible inventory files don't need complex variable declarations for ansible_ssh_private_key_file or ansible_port. You can simply list the host aliases:

[webservers]
prod-web
staging-web

[databases]
internal-postgres

If you need to rapidly scaffold infrastructure configs, match your host aliases inside our browser-only Ansible Inventory Generator to produce clean INI or YAML inventories that require zero manual SSH variable definitions.

Frequently Asked Questions

Where is the local SSH config file located?

In Linux and macOS, the SSH config file is located at ~/.ssh/config. In Windows, it resides in C:\Users\YourUsername\.ssh\config.

What permissions should my SSH config file have?

Your SSH configuration file must have strictly restricted permissions to prevent system security errors. Set it to read/write only for the owner by running chmod 600 ~/.ssh/config.

How do I connect to a server through a Bastion Host using SSH config?

Use the ProxyJump parameter. For example, configure ProxyJump bastion-alias beneath your target host block. The SSH client handles the intermediate TCP tunnel automatically.

Can I use multiple SSH keys for different GitHub accounts?

Yes. You can define multiple Host blocks (e.g., github.com-work and github.com-personal) that both point to HostName github.com, but specify different IdentityFile paths for each block.

What is the difference between ProxyJump and ProxyCommand?

ProxyCommand is the older, legacy method that relies on chaining external netcat (nc) commands to build a tunnel. ProxyJump (introduced in OpenSSH 7.3) is built natively into SSH, is much easier to read, and executes faster without external dependencies.

How can I speed up multiple SSH connections to the same host?

Enable SSH Multiplexing by adding the ControlMaster auto and ControlPath directives to your config file. This allows subsequent SSH connections to reuse the original, already-authenticated TCP socket, cutting connection times significantly.

How do I handle SSH timeouts and frozen terminals?

Firewalls and NAT routers often drop idle TCP connections without notifying the client, resulting in a frozen terminal. Use ServerAliveInterval 60 to force the SSH client to send a tiny ping every 60 seconds, keeping the connection active.

Does the order of rules in the SSH config matter?

Yes, absolutely. The SSH client parses the file from top to bottom and applies the first matched parameter it finds. For this reason, you should define specific Host blocks at the top of the file, and general wildcard blocks (like Host *) at the very bottom.

What happens if my SSH config file gets too large?

If you manage hundreds of servers, a single file becomes unmanageable. You can use the Include ~/.ssh/config.d/* directive at the top of your config to break it down into modular, environment-specific files (e.g., separating production, staging, and personal hosts).