In continuation of our DevOps blog series, we will explore how HashiCorp Vault SSH CA Dynamic Secrets Engine can be set up to manage developer access to instances.

We will walk you through how to set up the SSH CA Secrets Engine. We also provide fine-grained secure access to your team with two EC2 instances – development and production.

The Vault SSH secrets management provides secure authentication and authorization for access to machines via the SSH protocol. The Vault SSH secrets engine helps manage access to machine infrastructure, providing several ways to issue SSH credentials.

The Vault SSH secrets management supports the following modes.

  • Signed SSH Certificates
  • One-time SSH Passwords
We will talk about the One-time SSH Passwords (SSH OTP) in the next blog.

Problems

Managing SSH keys for hundreds of servers and users can prove to be an overwhelming task for the infrastructure operations team. Server access management is maintained with SSH keys across VMs that are up to date for each user. Managing these keys is even more difficult because certain users should not have access to certain VMs. To make matters worse, access must be revoked as soon as a user leaves. These problems can all lead to error-prone processes and are handled either manually, via bash scripts, config management, or through some other solution.

Imagine a Developer starts working in your company. We would like to give them access to a subset of machines based on their role.  At the same time, we wouldn’t want them to worry about managing the associated (long-lived) SSH key for the duration of their project involvement. Instead of managing a unique private key for each individual developer, we can set up Vault as an SSH CA. These create signed certificates for the contractors to SSH to target hosts. This simplifies the server access management of trusted certificates on hosts down to one SSH CA public key per role (contractors in this case) on a target host instead of an SSH key per user. See the below figure for more understanding.

Vault’s dynamic SSH keys can also be short-lived. This feature is built into OpenSSH and allows us to keep access windows to resources time-boxed. So instead of managing indefinitely lived SSH keys, we can just give out SSH keys that are good for say an hour (or less!). If users want to re-login to a destination box after key expiration, they must first re-authenticate to vault and request the generation of a new key for SSH (which the Vault audits).

How it works?

SSH leverages asymmetric public key cryptography to authenticate clients to hosts and vice-versa.

We can set up Vault to act as the Certificate Authority (CA) for our SSH certificates. In the case of client signing, we can distribute the CA public key to hosts once configured. We want users to be able to SSH into. That public key is added as a trusted user key in the destination box’s sshd configuration. Once users log in, sshd will verify access by checking the user’s key signatures against the trusted CA key.

We can take things a step further by creating multiple “roles” in Vault for restricting access to subsets of machines. For example, we may set up a web prod role and a development prod role. Each role has a separate CA public and private key. We then distribute the public keys to the proper destination machines. This greatly simplifies key management as we only need to manage the CA key instead of individual keys for every user.

In this scenario, we are going to set up Vault to sign SSH keys using an internal Certificate Authority (CA). We will configure the SSH secrets engine and create a CA within the vault.  We will then configure an SSH server to trust the CA key we just created. Finally, we will attempt to SSH using a private key, and a public key signed by Vault SSH CA.

Prerequisites

This guide assumes a fixed setup as given below:
  • A Vault Server
  • Two EC2 instances with Port 22 opened in the security group.
  • An SSH client machine for authentication.
  • Install Vault CLI on the client server
  • The client system must be able to reach the Vault server and the OpenSSH server.
  • The Vault user should have the capability to enable secrets and write policies to the Vault server.
 
Step 1: Export VAULT_ADDR in the client system.

This will be your vault address.

You can add the following to ~/.bashrc or ~/.bashprofile if you are using a macOS to make it persistent.

export VAULT_ADDR=”http://vault_server:8200

Step 2: Define User Policy

Create a file with your preferred text editor

vim ssh-signed-cert-user-policy.hcl
Add the following to ssh-signed-cert-user-policy.hcl and save the file.

path “ssh/sign/*” {

  capabilities = [ “create”, “read”, “update”, “list” ]

}

Run the following command to write the policy to the vault.
vault policy write ssh-signed-cert-user-policy ssh-signed-cert-user-policy.hcl
 
Step 3: Enable and configure the SSH CA secrets engine:
Enable the SSH secrets engine and mount it in an arbitrary path. This path will be used to sign Client SSH keys.
vault secrets enable ssh
Generate new keys in that path. You can also import keys from your existing PKI, or a Vault PKI engine.
vault write ssh/config/ca generate_signing_key=true
 
Step 4: Add the public key to the TARGET_HOST’s SSH Configuration.
curl -s http://vault_server:8200/v1/ssh/config/ca –header “X-Vault-Token: <YOUR_VAULT_TOKEN>” | jq -r ‘.data.public_key’ | sudo tee /etc/ssh/trusted-user-ca-keys.pem
 
Step 5: Add the path where the public key contents are stored to the SSH configuration file as the TrustedUserCAKeys option.
echo “TrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pem” | sudo tee -a /etc/ssh/sshd_config
Restart sshd service
sudo systemctl restart ssh
 
Step 6: Create a role in vault for signing client keys
Create a JSON file with required parameters

cat >> certsigner-role.json <<EOF

{

“allow_user_certificates”: true,

“allowed_users”: “*”,

“default_extensions”: [

     {

         “permit-pty”: “”

     }

],

“key_type”: “ca”,

“default_user”: “ubuntu”,

“ttl”: “30m0s”,

}

EOF

Write the role to vault with the created JSON file
vault write ssh/roles/clientrole @certsigner-role.json
 
Step 7: Client Authentication
Generate an SSH Key
ssh-keygen -t rsa -N “” -C “Vault” -f ~/.ssh/client_rsa_key
Ask Vault to Sign your key
vault write ssh/sign/clientrole \ public_key=@$HOME/.ssh/client_rsa_key.pub
Save the resulting signed public key to your local disk
vault write -field=signed_key ssh/sign/clientrole \ public_key=@$HOME/.ssh/client_rsa_key.pub > signed-cert.pub
(Optional) View enabled extensions, principals, and metadata of the signed key.
ssh-keygen -Lf signed-cert.pub
SSH to the Host Machine
ssh -i signed-cert.pub -i ~/.ssh/vault_rsa ubuntu@<TARGET_HOST_IP_ADDRESS>
 

Next Steps

Vault is a great secrets management engine with deep integration to a lot of tools including your cloud accounts. We will cover more of these use cases in the near future.

We would love your feedback and hear more about how you have improved developer productivity. If this is something you want to adopt in your environment, hit us up on devops@pacewisdom.com

Author – Shibily Shukoor (DevOps Engineer at Pacewisdom)
By admin
...........................................................

Leave a comment

Your email address will not be published. Required fields are marked *

RECENT BLOGS

Sustainable IT for a Greener Future

Sustainable IT for a Greener Future

Software Infrastructure for 5G

Software Infrastructure for 5G

How Pace Wisdom Solution Helps in Agricultural Advancement through App Development

How Pace Wisdom Solution Helps in Agricultural Advancement through App Development

A curation of interesting IT trends of 2021

A curation of interesting IT trends of 2021

Gamification of Software Development

Gamification of Software Development

IoT – The backbone of smart city projects

IoT – The backbone of smart city projects

Sales analytics – A growth accelerator

Sales analytics – A growth accelerator

Technology in Clinical Trials

Technology in Clinical Trials

Benefits of a Virtual CTO

Benefits of a Virtual CTO

What is Digital Transformation?

What is Digital Transformation?

IT outsourcing in the digital age

IT outsourcing in the digital age

Hiring Software Developers For Startups

Hiring Software Developers For Startups

Pace Wisdom Stands With HESA Global’s Fight Against Hunger

Pace Wisdom Stands With HESA Global’s Fight Against Hunger

Flutter Framework for your next App development Project?

Flutter Framework for your next App development Project?

Implementing Effective And Simple BCP In Pace Wisdom During COVID-19

Implementing Effective And Simple BCP In Pace Wisdom During COVID-19

Docker Fundamentals – 1

Docker Fundamentals – 1

DevOps at Pace Wisdom Solutions: Gitlab CI/CD (Part 1)

DevOps at Pace Wisdom Solutions: Gitlab CI/CD (Part 1)

The Next Revolution – Blockchain Supercharged With AI

The Next Revolution – Blockchain Supercharged With AI

Native Apps, Hybrid Apps or Cross Mobile Apps? Which To Choose?

Native Apps, Hybrid Apps or Cross Mobile Apps? Which To Choose?

FinTech Revolution – Transforming The Financial Sector

FinTech Revolution – Transforming The Financial Sector

Benefits Of The Cloud In IoT

Benefits Of The Cloud In IoT

Content-First Design: A New Web Order

Content-First Design: A New Web Order

Boosting UX With Visual Hierarchy

Boosting UX With Visual Hierarchy

Software Testing In IoT Era – Prepping Up For A New Ballgame

Software Testing In IoT Era – Prepping Up For A New Ballgame

How To Deliver Better UI/UX Experience

How To Deliver Better UI/UX Experience

Emerging Cyber Threats To Be Aware Of

Emerging Cyber Threats To Be Aware Of

Payments Industry Technology Trends in 2019

Payments Industry Technology Trends in 2019

E-commerce Services Which Will Reign in 2019

E-commerce Services Which Will Reign in 2019

Digital Healthcare Trends for 2019

Digital Healthcare Trends for 2019

IoT Devices We Will See in 2019

IoT Devices We Will See in 2019

Impact of 5G Network on Mobile Apps

Impact of 5G Network on Mobile Apps

Future Trends In Logistics And Supply Chain

Future Trends In Logistics And Supply Chain

Exploring Blockchain-Based Data Security

Exploring Blockchain-Based Data Security

Metrics To Make Software Development Right

Metrics To Make Software Development Right

Future Trends In Software Development That Will Dominate 2019

Future Trends In Software Development That Will Dominate 2019

How AI helps in the growth of E-Commerce?

How AI helps in the growth of E-Commerce?

Building CodeIgniter Multi Language Website  

Building CodeIgniter Multi Language Website  

Uses of Python Programming Language

Uses of Python Programming Language

Build progressive web apps with Ionic framework

Build progressive web apps with Ionic framework

Angular versus React JS. Which would you choose and why?  

Angular versus React JS. Which would you choose and why?  

Ionic and AngularJS superpowers for Mobile App Development

Ionic and AngularJS superpowers for Mobile App Development

Hybrid mobile app development using Onsen UI  

Hybrid mobile app development using Onsen UI  

Build mobile app with Ionic

Build mobile app with Ionic

Build the amazing native app with an Ionic framework programming language

Build the amazing native app with an Ionic framework programming language

PhoneGap Mobile app Development  

PhoneGap Mobile app Development  

Why is Node.js Better than Java and .NET?

Why is Node.js Better than Java and .NET?

How Artificial Intelligence is Driving Mobile App Personalization

How Artificial Intelligence is Driving Mobile App Personalization

Gaming mobile app development

Gaming mobile app development

A beginners’ guide to developing virtual reality apps.docx

A beginners’ guide to developing virtual reality apps.docx

4 best programming languages suited for creating VR applications

4 best programming languages suited for creating VR applications

Top 5 Editors for React Native Visual IDE Mobile App Development

Top 5 Editors for React Native Visual IDE Mobile App Development

AI in the field of Education / HealthCare / Hospitality

AI in the field of Education / HealthCare / Hospitality

Google Analytics vs Firebase Analytics for your App

Google Analytics vs Firebase Analytics for your App

Things that you are probably missing out on your E-commerce App

Things that you are probably missing out on your E-commerce App

Amazon SNS vs Other Push Notification Services

Amazon SNS vs Other Push Notification Services

The Do’s and Don’ts for Building Better Android Apps

The Do’s and Don’ts for Building Better Android Apps

Six Reasons to implement ERP Software in Business

Six Reasons to implement ERP Software in Business

Can React Native be the efficacious successor of Hybrid App?

Can React Native be the efficacious successor of Hybrid App?

Everything to Know about AngularJs and ReactJs

Everything to Know about AngularJs and ReactJs

How to write a Project Brief that truly works for everyone

How to write a Project Brief that truly works for everyone

Designing an API, Tips & Tricks

Designing an API, Tips & Tricks

The Biggest Blunders one should avoid while Pre and Post Launching an App

The Biggest Blunders one should avoid while Pre and Post Launching an App

An insight of the promising future of Enterprise Apps

An insight of the promising future of Enterprise Apps

How IoT and Big data are solving problems in the Educational ecosystem?

How IoT and Big data are solving problems in the Educational ecosystem?

Xcode: Setting up a Colour Style Guide for a project

Xcode: Setting up a Colour Style Guide for a project

IoT Tech Accelerates Among Manufacturers: Survey

IoT Tech Accelerates Among Manufacturers: Survey

A guide for game app developers

A guide for game app developers

How Hybrid Mobile Apps can benefit your small business?

How Hybrid Mobile Apps can benefit your small business?

5 Important Considerations for Enterprise Mobile App Development

5 Important Considerations for Enterprise Mobile App Development

How to Change Your Career from Graphic Design to UX Design?

How to Change Your Career from Graphic Design to UX Design?

Planning for a Successful iPhone App

Planning for a Successful iPhone App

Tips for Securely Moving Data to the Cloud

Tips for Securely Moving Data to the Cloud

Cloud computing challenges faced by companies, Enterprise & SME

Cloud computing challenges faced by companies, Enterprise & SME

Blockchain on Healthcare

Blockchain on Healthcare

Seeking an IoT platform? Know what to look for

Seeking an IoT platform? Know what to look for

How Cloud Computing is Powering Indian Start Ups

How Cloud Computing is Powering Indian Start Ups

Core Data – Object Graph Management & Persistence Framework

Core Data – Object Graph Management & Persistence Framework

Do you really need the Hamburger?

Do you really need the Hamburger?

ABC of iOS App Development

ABC of iOS App Development

Digital Services for tomorrow’s enterprise
Explore more