December 8, 2024 4 min to read

Passwords, Passwords, Passwords

Password 101

Proper password and secrets management is not just a recommendation—it’s a critical necessity, especially for all IT engineers who handle sensitive credentials across multiple systems and environments, for multiple clients, projects and companies.

In this blog, we will talk about common scenarios I have seen over the last couple of years working as a DevOps/Cloud/SRE engineer.

Common Password Management Mistakes

1. Exposing Credentials in Public Repositories

One of the most dangerous mistakes engineers make is accidentally committing sensitive credentials to public repositories. Here’s a real-world example:

# DON'T DO THIS - Example of what not to commit to Git
aws:
  access_key_id: AKIAIOSFODNN7EXAMPLE
  secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

This type of exposure can lead to immediate security breaches, with bad actors automatically scanning GitHub for such credentials. I can tell multiple stories on where we saw multiple instances spinning up in multiple AWS Regions because of this simple error.

2. Hardcoding Credentials in Application Code

# Bad practice
database_password = "super_secret_password123"
connection = mysql.connector.connect(user="admin", password=database_password)

Another fun one, and it seems to be more common than we would think of. In this scenario developers or DevOps working on Infrastructure as code can commit and push these credentials into the live applications and then make them accessible to every single person that has access to the application.

Best Practices for Password Management

• Use Secret Management Tools:- HashiCorp Vault- AWS Secrets Manager- Azure Key Vault

• Implement Environment Variables:

    # Good practice
    import os
    database_password = os.environ.get('DB_PASSWORD')
    connection = mysql.connector.connect(user="admin", password=database_password)
  

• Use .gitignore Properly:

    # Example .gitignore entries
    .env
    credentials.json
    **/secrets/*
  

Remember, .gitignore is you best friend!

Security Measures for DevOps Engineers

1. Implement Rotation Policies: Set up automatic credential rotation for all production secrets.
2. Use Multi-Factor Authentication: Enable MFA for all service accounts and user access.
3. Audit Regularly: Implement logging and regular auditing of access to secrets.
4. Practice Least Privilege: Only grant the minimum necessary permissions to services and users.

Emergency Response Plan

If credentials are accidentally exposed:

1. Immediately invalidate the exposed credentials
2. Rotate all potentially compromised secrets
3. Review access logs for unauthorized usage
4. Document the incident and update security procedures

Tools and Solutions

Monitoring and Compliance

There are multiple tools out there that can scan multiple services, applications, and repositories in search for secrets that shouldn’t be there. Implementing automated scanning tools to detect potential security issues is a very common practice nowadays in the CI/CD tools and pipelines of companies.

# Example: Using git-secrets to prevent AWS credential commits
git secrets --register-aws
git secrets --install
git secrets --scan-history

Conclusion

Proper password management is a fundamental aspect of DevOps security. By following these best practices and implementing appropriate tools, you can significantly reduce the risk of security breaches and maintain a robust security posture in your company and save tons of time and money if these credentials are exposed and granted to unauthorized entities.

Remember: Security is not a one-time setup but a continuous process that requires constant attention and updates!

Build On!