Hey I have been working on a solution to prevent secrets from committing to VCS repos, so far have prevented 10+ AWS keys from accidentally being committed. Github has an offering but it's very costly for our team. Does anyone of you want to try out ?
1) secrets in code are a symptom of broken dev culture not just tooling. fix the culture first, train devs early on using environment variables and iam roles. make secrets invisible to code.
2( relying on gitignore or vaults alone is band-aid. sdk picks up creds from environment or home dir automatically if you use aws properly.
3) automate secret scans in ci but dont trust them blindly, human reviews and rotating keys asap is still critical.
4) biggest risk is devs rushing and skipping processes, so build workflows that make mistakes obvious and costly to push.
5) at scale, even perfect tech fails without good process + education. focus there and tools become backup not main defense.
this mindset saves time and downtime. secrets leaks cost more than any fancy tool subscription.
My team has faced issues like this and other than ensuring any secrets are removed from your code and stored in a .gitignore'd config file (if you really need them to live so close to the codebase in the first place), you need to prioritize that everything goes through proper PRs, privacy/access is properly configured, and any compromised secrets are rotated immediately. We have some tools like Snyk and Trufflehog but even those don't catch a lot of things - human review is best.
Why are AWS keys anywhere near your code in the first place?
For instance in Python, you initialize an object using
When you use IAM identity center, you get temporary access keys which you assign to environment variables and the keys are automatically picked up.Even if you use “aws configure” and have long lasting keys (don’t do that), your keys will be stored in your home directory, nowhere near your repository and still usable locally.
When running your code on AWS, whatever you are using to run it on will get permission from the IAM role attached to the Lambda, EC2, etc.
It tends to happen more on front end I think, especially since it's in the tutorial and many haven't been given the training on what to do better. Not really AWS, but even the trained ones will put it in a .local.properties file or something and then forget to add it to gitignore
There is no sample code from AWS that has you read access keys in your code from a .local file. The SDK automatically picks it up from your local environment when you run AWS configure to store them in your home directory or better when you get temporary credentials via the IAM identity center and store them in environment variables
You really just need to not forget to do that. Isn’t it that simple?
A less snarky answer, and why AWS is largely a non issue these days is because the secrets were designed out of code And are effectively provided as an integral part of the infrastructure which includes regular and reliable expiration and rotation. So any chance you get, design secrets in this way.
The only thing ever in code are references to the correct roles or secrets. Only ever references to the location of the secret. Get in the habit of this and the problem is drastically reduced and becomes something you don’t have to think about.
In a ideal world yes developers should care about these issues but developers need access to AWS keys to locally test integration with AWS services like SQS and Dynamo so access to micro service keys needs to be provided.
The problem occurs when they forget and commit, that key needs to be rotated which has caused downtimes in the past, or scrubbed which involves a messy fight with VCS support teams.
The problem is not just AWS, in general for third party integrations with platform like banks developers needs to test locally but they forget removing those keys. Each keys committed is a potential SOC2 / PCI non-compliance avenue.
You never need to have your access keys in your repository or read them explicitly from any properties file. The SDK will automatically get the keys from your home directory when you run your code.
The easy but less secure solution:
Store all secrets in one file WITHIN your local repo and add that one secrets file to something like a .gitignore file. Then validate the file is excluded using git status.
The more secure solution is to store all secrets within a secrets vault and access that vault from application logic on application restart, provided the application is a service that rarely restarts.
No that’s not more secure. You never need to read your secrets for AWS into your code. Your secrets never need to be anywhere near your repository in a vault nothing.
The SDK will pick them up from your local computers home directory or from the AWS environment when running on AWS.
perplexity offered me those:
https://github.com/awslabs/git-secrets
https://www.infracloud.io/blogs/prevent-secret-leaks-in-repo...
https://www.reddit.com/r/git/comments/1h1r0ep/best_practices...
In addition GitGuardian cost something around $220/year per developer which is not too bad
How big is your team? GitGuardian is free for team below 25 dev. And it will make sure secrets don't make it to your code and highlight any historical leaks too.
I’m a little confused, is this mostly for public repos? Because for internal repos you’ll catch it in code review and then just revoke the creds?
Revoking crews is not easy in general