How to Scan AWS S3 for Secrets
What is AWS S3 and why does it have secrets?
Amazon Simple Storage Service (S3) is Amazon Web Service's (AWS) cloud storage offering. Developers use S3’s scalable object storage to store all types of data including source code, configuration files, database backups and more. Unfortunately, S3 buckets are often misconfigured and grant public access, allowing anyone to view files inside.
Researchers often find sensitive keys inside exposed S3 buckets. A couple years ago, Truffle Security discovered 4000 Public S3 buckets containing sensitive .env and .credentials files, with an average of 2.5 exposed secrets per file.
What is TruffleHog and how does it work?
TruffleHog is an open source secrets scanning tool that detects over 800 different types of secrets in a variety of sources, such as git repositories, local files, AWS S3, Docker images and more. It utilizes detector modules built for a large range of secret formats, and extracts matching data from plaintext files as well as rich text documents like PDFs. Then, it verifies the secret by checking the credential against the actual SaaS provider’s APIs, if available.
TruffleHog installation instructions can be found here. Now, let’s step through how to scan S3 for secrets.
Scanning a S3 bucket
TruffleHog natively scans s3 buckets. In fact, TruffleHog uses the AWS SDK to automatically authenticate with locally stored AWS credentials in ~/.aws/credentials or EC2 instance metadata.
To scan a bucket, simply pass the name of the bucket to the --bucket argument like so:
If your AWS account needs to assume a role in order to access the S3 bucket, you can do that with the --role-arn flag:
If you do not have AWS credentials stored locally, you can pass them directly to TruffleHog:
By default, TruffleHog will verify all leaked secrets it identifies using dynamic requests. Verification eliminates the vast majority of false positives, so security engineers and developers can focus on remediating true vulnerabilities and issues.
To only display verified results, add the --only-verified flag:
For a detailed explanation of how to most securely architect AWS permissions for scanning S3 buckets with TruffleHog, please see our enterprise S3 documentation (under the heading S3 with AWS IAM Role Assumption).
Lastly, if you’re looking for continuous secret monitoring of your S3 bucket data and not just a one-off scan, check out our enterprise version: https://trufflesecurity.com/trufflehog/.
