What is Git and why does it have secrets?
Git is a very popular source code control system. It’s also one of the largest sources of leaked secrets and API keys. Github, a source code sharing platform that uses git, stated in a blog post that over 1.7 million leaked secrets were detected in 2022 alone. Our own research into GitHub revealed 1,800 public pushes leak live API keys and passwords every day. Git repos often contain secrets due to sensitive key material hardcoded in code and configuration files (such as cloud API keys, SSH secret keys, database passwords, etc.).
What is TruffleHog and how does it work?
TruffleHog is an open source secrets scanning tool that detects over 800 different types of secrets in a variety of sources, such as git repositories, local files, AWS S3, Docker images and more. It utilizes detector modules built for a large range of secret formats, and extracts matching data from plaintext files as well as rich text documents like PDFs. Then, it verifies the secret by checking the credential against the actual SaaS provider’s APIs, if available.
TruffleHog installation instructions can be found here. And here’s a link to a git repository containing purposefully leaked keys, so you can test TruffleHog out. Now, let’s step through how to scan git for secrets.
Scanning a single Git repository
Remote repository
TruffleHog’s git subcommand is used for all Git related scans. To scan a remote repository, simply add the git repository’s URL as an argument. This works for any git repository host, be it GitHub, GitLab or any other self-hosted git servers.
Scan a Git repository remotely over HTTPS:
Scan a Git repository remotely over SSH:
By default, TruffleHog will verify all leaked secrets it identifies using dynamic requests (e.g. for AWS keys, the AWS API). Verification eliminates the vast majority of false positives, so security engineers and developers can focus on remediating true vulnerabilities and issues.
A verified AWS access key ID (in green) vs an unverified GitHub token (in white)
To only see verified results, add the --only-verified flag:
Local repository
To scan a local git repository, use the file:// URI prefix to point trufflehog to that git directory:
If the repo is in the current directory, use a relative path:
If the repo is in another directory, use an absolute path:
Scan specific branches or commits
The open-source version of TruffleHog provides command-line options to specify which branches or commits to scan. For example, if you want to only scan the dev branch, run the following command:
If you want to scan all changes in a new branch not yet merged into the main branch, run the following command:
Scanning Git cloud providers
TruffleHog has special commands for some source code control platforms that use Git. This is because leaked secrets can also be found in places like comments, issues and pull requests.
At the time of writing, TruffleHog open source provides github and gitlab commands to support pulling relevant git data from those APIs. Other git platforms such as BitBucket, CodeCommit, Gitea and Gerrit can be scanned using the normal git commands outlined above, with no support for platform specific data. Additionally, better native support for auto-enumerating repos in BitBucket and Gerrit are available through our enterprise offering.
GitHub secret scanning
Scanning a single GitHub repository
A single GitHub repository can be scanned by using the github subcommand with the --repo flag:
The above command will only scan the contents of the repository. Comments in issues and pull requests can be scanned by adding options --issue-comments and --pr-comments.
Found AWS keys in a GitHub issue
Scanning all repositories in a GitHub organization
TruffleHog supports scanning all repositories belonging to a GitHub organization. If the organization hosts a lot of repositories or has private repositories you want to scan, you can run an authenticated TruffleHog scan by passing a GitHub Personal Access Token (PAT) to the --token= flag.
Unauthenticated GitHub Organization Scan:
Authenticated GitHub Organization Scan:
Scanning the trufflesecurity organization for secrets on GitHub
GitLab secret scanning
Unlike scanning GitHub’s public repositories, GitLab requires users to authenticate before interacting with their API. As a result, all GitLab scanning requires users to pass an API token to TruffleHog via the --token flag.
Scanning all available GitLab repositories with an API token
Similar to GitHub, the GitLab API allows TruffleHog to enumerate all available repositories for a given API token. To scan all accessible GitLab repositories, run the following command:
If the GitLab server is self-hosted (instead of gitlab.com), the endpoint URL must be specified like so:
TruffleHog CI integration
TruffleHog can continuously scan a Git repository for newly introduced secrets via integration with CI (Continuous Integration) pipelines. This enables organizations to detect secrets automatically on new commits, without needing to run it manually.
TruffleHog can be integrated into CI via Github Actions, Circle CI, TravisCI and more. For additional details, see theTruffleHog enterprise documentation on Scanning in CI.
Lastly, if you’re looking for continuous monitoring and not just a one-off scan, or you need help deduplicating results, or are looking for a few more integrations than the open source version provides, check out our enterprise version: https://trufflesecurity.com/trufflehog/
