If most security breaches start with (or involve) stolen credentials, why don’t bug bounty programs incentivize researchers to find these credentials before attackers?
Credentials are one of the most challenging vulnerability classes to work into a bug bounty program, In this post, we’ll dig into why and share an exciting new update from HackerOne that gets us one step closer to sanity, though there’s still a lot of unanswered questions the whole industry is sorting through.
For the sake of this post, we’ll roughly put credentials into 2 broad categories:
User credentials
Corporate credentials
End user application credentials
Machine identity (Server/SaaS/Cloud/DB) credentials
What servers use to authenticate to things
User credentials
The most common way individual user accounts are compromised is through user credentials, credential re-use, credential stuffing attacks.
Despite all of the password managers on the market, the average user still re-uses the same password across multiple services. When an attacker compromises a username and password from one organization, they will attempt to access other services using the same credential.
If you ever played World of Warcraft back in the day and had all your gold stolen, this was likely how it happened:
This situation is exacerbated by the fact that hundreds of companies have had their databases stolen and leaked online, resulting in the creation of vast datasets that encompass nearly every internet user, and their previous passwords.
It’s typically (but not always) off-limits to validate keys found in data breaches and submit them to bug bounties, due to the ethical gray area of what’s required to obtain these passwords. If threat actors can profit from stolen credentials via bug bounty programs, this creates a financial incentive to obtain stolen data.
Machine identity credentials
Machine identity credentials such as AWS keys, database passwords, encryption keys, and API keys are often exposed in a way that doesn’t require any type of shady stolen data to access. For example, data containing credentials uploaded to Postman is governed by Postman’s terms, which allows Postman to share those credentials in the Postman Terms of Service:
Similarly, code uploaded to Github containing secrets is governed by Github’s Terms of Service:
This makes this type of key much cleaner to report to a bug bounty at its surface. It might seem like this is case closed for these types of keys, unfortunately the trouble only starts at reporting.
Another typical bug bounty response to reporting a key
What should a security researcher do, if every time they submit a key, they get back an answer from the provider saying “This key is non sensitive” or “This key is not active." What’s worse is sometimes this actually lowers the researcher’s reputation/impact on Bounty platforms!
Is it okay for researchers to validate the key before reporting it? Is it okay to see what the key can access? What happens if the researcher’s assessment of impact is different from the triager?
Every day triagers contend with many different submissions, across multiple vulnerability classes. With thousands of potential key types, expecting triagers to perfectly assess the validity and impact of a leaked key is a tall order. But then, the burden of proof lies heavily on the researcher to “prove impact”. But how do you do that, in a way that’s safe?
Let’s start with the ugly: making the wrong move here can literally put you in jail. In 2019 two people who accessed Uber data from an AWS key they found on GitHub were sentenced. This is despite the fact Uber had paid them out as bug bounties.
Uber convicted hackers use AWS key to access data, and found guilty.
In 2016 a pair of “hackers” used an AWS key to download all of Uber’s backups, requested $100,000 dollars of bug bounty money, and kept a copy of the data they downloaded after being requested to delete it. This credential was also found in a private repository that was only accessed through credential stuffing in the first place.
How do we balance these factors?
On the one hand:
Busy triagers that don’t want to test each individual key, put the burden of impact and proof on the researcher.
On the other hand:
Researchers sometimes push things way too far: they use one credential to find another credential, download all production data, and retain access to the data after being requested to delete it.
One solution is to limit the extent of what security researchers can do with credentials to prove they’re active (ie. don’t access someone else’s data, just verify you can authenticate with it). Is this perfect? No. Triagers will still mark a reported credential as informational until the researcher proves sensitive data is accessible. But permitting researchers to test these keys, formally, in the program guidelines, gets us to a much better place.
A year ago, the justice department put out guidelines saying if the hacking you’re doing is “In Good Faith”, it’s not a violation of the Computer Fraud and Abuse act.
That’s not good enough. While we believe this provides some protection, we need specific guidelines outlining what in good faith actually means. Thankfully HackerOne recently updated their policies to clarify what you can do with these keys.
We had the privilege of chatting with them about their changes.
HackerOne’s guideline updates
The standard bug bounty (BB) program brief highlights in-scope targets and vulnerability classes to focus on, but it’s extremely rare to see “leaked credentials” mentioned anywhere on these briefs. In fact, language like “Any domain/property of <company> not listed in the targets section is out of scope” might even dissuade hackers from searching for leaked credentials on services like GitHub where credentials commonly leak out.
Fortunately, our experience suggests that BB platforms will usually accept a one-off report about a leaked credential, but it’s clear the focus has historically been on other types of vulnerabilities based on their classification system.
Take a look at what the CISO of HackerOne said, when asked about this challenge on their platform:
"We've had customers try to penalize researchers for "testing credentials". On the flip side, we've had customers rejecting findings because the impact isn't proven."-HackerOne representative Chris Evans
That’s why it was so exciting to read about HackerOne’s upcoming Platform Standards change.
HackerOne’s Update
As of April 2, Leaked Credentials officially become a standard vulnerability category across most HackerOne bug bounty programs. HackerOne recognized a gap between vulnerabilities impacting their clients and incentives within their hacker community. By including leaked credentials, hackers will be motivated (and legally protected) to report exposed secrets throughout most programs on their platform.
Importantly, HackerOne documented how hackers should validate credentials:
“Hackers should submit the leaked credentials to the program and should NOT test their validity beyond authenticating and then immediately deauthenticating - without exercising any functionality.“
Does this fix all the problems outlined above? No. There’s still areas where triagers will close reports for live keys saying not enough impact is demonstrated. Also none of this applies to stolen user credentials:
“There are many concerns about the provenance of credentials. For example, there's near universal agreement that it's not ok to buy dark web data in order to prove leaked credentials.” -HackerOne representative Chris Evans.
But within the scope of leaked machine credentials posted on platforms that actually license you to view them, it’s a lot more clear testing their validity (without downloading user data) is in good faith.
Thankfully there are awesome open source tools, like TruffleHog, that allow you to do exactly that: test the validity of credentials without downloading any data.
At Truffle Security, we’ve championed credential verification as the only way to truly reduce false positives in secret scanning. By making one non-state changing request to the service provider using the discovered credential, researchers can ensure a suspected secret is valid prior to reporting it.
Imagine having to identify a non-sensitive, non-state-changing URL for each token type. When using TruffleHog this becomes as simple as:
TruffleHog will never return or display sensitive data as part of its verification process, and it will never do anything stateful.
The team at TruffleHog has meticulously documented verification endpoints for the 800 different key types it can identify. And this number is constantly growing thanks to contributions from our community!
HackerOne’s guidelines explicitly recognize this method and encourage hackers to use tools that verify the authenticity of secrets (like TruffleHog) prior to submission. And this makes sense: the first thing a triager would do when receiving a report about a leaked credential is … verify the leaked credential.
Later on in HackerOne’s guidelines, they write “The severity should be based on the type of access a given leaked credential would grant an attacker…”.
This is great, in theory, but the details leave room for disagreement among the client, triager and hacker. If a hacker cannot validate the type of access a leaked secret grants an attacker, then they have to rely on the triager and client to accurately assess this impact. This process might not sit well with the hacker. But on the other hand, the client doesn’t want a hacker making hundreds of potentially state-changing HTTP requests to figure out what an API key can access. This seems like the status-quo for now and we’ll be interested to see how the triage process plays out.
What’s Next for Bug Bounty Hunting Leaked Credentials?
As a secret scanning company, we have lots of thoughts about how credential bug bounty programs could be improved, but here are just two.
The first is simple: create a leaked credential vulnerability standard similar to HackerOne. All other VDP and BB platforms should adopt leaked credential standards.
Second, organizations that give customers access to an API should encourage researchers to report leaked keys for their API. OpenAI’s program on BugCrowd is the best example that we’ve seen so far of this:
The OpenAI team recognized that ChatGPT’s popularity would mean their own users would accidentally expose API keys all over the internet. To prevent threat actors from using them (and their users from racking up huge bills), the OpenAI security team decided to incentivize researchers to find them.
Start Scanning for Credentials with TruffleHog
On HackerOne and want to start scanning for credentials? Check out TruffleHog.
