GitHub Real-time

10 min

GitHub Real-time

Edition: Enterprise only

The GitHub Real-time integration scans GitHub push events as they occur, detecting credentials and other sensitive data the moment code is pushed. For non-real-time scanning of repositories, gists, issues, and pull requests, see the GitHubGitHub source instead.

Dependencies (Self-Hosted Deployment Only)

This setup requires specific tools for effective operation. Git is essential for repository management, while rpm2cpio, binutils, and cpio are necessary for extracting files from .rpm and .deb package formats.

Git: For cloning repositories.

rpm2cpio: To extract content from RPM packages.

binutils: Includes the "ar" tool, crucial for extracting contents from .deb files.

cpio: A versatile file archiver utility, compatible with various archive formats including .rpm and .deb.

Installing Dependencies on Ubuntu

To install these dependencies on an Ubuntu system, follow these steps:

Open a terminal.

Update your package lists to ensure you get the latest version available:

$ sudo apt update

Install the required packages:

sudo apt install git rpm2cpio binutils cpio

Configuration

The GitHub Real-time integration is configured via a local configuration file. Web configuration in TruffleHog is not available for this source.

GitHub Real-time requires setup in two places:

A local configuration file (covered in Local configuration below).

A webhook in GitHub that sends push events to TruffleHog (covered in Configure GitHub to send push events below).

Local configuration

GitHub Real-time supports the same authentication methods as the GitHubGitHub source — token-based and GitHub App. For details on creating tokens or GitHub Apps, see the GitHub source documentation; the rest of this section covers what's specific to Real-time.

Both modes require a webhookSecret — a high-entropy value shared between TruffleHog and GitHub that's used to validate incoming push event deliveries.

Access token

Use this with a GitHub personal access token.

sources: - connection: "@type": type.googleapis.com/sources.GitHubRealtime token: XXXXXXXXXXXXXXXXXXXXXXXXXX webhookSecret: XXXXX name: GitHub Real-time type: SOURCE_TYPE_GITHUB_REALTIME verify: true

GitHub App

Use this with a GitHub App.

sources: - connection: "@type": type.googleapis.com/sources.GitHubRealtime githubApp: appId: XXXXX installationId: XXXXX privateKey: | -----BEGIN RSA PRIVATE KEY----- XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -----END RSA PRIVATE KEY----- webhookSecret: XXXXX name: GitHub Real-time type: SOURCE_TYPE_GITHUB_REALTIME verify: true

Configure GitHub to send push events

In addition to configuring TruffleHog to receive push events, you must configure a GitHub webhook to send them. Webhooks can be configured at the repository, organization, or GitHub App level. Any webhook type works with either authentication method.

Configure the webhook with the following values:

Field	Value
Payload URL	https://<your-trufflehog-domain>/sources/github/webhook
Content type	application/json
Which events	Push events only
Secret	A high-entropy value you generate. Copy this into webhookSecret in your TruffleHog configuration.

A single GitHub Real-time integration can accept events from multiple webhooks of different types — for example, two repository webhooks and one organization webhook. All webhooks feeding the same integration must share the same secret, and the integration's configured credentials must have access to every repository that sends push events to it.

Multiple integrations. If you run multiple GitHub Real-time integrations, give them the same name so scanning work is distributed across them. Different names cause each integration to scan every event, duplicating work.

Configuration options

Field	Type	Required	Description
webhookSecret	string	Yes	Secret shared with GitHub, used to validate push event deliveries. Treat as a sensitive value.

For all other fields, see the GitHubGitHub source documentation.

Capabilities

Feature	Supported
Real-time scanning	✅
Scan archive files	✅
Scan base64-encoded data	✅
Scan binaries	✅
Scan gists	✅
Scan forks	✅
Scan history	✅

Notes

The integration's interface and behavior may change as it matures.

Only the first 2,048 commits in each push event are scanned. Additional commits are skipped.

Comments and include/exclude filters are not supported. The integration scans every push event delivered by configured webhooks.

GitHub does not send push events in certain uncommon circumstances. See the GitHub webhook documentation for details.

If a scanner is stopped and resumed within seven days, it scans every push event that arrived while it was stopped. If the scanner stays stopped for more than seven days, the interim events are ignored.