The CI/CD (Continuous Integration & Deployment) tools are a widely adopted part of automated delivery processes in the software development industry. Tools like Travis CI are used for building, testing, and deploying software automatically. They’ve also become a recognized attack surface for threat actors, for which logs from such automated builds may be particularly interesting in search for exposed secrets. Securing build process outputs, particularly in the case of public repositories, is a factor each developer or team should consider when setting up their CI/CD recipes. The role of CI/CD tooling is to support the security of the build pipeline outputs.
Security practices related to the automated CI/CD process should be considered or observed, depending on the developer team’s particular security policy, to ensure a secure software supply chain. However, it may not be enough to prevent secrets from getting exposed in CI/CD build job logs, either due to human error or malicious commits slipping past code review. So we’re happy to announce that additional means were added to Travis CI to better cope with such incidents.
Additional log scanning and censoring
Travis CI already censors secrets configured for the build jobs on-the-fly, to prevent their open-text exposition in the build job log. Still, it is possible to bypass that dynamic filter, and if there are no errors in the build jobs themselves, the build job logs may not be inspected in detail once the build pipeline stabilizes. If a threat actor gets access to the build job logs, it is possible to review these in search of specific secret formats, like tokens, passwords, keys, etc.
With new additional log scanning, every build job log will be scanned shortly after the build job is finished. The scan is performed using two independent Open Source scanners. If the post-job scanning process finds an unmasked secret-like entry in the job log, Travis CI, as a preventive action, will censor the full line in the job log and produce a log scan report. The scan report is available to the repository administrators for 7 days. The job log scan report contains details on found potential secrets, referring to the line numbers in the raw job log file, and is meant to help review and find the source of the potential leak and if this proves to be the actual exposition of the secret, the scan fixes that.
Whenever an additional post-job scanning process finds secret-like entries in the build-job log, a graphical status in the Travis CI Web UI will present the log scan results. You may find more details in our documentation.
My log scan status shows “failed”, what should I do?
The scanners aren’t perfect, and we expect false positives to pop up in the process. Please review the report and the build job process to assess the situation and, if needed, undertake follow-up actions.
Is the job log scan bulletproof?
No. The scan greatly improves the effectiveness of securing the job log content on top of the existing dynamic filter, but by no means is it perfect. Organizations should instill their own policies to ensure that their job logs meet their own security measures.
Hey, what about legacy job logs available via Travis CI API or UI?
We are in the progress of reviewing the 2022 job logs to adjust them based on these new policies. This activity will be executed over the course of the next calendar quarter. Repository administrators may see additional log scan reports popping up for legacy build jobs.
Is the graphical log scan status visible to everyone who can view the build history?
This is a consideration for public repositories or, in general, repositories with a wide group that can access the build job logs. As for an initial release, we’ve decided that the ability to mark to the visitors with access to the repository that there’s something found in the build job log while limiting access to the scan report balances signaling with access to details. Having said that, we will closely review the feedback and adjust accordingly. Please mind that repository administrators may always limit access to the actual build job logs for the write-access users only.
Why not notify via email?
In this case, the amount of email notifications could be counterproductive and might result in blocking these emails, depending on your mail service spam algorithms. As the study earlier this year has shown, many secret-like patterns are present in the legacy build job logs, usually exposed unintentionally. We will be closely monitoring the number of warnings raised by the scan process and decide on enabling further notification options.
Can I opt out of the additional post-job log scanning and censorings?
No. As much as we understand it may be less convenient, there’s been a lot of accidentally leaked secrets in the build job logs in the past, and there’s always a risk of a threat actor attempting to expose them in the future.
