listen to this article:

Integrating Code Signing with CI/CD

If you were to look up the term, “developer”, in the Marriam Webster dictionary it says, “a person or company that develops computer software”.  While true, this is a rather simplistic take in today’s world where developers are the very heart of IT.  Today’s developer should be referred to as, “a person or company that designs, develops, builds, signs, tests and releases computer software”.

A developer has inputs across the entire software development lifecycle and any assistance in accelerating that lifecycle is highly encouraged. Unlike today, in the early 2000’s, a lot of the automation was through in-house, custom-built tools. Now, a plethora of purpose-built software exists in open source and commercial spaces to aid automation and deliver of software better, faster and more secure.

Watch Unbound’s Webinar on their next generation of Code Signing solution

Why Cryptographic Agility is Critical to CI/CD

As Continuous Integration/Continuous Delivery (CI/CD) toolset matured and code got built and tested faster, the need for cryptographic agility (sign faster and securely) became critical. It was understood that the signing process should not become a bottleneck and slow down the entire delivery lifecycle. Developers started looking for code signing solutions with increasing maturity, to have the following three tenets:

  • High performance: Ability to sign artefacts “very quickly”, especially when integrated with CI/CD toolsets.
  • Strong security: Protect the cryptographic keys and digital certificates used for signing. Support code scanning with various anti-malware engines.
  • User experience: Ease of integration with CI/CD systems, interface with HSM or other mechanisms etc., along with ability to access cryptographic keys on-demand.

What Types of Code Need Code Signing?

Developers today deal with a wide variation of ecosystems that need code signing. They operate in complex technology ecosystems, sometimes spanning across multiple diverse stacks in a large Enterprise. As part of the build process, the code in most technologies needs to be signed using digital signatures before it can be shipped off.

Let’s examine some of the stacks that a developer could be coding in. We will explore the classical methods of signing a developer might use for each of these stacks:

Traditional Java, .Net: As Java website describes, Java is still in heavy use, running more than 51 billion Java Virtual Machines worldwide. In microservices world, Spring Boot is a popular framework. Developers use jarsigner/keytool (part of JDK) for code signing. CodinGame Developer Survey 2021 ranks .Net Core as the top three most popular development frameworks.  In this world, developers use signtool.exe (part of Microsoft SDK) for code signing.

Interpreted languages (Python, Rube, R etc.): These interpreted languages have gained a lot of popularity with the rise of AI/ML, data science and High-Performance Computing (HPC). One way of code signing here is to “freeze” your code, i.e. create a single-file executable file to distribute to end-users, that contains all of your application code as well as the interpreter. Another approach is to create a Linux distro package (e.g. .deb files for Debian or Ubuntu, or .rpm files for Red Hat and SuSE.). Ruby gems can be cryptographically signed by gem cert command. However, all these approaches are not very prevalent in the open-source world, primarily due to the cost and efforts involved.

Front End code (JavaScript, React, HTML/CSS etc.): These have been the bane of web development for a while. Since these are not a binary format itself, there are only indirect ways to sign them. Developers usually prefer the secure medium of transmission through HTTPS.

Mobile code (iOS, Android etc.): Any application worth its salt needs to have a mobile app – either native iOS/android, or hybrid Xamarin/flutter etc. iOS code signing is similar to java’s jarsigner, but with an Apple root CA. Android’s sign process can be viewed as a light version of java signing.

Container systems: The next generation to virtualization was the advent of docker and container orchestration systems. Docker Content Trust (DCT) provides the ability to use digital signatures for data sent to and received from remote Docker registries. The hyper scalers have their Container Registries implement DCT model.

Serverless computing: The hyper scalers offer technologies for running code, managing data, and integrating applications, all without managing servers. From a code signing perspective, developers can use AWS’s Signer, a fully managed code signing service for signing, lay lambdas. Azure has Azure Key Vault, a cloud service for securely storing and accessing cryptographic keys. Developers use signtool.exe for code signing, or open source tools like “Azure Sign Tool” that integrate with Azure Key Vault for performing the signing process.

Developers’ Challenges with Integrating Code Signing with CI/CD

Support of Diverse Technology Stacks

As illustrated above, a developer needs “one-size-fits-all” solution for code signing, for ease of design and maintainability. Using different code signing methods and cryptographic keys is a logistic and maintenance challenge.

Support Local/Remote Signing

A developer would expect two types of integrations with CI/CD pipelines:

  • Local sign: Integration between OS/Platform signer utility and security provider (HSM or vendor);
  • Remote sign: CI/CD pipelines that use REST api’s to send file to vendor, who will sign it and return.

Lack of Comprehensive Security Policy Across Build and Delivery Pipelines

Code signing should not be towards the end of your pipeline, since attackers can potentially “move left” and insert malicious content earlier into the pipeline.

Securing Cryptographic Keys

Developers typically tend to store cryptographic keys on build servers or file shares, to aid the automation tools. This is very risky and can potentially expose these sensitive keys.

Scan Before Sign

Developers want to scan their code just before signing it. This ensures that they are not signing something that’s already corrupted.

One Key, Many Signs

It’s not a good idea to use one key for all your signatures. Developers want to distribute the risk with multiple certificates.

Securely Streamlining Code Signing for Devops

In this article, we viewed a developer’s perspective and the need for code signing across the technology stacks they might work on. We also analyzed challenges developers have in integrating code signing with CI/CD pipeline.

In summary, the need of the hour is to offer developers the best of both worlds: strong security and superior performance, in addition to client integrations to all the tools used in their environment today.