listen to this article:

DevSecOps

On October 4, 2021, a rather misfortunate event occurred  that nearly stopped the social world from turning.  Well, not really, turning but it did wreak havoc across the social and investment spheres. Facebook and its allied portfolio of services – Instagram, WhatsApp, Messenger, and Oculus were down for approximately six hours which sent more than just social influencers into a flurry. According to The Guardian “$50B was wiped off the company’s market value by jittery investors, founder Mark Zuckerberg’s own paper fortune shrunk by $7B and more than $13M of the advertising dollars that are its lifeblood disappeared each hour the platform was offline.”  

Beyond these speculative numbers, the impact of this global outage was very real, and it’s time we look deeper at our reliance on these services and their intersection with our global economy.  Take for example WhatsApp, which has become a critical piece of communications infrastructure in many countries – routinely used to connect doctors and patients, intercompany communications, as well as   used by many for payments. It’s important to not just understand the reliance, but to also be technically prepared should a dependent system or service fail.  Millions of people rely on Google DNS servers to reach every server on the planet. Now consider the impact of those servers going down for an extended period of time. That wouldn’t just affect consumers, it would disrupt commerce, production, communication, and your overall IT infrastructure. 

Outages like these draw our attention to how vulnerable the entire world is to Enterprise malfunctions – whether it’s related to processes, access, security, and system gap one, thing is clear and that is the criticality of the basics – security and consistent processes embedded through the software development lifecycle. 

The New Normal for DevSecOps 

In the early 2000’s, I used to work in the “Software Management and Release (SMD)” group of a large Enterprise. We used to create builds that took 18+ hours to compile, running on four parallel blade servers. We used to build on 8 versions on Unix and 3 versions of Windows. For security, the Release Manager would meticulously match the software BOM (Bill of Material) from their treacherous (extreme color coded) excel sheet. Testing was the bunch that sat on the 3rd floor and who were always whining that they had so much to test, and so little time do it properly. Developers would always claim – “but it worked on my machine”. That’s what early-stage DevOps looked like back then and we didn’t know to call it what it is today, DevOps! 

Jump to early 2010’s when I was doing an independent consulting assignment for a start-up and the goal given to me was simple – “take our line of code from Git to Production in less than 20 min!”. Now, that directive may sound simple but if you unpack that statement, there’s a lot to be considered, understood and then implemented in that request and statement.   

Jump another ten years and the 2021 Accelerate State of DevOps by DORA (Google Cloud’s DevOps Research and Assessment (DORA)) states the following about Elite performers: 

Elite performers now make up 26% of teams in our study and have decreased their lead times for changes to production. The industry continues to accelerate, and teams see meaningful benefits from doing so. 

Elite Performers are the Enterprises who meet the following metrics: 

  1. Deployment frequency – On Demand (multiple deploys/day) 
  2. Lead time for Changes – less than one hour 
  3. Time to restore service – less than on hour 
  4. Change failure rate – 0%-15% 

The drivers for this rapid agility are not surprising – accelerated digital economy, creation or migration to modern cloud native applications, cloud-centricity, hybrid cloud operations, hyper automation, and the list goes on. 

Why “Sec” in DevOps is Becoming More Important 

The Peter Parker Principle of Spider Man fame states – “With great power comes great responsibility”. As software releases become better and faster, there is a greater responsibility to make them secure and resilient. The Accelerate State of DevOps report also confirms that you must consider a critical fifth metric – Reliability” to the previous four metrics called out in the report section above. It represents the degree to which a team can keep promises and assertions about the software they operate. 

A key tenet of Reliability is the Security Reliability. That’s the “Sec” in DevSecOps. It’s the ability of an Enterprise to enhance and protect their security posture. The 2020 SolarWinds Orion IT management software attack or 2019 malicious Asus update and several such high incidents are often traced back to a compromised software supply chain. Software Supply Chain is the collective term used to describe the stages of software lifecycle from source to deployment, with all the tooling included. As Enterprises become more cloud native, microservices based, they tend to include more of dependencies from open source and vendor projects, thus increasing the attack vector. It would not be a stretch to state that – 

 “Software supply chain is the new food chain” 

A disturbance in the food chain disturbs the entire life ecosystem. An attack on anyone’s software supply chain impacts the entire digital ecosystem. Each one of us are impacted in more ways than we can begin to imagine, since were all a part of this connected ecosystem. 

Supply chain attacks often work by breaking the code-signing process 

It is crucial that a code signing solution would be agile and evolve with the ever-growing enterprise needs. A robust solution would be one that constantly adds support for: 

  1. New artifacts 
  2. New CI/CD tools – on-prem or on the cloud 
  3. New cryptographic algorithms such as post quantum crypto 
  4. New functionalities such as scan-before-sign 

How Can your Organization Achieve True DevSecOps?

True DevSecOps was initially seen as the mirage, an illusion that did not exist! However, with the Business demanding faster, cheaper, secure releases coupled with the maturing of toolsets and an evolving culture, this mirage seems to be becoming a reality for more and more Enterprises. 

Here are six practical steps that you can take to accelerate your journey towards true DevSecOps: 

Define Your North Star 

Like they say – knowing where to go is half the getting there! 

Not every Enterprises needs to be FANG replica (Facebook, Amazon, Netflix, Google), nor should they be. Maybe the business is not such, maybe the requirement is not such, maybe the Enterprise is just not ready yet. Analysing the successful implementations of true DevSecOps and creating your own version, your own north star is the most critical step. And often times, it takes external expertise to create this. They will probably be able to identify your blind spots and create the relevant custom implementation.

Audit Your Security

Don’t forget the “Sec” in DevSecOps! Have special focus on security, including auditing each step and tool of your Software Supply Chain. Audit the use and application of cryptographic solutions, including unified key management and protection. And do this for “ALL” Products/Services in Production.  

In one of my assignments for a South African insurer, we found two Windows NT boxes in load balanced UPS’s, serving a couple of DCOM components, live in Production. The guys who had built these components had retired last year! Take special care of such delicacies!

Know The Pipelines

The software supply chain pipeline is typically created to ship code. However, that’s only the partial deliverable for the Enterprise. The pipelines should be starting from the infrastructure later (obsoleting the earlier, fragmented legacy cryptographic infrastructure), extending to config, code, database, and security. That’s a well-defined pipeline – “Everything as code (XaaC)”

Security Enables Velocity

The traditional view is that “waiting” for security reviews through the software lifecycle slows it down. This is not the case with modern tooling, which can be integrated right from Developer’s IDE to CI systems to Release Cycles. Code signing is another critical element to thwart attempts to distribute malicious software. Security is not a blocker to velocity, it’s an enabler by giving you confidence that what you’re shipping is safe.

Identify Gaps and Iterate

DevSecOps embodiments continuous improvement, as you would have realised. You start with where you are and push for automation and security. And keep pushing the envelope to further levels of maturity. It’s a continuous process of identifying the gaps and iterating with the solutions. 

Security is Everyone’s Responsibility

Gone are the days when security checks were done by a team as part of Pre-Release checklist. Today, that “checklist” has shifted left into the Developers IDE, the QA Analyst’s repertoire of tools, into the DBA’s daily diligence and into the DevOps Engineers implementation pattern. Cryptography is used in authentication, encryption of data in many different scenarios (in databases, on storage, as virtual machines), for signing on business transactions to ensure integrity, for signing on code to prevent the propagation of malware, to protect new digital assets (like crypto assets), and much more. Security is indeed everyone’s responsibility – that’s the only way it “gets done”.

The Benefits of Adopting True DevSecOps 

The benefits of adopting a true DevSecOps —is that there is a model for it that allows for the proper integration of security, development and operations. . One important offshoot of true DevSecOps, other than the obvious time-to-market, defect reduction, etc., is the realization of significant risk reduction.  

ZeroNorth surveyed over 250 security professionals, engineers, developers and other IT pros from organizations involved in application development and found that security vulnerabilities and flaws which were detected and addressed earlier in the development processes correlated with enhanced user experience and better protection of Enterprise and users from attacks. 

With the growing trend towards migrating to cloud and its resulting intricate, heterogenous infrastructures, cybersecurity, hence cryptographic practice, is a key attribute of organizational success as is speed and agility and should be addressed and prioritized accordingly.