Presentation: Legacy-SecDevOps (AppSec Management Debrief)

Here is a presentation I created last year as a debrief to C-Level execs

It is quite strong, but they took it quite well and agreed with most of it :)

Let me know what you think of it (I'm sure you've seen many similar projects and organisations)

The Authentication micro-service cache incident

A good example of why we need tests across the board, not just normal unit tests, but integration tests, and tests that are spawned as wide as possible, is the story of a authentication module that was developed as an re-factoring into a separate micro-service.

When the module was developed, it contained a high degree of code coverage, in fact it had 100% unit test coverage. The problems arose when it went live, and several issues occurred. One of the original issues occurred because the new system was designed to improve the way the database or the passwords were stored. This meant that once it was fully deployed some of existing dependent services stopped working.

Risk Dashboards and emails

It is critical that you create a suite of management dashboards that map the existing security metrics and the status of RISK tickets:

Why GitHub and JIRA

My current experience is that only GitHub and JIRA have the workflows and the speed that allow these risk workflows to be used properly in the real world.

I know there are other tools available that try to map this and create some UIs for risk workflows, but I believe that you need something very close to the way developers work. GitHub and JIRA meet this essential requirement, as they are both connected to the source code.

Linking source code to Risks

If you add links to risk as source code comments, you deploy a powerful and very useful technique with many benefits.

When you add links to the root cause location, and all the places where the risk exists, you make the risk visible. This reinforces the concept of cost (i.e. pollution) when insecure, or poor quality, code is written. Linking the source code to risk becomes a positive model when fixes delete the comments. When the comments are removed, the AppSec team is alerted to the need for a security review. Finally, tools can be built that will scan for these comments and provide a 'risk pollution' indicator.

(from SecDevOps Risk Workflow book, please provide feedback as an GitHub issue)

Employ Graduates to Manage JIRA

One of the challenges of the JIRA RISK workflow is managing the open issues. This can be a considerable amount of work, especially when there are 200 or more issues to deal with.

In large organizations, the number of risks opened and managed should be above 500, which is not a large quantity. In fact, visibility into existing risks starts to increase, and improve, when there are more than 500 open issues.

The solution to the challenge of managing issues isn't to have fewer issues.

Can't do Security Analysis when doing Code Review

One lesson I have learned is that the mindset and the focus that you have when you do security reviews are very different than when you work on normal feature and code analysis.

This is very important because as you accelerate in the DevOps world, it means that you start to ship code much faster, which in turn means that code hits production much faster. Logically, this means that vulnerabilities also hit production much faster.

Threat Model Confirms Pentest

A key objective of pentest should be to validate the threat model. Pentests should confirm whether the expectations and the logic defined in the threat model are true. Any variation identified is itself an important finding because it means there is a gap in the company's understanding of how the application behaves.

There are three important steps to follow:

1. Take the threat models per feature, per layer and confirm that there is no blind spots or variations on the expectation
2. Check the code path to improve the understanding of the code path and what is happening in the threat model
3. Confirm that there are no extra behaviours

(from SecDevOps Risk Workflow book, please provide feedback as an GitHub issue)

Threat Model per Feature

Creating and following a threat model for a feature is a great way to understand a threat model journey.

First, take a very specific path, a very specific new feature that you are adding, or take a property, such as a new field, or a new functionality.

Using Git as a Backup Strategy

When you code, you inevitably go on different tangents. Git allows you to keep track of all those tangents, and it allows you to record and save your progress.

In the past, we used to code for long periods of time and commit everything at the end. The problem with this approach is that sometimes you follow a path to which you might want to return, or you might follow a path that turns out to be inefficient. If you commit both early and often, you can keep track of all such changes. This is a more efficient way of programming.

Feedback Loops

The key to DevOps is feedback loops. The most effective and powerful DevOps environments are environments where feedback loops, monitoring, and visualizations are not second-class citizens. The faster you release, the more you need to understand what is happening.

DevOps is not a silver bullet, and in fact anyone saying so is not to be trusted. DevOps are a profound transformation of how you build and develop software.

DevOps are all about small interactions, small development cycles, and making sure that you never make big changes at any given moment. The feedback loop is crucial to this because it enhances your understanding and allows you to react to situations very quickly.

Good Managers Are Not The Solution

When we talk about risk, workflows, business owners making decisions about development, and QA teams that don't write tests, we often hear the comment, "If we had good managers, we wouldn't have this problem".

That statement implies that if you had good managers, you wouldn't have the problem, because good managers would solve the problem. That is the wrong approach to the statement. Rather, if you had good managers, you wouldn't have the problem, because good managers would ask the right questions before the problem even developed.

Horizontal DevOps

The best model I have seen for integrating DevOps in a company is where teams are created that span multiple groups. Instead of having a top-down approach to the deployment of operations, where you create the central teams, the standards, the bills, etc., and then push it down, the central DevOps team hires or trains DevOps engineers and then allocates them to each team.

The logic is that each team spends a certain amount of time with a DevOp engineer, who trains the teams in DevOp activities and best practices, and thereby embeds the best practices in the development life cycle.

Is the Decision Hyperlinked?

I regularly hear the following statements: "The manager knows about it", "I told you about this in the meeting", "Everyone is aware of this", and so on. However, if a decision is not in a hyperlinkable location, then the decision doesn't exist. It is vital that you capture decisions, because without a very clear track of them, you cannot learn from experience.

Capturing decisions is most important for the longer term. When you deal with your second and third situations, you start building the credibility to say, "We did this in the past, it didn't work then, and here are the consequences. Here is the real cost of that particular decision, so let's not repeat this mistake".

Involve Security Champions in Decision-making

Once a program starts being placed, security champions will often give feedback that they are not involved in the workflows and decisions. The job of the security champion is to ask, "What is this? Do I trust this? What happens with this?", but they often don't get the opportunity to ask these questions, because decisions are made without their input.

To illustrate this problem, a situation occurred recently where the security champion started to create threat models across a product, and thereby managed to retroactively involve himself in some of the decision-making.

Risk Workflow for Software Vendors

A software vendor is someone who delivers a software application that is executed by a client. The same concept applies to web applications and web apps, but let's start by looking at a traditional software package.

The risk workflow in this case is very important, and there are multiple angles to consider. Let's start with a simple one.

The first items to consider are the issues that evolve during the development of the software. Already, two types of risks exist. There are the risks that exist on the application, which should be known and captured on the risk register. The business owner must accept these risks, because ultimately he/she must decide how to prioritize the risks, and whether to fix them or not, depending on the priorities of the business.

The Pollution Analogy

When talking about risks, I prefer to use an pollution analogy rather than technical debt. The idea is that we measure the unintended negative consequences of creating something, which in essence is pollution.

In the past, pollution was seen an acceptable side effect of the industrial revolution. For a long time, pollution wasn't seen as a problem in the same way that we don't see security vulnerability as a problem today. We still don't understand that gaping holes in our infrastructure, or in our code, are a massive problem for current and future generations.

We are still in the infancy of software security, where we are in the 1950s in terms of pollution. David Rice gave a great presentation[^david-rice-pollution] where he talks about the history of pollution and how it maps perfectly with InfoSec and AppSec.

Published "SecDevOps Risk Workflow" book (v0.66)

Here is the text I just sent to the current 215 readers of my SecDevOps Risk Workflow book

---

Hi, here is v0.66

The reason you have not seen an update for a month is because I focused my writting time on the 'Hacking Portugal' book which you can get from Amazon (https://www.amazon.co.uk/Hacking-Portugal-Making-Software-Development/dp/1540743632) or Leanpub (https://leanpub.com/hacking-portugal)

That book is an expanded version of the keynote presentation I delivered at BSidesLisbon (see http://blog.diniscruz.com/2016/11/presentation-hacking-portugal-and.html) and it is my first book published on Amazon :)

Relationship with existing standards

It is important have a good understanding of how a company's Risk profile maps to existing security standards alike PCI DSS, HIPAA, and others.

Most companies will fail these standards when their existing 'real' RISKs are correctly identified and mapped. This explains the difference between being 'compliant' and being 'secure'.

Increasingly, external regulatory bodies and laws require some level of proof that companies are implementing security controls.

I don't know the security status of a website

Lack of data should affect decision-making about application security.

Recently, I looked at a very interesting company that provides VISA compatible debit-card for kids, which allows kids to get a card whose budget can be controlled online by their parents. There is even a way to invest in the company online via a crowdfunding scheme.

I looked at this company as a knowledgeable person, able to process security information and highly technical information about the application security of any web service. But I was not able to make any informed security decision about whether this service is safe for my kids. I couldn't understand the company's level of security because they don't have to publish it and, therefore, I don’t have access to that data.