Presentation: Veracode Automation CLI (using Jenkins for SDL integration)

Here is a presentation about an secure CI workflow that I'm working on.

The key parts are the Veracode CLI developed (see veracode-api) and the couple Jenkins projects which use the Veracode engine in a 'concurrent scanner' model.

Let me know what you think of it:

Adding Custom Descriptions and Content to Fortify using the RulePack's CustomDescriptionRule element

After Cat.Net and Checkmarx, the next TeamMentor integration target is HP's Fortify

The idea is to add references and cross-links between Fortify results and TeamMentor Guidance. Since that way we get the best of both worlds: 

• SAST Findings
• Targeted and customised Security Guidance

I'm currently using the 5.16 version of Fortify which contains a nice Rule Editor that can be used to create well formatted RulePack files (this tool also as a nice wizard to help with the creation of 'proper' custom rules (for example to map out new Sources or Sinks)):

Another example why SATS technology needs custom rules (re: Detecting SQL Injection on .NET Entity framework)

In Darren's great SQL Injection with Entity Framework 5 and Static Code Analysis post, he mentions how all SAST tools he used failed to find this security vulnerability:


The reason is simple: None of the tools he used had the Database.SqlQuery Method marked as a Sink (and btw last time I checked FxCop is not a SAST tool)

One of the things I wanted to do this week was to add a rules manager/editor to CAT.net (see my current research on this SAST tool from Microsoft here), and this could a good example (i.e. show how to write a custom CAT.Net rule to detect this security issue).

Should we use Bayesian or Neural Networks for SAST? yes, but not yet (we are not ready for it)

The super sharp OWASP Leader Johanna Curiel, while trying to get her head around the O2 Platform, asked me earlier today :

in your research, have you try static code analysis using any form of artificial intelligence such as Bayesian or neural networks ?

let know, while I was studying, I was researching this stuff. I just would like to hear from you if you had any experience with this

The short answer is NO, I have not really looked at Bayesien or Neural Networks for SAST (Static Analysis)

The longer answer is We Dont need it (yet), since there are many bigger limitations of the current SAST technology and tools, which we need to solve first before we look into that type of advanced analysis and techniques.

My comments on the SATEC document (Static Analysis Tool Evaluation Criteria)

(submitted today to the wasc-satec@lists.webappsec.org list)

A bit late (deadline for submission is today) but are my notes on the version currently at http://projects.webappsec.org/w/page/41188978/Static%20Analysis%20Tool%20Evaluation%20Criteria

My comments/notes are marked as Conted to add in underscore, bold and Italic or [content to be deleted in red]

When I wanted to make a comment on particular change or deletion, I did it on a new line:

DC Comment: ... a comment goes here in dark blue

Of course that this is my opinion, and these notes are based on the notes I took in 'analogue mode' (i.e. on paper :)  )

Viewing the C# MethodStream for a WebMethod

I was debugging an issue with TeamMentor WebServices and created a view that gave me the MethodStreams for all its source code

MethodStreams are virtual files that contain all relevant 'call-flow source-code' for a particular starting methods (note: MethodStreams are one of O2’s biggest innovation on the SAST world)

Feedback needed on 'Static Analysis Tool Evaluation Criteria'

Sherif Koussa is looking for feedback on the Static Analysis Tool Evaluation Criteria document he is working on.

This is a really important document/criteria which would help SAST users to know which is the best tool for their needs.

It looks good and I will try to read it in the next week or so.

There is a mailing list which you should join if interested in this topic.

Would I recommend Checkmarx as a SAST engine?

I was asked this question twice in the last couple days, and my answer is YES!

Although CheckMarx is still not as open and easy to engage as I would like them to be, they are actually one of the best ones out there.

And there is one asset that CheckMarx SAST engine has that is REALLY GOOOOOOODDDDDDD!!!!

Their rules are written in C# and if you (like me) like to write custom rules, they have a nice REPL interface that can be used by power users (with access to a lot of the metadata and code-transformations created during the analysis phase)

I'm currently integrating TeamMentor with Checkmarx (for a joint customer) and I really like it. You can see our latest PoC at http://checkmarx.teammentor.net, which includes a view that shows a mapping between:

• a security finding, 
• it's CWE description
• the TeamMentor landing page (for that finding)
• the C# CheckMarx rule that triggered that finding

Eric Lippert joining Coverity

This is VERY interesting, since Eric is one of the C# Microsoft Gurus.

You can read his announcement on is his last MSDN post A new fabulous adventure and on his new blog

Good on Coverity for catching such good talent, and I wonder if they are (finally) going to enter the SAST space for C#/.Net

Using SAST technology on Android to detect SSL implementations issues

Arvind called my attention to this pdf Why Eve and Mallory Love Android: An Analysis of Android SSL (In)Security which (looks like) is a great read and it includes this really interesting paragraph:

8. COUNTERMEASURES
8.1 OS SolutionsMalloDroid 

Installation Protection. (page 59)

MalloDroid could be integrated into app installers, such as Kirin [9], to perform static code analysis at install time. This analysis performed directly on a phone could warn of potentially unsafe applications. Users would then have to decide whether they wish to install the app irrespective of the warning.

I ABSOLUTELY agree with this!!!!!

There is a point where the complexity level of what is being implemented is so high and there are so many moving parts to take into account (or to do them consistently in deployment) that we need a way to codify what is (or should) happen.

First video of Eclipse's real-time static analysis plugin for Java security

Diarmaid McManus has recently published a video of his Eclipse's real-time static analysis plugin for Java security

Here is the video of it in action (showing an error on a StackTrace information disclosure issue)

What are the challenges with SAST that don't need a better engine

Earlier today I commented on twitter "I've been trying to tell the Commercial SAST vendors for ages that current problems is not the engine" and was asked 'What do you see the problem being?'.

So here is my very quick list of problems I see with SAST engines, which have nothing to do with the core engine

What am I doing with Cat.NET?

I was asked this today, so here is the answer (in Oct 2012).

I'm:

• showing what a SAST should do 
• 'making it work' 
• showing how a 'real-time' SAST scanner is possible and can be made to work (namely educating the 'buying audience' that this possible and they should demand it from their SAST vendors/suppliers)
• figuring out the workflow needed for SAST to be effective used in SDLs
• expanding the SAST usage by getting developers and software teams realize the value of SAST
• educating developers and security consultants on:
• how to use SAST technology
• what secure code looks like
• what vulnerable code looks like
• show how SAST can be integrated with other tools. For example with the O2 PoCs, I already show integration with:
• VisualStudio 2010 
• Roslyn
• MSBuild
• a stand-alone exe (i.e. running/distributing a scanner in one stand-alone exe)
• TeamMentor (ie. show how its guidance can be used by an developer in an IDE)
• developing public schemas for findings, rules, coding artefacts, etc...
• creating a community to publish, share and comment on SAST rules
• creating an environment where commercial (or OpenSource) SAST vendors/products/services, can come together and collaborate.

The last point is the most important, I have no interrest in developing a SAST engine or solve the hard scalability challenges that come with it. 

What I really want is to get us all working together so that we can really make a difference in the Web Application Security world.

Unfortunately, in Oct 2012 there is not one SAST company that is playing the game ... but hopefully once we build it they will come :)

So, ultimately, the reason I'm using Cat.NET is because I CAN :)

Cat.NET is the only free SAST engine that I can easily customise and use! 

As mention in Providing licenses to security consultants , the SAST vendors still have no idea/solution/program to deal security consultants like me. Which is not very clever, since they are losing a lot of mind-share by being over protective of their technology (and those 'super precious rules')

If you are interested in SAST and want to be involved, here is a list of areas that need energy: What are the challenges with SAST that don't need a better engine

Help in sorting out the Cat.NET's EnvDTE dependency

I've been under a rock for the past month coding away (more details later) and one of the modules I created was a stand-alone O2 Tool (one file) that exposes the Cat.NET engine outside VisualStudio.

You can get it from the latest version of O2 (GitHub) or from this DropBox download: Tool - Cat.Net outside VisualStudio v1.3.exe

You can see below how it is supposed to work, but I'm having an issue at the moment which would be great if somebody could help in.

Basically the problem is that there is an dependency on EnvDTE which only happens on certain installs.

I have been able to run this tool on VMs with and without VisualStudio installed, so there must be something else that installs EnvDTE on a box (like the one I was using at EC2).

The problem is that one of my users is not able to run the O2 scripts based on the O2's Cat.NET APIs, and we need to figure out what needs to be added (or removed) in order to make this work.

In SAST the issue is 'Trace Connection', not 'Scan Size'

One of the 'wrong problem to be solving' paradox that happens in the SAST world is the focus on making their engines able to 'scan large code bases'. It is not a coincidence that the key question I got from SAST engine guys on the Real-time Vulnerability Creation Feedback inside VisualStudio (with Greens and Reds) was 'Humm.... interresting but will is scale to large applications?' 

I actually blame the SAST clients for this, since they are the ones asking (and paying for) the wrong question:

 "How can you 'vendor xyz' scan my million lines of code application"

Instead they should be asking:

 "When you scan my code, can you connect the traces?"

'Connecting the traces' means that you are able to scan parts of the application separately and then connect them at a later stage.

O2 .NET SAST Engine: MethodStream and CodeStrams for a WebService Method

One of the hidden Gems of the O2 Platform is its SAST engine that is able to create what its called MethodSteams and CodeSteams (from C# code).

MethodSteams are a code representation of an entire call-tree, i.e. one file that contains the original method and all the methods it calls (recursively).

CodeSteams are all the possible 'taint follow' paths that exist on a particular MethodSteam

Here are they in action in the HacmeBank's source code:

This is how we have to show security vulnerabilities to developers (in real time as they are created)

I posted a PoC today that represents my vision for O2 and what I have been trying to do for the past 5 years.

You can see the video at Real-time Vulnerability Creation Feedback inside VisualStudio (with Greens and Reds) where every time the user makes a change to the code there is an auto-compilation (using Roslyn's C# compiler) and a SAST scan (using Cat.NET)

What I like the most about this, is that I now get to think about 'the best workflow to present developers the security guidance they need'.

Although this PoC is quite agressive (I do a compilation and scan on every keystoke which is a bit OTT), here is another video that shows a bigger compilation+scan on save: Real-Time C# Solution Compilation and Security Scanning (using Roslyn and Cat.NET) 

What do you think?

Real-Time C# Solution Compilation and Security Scanning (using Roslyn and Cat.NET)

On the Real time Vulnerability Scanning using Cat.Net and Roslyn (SAST) example, the compilation and scanning happened on a single file, which was cool, but not that realistic.

The next step was to be able to compile and scan an VS Solution file (*.sln), like one from the 'vulnerable by design' SuperSecureBank site.

In the video below, I use Roslyn's project loading and monitoring capabilities to trigger a compilation every time there is a change made to a solution file. And if the compilation is ok, a security scan (using Cat.NET) is triggered (both steps done in a couple seconds):

Running Cat.NET SAST Scanner outside VisualStudio

One of the big limitations of Cat.NET was its need to be executed inside VisualStudio.

Yes VisualStudio is one of the places where we want to use Cat.NET's engine and GUIs, but it is also very important to be able to run it as a stand-alone tool (and to inject it into other .NET applications).

Here is a video that shows a GUI/script I created with the O2 Platfrom, which shows the execution of Cat.NET GUIs + Scanning engine outside VisualStudio, and a couple extra features added.

Video: Real time Vulnerability Scanning using Cat.Net and Roslyn (SAST)

Here is a pretty cool PoC of creating an environment for developers that helps them to know when they create a security vulnerability (in real time).