The scenario is a very typical one for any SAST client (namely Ounce/AS.Source or Fortify):
- They have a large .NET application, made of a large number of projects (i.e. big code base)
- The app is quite complex and makes use of multiple messaging-buses to move data around (think of this has a more obscure version of getters and setters pairs)
- There where a good number of AS.Source results (about 19k findings with 11k traces), but very few that actually make sense (to them)
- The security team is made of really cleaver guys (who know how to code in C#) and with good understanding of how the app works (add as a bonus the fact that they actually want to improve the apps security (vs being compliant) and that they want to get good value from SAST+DAST tools)
- Since the security team actually knows the app and what type of issues exist in there, they where very quickly able to look at the initial AS.Source findings and say
".... that is not good enough!'
a) there can't be THAT number of 'findings' and
b) I know for a fact that a number of those 'findings' are false' ..."
So after trying to get a number of good results via the 'official way' (i.e. using a bunch of IBM/AppScan guys), the IBM sales rep decided to bring in a company that he trusted in Boston (Security Innovation) which happened to have a guy in there that really knows how to get the most out of AppScan Source (me) with a toolkit that allows the easy development, test and deployment of the analysis customization that will be required to get this to work: the OWASP O2 Platform
The model that we are going to use on this engagement is one where:
- I am going to create a customized version of O2 for the client which is going to be fully supported by SI
- IBM makes a product sale
- SI makes a service sale
- and O2 gets a couple more tools and scripts (any new code that doesn't contain the client's apps information/structure will be committed back to the O2 code base).
I was there for most of the day, and here is what happened:
- After the intros, the IBM and SI reps defined the working model where IBM would be focused on the product side and SI would be focused on the customization side
- This meat that instead of having to deal with the normal road-blocks created by trying to get the AS.Source to work, we could move strait away into the solution part, ie using O2 :)
- I started to explain my methodology, namely how I use O2 to analyse an application's structure and connect the dots, but since clearly they had no idea of what I was talking about looked like (and they had never seen O2), I decided to do a number of demos:
- After firing up a projector, I showed them the O2 scripts with the analysis of the JPetStore (see Current O2 support for analyzing Spring MVC). This is the order that I showed this scripts:
- launched the main O2 Script and showed how I could click on a link to start the local apache and hqsql server
- make a couple simple browser-automation requests to the server to make sure it is working ok
- opened up the O2 script with the exploits
- executed the multi-step shopping card purchase without any payloads (just the normal JPetstore app behaviour)
- executed the same shopping card purchase with the payload that allowed the modification of the total purchase price
- executed the other exploit scripts (variations of the above)
- then I opened up the script that showed the Spring MVC mappings, namely
- all urls
- urls mapped to source code
- correct identification of the Command Class used on the Spring Autobinding
- correct visualization of the real size of the CommandClass used on the shopping card (which is huge and allows for the manipulation of many more fields than the ones exposed in the web gui, like for example the setTotalPrice)
- .... Now they were getting into O2 .... and since the main guy from their security team knew how to code in C# he was able to write most of the scripts I talk about below (with my guidance and help on the best O2 APIs to use :) )
- While we (me and a great friend from IBM that was there as a technical resource from IBM) grabbed a coffee, they installed O2 and found a test version of the target website that we could use (he also copied the O2 Click-One installed files and O2 scripts into another Box which didn't had direct internet access (ie. you can run/install O2 without the click-once installer))
- Now we were ready to go and we started with the O2 Quick Development GUI writing a script that:
- retrieved all files from a particular directory
- added a treeview to the 'panel' control (from the O2 Quick Development GUI)
- added the files to the treeview
- added a source code viewer to the panel control
- configured the treeview to show the file contents when the user clicked on one of its nodes
- made the script run on a popup window (instead of the panel control)
- saved the script on a local folder (as an *.h2 file)
- clicked on that *.h2 file to see how this script had just become a stand-alone tool
- By now they were staring to get the really power of O2 for fast prototyping and mini-tool development
- Then we moded into the O2 IE Automation control script and:
- created a script that would open their website
- retrieved the list of fields
- found the field that controlled a search function
- populated that field (programatically) with a specific value
- tried to find the submit button but it was not there (it was an image with a link)
- injected jQuery and FirebugLite into the IE object so that we could use it to find the search button (i.e. image with link) id
- once we knew that id (back in O2) get its reference and click it (programatically)
- now that we had a working search automation, we created a inner lambda method so that it could be invoked with just a method call: search('....')
- tried to show how O2 FuzzDB could be used to easily run a batch of XSS or SQLi test strings on that automation, but their firewall rules prevented the download of it :)
- ok... and that was just on a couple hours, ... with him coding now me ... :)
- The next script was more interesting, we went back to the first script (the one where you could click on a treeNode and see its contents) and did this to it:
- changed the location path to be root of the app we were looking at
- changed the search extensions to be *.asmx, *.asmx, *.ashx
- added a web browser so that we could also see the web page that was related to that page (this was done by simply doing a search and replace of the root path with a url)
- added an extra source code viewer so that we also could see the source code of the .cs page, ie:
- user clicked on c:\...path...\webroot\page.aspx (in the treeview on the top left) and
- open in browser (top right) the web page http://....test...site.../page.aspx
- open in source code viewer #1 (bottom left) the file c:\...path...\webroot\page.aspx
- open in source code viewer #2 (bottom right) the file c:\...path...\webroot\page.aspx.cs
- ... Now they were getting really excited... . What what happening was that they were realising/visualizing that the real power and uniqueness of O2, is not on the ability to write such scripts (you can do that in Visual Studio or Eclipse given enough time), but in the ability to do it in such interactive environment and speed (during the security review engagement)
- Next we went back to the AS.Scan results and started reviewing them (using O2's *.ozasmt file viewers which are super fast and interactive)
- Once we had a couple interesting targets and realized that we had good traces for the web pages we were looking at before, we made the following changes to the previous script
- add support for loading up *.ozasmt files
- add support to cache the loading up of *.ozasmt file (which not only made the multiple 'code changes + auto compile + execution' cycles much faster, but it also keeped the memory consumption of the O2 process under control :)
- added a *.ozasmt findings viewer to the main gui (we put it on the top right)
- created a Lambda function that was able to create a filtered view of the findings loaded based on a file name (this was basically a foreach loop that returned a match for findings that had a trace from that file)
- modified the afterSelect event of the treeview to also call the findings filtering lambda function (and viewer), and
- .... finally we ended up with a pretty sweet tool, that when the user selected a particular *.aspx, it would :
- show its web pages
- show both is source code files (*.aspx and *.aspx.cs)
- showed all findings that were related to that file
- By now we were running out of time, but I got the feeling the they really got a taste for the type of analysis that can be done with O2 and AS.Source
- They also started to see the power of having a large number of AS.Source findings since that can be of enormous value when looking/analysing sprecific parts of the application (btw 11k traces are nothing for O2 , it can handle 100k or even 1M traces :) )
- One of the steps that needs to happen over the next week or so, is that my good IBMer friend will work with them to add the missing taint rules (namely Sources and Sink) and rerun the scans (one key problem that was happening is that due to the large number of abstraction layers the AS.Source was missing both sources and sinks on lots of key parts of the application (i.e. creating no traces))
As a side note, they were also having lots of problems with a big 5Gb AppScan Standard file.
I wanted to take a look since O2 has a parser for it, and part of the exercise is to connect the AppScan Source findings with the AppScan Standard findings (i.e. urls to source code).
When I asked him to unzip that 5GB (*.scan) file and just give me the internal findings file (which is a FireBird DB), he asked me '... unzip what?...' (i.e. he never looked inside an AppScan Standard results file!)
This is quite crazy, since there is so much good stuff in those files that I'm sure will be useful for him when performing manual+automation app reviews (I wonder how many AppScan Standard users actually know that it's possible to access those resources)
Btw, the O2 script that we wrote stayed at their offices, but since that is quite a generic need (map *.aspx pages to website and source) I will write a generic version of it to show what it looks like (in fact O2 already has a more advanced and complex version of that script, which also maps *.aspx to *.ascx which I used and developed in the initial stages of my TeamMentor coding (i.e. when I was trying to figure out what was going on))
I know that some people will look at what we did yesterday and wish that was not needed, unfortunately in the real world all apps are different and if we don't have the ability to create workflows like the one described here, we will always struggle to understand how the app work (and get the best out of the tools we have).
As a final point, what also happened yesterday (and always happens when I show these capabilities/scripts to security teams or devs), is that they were also seeing the opportunities to use this for more than security (specially to create bridges+dialogues with development/QA teams)
But that is topic for another blog post.....
:)
