Multiple Roslyn based tools (all running Stand-Alone outside VisualStudio)

Over the past months I have been using Rolsyn in a number of scenarios, creating in the process multiple O2 Platform tools which might be useful to new Roslyn users.

Here are the stand alone exes, all about 9Mb since they contain the Roslyn dlls and the O2 REPL Script environment as embedded resources.

• PoC - Roslyn C# ScriptEngine Execute v1.0.exe  - provides a REPL for Roslyn
• Util - View Roslyn AST Graph v1.0.exe - good to see the AST tree created by Roslyn
• Util - Exe Creator (using Roslyn Compiler v1.0.exe - Nice tool to create stand alone exes
• Util - Roslyn - Compile Solution and view errors v1.0.exe - compile entire solutions
• Util - Roslyn - OnSave Compile Solution (with REPL) v1.0.exe - adds a REPL for Roslyn objects
• PoC - Roslyn refactor on InitializerExpressions v1.0.exe - shows an refactoring example
• Util - Roslyn AST Refactoring via REPL Script v1.0.exe - gui to develop refactoring scripts
• Util - Roslyn Cat.Net- Compile and Scan Solution v1.0.exe - shows integration of code compile and security scanning (Cat.NET will be download the first time this tool is executed)

The titles of these tools should give a good clue on what they do, but if you want me to expand on them (or how the script works), just add a comment to this post with your request/questions.

Note that you can run all these scripts using the latest version of the O2 Platform (just search for 'Roslyn' scripts).

If you are trying to get your head around Roslyn, the best one to try is the first one (PoC - Roslyn C# ScriptEngine Execute v1.0.exe) which is a nice REPL environment. Here is a post I wrote a while back that shows this tool in action: Video: Using O2's Roslyn-based C# REPL Script environment

This is how we have to show security vulnerabilities to developers (in real time as they are created)

I posted a PoC today that represents my vision for O2 and what I have been trying to do for the past 5 years.

You can see the video at Real-time Vulnerability Creation Feedback inside VisualStudio (with Greens and Reds) where every time the user makes a change to the code there is an auto-compilation (using Roslyn's C# compiler) and a SAST scan (using Cat.NET)

What I like the most about this, is that I now get to think about 'the best workflow to present developers the security guidance they need'.

Although this PoC is quite agressive (I do a compilation and scan on every keystoke which is a bit OTT), here is another video that shows a bigger compilation+scan on save: Real-Time C# Solution Compilation and Security Scanning (using Roslyn and Cat.NET) 

What do you think?

Real-time Vulnerability Creation Feedback inside VisualStudio (with Greens and Reds)

Here is a pretty cool PoC of a Real-time Vulnerability Creation Feedback inside VisualStudio where I show how a number of security vulnerabilities (XSS, SQL Injection, Path Transversal, Unvalidated Redirects) are detected as soon as they are created.

I'm trying to find out the best visual way to present this info to the user, and Greens/Red are usually a good indication of good/bad stuff (also note how the squiggly lines show the traces (i.e. the places where there is a vulnerability)

If you want to comment on this PoC or idea, please use this reddit thread 

Related posts:

• Real-Time C# Solution Compilation and Security Scanning (using Roslyn and Cat.NET) 
• Running Cat.NET SAST Scanner outside VisualStudio 
• 1 line to compile, create and execute: O2 Script to use Roslyn to Dynamically compile and execute a method 
• New Reddit Community for Cat.Net 
• Using/Consuming Cat.Net's engine inside the O2 Platform (and outside VisualStudio) 
• Using Roslyn to Load and (quickly) Compile C# Solution files (outside VisualStudio) 
• Video: Real time Vulnerability Scanning using Cat.Net and Roslyn (SAST) 

Roslyn NerdDinner compilation errors (on June 2012)

Using the Util - Roslyn - Compile Solution and view errors.h2 O2 Platform Script/GUI, I was able to quickly try to compile the latest version of NerdDinner (just drag and drop the NerdDinner.sln file: 

As you can see by the screenshot above, there are the current errors, some of which are a bit weird.

Here are all the errors in an easier way to consume:

Real-Time C# Solution Compilation and Security Scanning (using Roslyn and Cat.NET)

On the Real time Vulnerability Scanning using Cat.Net and Roslyn (SAST) example, the compilation and scanning happened on a single file, which was cool, but not that realistic.

The next step was to be able to compile and scan an VS Solution file (*.sln), like one from the 'vulnerable by design' SuperSecureBank site.

In the video below, I use Roslyn's project loading and monitoring capabilities to trigger a compilation every time there is a change made to a solution file. And if the compilation is ok, a security scan (using Cat.NET) is triggered (both steps done in a couple seconds):

Running Cat.NET SAST Scanner outside VisualStudio

One of the big limitations of Cat.NET was its need to be executed inside VisualStudio.

Yes VisualStudio is one of the places where we want to use Cat.NET's engine and GUIs, but it is also very important to be able to run it as a stand-alone tool (and to inject it into other .NET applications).

Here is a video that shows a GUI/script I created with the O2 Platfrom, which shows the execution of Cat.NET GUIs + Scanning engine outside VisualStudio, and a couple extra features added.

Using Roslyn - Refactoring AddingMethodToClass (while creating an API)

Here is a step-by-step refactoring activity where I used of the Roslyn example methods (the AddingMethodToClass which shows how to "replace a class inside a namespace") and rewrote it using O2's Roslyn APIs (adding a bunch of Extension methods on the way) 

This is what the refactored code looks like:

This is what the original code looks like:


This is a smaller version of the refactored code:

Here is a document that shows how the refactored code was created:

1 line to compile, create and execute: O2 Script to use Roslyn to Dynamically compile and execute a method

Using the code sample from Loading an assembly generated by the Roslyn compiler here is how to write it using O2's Roslyn API.

Version 3 (doing it all in 1 line : ) )

Version 1 (replicating the original script with some error detection):

Code Samples

Version 1:


Version 2 (simpler with no error handling):


Version 3 (in one line):

Simple O2 script to show Roslyn's Ast Error detection capabilities

Here is an simple O2 script that shows Roslyn's C# AST parsing capabilities in action, including how to show squiggly lines in the source code on the locations of those errors.

Note how fast the whole process is, and how Roslyn is able to handle broken C# files (my plan is to use this inside O2's REPL environment in order to give much better error messages when the AST is broken)

Video:

O2 Script:gist
https://gist.github.com/2903012

Using Roslyn to Load and (quickly) Compile C# Solution files (outside VisualStudio)

Now that we can use Roslyn to compile (and create assemblies) in real-time the next step is to use it to open up and consume C# Solution files (something that I've always wanted to do properly without needing to use VisualStudio or deal with the xml format used by the *.sln files).

Weird Roslyn.Utilities.SerializableDataStorage casting issue when using new version

While writing a couple Roslyn scripts I started having a weird problem when consuming the June 2012 CTP dlls.

I was having the exception:

Unable to cast transparent proxy to type 'Roslyn.Utilities.SerializableDataStorage'. Unable to cast transparent proxy to type 'Roslyn.Utilities.SerializableDataStorage'.

3 pdfs on how I added Roslyn support to the O2 Platform (lots of Roslyn Script examples)

As you saw on Video: Real time Vulnerability Scanning using Cat.Net and Roslyn (SAST) and Video: Using O2's Roslyn-based C# REPL Script environment I was able to add a good number of Roslyn capabilities to O2.

Video: Real time Vulnerability Scanning using Cat.Net and Roslyn (SAST)

Here is a pretty cool PoC of creating an environment for developers that helps them to know when they create a security vulnerability (in real time).

Video: Using O2's Roslyn-based C# REPL Script environment

Roslyn is a new Microsoft's C# compiler written in C# (with also a VB flavour)

It is actually quite a spectacular piece of technology and in a way there is a LOT of similarities between the REPL environment I created in the O2 Platform and the one Roslyn is trying to do. In fact I'm thinking of moving some of O2's scripting capabilities from the current SharpDevelop parser to this new Roslyn one since their object model seems to be MUCH more powerful (of course that I'm bias, but I think that the O2's REPL environment as a couple killer features missing in current version of Roslyn REPL, so let's create the best-of-both-worlds REPL environment :) )

I've started using Roslyn in O2, and the first thing to do was to create an REPL environment which you can see in action in this video.

I am documenting the 'scripts evolution' (as drafts), and here are two I just created:

• Using Roslyn (first tests).rtf  (series of scripts that ended up with the Roslyn-based REPL enviroment shown above)
• Using Roslyn (consuming Static and Sematic data).rtf (execution of the scripts from the high-recommended Whitepaper: Roslyn Project Overview )

I think there is HUGE potential with this technology, and one area that I'm specially interested in is doing Security-focused Static Analysis, with integration with TeamMentor and Cat.NET (see Video: Injecting TeamMentor into Cat.Net running inside VisualStudio )

You can get Roslyn from O2 (it will install the package from NuGet), but if you want to try the visual studio integration you need to get the Microsoft “Roslyn” October 2011 CTP

If you want to comment this post or video, please use this reddit thread

Some Roslyn References and articles:

• http://msdn.microsoft.com/en-us/roslyn  (main site)
• http://social.msdn.microsoft.com/forums/en-us/roslyn (forums)
• Two draft docs published by Roslyn team 
• REPL and Code Semantics: http://tinyurl.com/3kprrlk
• Scripting APIs: http://tinyurl.com/3ezykda
• Channel 9 videos:
• The Roslyn Project - Meet the Design Team 
• Going Deeper with Project Roslyn: Exposing the C# and VB compiler’s code analysis 
• http://blogs.msdn.com/b/visualstudio/archive/2011/10/19/introducing-the-microsoft-roslyn-ctp.aspx
• http://blogs.msdn.com/b/csharpfaq/archive/2011/12/02/introduction-to-the-roslyn-scripting-api.aspx
• http://visualstudiomagazine.com/articles/2011/11/16/the-roslyn-scripting-api.aspx