Using NGit to create native Git support in Azure deployed app (with automatic pushes and pulls)

This entry will show a pretty powerful new feature in TeamMentor (TM) which I’m very proud and excited about!

This feature is so important, that it literary caused a delay on the release of TM 3.3 for about 1 month (my instinct was pushing me on this direction since I 'knew' that this could be done, and that it would be a killer feature). Btw, there is a lot more NGit/Git support than what is shown here, but I’m sure you will see the power in the workflow described below.

Basically, TM’s backend engine will now automatically perform:

• a git pull  - when the TM server starts (or it cache is rebuilt) 
• a git commit followed by a git push - on every library edit (on both content and structure changes).

To read: Reddit’s thread on ‘We are the Mozilla Security Community, Ask Us Anything!’

This looks like a great Q&A from Mozilla’s (quite impressively big) security team:

http://www.reddit.com/r/netsec/comments/1b3vcx/we_are_the_mozilla_security_community_ask_us/

Is Dove's Trojan horse illegal and if not, why not?

Assuming this article is actually true Dove Canada Uses Photoshop Trojan Horse to Shame Potential Body-Shamers , isn't this a variation of an 'unauthorized action' against a user's computer?

For example here in the UK,  couldn't this be used on an CMA (Computer Misuse Act) case?

After all, Dove knew they were doing an unauthorized action, and Dove is profiting from it.

My view is that the CMA (and similar laws) are completed crazy and make any Internet user a (potential) criminal. So with that in mind, what is the deal with this Dove trojan horse? Is that legal?

Here is a reddit thread about this article, that doesn't mention the legal side of things: Not a IFF post, but an excellent message from Dove about body image and Photoshop.

Extracting content files from a Azure deployed version of TeamMentor (pre 3.3 git support), starting with a failed SFTP attemp and ending with a CSharp REPL script

I was asked by Serge to retrieve some changes he made to a test version of TM hosted in Azure.

This site was hosted at https://tm-hashes.azurewebsites.net and since this was a version before the built-in Git Support (where git TM Libraries are natively supported by TM), the only way to get the files was to copy them from the live server.

So my first attempt was to use SFTP (which Azure supports) to connect directly to the web root.

Using Git Branches to fix Issues added to TeamMentor's GitHub repository

This is the currently workflow that I’m following when coding/fixing TeamMentor Issues added to the TeamMentor/Master/Issues list.

• Find issue to address
• Create and checkout new branch (with the issue ID on its title)
• Apply the fixes (on the new branch)
• Commit the changes (on the new branch)
• Checkout master branch
• Merge changes (from new branch) into master , using the --no--ff (no fast-forward) option (this is very important, see here and here for a good explanation why)
• Push to GitHub

Lets look at this in action.

Testing TeamMentor's password reset feature (now with token stored as a Hash)

In the The Power of UnitTests when refactoring code (for example Security Pages) post I showed the new TeamMentor feature of password reset.

This post shows an updated version of it which now stores the password reset tokens using PBKDF2 hashing.

Creating a version TeamMentor which uses the new GitUserData.config file

Introduced in the 3.3 version of TM is a new feature to load the UserData repository from an external location (GitHub or local folder).

This post shows how to set it up.

First step is to get the latest version of TeamMentor from GitHub, where we can clone it locally or download the zip file

Improving website performance via GZip compression and 304 redirects

If you look at TeamMentor’s traffic (using for example Chrome developer tools), you will see that (for the version https://tm-www.azurewebsites.net ) on the 2nd load, there will only be about 307k downloaded in about 8secs:

TeamMentor 3.3 RC4 - Final tests, please have a go

Hello, we're on the final stretch for releasing a major update to TeamMentor and it would be great if you give it a test drive!

The current release is called TM 3.3. and is mainly focused on a number of changes we developed specifically for our TeamMentor.net version (i.e. the public read-only version of TeamMentor). The source code is published on the main TM repository, at the 3_3_Release branch (see  https://github.com/TeamMentor/Master/tree/3_3_Release )

Changing the ‘View TM article by anonymous users’ status via GitHub

From the 3.3. release of TeamMentor (TM) it is now possible to change configuration settings of live servers directly from GitHub.

For example I just published a QA version of the https://services.teammentor.net site on Azure’s https://tm-services.azurewebsites.net

Here is what https://services.teammentor.net  (on version 3.2.3) looks like:

My comments on the SATEC document (Static Analysis Tool Evaluation Criteria)

(submitted today to the wasc-satec@lists.webappsec.org list)

A bit late (deadline for submission is today) but are my notes on the version currently at http://projects.webappsec.org/w/page/41188978/Static%20Analysis%20Tool%20Evaluation%20Criteria

My comments/notes are marked as Conted to add in underscore, bold and Italic or [content to be deleted in red]

When I wanted to make a comment on particular change or deletion, I did it on a new line:

DC Comment: ... a comment goes here in dark blue

Of course that this is my opinion, and these notes are based on the notes I took in 'analogue mode' (i.e. on paper :)  )

Using RazorSharp page to create CSV list of TeamMentor users

One of the feature requests for the 3.3. release of TM was to have a CSV exported list of users (see Provided CSV of user accounts for Tom)

In the previous version of TM adding this feature would required a number of server-side+GUI code changes, or the use of ASPX pages (which I specifically didn't want to use).

But since 3.3 supports RazorSharp (*.cshtml files), we can now create such CSV export page just like this:

Writing RazorSharp script to import TeamMentor users

Now that we are in the final stages of publishing TM 3.3. out of the door, one of the tasks I had to do is to import the old user schema into the new format.

To to that I wrote this RazorSharp script (which can be coded in a REPL like environment) to do the import:

HSTS in TeamMentor

The latest version of TeamMentor adds the extra HSTS protection (see issue teammentor.net should use HSTS)

But what does this mean in practice?

Util - Current Font Families.exe

Today I needed to quickly see a list of available fonts in my current VM, so I quickly wrote this script:

Util - Jni4Net - Java BeanShell REPL v1.0.exe

Using the technique shown in the Invoking Java BeanShell from .Net CLR  post, here is REPL that allows the quick execution of Java BeanShell command in a C# GUI

You can download this stand-alone O2 tool from: Util - Jni4Net - Java BeanShell REPL v1.0.exe

And this is what the default GUI looks like:

Why does Java/Oracle keep trying to install CrapWare on by VM? And why their sales team love java vulnerabilities!

Everytime I do an update to Java (which is quite a common event these days), I get something like this:

ScreenHero for desktop sharing looks pretty powerful

Check it out at http://screenhero.com

Download of O2 Platform Stand-alone-tools that run on OSx

Continuing on the OSX Theme (see last couple posts) I updated the O2 Platform download page with a number of Stand-alone-exes that work on both windows and OSX:

Running O2 Platform's main C# REPL script on OSX (wasn't working before)

After the last couple posts on O2 and OSX I decided to have a quick go at running the main O2 C# REPL on my Mac and was very pleasantly surprised when it worked!!!

Here is a screenshot of the PoC - Roslyn C# ScriptEngine Execute v1.0.exe (which you can download from here) :

Problem running O2's Exe on OSX 10.8 , fixed using XQuartz

I just tried to run a simple O2 Platform Exe on OSx (which used to work) and got this:



I.e. nothing happened!

Installing Mono and MonoDevelop/Xamarin on OSx

If you want to run O2 Platform scripts/exes on OSx, you need to start by installing Mono, Xamarin and (most likely) XQuartz

Go to http://www.go-mono.com/mono-downloads/download.html and click on the Beta version 3.0.6 Mac OSX image

Getting list of Jars loaded in SystemClassLoader (using Jni4Net)

I just created a couple extension methods for Jni4Net that allow (amongst other things) the listing of the jars currently loaded in the SystemClassLoader (see API_Jni4Net.cs for the code of these .NET Extension Methods)

The objective is to simplify the use of Jni4Net, and to hide the complexity in consuming Java code from .NET:

Here are a couple examples of these Extension Methods in action:

What do the Twitter backups downloadable files look like

After about a minute from sending the request shown at the end of Feature request: Tweet backups to Git/GitHub ,  I received an email with:

Feature request: Tweet backups to Git/GitHub

Here is a feature that would help to make the most of the amazing knowledge that is daily shared on twitter.

The idea is to have tweet data stored in a git repository which would contain:

• Tweets (all available info for each tweet, which is a LOT more than what is currently exposed on the multiple tweet client apps)
• Followers
• Following
• Connects (when others reference the account)

Ideally there should be a commit for each item , but a daily commit should also work.

Loading OWASP ESAPI jar and its dependencies from C# (using jni4net)

Here is a pretty cool PoC where I was able to load an jar file and its dependencies into an 'Jni4Net created' JVM

Invoking Java BeanShell from .Net CLR

Here is a very rough PoC of how I was able to execute a JavaBean shell script from inside the O2 Platform (with the java code executed under a JVM)

Executing "return 2+2;" as a java beanshell comand (see result on the bottom right Output pane)

Invoking an OWASP AppSensor Java method from .NET C# REPL (using Jni4Net)

On the topic of AppSensor, you might find the code snippet below interesting. 

Inside an O2 Platform C# REPL editor (which is running in .Net's CLR), I was able to:

• load the AppSensor jar in a new class loader, 
• access/view its classes in a GUI 
• create an instance of org.owasp.appsensor.trendmonitoring.TrendEvent 
• execute the getTime method). 

Note that the AppSensor code is running on the Java's JVM (loaded in the same process as the .Net's CLR)

The code is still in very rough status, but it works :)

Where to have AppSec Q&A threads (what about Reddit?)

Note: I wrote this a while back but somehow was stuck on my 'Drafts' folder (but the question is still relevant in March 2013)

---

So it looks like StackExchange Security is not going to work for WebAppSec and OWASP (since this question is exactly the type of question we should would like to see there How to implement url encryption on .xsl page using OWASP ESAPI?  and that has been closed)) . That said, there are a couple good Q&A on the OWASP tag: http://security.stackexchange.com/questions/tagged/owasp

Putting O2 content on Google Code's wiki (just like ZAP)

I really like what Simon is doing with Zap at https://code.google.com/p/zaproxy/wiki/Introduction?tm=6 and I think we should do the same with O2 (I would like to have used the main OWASP wiki, but it's to messy, heavy and lacks the ability to create a side navigation)

It will be at https://code.google.com/p/o2platform/wiki

The idea is that the O2 related blogs entries are used for how-to articles, and the wiki pages contain consolidated content and references links (to those blog entries)

This would give O2 users good kickstarters on particular topics, for example https://code.google.com/p/o2platform/wiki/Browser_Automation :)

If you want to help editing these pages, send me your google account id and i'll make you an editor

Writing an IE Automation script to login into UK’s Wifi (using O2 Platform’s WatiN ExtensionMethods)

Here is an example of how to write an O2 Platform IE Automation script that will login a user into a wifi connection that needs a username and password.

Open the IE Script tool which you can get from this stand-alone version (see Packaging an O2 Platform Script as a stand alone tool (in this case the WatiN based ‘IE Script’ tool) )

Packaging an O2 Platform Script as a stand alone tool (in this case the WatiN based ‘IE Script’ tool)

If you grab the latest version of the O2 Platform and try to run the IE Script tool

Creating a new TeamMentor test site using TeamCity, GitHub and Azure

Serge just asked me to create a new TeamMentor (TM) website for him using a particular TM library, so here are the steps I took (note: some of this will be automated in the next TM release)

Prob with (older version of) NGit where it was failing to create Git repositories in Azure/TeamCity

Using an NGit version from a couple months ago.

What happens when Asp.Net not installed on Windows 8 server

If you get an error like this:

Manually adding a code complete reference to the FLuentSharp C# REPL editor (using a 'Script the Script' editor)

Let’s say that you are in the C# REPL editor and you want to manually add a dll to be taken into account by the code complete engine (useful in the 5.1 version of the O2 Platform which had a bug that prevents some references from loading)

For example, let say you added the O2_FluentSharp_NGit.dll reference:

Setting up a apache (httpd) based git server (using an O2 Platform script)

Following from the instructions on this blog post Hosting a Git server under Apache on Windows and after installing git and apache locally

I wrote this O2 script:

Really nasty bug created by different behavior of WCF Security Principal in Azure (vs Locally in IIS or Cassini)

A couple days ago, I spent most of one day 'bashing my head against a brick wall'  due to an authorization bug that only happened in Azure!

Here are the rough screenshots I took when I was debugging it (the extra logging entries where added to help me understand what was going on (in Azure and locally))

Catching an Exception in RazorSharp

Here is a cshtml RazorSharp page that shows how to catch an exception (note the multiple mixes of code and HTML)

The Email RegEx that (could had) DOSed a site

While I was writing the UnitTests for TeamMentor's NewUser validator (see Validating a POCO DataContract using .NET's DataAnnotations Validator ), I had a weird result in one of the tests.

I basically got a 'never ending execution' scenario on this UnitTest:

Validating a POCO DataContract using .NET's DataAnnotations Validator

In order to make sure that the TeamMentor server only creates users with valid data, here is how I implemented data validation into the NewUser class using .NET's DataContract annotations.

The first step was to add the annotations to the NewUser object, which originally looked like this:

Another GitHub ‘Normal’ status that doesn’t allow me to Push

It took me 5 attempts over about 10m (from 7:40pm on 11th Mar 2013) to get a commit pushed into GitHub’s servers:

What happenned to my CPU and Memory when I restarted VisualStudio 2010

When I shutdown VisualStudio 2010

How PBKDF2 interactions affected UnitTests performance by 2.5x

With PBKDF2 support enabled all the time: 1min 20sec

Adding an Id field to an HTML element using jQuery (to help UnitTesting)

Michael just asked me to add an Html ID attribute to help him with the UnitTesting of teamMentor:

Is this a safe way to do a .NET Server Redirects? (and deal with A10: Unvalidated Redirects and Forwards)

The objective is to prevent A10: Unvalidated Redirects and Forwards in TeamMentor (version 3.3 had an issue with it)

Here is the code that does the redirection from user import (LoginReferer parameter):

Why we use Cassini instead of IIS Express

Following the IIS vs Cassini different behaviors on ASP.NET Server.Transfer  post, Barry commented on Twitter that I should use IIS Express instead:



so I decided to try it (which you can read bellow)

GitHub.com needs to improve their the 'Normal' status definition and error reporting dashboard

At the moment (i.e. when I wrote this post), all should be 'Normal' with GitHub, since according to their status, their world looks like this:

IIS vs Cassini different behaviors on ASP.NET Server.Transfer

Here is the problem.

Opening the page http://localhost:3187/passwordForgot (which is served by Cassini) works OK:

Seeing an NGit Diff by using reflection to access the internal Sharpen.ByteArrayOutputStream Class

I was trying to get the NGif diff output stream, but hit on an issue that the Sharpen.ByteArrayOutputStream class is internal

The Power of UnitTests when refactoring code (for example Security Pages)

Following a use-case usability requirement, I just changed the new TeamMentor ’password forgot’ page ...

Viewing the C# MethodStream for a WebMethod

I was debugging an issue with TeamMentor WebServices and created a view that gave me the MethodStreams for all its source code

MethodStreams are virtual files that contain all relevant 'call-flow source-code' for a particular starting methods (note: MethodStreams are one of O2’s biggest innovation on the SAST world)

Using Chrome inside a native VisualStudio pane (using Window Handle Hijacking)

To help me debug and visualize an AngularJS page I was developing, I used the O2’s Window Handle Hijack technique to insert an Chrome window inside VisualStudio 2010.

Here it is in action: