MAYA and Trillions miss the point of OpenSource

So I've been reading the Trillions book at night and am starting to realize that the authors have a pet-hate with OpenSource.

They fall for all the cliches when criticising OpenSource and in a really weird and surreal way, they have not noticed that the Open Source development model is a real-world example of the Trillion node network they talk about (see When I think of Trillions, I think of Source Code Blocks).

Some of their comments on the fact that Open Source code is 'created by amateurs', 'have very little quality', 'does not foster innovation', are 'the reason our our TV crashes', etc...  are just silly  (it's like those chapters where written in early 2000s).

I guess nothing like that happens on proprietary world!!! which is famous for creating flawless software!

The bottom line is that there is good and bad with both worlds, and both (open and proprietary code) suffer from the 'Complexity and lack-of-engineering' problem, correctly identified by the authors.

I guess its the old saying '...Its hard to understand something when your income depends on not understanding it....'. 

Maybe the fact that MAYA is a commercial research center and they are paid to do research, means they don't like the open world of Open Source. Humm ... a search for Open Source and GitHub at Maya.com doesn't produce a lot of hits.... I wonder why?    :)

For me Open Source is much more than just sharing code. Open Source (even more than Free Software) is a way to share information, use the best technology out there and learn from the masters.

Btw, don't get from this post that I think the Trillions book is bad. Trillions is one of the best books I have read in a while (and you should read it), it is just that they Open Source ideas are completely bias and stereotypical

When I think of Trillions, I think of Source Code Blocks

The guys at MAYA when they talk/write about Trillions they are thinking of information package like a container , tagged with a U-Form UUID and with liquid properties

But for me, what I'm thinking of is Source Code Blocks (Methods, Classes, Modules, Assemblies) and what they do (parsers, filters, data transformation, data presentation, business-logic activities, workflows, user interfaces, etc....)

One of the problems with have with the 'software-driven applications' that we create every day is that after a while, there is nobody that really understands how the whole system actually behaves.

And the reason is simple: Too much Complexity.

Unless an application is built in Assembly, the code written by the programers is executed against a number of abstraction layers, each with its own behaviour, reality and side-effects.

And since we currently don't have a way to model that behaviour, we end up with the current situation where we 'Code and Execute it to see what happens (i.e. see if it does what the programmer/manager/architect/buyer is thinking that it will do)'.

SAST technology and run-time-analysis are the key since we need to be able to model an application's behaviour and create rules that describe the expected (or not expected) traces, activities, practices, etc...

But for that we need to approach application behaviour analysis (which is what SAST is doing) in a different way.

We need to apply the Trillions concepts and look at a piece of software that has trillions of nodes (i.e. code blocks).  And like the the MAYA guys like to say, this has already been done by nature , we just need to apply the same concepts :)

Btw, I really like the idea of applying UUIDs to bits of code. This is one of the key missing pieces of the current Sandboxing puzzle and one of the ways we can scale.

U-Forms and Information Liquidity

Here is a great presentation from Maya's Mickey McManus on Information Liquidity:

At the end of the presentation the idea of U-Forms is presented as the 'container' for information (more info here and here)

It looks like an U-Form is made of an ID and set of value-pairs atributes, which is a nice a simple set of pointers.


Related posts:

• Design for Fork and the liquididy of OpenSource/Git 
• Trillions from MAYA (see the video, buy the book) 
• In SAST the issue is 'Trace Connection', not 'Scan Size' 

Design for Fork and the liquididy of OpenSource/Git

If you are designing an application that you will share the code on GitHub (open source or not), the most important advise I can give you is to 'Design for Fork'

What I mean is that the idea of 'creating a fork (or clone) and starting a new instance of the app' should not be something that is left to the Open Source Gods, as in '...hey the code is available so you can go out and run your own instance...'.

If your app is not 'Fork/Clone Friendly' then it will always be a pain to deploy and use, and you will be missing a very key mechanism to keep complexity at bay. Deployments should be measured in seconds/minutes and with only a couple clicks required.

Maya talks about Information Liquidity in their Trillions concept where information is supposed to flow like a liquid and be able to move over rough , uneven or new surfaces by being liquid. Note that although there is a common carrier on any liquid (water/H2O) there are lots of different types of liquids (i.e. information) that can be carried, packaged, exchanged, consumed, etc...

This is liquidity is exactly what we should aim to get from Open Source or GitHub repositories. Its code AND content, must be easily consumed, exchanged, manipulated, modified, etc...

For me, this concept is not something theoretically, it is something that I put in real practice while developing TeamMentor and the O2 Platform

One of the areas of TeamMentor that I'm more proud of, is how the latest version is very light to install, deploy and set-up (the only requirement is .NET 4.0 and in some cases IIS ). There are no databases to install/configure, the source code is included with the deployment, the content/articles is all stored on XML files and we use Git/GitHub not only as version/control control , BUT as a distribution medium.

The idea of storing the content on the filesystem is very important since part of making your data liquid is to have NO database (as in MSSQL, MySql, NoDB, etc...).

My recommendation is to store the data on file system (as files) and use GIT (and GitHub) as your database. You will not create something more powerful and flexible than Git and it is a very scalable and robust solution for content control.

Trillions from MAYA (see the video, buy the book)

The Trillions Video is one of the most important videos that I have seen over the last couple years and one that gave me a nice warm felling that I'm doing the right thing with my O2 Platform development strategy.

They have now released a book Trillions: Thriving in the Emerging Information Ecology which I have started to read (on IPad's Kindle) and if you want to understand what will happen next, you NEED to read it.

A key message in the video and book is that to deal with new paradigms and systems,  we need complete new strategies, approaches, tools and ideas.

And that is exactly what I'm doing with O2 Platform. Instead of doing what just about every other Security tools vendors is doing (i.e. 'trying to create a 'blackbox' solution with some customisation features on top'), I'm creating an environment/platform where Scripting and Customisation are first-class citizens. In fact most of the O2 Platform is already 'scripts' and the expectation is that when facing the target application/website, the question is not 'do we really need to customise our technology/tools/approach?' but 'how fast can we customise our technology/tools/approach so that it actually represents reality?'. 

It's the customisation-time-delta that matters, and of course that the faster that happens, the more we (Application Security Knowledge) will scale :)

Back to Trillions, I see Application Security (and its complexity) as they see Trillions. Each node (from source-code to app's behaviour) is something that needs to be analysed, modeled, managed, controlled and (sometimes) fixed.

In fact, a business model that still yet to take hold in our industry is 'Security Tools/Technologies/APIs Customisation Services' (with clients paying for it and service companies providing it)

Btw, MAYA company and research is simply amazing and their focus on Design is a great inspiration  (what a great place to work that must be). Checkout their other videos (http://vimeo.com/mayanmaya) and research (http://www.maya.com/practices/research).

Even their name is really powerful, since MAYA means Most Advanced Yet Acceptable.

Finally, if you want to explain what 'Is An API?' to a non-developer audience, point them to MAYA's latest video on Containerization (I would love to have videos like this to example how SAST, DAST and even O2 works :)  )

In SAST the issue is 'Trace Connection', not 'Scan Size'

One of the 'wrong problem to be solving' paradox that happens in the SAST world is the focus on making their engines able to 'scan large code bases'. It is not a coincidence that the key question I got from SAST engine guys on the Real-time Vulnerability Creation Feedback inside VisualStudio (with Greens and Reds) was 'Humm.... interresting but will is scale to large applications?' 

I actually blame the SAST clients for this, since they are the ones asking (and paying for) the wrong question:

 "How can you 'vendor xyz' scan my million lines of code application"

Instead they should be asking:

 "When you scan my code, can you connect the traces?"

'Connecting the traces' means that you are able to scan parts of the application separately and then connect them at a later stage.