Saturday 27 October 2012

New GitHub structure for the multiple TeamMentor development and deployment scenarios

After a lot of internal debate we finally arrived at the following GitHub architecture (the text below is based on Jason's last email):

Current set-up and logical Divisions:

  • TM code and app are free, we can give anyone access
  • Tool integrations (like CX or CAT.NET) are free and delivered separately
  • OWASP content is free (released under a CC license), we can give anyone access
  • Top Vulns library is used by the services team and is free
  • For each of the other libraries SI Customers have to pay for and they can be delivered as a combination or one at a time.
  • There are internal tools, such as PowerTools and Unit Tests that aren't normally delivered to customers, but can be given for free

For that reason we use a repo for the TM code, a repo for internal tools and unit tests, a repo for each tool integration and then a repo for each library.:TM code,  TM Docs Library, Tm4tm library, Top Vulns Library and OWASP Library are public, the rest are private.

Customer Purchase:
When a customer purchases TM they need their own source tree to work within and to customize. It should not be visible to anyone else since it'll contain content that is their internal IP. We give each customer a repo that contains the code we delivered to them and then each time we ship a new version we'll push to that repo and merge changes.

GitHub Organization:

  • SecurityInnovation Organisation: 
    • Contains what's there now minus TeamMentor
  • TeamMentor Organization (the 'Code' repos) 
    • Main TeamMentor codebase, but no content. 
    • Includes unit tests, power tools, tool integrations
    • Example of repositories:
      • Master , Unit Tests, Cat.NET, Power Tools
  • TMContent  Organisation (the 'Content' repos)
    • All TeamMentor content libraries (there should be no code in these repos)
    • Example of repositories (each a TM Library):
      • OWASP (with CC License)
      • TM Docs (public)
      • Tm4tm Library (public) - Tm4tm = TeamMentor for TeamMentor
      • Top Vulns (public)
      • SI Library (private)
      •  .NET 2.0 , .NET 3.5 , .NET 4.0
      • Android and iOs
      • C++
      • Java
      • PHP
      • CWE and PCI
      • etc...
  • TMClients Organisation (the 'Content + Data' repositories): 
    • Repos containing both TM source code and TM Library (in order to make it easier to track deployments and custom changes)
    • Repos for each TeamMentor 'client; (which includes SI sites like the fork) 
      • TM + OWASP Library (the 'eval' version)
      • TM + SI Library
      • version
      • Customer A
      • Customer B
      • Customer C , etc..

One of the core ideas/objectives of this structure was to have a clean separation between Code and Data (i.e. TM Source code and TM Libraries). We arrived at a solution based on multiple GitHub Organisations because their website works better that way.

It also helps in the separation of duties/focus since the TeamMentor GitHub organisation is for developers, the TMContent is for content writers/editors and the TMClients is for TM Consumers (and DevOps). Each has its own timings, dynamics and workflow.

Interestingly there is another type of data that we will be soon creating a lot of, which is 'User specific' information. For example if we were to have detailed logs of user's activities, or allowed the user to perform particular actions to an article (see First pass at creating ‘user activities’ (to be removed from 3.0 release) ) where would we store that information?