An Odyssey into Open Technologies

I’m Back!!!

I have returned from a two month “Technology Walkabout”.

My ritual journey began with a discussion with a colleague about emerging technology trends.

As a Microsoft-centric Technologist for more than twenty years and a past owner of a Microsoft Certified Solution Provider in the Silicon Valley, I engaged in the conversation as an evangelist for the ASP.NET / C# religious congregation.

The longer I engaged in the intellectual idea exchange the more I realized that I had become a “Technology Prisoner” within the confined walls of the Microsoft world.

This post is not a slam on Microsoft …

… It is a journal of an experience of sudden and striking realization

… An Epiphany

I have made a very good living with Microsoft and it has provided for my family consistently over the years.

For years I have minimized the “Noise” of the advantages of “Open Source Technologies” over conventional Technology Stacks.

After the discussion with my colleague about emerging technology trends and the Open Source Community I could no longer, in good conscience, dismiss his rather passionate arguments.

Semantic Design and the JavaScript World

I have resisted mastering JavaScript for the better part of my Web development career.

I was comfortable in the belief that as a Microsoft Server stack and back-end architect that I would leave the complicated aspects of JavaScript to the Front-end designers.

As a primarily MVC centric architect and developer I could rely on the Razor Engine and “Helper Methods” for validation and other JavaScript Client code.

This naive position allowed me to justify my position as just a “Consumer” of JavaScript and not become a real JavaScript programmer.

Don’t get me wrong, I could write competent, Object Oriented Programming structured code with namespace, JavaScript Closures using Prototype Inheritance when required, but it was generally from a scientific perspective and certainly not from an artistic perspective.

It was my vision of the TMD Semantic Design using Roy Fielding’s HATEOAS concept and the level three Leonard Richardson’s Maturity Model REST requirements that I had my epiphany.

The Traditional Data Impedance Mismatch

I have been designing and consuming data originally sourced as JavaScript Object Notation (JSON) for years without realizing the hoops I had to jump through to manage the information throughout my stack layers.

To me is was natural to serialize and deserialize the JSON data from the MVC Controllers, to create C# Data Transfer Objects (DTOs) for Plain Old CLR/C# Objects (POCOs), and to use an Object Relational Mapping (ORM)framework to transform the DTOs into something a relational database such as SQL Server could understand in a formal relational Write Schema.

JSON  →  DTO  →  ORM  →  SQL

This “Data Impedance Mismatch”, when explained to me and after some attempt to justify it, I suddenly realized how much time, and client’s money, had been spent in managing the inherited artifacts associated with this mismatch.

A career changing “Moment of Clarity” occurred when I tied what superficial knowledge I had with Big Data and HADOOP with the concept of document-based persistence.

NoSQL and the JSON Data Impedance Solution

Here is a question I have asked numerous times during my walkabout:

When asking technologists that make a living in the relational SQL database world the answer was always: On Write, no exceptions.

When asking technologists that make a living in the NoSQL database world the answer was: On Read or never.

The belief that schema is required on write, in my opinion, is one of the reasons that Thought Works January 2014 Technology Radar, a Martin Fowler company, has placed Enterprise Data Warehouse Technology in the “On Hold” maturity ring.

The issue of Write Schema reveals itself within large and small relational storage schemas when assumptions are required for the schema designs that change during the development and before the deployment of those schema.

A well-known Health Care Company Commissioned a New Enterprise Data Warehouse …

… Spent $10 million and Two Years of Development

Only to Realize the Schema No Longer was viable for their Business Requirements

… They abandoned the entire project after failed attempts to generate useful Business Intelligence

The Health Care industry changed so drastically during the development that the very business justification for spending $10 million could not be achieved with the constraints of the deployed schema without adding additional complexities and data impedance  mismatches.

Had the raw data simply be archived with an ID field and the related proper metadata and data as a single, non-relational, piece if data the information could have been harvested and organized based on the temporal requirement.

A schema for presenting the relational data could have been delivered generating useful Business Knowledge that would have create valuable Business Wisdom at the time of the request.

… Data Schema is ONLY Required on READ …

…… WRITE Schema create Constraints on the Data

… Based on the Requirements at Design Time

…… Not the Requirements at Run-time

 The concept of processing JSON end-to-end,
from the HTML Form to the NoSQL Database
and back again was a major paradigm shift
in my architectural world

The Technical Radar for TMD Semantic Design

The TMD Semantic Design Technical Radar represents my in-depth study of the currently available technologies within the Open Source Community.

I have selected the forty technologies represented in the first three maturity rings as the results of my study.

I tried most of the technologies and compared what is trending, well supported and accepted within the Open Source Community as best I could in my two month excursion.

The Maturity Rings and Category Quadrants

The inner ring represents the core technologies for each of the four quadrant categories:

1. Languages

2. Tools

3. Hosted Services

4. Frameworks

The next ring lists the Technologies that are trending and have been selected as core technologies at this time.

The third ring shows technologies that show great promise and are part of the current Semantic Design framework.

The outer ring lists viable technologies that are primarily not Open Source but can be part of hybrid solution stacks, if required.

Select the link on the image to be directed to the hosted “TMD TecDar”

Hover over any “Radar Blip” to view the technology

Select the blip to be directed to a Website that details the selected technologies

The Selected Core Technologies

The Three Main Core Power Trio Groups:

1. Client Frameworks

   1. AngularJS

   2. Bootstrap

   3. BreezeJS

2. Server Frameworks

   1. Node.js

   2. Node Express Web Services

   3. Node Jade

3. NoSQL Frameworks

   1. MongoDB

   2. Mongo Mongoose Schema Framework

   3. MongoDB Monitoring Service

The Five Main Technology Power Trio Groups:

1. Content Display

   1. HTML 5

   2. CSS3

   3. JavaScript

2. Project Technologies

   1. Node NPM Package Manager

   2. Bower Dependency Manager

   3. WebStorm Development Environment

3. Code Hosting Technologies

   1. GIT Code Management

   2. GitHub Code Repository

   3. Dropbox Document Hosting Service

4. Project Hosting Technologies

   1. MongoLab Mongo Hosting

   2. MongoHQ Mongo Hosting

   3. Heroku Cloud Hosting

5. Code Quality Technologies

   1. Protractor End-To-End Testing with Selenium WebDriver

   2. Karma Node Test Runner

   3. Jasmine BDD JavaScript Testing Framework

In Conclusion 

Future posts will detail how these technologies will play into the semantics of the design model.

The selected technologies represent what has been coined the MEAN stack using BreezeJS as a Client-side data management framework:

The B-MEAN Stack 🙂

I have created a detailed spreadsheet of all the selected technologies with summarized information and the links that are provided on the TMD TecDar Website.

Core Technologies

Stay tuned for more on the implementation of my Technology Journey into the exciting world of Open Technologies

Wisdom Pearl # 140 – Tools of the Trade

The Right Technology Tool

… For the Right Technology Solution Job

…… Always be Diligent in Keeping Your Development Toolbox Up to Date