An Odyssey into Open Technologies
I have returned from a two month “Technology Walkabout”.
My ritual journey began with a discussion with a colleague about emerging technology trends.
As a Microsoft-centric Technologist for more than twenty years and a past owner of a Microsoft Certified Solution Provider in the Silicon Valley, I engaged in the conversation as an evangelist for the ASP.NET / C# religious congregation.
The longer I engaged in the intellectual idea exchange the more I realized that I had become a “Technology Prisoner” within the confined walls of the Microsoft world.
This post is not a slam on Microsoft …
… It is a journal of an experience of sudden and striking realization
… An Epiphany
I have made a very good living with Microsoft and it has provided for my family consistently over the years.
For years I have minimized the “Noise” of the advantages of “Open Source Technologies” over conventional Technology Stacks.
After the discussion with my colleague about emerging technology trends and the Open Source Community I could no longer, in good conscience, dismiss his rather passionate arguments.
It was my vision of the TMD Semantic Design using Roy Fielding’s HATEOAS concept and the level three Leonard Richardson’s Maturity Model REST requirements that I had my epiphany.
The Traditional Data Impedance Mismatch
To me is was natural to serialize and deserialize the JSON data from the MVC Controllers, to create C# Data Transfer Objects (DTOs) for Plain Old CLR/C# Objects (POCOs), and to use an Object Relational Mapping (ORM)framework to transform the DTOs into something a relational database such as SQL Server could understand in a formal relational Write Schema.
JSON → DTO → ORM → SQL
This “Data Impedance Mismatch”, when explained to me and after some attempt to justify it, I suddenly realized how much time, and client’s money, had been spent in managing the inherited artifacts associated with this mismatch.
A career changing “Moment of Clarity” occurred when I tied what superficial knowledge I had with Big Data and HADOOP with the concept of document-based persistence.
NoSQL and the JSON Data Impedance Solution
Here is a question I have asked numerous times during my walkabout:
When asking technologists that make a living in the relational SQL database world the answer was always: On Write, no exceptions.
When asking technologists that make a living in the NoSQL database world the answer was: On Read or never.
The belief that schema is required on write, in my opinion, is one of the reasons that Thought Works January 2014 Technology Radar, a Martin Fowler company, has placed Enterprise Data Warehouse Technology in the “On Hold” maturity ring.
The issue of Write Schema reveals itself within large and small relational storage schemas when assumptions are required for the schema designs that change during the development and before the deployment of those schema.
A well-known Health Care Company Commissioned a New Enterprise Data Warehouse …
… Spent $10 million and Two Years of Development
Only to Realize the Schema No Longer was viable for their Business Requirements
… They abandoned the entire project after failed attempts to generate useful Business Intelligence
The Health Care industry changed so drastically during the development that the very business justification for spending $10 million could not be achieved with the constraints of the deployed schema without adding additional complexities and data impedance mismatches.
Had the raw data simply be archived with an ID field and the related proper metadata and data as a single, non-relational, piece if data the information could have been harvested and organized based on the temporal requirement.
A schema for presenting the relational data could have been delivered generating useful Business Knowledge that would have create valuable Business Wisdom at the time of the request.
… Data Schema is ONLY Required on READ …
…… WRITE Schema create Constraints on the Data
… Based on the Requirements at Design Time
…… Not the Requirements at Run-time
The concept of processing JSON end-to-end,
from the HTML Form to the NoSQL Database
and back again was a major paradigm shift
in my architectural world
The Technical Radar for TMD Semantic Design
The TMD Semantic Design Technical Radar represents my in-depth study of the currently available technologies within the Open Source Community.
I have selected the forty technologies represented in the first three maturity rings as the results of my study.
I tried most of the technologies and compared what is trending, well supported and accepted within the Open Source Community as best I could in my two month excursion.
The Maturity Rings and Category Quadrants
The inner ring represents the core technologies for each of the four quadrant categories:
The next ring lists the Technologies that are trending and have been selected as core technologies at this time.
The third ring shows technologies that show great promise and are part of the current Semantic Design framework.
The outer ring lists viable technologies that are primarily not Open Source but can be part of hybrid solution stacks, if required.
Select the link on the image to be directed to the hosted “TMD TecDar”
Hover over any “Radar Blip” to view the technology
Select the blip to be directed to a Website that details the selected technologies
The Selected Core Technologies
The Three Main Core Power Trio Groups:
Node Express Web Services
Mongo Mongoose Schema Framework
MongoDB Monitoring Service
The Five Main Technology Power Trio Groups:
Node NPM Package Manager
Bower Dependency Manager
WebStorm Development Environment
Code Hosting Technologies
GIT Code Management
GitHub Code Repository
Dropbox Document Hosting Service
Project Hosting Technologies
MongoLab Mongo Hosting
MongoHQ Mongo Hosting
Heroku Cloud Hosting
Code Quality Technologies
Protractor End-To-End Testing with Selenium WebDriver
Karma Node Test Runner
Future posts will detail how these technologies will play into the semantics of the design model.
The selected technologies represent what has been coined the MEAN stack using BreezeJS as a Client-side data management framework:
The B-MEAN Stack
I have created a detailed spreadsheet of all the selected technologies with summarized information and the links that are provided on the TMD TecDar Website.
Stay tuned for more on the implementation of my Technology Journey into the exciting world of Open Technologies
Wisdom Pearl # 140 – Tools of the Trade
The Right Technology Tool
… For the Right Technology Solution Job
…… Always be Diligent in Keeping Your Development Toolbox Up to Date
The following two tabs change content below.
I am a Principal Architect at Liquid Hub in the Philadelphia area specializing in Agile Practices as a Certified Scrum Master (CSM). I use Test Driven Development (TDD) and Acceptance Test Driven Development (ATDD) with Behavior Driven Development (BDD) as my bridge to Agile User Stories Acceptance Criteria in a Domain Driven Design (DDD) implementing true RESTful services