I wanted to leverage some of the thoughts in medium.com/digital-anatom… to break down what Flow Architecture is and why it is important. 1/18
The term "flow" in this context is meant to illicit images of constant movement in multiple planes; constrained by context and physics, but filling the space available. It is not a new concept--we depend on the science of flow dynamics for liquids and gasses in every day life. 2/
Even data flow architectures have been defined since the mainframe days. A decent overview of key concepts can be found at tutorialspoint.com/software_archi…. However, most of those architectures are batch oriented or focus on constraints in available processing and network resources. 3/
Cloud changes all of those constraints. Doesn't necessarily eliminate them, but certainly moves the bar several degrees of magnitude. Compute can be had on demand (for a price). Network bandwidth is much less of a constraint than it once was. Storage is ridiculously cheap. 4/
All of which leads to new dynamics that were once crazy to consider. The most important of these things, in my mind, is the ability to process what was once batched individually in new real-time architectures. Reduce reaction from hours or days to seconds. 5/
How I came to realize the importance of this is an interesting story that illustrates much of what I'm arguing here. I worked at SOASTA (now a part of Akamai) for about 18 months. One of their leading products was a Real User Measurement (RUM) product called mPulse. 6/
mPulse collects data in real time from every browser that interacts with your web site for things like the page loaded, the load time for each element on the page, what linked resources were loaded, etc. It then provided broad analytics capability to use on that data. 7/
It was an incredibly powerful tool. But its power multiplied when the analytics engine allowed you to display external data in the same graph as the page data. One of the first things done for retail customer was putting conversion data on the same graph as performance. 8/
Suddenly, you could see in a very graphic way what performance did to sales. Other "correlations" were also discovered that demonstrate the impact of tech metrics on business outcomes. Now SOASTA did something really smart… 9/
They created an integrated display system that would turn a wall into a highly visual display of almost anything the performance team deemed important to monitoring site performance, including mPulse graphs. But some teams added news feeds, current page displays, etc. 10/
For the first two customers in which they installed the prototypes of this display system, something really interesting happened. 11/
Within two days, both had senior business executives (marketing and digital) standing in the doorway asking if they could get their people involved. In other words, user outcome data had bridged the divide between tech and business. slideshare.net/jurquhart/the-… 12/
The icing on the cake, however, was that this data was being displayed with a maximum of a 10 second delay (well the mPulse data, anyway). It was, for all intents & purposes, a real-time feedback mechanism for how a campaign was going, how an article was being consumed, etc. 13/
When I saw that, it hit me. Real time data feeds were going to change the pace at which business would be able to react to the events around it. This was already happening in many places of course. High speed trading, for instance. But there were many more opportunities. 14/
So, what needed to happen to shift the tide to real-time? Well, the aforementioned benefits of cloud is one facet, but what was really critical is the (re)introduction of event driven mechanisms via "serverless" services. 15/
Being able to write a function to react to an event without deploying infrastructure is game changing. Being able to quickly queue arbitrary events for use cases that need that is game changing. It is increasingly possible to adopt a "real-time first" arch philosophy. 16/
Much more is needed before real-time can be a true default, however. There is no "standard" event subscription mechanism to allow orgs publish events for others to securely consume. But when there is, watch out. The data will flow. 17/
More of my thoughts about Flow can be found on my blog at medium.com/digital-anatomy. (The name, BTW comes from an analogy of data flow to blood flow.) As always, I welcome any and all feedback, as I write to learn. 18/18
