Hybrid Transactional / Analytic Processing with Temporal Linked Data®

This weblog is about auto-generating a high-throughput, low-latency, resilient, reliable, scale-out, infrastructure-saturating microservice application, with realtime projections for graph analytics, based solely on a set of bespoke RDF data models.  The history of all writes are preserved along side their respective aggregates, providing for a temporal representation of all changes—application time travel for free.  


CMB Radiation View of the Universe’s Original State, Courtesy NASA / WMAP Science Team


We call this clustered transactional capability Temporal Linked Data® (TLD).  Runtime and persistence is provided through the BEAM ecosystem by way of Docker containers and container orchestration.


TLD generates well-written Elixir on world class BEAM web frameworks.  Auto-generating the backend solution is accomplished by reading a set of RDF data models that represent aggregates of OWL Datatype Properties and that contain concrete links between aggregates by way of OWL Object Properties.  These self-describing RDF models enable us to generate: 1) router endpoints, 2) a RESTful API with JSON payloads, 3) Elixir data modules, 4) OTP GenServer and Elixir process registry modules by way of servers and workers in a well-supervised distributed process hierarchy, and 5) change-data projections into scale-out big-data graph analytics platforms.  The result is a highly concurrent, highly reliable enterprise backend.  



Iteratively Deployed TLD Microservices with Projections to Big Data Graph Analytics


The above sketch depicts autonomous TLD microservices asynchronously projecting writes to a common, scale-out, big-data graph analytics platform.  TLD services are intended to be delivered early and often, over time building up heterogeneous data for unique insights.


This is an auto-generated “graph throughout” solution architecture that is best described as a specific type of Hexagonal or Port-and-Adapter architecture where, by default, the transactional graph data structures flow through to the analytic graph structures to provide a realtime 360 view of the business, for The Business.


Alan Strait