Credits & Acknowledgements¶
Building something as audacious as Macrometa’s GDN cloud would not have been possible without the extraordinary contributions of many people. We are deeply indebted to the following people for the extraordinary body of research work and publications that have enabled us to build Macrometa.
Christopher Meiklejohn (Consulting Scientist to Macrometa)¶
Christopher’s research into state and operational Conflict Free Replicated Data Types (CRDTs see next section) & implementations such as LASP, Partisan and work on AntidoteDB, enabled us to bring the best ideas from CRDTs to our vision for a geo distributed database. We lucked out when Chris agreed to help us with formally verifying our approach and provided lots of valuable input on our operational CRDT and garbage collection strategies. In addition, Chris is just one of the most humble and great people we have had the good fortune of collaborating with. http://christophermeiklejohn.com
Conflict Free Replicated Data Types (CRDTs)¶
Marc Shapiro’s research at INRIA, France lead to the creation of CRDTs, Strong Eventual Consistency, and its applications in wide area and disconnected applications. His papers and videos helped us several years back with our initial investigation of replicated data types. https://dl.acm.org/author_page.cfm?id=81100431017
Carlos Baquero’s “Pure Operation-Based Replicated Data Types” paper enabled us to think about the right approach to multi master replication and causal broadcast. https://arxiv.org/pdf/1710.04469.pdf
Martin Klepmann’s ground breaking book “Designing Data-Intensive Applications” opened up many exciting approaches and techniques for us to consider when building our geo-distributed cloud service at scale. We continues to source inspiration from his work, presentations and papers on streams and consistency. http://dataintensive.net/
Leslie Lamport invented Latex, but even more importantly gave the world a way to think about time keeping that results in correctness, performance and reliability in geo distributed systems. His paper “Time, Clocks, and the Ordering of Events in a Distributed System” helped us create our logical clock and causal ordering implementations. https://lamport.azurewebsites.net/pubs/time-clocks.pdf
We leverage following open sources in our geo-distributed platform.
Macrometa uses ArangoDB’s query engine and AQL syntax as the foundation for its database query functionality. ArangoDB is an apache licensed NoSQL database that supports multi-model polyglot persistence. https://github.com/arangodb/arangodb
Macrometa utilizes Apache Siddhi library for pipelines processing.
Macrometa uses Apache Bookkeeper & ZooKeeper for a scalable, fault-tolerant, and low-latency append-only log optimized for real-time messaging. http://bookkeeper.apache.org/