This is a good list, but I'm trying to understand the following in better detail:
> [12] Be conservative on APIs and liberal with implementations.
> [18] Maintain multiple implementations in test for APIs; compare results between them. The cost is worth it (it will help with correctness, and also prevent leakage of implementation detail).
The word implementation is thrown out a lot. What does implementation mean in these quotes? The databases used (ie. dependency injection), the endpoints (ie. /user, /user/123), or something else?
I'm trying to visualize this in my head and am having trouble.
Co-creator of Delos here, Let me give a concrete example for [18].
We have a "StorageEngine" interface that wraps around RocksDB, i.e. "RocksDbStorageEngine" to provide key-value interfaces; however we are worried that someone would leak the implementation detail (RocksDB is so powerful) from the "StorageEngine" interface.
To solve the problem, we write another fully in memory storage engine implementation called "MemoryStorageEngine", which acts as a specification of the "StorageEngine" interface.
We run all storage engine tests against both implementations (rocksdb/memory), except for the durability part; we also configured the system to run over both storage engines.
By doing this:
- we can use the memory based one to enforce the behavior RocksDB based one by running unit tests against both implementations.
- no one can easily leak some implementation details of RocksDB from the StorageEngine interface; to do that, you need to first add those advanced feature into memory based StorageEngine as well!
This is just one example, and we have a list of such abstractions, such as MetadataStore, Logs etc. We created multiple implementation of each of these interfaces and ensure the system could run on any combination of those implementations.
Once people use your APIs you cannot trivially modify the bits being used or take things away.
A large API likely has complexity, security implications, messy transaction boundaries, etc... keep it small and simple, think of UX of an API, make it intuitive and capable.
That's the conservative bit... now the liberal bit.
Make clients, make one in Python and look at date parsing, make one in Go and consider struct re-use and how to deal with sets/arrays of things, make one in JavaScript and consider how easy it it to re-use parts of the code. If you have an endpoint that returns a list of things... is it supposed to be paginated over? How easy is that? Is it an infinite list? How easy is that? What would a client want to achieve? Is it possible?
When you produce prototype applications in a variety of languages the majority of the small inconsistencies and issues are super obvious.
And remember, once the API is out there it's hard to change it and you can't remove things easily... so discovering all of this early helps you get it more right than wrong the first time round.
To take Stripe's example, the API is as documented here[1]. Essentially a documented contract between a platform provider and their clients.
Implementation is things like, as you noted, DB used, programming language, and so on.
By "Be conservative on APIs" they mean be careful what you promise to your clients as communicated and documented through APIs. Once promised it's almost impossible to go back; you will lose your clients' trust and piss them off. In case of Stripe they are committing to support idempotent requests[2] which is sort of a big deal. And I don't expect them to drop that support in any of their future APIs because I imagine hundreds of millions of $$ business would be relying on that idempotency support.
> [12] Be conservative on APIs and liberal with implementations.
> [18] Maintain multiple implementations in test for APIs; compare results between them. The cost is worth it (it will help with correctness, and also prevent leakage of implementation detail).
The word implementation is thrown out a lot. What does implementation mean in these quotes? The databases used (ie. dependency injection), the endpoints (ie. /user, /user/123), or something else?
I'm trying to visualize this in my head and am having trouble.