Some Thoughts on Compatibility, Interoperability and Integration

Some Thoughts on Compatibility, Interoperability and Integration

We use these terms interchangeability at our peril… as I discovered earlier this year…!

I published an article about the Corda Settler yesterday. The Settler is a new way to think about interoperability between blockchains and other platforms. And as I was writing it, I realised that part of the confusion in the interoperability debate is caused by our using different words for the same concept, and also using the same word to mean lots of different things.

I wrote the following to help clarify my own thinking and hope others may find it useful too.

Integration: the solved problem of moving information around

First, let’s get the idea of “integration” on the table. This is the problem of moving information between systems, without any concern for how it will then be used.

For example, can your blockchain node publish information to other systems inside the firm that runs it? Can it make a call to a web service to obtain an FX rate for a proposal to a counterpart? Can it query a database?

The answer is probably yes. This is a problem the IT industry already solved, of course; it is a capability that blockchain nodes need to possess. I would (of course!) argue that Corda does integration better than other platforms but it’s a matter of degree and reasonable people may disagree about the differences in capabilities between platforms.

But, importantly, it says nothing about whether some other node on the network should believe the information obtained from a remote integration point or even be able to understand it. Integration is just a question of moving information around.

So, integration is table-stakes but isn’t massively unique to blockchains.

Compatibility: the defining characteristic of a blockchain network

At the other end of the spectrum is what I think we should call ‘compatibility’.

I wrote a piece earlier in the year about the advantages you get from deploying multiple applications to the same blockchain network. The argument I was making was that if two applications are deployed to the same blockchain network, then they can probably transact without having to do anything special at all… you don’t have to worry about whether the apps can work together. You get this capability for free.

Nobody worries if one contract deployed to the Ethereum mainnet can call another contract on the Ethereum mainnet, right? It just works. So it is for Corda Network and so forth.

It’s pretty hard getting it to work, of course. After all, you’re dealing with nodes deployed across the world by different people executing an ongoing process of consensus to ensure that for facts any of them hold in common, all parties see the same thing:

“I know that what I see is what you see”.

And for this to work, it means that if one node ever tries to introduce a new piece of shared state — a new fact, if you like — or update an existing one, we need all relevant participants either to accept the change or all to reject it. Things don’t work if different nodes reach different conclusions about the same information!

This obviously has to be the case when you think about it: if I could publish a transaction that you thought was valid but your neighbour that was invalid then we’d have just violated the key principle: we wouldn’t all see the same thing.

To achieve this, the nodes on the network need to share a lot in common. First the nodes need to speak the same protocol. But they also need to agree on any optional settings in that protocol. After all, it’s not enough for two Bitcoin nodes to understand the Bitcoin block format; they also have to agree on what the maximum allowable block size is, for example. Bitcoin nodes that don’t agree on this question aren’t merely in disagreement; they’re on different networks!

Importantly, nodes on a blockchain network — and this surprises a lot of people — also need to agree on what the rules were at every point in the past…

To see why this must be the case, recall that blockchains operate on the principle of “trust but verify”. I don’t take what you tell me at face value. Instead, I independently assess the evidence — historic transactions, the backchain, etc — for myself. So if you and I disagree on what the rules were at some point in the past then we could happen to disagree on whether any given piece of historical evidence is valid or not. And hence end up with different views of what the true state of today is.

Getting to the point where multiple nodes can agree on all the consensus rules that matter today and all the consensus rules that mattered in the past as well is a difficult endeavour but it also comes with a big prize: perfect compatibility.

That is: if my node and your node speak the same protocol and agree on all the protocol settings that could otherwise cause them to disagree on anything then they are, by definition, perfectly compatible. And this means that they can, in principle, transact with each other, for any purpose, at any time, using any app, relying on any historical shared data. Data generated from one application on that network could be used directly by another application with full provenance and seamless movement of that data.

A really good situation to be in!

I talked about how the Ethereum mainnet attempts to achieve this in a public setting. And it is also the basis of Corda Network.

Compatibility is a really high bar but it’s also one worth striving for because the benefits — freely moving assets, shared infrastructure, elimination of information silos, etc — are so great. It’s why I took such an aggressive shot at the vendors who were NOT striving for universally compatible networks and were, instead, deploying independent, incompatible networks of nodes using their software.

But… there will of course also be situations where incompatible nodes do have to transact. Situations where a Corda network needs to interact with a Fabric network. Or even where two independent, and hence incompatible, Corda networks need to interact. So what word do we use for that and how do we actually achieve it?

Interoperability: The problem of exchanging and making use of information between incompatible networks

So if “compatibility” is the word we use when talking about sharing information between nodes on the same network, what do we use for the problem of sharing information in a meaningful way between nodes on different networks? Networks that are, by definition, in some way incompatible?

If we just want to move information between the platforms — integration — then no problem… that’s easy. But what if we want one or both sides of the communication process to understand or trust the information they’re receiving in some way?

That’s a harder problem… we need somehow to make use of the information we receive. Which, as I wrote on my article about the Corda Settler, is the dictionary definition of “interoperability”.

To take a concrete example, we’re talking here about situations like having a Corda node receive some information from a Fabric node, say, and use it to decide whether to make a binding update to the Corda network, and do so in a way that other nodes on that Corda network will agree was correct.

So this is what I think we mean when we talk about interoperability:

How do we ingest information from a “foreign” network and use it to decide whether to perform some action on a “local” network in a way that everybody on the local network agrees was correct and final?

Key insight: if somebody isn’t running a full node on your network, they don’t and shouldn’t care if you’re a blockchain.

It may seem like I’ve used a lot of words to not say very much but the argument above has taken us somewhere very interesting, because we have, I think, driven to the essence of the interoperability problem: importing ‘foreign’ information into a blockchain node and assessing it in a way that all other relevant nodes on that blockchain network will agree was correct.

And to reiterate, the foreign information invariably is a piece of information that purports to ‘prove’ that some irreversible action has taken place someplace else.

The irreversible actions we typically care about are things like “€1bn moved from account A to account B, with no possibility of reversal” or “5BTC moved into the following address, secured under six blocks” and so forth.

And our problem, from the perspective of the receiving blockchain node is to decide:

“do I believe the thing I care about actually happened? And do I believe it is final?”

We need to answer both questions because what I invariably do with this information is make a similarly binding update to the local ledger. And if this update pays away $1.13bn dollars on the local ledger because I believe €1bn arrived, on the other platform, I’d better be sure that €1bn won’t vanish out from underneath me!

So, continuing this example, we need to first figure out if the information we’re looking at does represent the right €1bn payment. But we also need some way of deciding whether we believe that the foreign network won’t reverse it.

And this leads us to the massively simplifying assumption because: these two questions have nothing to do with blockchains!

They are questions we are interested in whether the foreign ledger is Bitcoin, Hyperledger Fabric or a central bank’s Real Time Gross Settlement system.

“Did the transaction happen?”

“Is it final?”

These are universal questions.

Now… you may be asking the questions for a variety of reasons. Perhaps you’re trying to decide if some money owing has been paid so you can mark a debt as settled. Perhaps you’re more ambitious and are trying to orchestrate an atomic swap of assets and so need to know the asset on the other ledger has been safely moved into escrow or encumbered in some other way.

But, regardless of the reason, the core building block you need is some way of knowing what happened in the other place.

So this leads us to what I think should be the basis of any serious interoperability work we do together as an industry: if the questions we’re asking about a foreign ledger are universal, our interoperability solutions should be too.

In other words, the key to making blockchains interoperate is to forget they are blockchains!