What's wrong with how we think about blockchain?

Often, when a blockchain solution is contemplated, it is regarded as an alternative to using established database technology. A better way to look at it is that blockchain complements traditional databases by providing capabilities that a centralized database can’t. What blockchain adds is a greater assurance of trustworthiness, and that can be a big deal.

Combining blockchain and database technologies

As opposed to using blockchain for the main distributed database, we suggest combining the power and efficiency possible from relational databases with the fraud resistance and assurance of immutability provided by a blockchain ledger. In financial processing, using a small number of blockchain transactions done at critical points, one can reliably document and secure a much larger number of actual database calculations and transactions.

Blockchain allows for the creation of distributed ledgers that can detect tampering against the database, and for example, pause payment as a defensive response. For arms-length security, this approach depends on the use of an independent, cloud-based, blockchain service provider, such as Hyperledger Fabric from IBM, Microsoft, Oracle, or Amazon. This would be used to store distributed copies of financial records as they are processed and also to store protected copies of the validation “chaincode” itself. In short, the addition of blockchain provides a way to keep a centralized database honest.

We recommend a unique model for integrating blockchain transactions and database transactions with an emphasis on scalability, security, and rapid auditability. The model is a good fit for automating the financial side of many complex SaaS web applications, particularly when large amounts of money are flowing between many independent participants. Blockchain ledgers can make promises that the databases can’t, but databases are easier to program, faster, and do useful things that blockchains can’t.

When intermediaries play a major role, blockchain is needed

We’re concerned with financial applications involving a primary intermediary and large amounts of money flowing between many parties. Whatever technology is used, be it relational or non-relational, distributed or not, some kind of central database is necessary. Someone must have physical, administrative control of that database, and the intermediary is the logical choice. The crux of the problem is that traditional database technology by itself can’t prevent abuse by the principal administrator. Thus, the key point of blockchain is to provide an added level of protection against abuse, even by the intermediary.

Overcoming scalability limitations in blockchain systems

Blockchain technology comes with a lot of overhead compared to traditional databases (extra network traffic, exchanges of certificates, encryption / decryption, redundant storage, multi-phase commit logic, etc.). It is widely thought that one must accept these inefficiencies in order to gain the security that blockchain affords - but this is not the case.

We propose a more selective basis on which to use blockchain to achieve its security benefits without a substantial performance penalty. In our model, the blockchain has a copy of a small but sufficient subset of the database and validation criteria to audit the database processing. There is much less work done on the blockchain than on the database itself, and the blockchain’s data and work (additions and updates) is always an exact copy slightly behind the database. Nevertheless, the blockchain’s stripped-down contents and requirements suffice for each node to have everything needed to guaranty the transparency and integrity of the relevant portion of the database.

The proper role of blockchain validation

Validation logic for financial transactions can be very complicated. This kind of logic is much more easily and efficiently performed through well-established database technology than through the clumsy and restrictive facilities of blockchain. There are many types of validation that are simple to program against a database, but not straightforwardly expressible in chaincode. For example, such checks might be referential constraints (does this code, item number, or ID actually exist), uniqueness checks, determining that an item is still available, etc.

The more sensible role for blockchain validation is to perform those simple checks that are financially material and critical to the participant who owns a given node. For example, a customer wants to know that he has approved the same version of the invoice before paying the supplier. Any participating entity can have a copy of the ledger, with the guarantee that all transactions have passed validation rules installed on the blockchain node under its control.

Limitations in blockchain transaction processing

Blockchain transaction processing is oriented to one record at a time, whereas financial transactions usually need to span multiple records. This makes using blockchain transactions for complex processes and recovering from failure difficult. On the other hand, in database processing, changes to multiple records easily can be programmed to appear atomically, i.e. as an uninterruptable, simultaneous operation. (Failures may occur, but modern databases are designed to guarantee that complex transactions are never left partially completed.)

The integrity of a financial application can be assured through database transaction processing without reliance on blockchain transactions. Because blockchain transactions simply replicate a subset of the database, recovering from any blockchain failures is greatly simplified.

Limitations of blockchain security

Traditionally, blockchains use separate communication “channels” to initiate and execute transactions between individual parties. On the other hand, it is possible to use a central (possibly distributed) database as the starting point and principal arbiter of transactions. Such a database-centric model can help with the combinatorial access control problem that results from distributing data across multiple, independent blockchain nodes. Furthermore, a database-centric approach can reduce unnecessary exposure of sensitive data.

When the initiation and primary execution of transactions depends on the blockchain, it becomes necessary either to have separate channels or to share channels between every pair of interacting parties. In other words, even a few hundred parties can lead to the need for tens of thousands of private channels, each having an independent set of party-to-party permissions.

The more scalable solution is for every blockchain transaction to be just a copy of an underlying successful database transaction. In this model, there is no complex combinatorial problem, because each party communicates directly with the database. The blockchain simply serves as a mechanism to assure auditability. The node controlled by each party need only contain information pertaining to that party. Both parties involved can have their own private blockchain copy of a given transaction. Thus, our database-centric approach also eliminates unnecessary data exposure.

Blockchain processing with a Virtual Supercomputer

A good way to build large-scale financial blockchain applications would be to use what we call a “Virtual Supercomputer” architecture. In this design, the database is used to coordinate a network of loosely coupled web and application servers. Each application engine looks for work by searching through records retrieved from database queues. This model can effectively combine the available processing power and resources of a large number of ordinary commodity and cloud virtual machines.

Regardless of the choice of database development tools, here are some of the features that can simplify building real-world blockchain applications:

- Data dictionary with integrated security and distributed computing

This simplifies building sophisticated, custom database applications.

- Efficient transactional queuing

Tight integration with standard database transaction processing assures rigorous multi-user synchronization and recovery from failures.

- Scalable distributed processing facilities

A database-centric architecture provides the basis for building an easily extensible Virtual Supercomputer. Loose coupling to asynchronous blockchain services avoids degrading database performance.

Our open source toolkit, Base One Foundation Component Library (BFC), provides these capabilities.

Conclusion

Blockchain ledgers can greatly reduce the potential for fraud, and this can be achieved with little performance penalty if the right design is chosen. By avoiding a flawed transaction processing model, a more database-centric approach can achieve greater security and scalability, while simplifying programming. A surprising number of potential blockchain applications worthy of consideration become possible.