Topic: Practical application of blockchain as a distributed data warehouse (Read 223 times)

About us

We are Sevastopol IT company "SolarLab", specializing in the field of electronic trading platforms and electronic document management. The company was founded in April 2016, when the first contract was signed and the office consisting of 10 employees became operational. From the very beginning of its existence our company has been growing actively, for example, by the end of 2016 "SolarLab" staff numbered approximately 60 people. This became possible due to both the high professional competence of our developers and the ability to implement successfully all the tasks assigned to us by customers. By the end of 2016, it became clear that the company was growing very fast and in order to offer customers the best solutions, we decided to open the R&D department. Within this department, the company actively works with machine learning technologies, bot-technologies, and blockchain. Below we will discuss the blockchain system applying without releasing coins.
  
The task

Recently our largest customer, the company that provides services for the site on electronic public purchases, contacted us with the idea considering its business improvement. Registration process on the site for public purchases is quite bureaucratic. To fulfil the requirements of the law and implement the principle of Know your customer (KYC) sites and certifying centers have to collect a lot of documents confirming the identity of the client and the origin of its funds. Our customer had an idea to create one-time accreditation service at all public purchases sites regulated by different laws thus, this would allow collecting necessary documents only once. Such service would be useful to all, because users after registration on any site would have the opportunity to work with everyone else without additional delays and sites would receive new users due to simplified registration. In the future it would be possible to attract competing sites and other related organizations to this platform.

The implementation

In fact, we faced the task of creating a decentralized data exchange system (hereinafter referred to as the Platform) within the consortium - i.e. closed community of participants. Although in our case it was a system in which data access could be provided to all participants, we decided to look at the task from different angles and create the system in which users would be able to decide for themselves to whom of the system participants they want to give access to the data.
Multichain was chosen as the technological base of the project, because this platform was originally designed with the support of consortium as the main mode of operation and it has convenient tools for working with the blockchain as a data warehouse. The Platform itself implements the interface for data exchange and means for delineating data access.
The main characteristics of the implemented project are:
1. Data is distributed among all nodes of the system, none of the participants in the system can falsify or delete data once issued to another participant, and there is no single point of failure. Even if the participant node is physically disconnected from the network, the data received before the disconnection remains on this node and is available for work, and when the connection is restored, the data that appeared during the absence of the node in the network is obtained.
2. The closed blockchain (consortium) is used - i.e. connection requires either approval of the coordinating node, or, if a consensus model of administration is chosen, it should be approved by all network participants.
3. After connection, the consortium member can create both public (accessible to all participants) and private (available to the selected circle of participants) data collections. The security of the private data is provided by means of cryptographic protection of data using GOST algorithms.
4. Only the creator can control access to the collection.
5. After adding of new participant to the existing collection access list, previously published data becomes available to him automatically.
6. If a participant is removed from the access list, the new published data will not be available to him.
7. The Platform supports the structuring of data in a collection in JSON format and the validation of the document format in it according to the JSON-schema.
8. The Platform provides an API that isolates certain user from the details of the blockchain implementation and allows him to work with higher-level concepts ("document" and "collection").
9. The Platform does not provide data deletion; no one can delete a document from a collection that is accessible to other participants.
10. When you update a document, the full history of its changes is saved, and it is possible to receive any of the versions of the document.
11. The Platform supports sending of customized notifications about changing / updating data in the collection of interest to the participant's external systems, thus, the information systems of one participant can respond quickly to the updates of data made by the information systems of another participant.

As you can see, the list is quite impressive. Of course, the system has some disadvantages that we know about: for example, if we give someone access to the data once, we can assume that the data can be copied and can be given to someone else, but this is out of our system’s control. However, this kind of risks exist in any system. Another drawback of the system, which exists due to the blockchain system use and the need for data encryption / decryption on the fly, is a total throughput of ~ 30 transactions per second, which is more than enough for a particular task, but may be a limiting factor for the application of the Platform in a number of other scenarios.
The operation principle of the data access managing mechanism is based on a hybrid encryption scheme:
1) A registered member of the system receives a public and private key. The public key of each participant is published in the blockchain.
2) Participant # 1 creates a new collection and determines with whom of the participants in the system he would like to share its contents (for ex. Participant # 2 and # 3), then he adds a document to this collection.
3) The Platform encrypts the document with the generated symmetric key (this means that the same key as for encryption will be required for decryption). This key has much smaller size than the encrypted document itself.
4) The Platform extracts the public keys of participants # 2 and # 3 from the blockchain and encrypts a symmetric key with them, than it adds the results to the blockchain.
5) Participants # 2 and # 3 access the system and try to look through the contents of the collection. The Platform decrypts the symmetric key with the private keys of participants # 2 and # 3, then decrypts the document with symmetric this key. Thus, participants # 2 and # 3 get access to document’s contents.

This is how the scheme works in general. The specific implementation made for our customer looks like this:

1) The participant  creates an account in the Accounting Center (hereinafter referred to as AC # 1), submitting all necessary documents and data.
2) The accounting center issues an electronic signature for the participant and binds with his profile all the data and documents provided by him, adding all this to the blockchain.
3) When a participant accesses the site on state purchases (hereinafter referred to as Site # 1), it extracts his registration data from the Collection created by AC # 1.
4) There may be a situation when the site does not have enough of the data and it will request additional information or it will be necessary to store other information (for example, the history of participation in the auction) for a particular client to share it with another friendly site. In this case, it creates a new collection with these additional data.
5) When the Participant decides to register at Site No. 2, it takes his registration data either from the collection of the AC No. 1 or from the collection created by Site No. 1

The result

Our solution has been implemented and has been working successfully since. If you have any questions or remarks, please write in the comments, we will be happy to answer or discuss. Since we are ready and willing not only to develop projects on a by-order basis, but also to make our own products, now we are working on one more project based on the blockchain technology. We will discuss it in our next article.