NEO Blockchain Concepts: Delegated Byzantine Fault Tolerance

In my last post, we had a look at NEO-CLI. Today, we will have a look at the basic concept of consensus NEO implements inside. There are a lot of ways to implement consensus in any blockchain platform you see around. All of them tends to solve the historical problem named “The Byzantine Generals’ Problem”. If you want to read the original paper on the problem, here it is.

 

The generals and their problems

The simplest way one could possibly describe this problem would be to describe it as the problem of determining the true outcome of a vote. One might wonder how that can be a problem since counting the votes and tallying them up should easily tell the result. Let’s imagine a scenario where we have 5 generals. They want to attack a city in the middle but all of them are in different geographic location. In a simplistic picture, the setup would look something like:

Byzantine Generals

They need to attack or retreat in sync or according to a predetermined plan. To act in unison, they proposed a way to vote for the next move. They decided to vote daily and the majority vote wins. Since they are not in the same geographic place, they use couriers to convey their vote amongst each other.

One doesn’t need to think for long to land on the conclusion that this whole plan has some inherent flaws. First, the couriers are humans and not that reliable, they can get captured, bribed to work for the enemy. They can get killed, lose the message or deliver the wrong message. Secondly, the generals themselves can be bribed in the same way to betray each other. And last but not the least, any general can make an inappropriate decision due to stupidity and failure to assess the situation or both.

Distributed computing by default inherits this problem since it works with a similar setup. Actors in the system can fall into the same trap as these generals. They can malfunction or act in an untrustworthy way to threaten or destabilize the system.

 

Delegated Byzantine Fault Tolerance

As it was mentioned before that there are numerous ways to solve this problem. Bitcoin paved a way of solving this problem through its Proof Of Work mechanism. Hyperledger Fabric improved upon the same principles through Practical Byzantine Fault Tolerance mechanism. NEO, on the other hand, proposes Delegated Byzantine Fault Tolerance.

Let’s assume we are living in Zion and there is a leader, preferably NEO. In this case, NEO is a delegate of Zion. In this world, he just can’t assume this role by his heroics, he has to gain votes from the citizen of Zion. All citizen votes for delegates like NEO and there are multiple delegates. The delegates are responsible for making the laws of Zion.

In any moment of time, if the citizen of Zion is not satisfied with a delegates proposal of laws, they can vote for a different delegate next time. Delegates are constantly notified of the demands of the citizen. They document it on a ledger, and yes you thought it right, this ledger is the analogy of our blockchain. These demands are added one after another and laws are passed focused towards keeping the citizens of Zion happy.

 

Zion-Delegates

 

But a single delegate can’t pass a law anytime he wishes to. The laws are passed in a periodic manner. When it is time to pass a law, a random delegate is chosen, let’s call him the speaker. The speaker proposes the law based on the demands of the citizens. The job of the speaker is to show how the proposed law increases the happiness index of Zion. And then he shares the law he is proposing to the other delegates. The delegates then have the job to decide whether the speaker’s calculations were correct and the laws are aligned to the common goal. They also verify the newly improved happiness index calculation. If 66% of the delegates believe that the calculation is correct and it is an improvement, the law gets finalized and passed.

What if the new law has failed to get 66% of delegates approval? Then the whole process starts over and a new speaker is selected.

How this is translated in NEO blockchain

This whole concept is reflected in NEO blockchain. Anyone who owns NEO is a stakeholder, thus a citizen. They want to create transactions, transfer, exchange assets. They do not participate in the bookkeeping of the blockchain. To do that one node needs to be a delegate as in NEO blockchain world, a bookkeeping node. There are certain criteria to be fulfilled to be a bookkeeping node in NEO blockchain. This includes dedicated internet connections and a certain amount of GAS. The demands are analogous to transactions. When they are put together by a bookkeeping node, a new block is created, which is analogous to a law. The happiness number is the hash of the aforementioned block.

Common Pitfalls: The dishonest speaker

What if the speaker is dishonest? What if he sends wrong versions of the law to some of the delegates?

Evil Speaker

In this scene, the evil speaker sends different versions of the law to different delegates. So, when the delegates communicate with each other they send different versions of the law. In this case, the leftmost delegate will get the accurate version of the law and will be able to calculate the happiness index properly. The rest of the delegates will fail to do so, failing the consensus.

Common Pitfalls: The dishonest delegate

The dishonest delegate scenario should be self-explanatory now.

Evil Delegate

In this scenario, we have an honest speaker who sends the same version of the law to every delegate but the rightmost delegate is dishonest. He sends a different version B to everyone else. In this case, 2 of the delegates will be able to calculate the happiness index properly. But the will only be able to verify it from the speaker’s version. It will reach 66% consensus and the law will be passed. But the result of this session will deem the rightmost delegate to be faulty. This data can be used for the next time of citizen vote where they chose their delegates.

By the definition here the minimum threshold for faulty delegates in the system is f=(n-1)/3 where n is the number of active delegate nodes.

To read more about this whole process, please go through the official NEO doc.

Hope this helps clear up how NEO views consensus. Next, we will see how this can be demonstrated between NEO-CLI nodes.

 

 

 

Advertisements

Neo Blockchain On Azure: Introduction to NEO-CLI

In my previous post, we started with a small NEO private net. Today, we will take a quick look into NEO-CLI and what it offers. Although it is named NEO-CLI, in practicality, this is a full blown NEO blockchain node instead of just a CLI tool to communicate with it. NEO offers two node types – GUI and CLI. I think the suffix comes from that and I wanted to explicitly mention it since it is a tad confusing.

At first, we will try to connect to our newly created private net. To do that, we will start with installing a separate installation of NEO-CLI. Installing NEO-CLI is pretty straight forward. You will need .Net core installed in your machine. If you don’t follow the instruction here.

Installation

I’m currently using an Ubuntu 16.04 as a reference OS. After installing .Net core framework you will need to install the NEO-CLI package. And since Im on a debian it was quite easy to do so the following way:

sudo apt-get install libleveldb-dev sqlite3 libsqlite3-dev

Configuration

We are testing and my local machine doesn’t have a frame of reference of the test private net we just created over Azure. To give this node a frame of reference we need to configure its SeedList to point to our own private net. What is a seed list? Simply put, it is nothing more than a list of URLs as described in the official NEO documentation.  This is the first set of nodes NEO-CLI will try to connect to when it boots up.

To configure the aforementioned SeedList, we will modify the protocol.json file, under the neo-cli directory.

We need to update the SeedList section of the configuration the following way:

“SeedList”: [
     "IP_or_FQDN_of_Azure_Private_Net_Host:20333”
 ],

If you opt to use the public test net, rename the protocol.testnet.json to protocol.json and you should be good to go.

Booting up the node

Now, it is time to start the node, we are going to invoke:

dotnet neo-cli.dll --log --nopeers

The log option will log the smart contract information and nopeers makes the node only connect to the seed nodes from the configuration file. this is something we want since this is a private network.

Creating a new wallet

Let’s create a new wallet then.

neo> create wallet mywallet.db3

NEO-CLI will ask for password twice for the wallet, pick your desired password. And copy the address and pubkey to keep it a safe place. If you forget the public key you can use list key command to see it.

More on protocol.json

Before we end this one, we will have one last look at the protocol.json configuration file for our node.

{
  "ProtocolConfiguration": {
    "Magic": 56753,
    "AddressVersion": 23,
    "StandbyValidators": [
        "02b3622bf4017bdfe317c58aed5f4c753f206b7db896046fa7d774bbc4bf7f8dc2",
        "02103a7f7dd016558597f7960d27c516a4394fd968b9e65155eb4b013e4040406e",
        "03d90c07df63e690ce77912e10ab51acc944b66860237b608c4f8f8309e71ee699",
        "02a7bc55fe8684e0119768d104ba30795bdcc86619e864add26156723ed185cd62"
    ],
    "SeedList": [
        "127.0.0.1:20333",
        "127.0.0.1:20334",
        "127.0.0.1:20335",
        "127.0.0.1:20336"
    ],
    "RPCList":[
      "http://127.0.0.1:30333"
    ],
    "SystemFee": {
        "EnrollmentTransaction": 1000,
        "IssueTransaction": 500,
        "PublishTransaction": 500,
        "RegisterTransaction": 10000
    }
  },

  "ApplicationConfiguration": {
    "DataDirectoryPath": "Chains/privnet",
    "NotificationDataPath": "Chains/privnet_notif",
    "RPCPort": 20332,
    "NodePort": 20333,
    "WsPort": 20334,
    "UriPrefix": [ "http://*:20332" ],
    "SslCert": "",
    "SslCertPassword": "",
    "BootstrapFile":"",
    "NotificationBootstrapFile":"",
    "DebugStorage":1
  }
}
  • The Magic field contains a uint value that denotes the source network of the message.
  • The StandbyValidators field are the validating nodes in the private node. It is the list of public keys of aforementioned validating nodes. We created 4 wallet here in this specific example and thus we have 4 entries here. 4 is the minimum number of nodes here to be listed for reaching a consensus.
  • SeedList is configured to localhost in this example configuration since NEO-CLI is booting up against the localhost node.
  • SystemFee section is the section that defines the system fee. As the configuration states, the registration fee for assets is 100000 GAS depicted by the RegisterTransaction field. EnrollmentTransaction field defines the registration fee for book-keepers. IssueTransaction is the fee for distributing assets. Finally the PublishTransaction is the fee for smart contracts.

 

That sums it up for this time. Next, we are going to have a look at how consensus works in NEO. And finally we will write a smart contract on NEO in C#. 🙂

Neo Blockchain On Azure : Starter Guide

If you are a blockchain enthusiast in these days, there is a good chance you have heard about NEO. NEO, arguably dubbed the “Ethereum Killer” promises to be a tailor made dApp platform for digitizing assets. This blog is not scoped to give everyone an introduction on NEO itself. So, I highly suggest reading the white paper from here.

What interests me mostly apart from NEOs approach with GAS and NEO to drive an economy for digitizing and managing assets is how it is built. Since it has its “ties” with Ethereum, my initial idea was it would probably be another Ethereum fork. And to my surprise, NEO is actually built from scratch. It has been on github since 2015 and totally built on C#. To be specific, net core. A quick look at the .csproj file tells me this is running on Asp.net core 2.0.2 now. Apart from whatever the promise it brings, this alone justifies my interest on how this is built.

Another thing that interests me here is how NEO approaches developers. Instead of having tailor made languages for smart contracts, NEO allows developer to write smart contracts with the the popular programming languages already. This list includes C#, Java, Golang and JavaScript. NeoVM, the small virtual machine that allows this to happen is .net core driven and hosted here.

The goal of today here is to deploy a NEO private net on Azure. Don’t get confused since for this time I won’t be using Azure Blockchain As A Service. Although the Azure Blockchain Workbench is indeed intriguing, that is a story for another day since Azure BAAS doesn’t support NEO out of the box yet. So we will go ahead with an Ubuntu Server 17.10.

1. Creating A Virt

According to NEO docs, we need to have access to a certain number of RPC ports in our newly created VM. The port description from the doc is here.

We will use the test net ports here, not the main net. We are going to add port 20332-20336, 30332-30336 to the inbound rules.

3. Allow Incoming PortsWhen that is taken care of, we will move to the final validation stage and create the VM. Now from the information found newly created resource page we would go ahead and ssh into the newly created VM.

4. Resource Properties Page

If you are on a OSX, Linux or basically any *nix you could use your native terminal to ssh to the new VM. If you are on windows, my choice is usually WSL.
The next thing we will need is Docker. From the Docker CE installation guide for Ubuntu 16.04 and newer, I devised the following steps.
$ sudo apt-get update
$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo apt-key fingerprint 0EBFCD88
$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
$ sudo apt-get update
$ sudo apt-get install docker-ce

To verify docker-ce is running, try the official hello world or ask systemctl.

$ sudo docker run hello-world
$ sudo systemctl status docker

After all said and done, we will pull the docker image cityofzion/neo-privatenet. This is a 4 Node NEO private net with 100M NEO and a lot of GAS. It also comes with a pre-loaded wallet. To pull and run the docker image, do the following:

$ sudo docker pull cityofzion/neo-privatenet
$ sudo docker run -d --rm --name neo-privnet -p 20333-20336:20333-20336/tcp -p 30333-30336:30333-30336/tcp cityofzion/neo-privatenet

Now, we are going to check out that preloaded wallet inside that docker container. To do that, execute the following:

$ sudo docker exec -it neo-privnet /bin/bash
* Consensus nodes are running in screen sessions, check 'screen -ls'
* neo-python is installed in /neo-python, with a neo-privnet.wallet file in place
* You can use the alias 'neopy' in the shell to start neo-python's prompt.py with privnet settings
* Please report issues to https://github.com/CityOfZion/neo-privatenet-docker
root@b6477e009639:/neo-python#

The banner will tell you that the screen sessions are running the consensus nodes and neo-python is already installed.

Execute a ls and you will see that neo-privnet.wallet is present there. We will run the pre-installed neopy and open the wallet.

neopy
neo> open wallet neo-privnet.wallet

The password for this is coz. To check the balance, execute:

wallet

5.Wallet Balance

We have around 100m NEO and 16K gas as promised!

There you go! You have your own NEO private net running over azure. To connect to the hosted private net from a remote client, we need to modify our neo client configuration. That is a story for another day.