Creating a Cryptocurrency in the Crystal Programming Language
Crystal is an up and coming programming language that should appeal to Ruby veterans in demand for more performance and flexibility. It may even be the most promising programming language of 2018.
In this tutorial, Toptal Ruby Developer Eqbal Quran demonstrates how you can harness Crystal’s potential to create your own blockchain and cryptocurrency.
Crystal is an up and coming programming language that should appeal to Ruby veterans in demand for more performance and flexibility. It may even be the most promising programming language of 2018.
In this tutorial, Toptal Ruby Developer Eqbal Quran demonstrates how you can harness Crystal’s potential to create your own blockchain and cryptocurrency.
Eqbal is a senior full-stack developer with more than a decade of experience working in web and mobile development.
Expertise
PREVIOUSLY AT
This post is my attempt to understand the key aspects of the blockchain by exploring the internals. I started by reading the original bitcoin whitepaper, but I felt the only way to truly understand the blockchain is by building a new cryptocurrency from scratch.
That’s why I decided to create a cryptocurrency using the new Crystal programming language, and I dubbed it CrystalCoin. This article won’t discuss algorithm choices, hash difficulty, or similar topics. Instead, the focus will be on detailing a concrete example, which should provide a deeper, hands-on understanding of the strengths and limitations of blockchains.
If you haven’t read it yet, for more background on algos and hashing, I suggest you take a look at Demir Selmanovic’s article Cryptocurrency for Dummies: Bitcoin and Beyond.
Why I Chose the Crystal Programming Language
For a better demonstration, I wanted to use a productive language like Ruby without compromising the performance. Cryptocurrency has many time-consuming computations (namely mining and hashing), and that’s why compiled languages such as C++ and Java are the languages of choice for building “real” cryptocurrencies. That being said, I wanted to use a language with a cleaner syntax so I could keep development fun and allow better readability. Crystal performance tends to be good anyway.
So, why did I decide to use the Crystal programming language? Crystal’s syntax is heavily inspired by Ruby’s, so for me, it feels natural to read and easy to write. It has the added benefit of a lower learning curve, especially for experienced Ruby developers.
This is how the Crystal lang team puts it at their official website:
Fast as C, slick as Ruby.
However, unlike Ruby or JavaScript, which are interpreted languages, Crystal is a compiled language, making it much faster and offering a lower memory footprint. Under the hood, it uses LLVM for compiling to native code.
Crystal is also statically typed, which means the compiler will help you catch type errors in compile-time.
I’m not going to explain why I consider Crystal language awesome because it is beyond the scope of this article, but if you don’t find my optimism convincing, feel free to check out this article for a better overview of Crystal’s potential.
Note: This article assumes you already have a basic understanding of Object Oriented Programming (OOP).
Blockchain
So, what is a blockchain? It’s a list (chain) of blocks linked and secured by digital fingerprints (also known as crypto hashes).
The easiest way to think of it is as a linked list data structure. That being said, a linked list only required to have a reference to the previous element; a block must have an identifier depending on the previous block’s identifier, meaning that you cannot replace a block without recomputing every single block that comes after.
For now, think of blockchain as a series of blocks with some data linked with a chain, the chain being the hash of the previous block.
The entire blockchain would exist on each node that wants to interact with it, meaning it is copied on each one of the nodes in the network. No single server hosts it, yet all blockchain development companies use it, which makes it decentralized.
Yes, this is weird compared to conventional centralized systems. Each of the nodes will have a copy of the entire blockchain (> 149 Gb in Bitcoin blockchain by December 2017).
Hashing and Digital Signature
So, what is this hash function? Think of the hash as a function, that returns a unique fingerprint when we give it a text/object. Even the smallest change in the input object would change the fingerprint dramatically.
There are different hashing algorithms, and in this article, we’ll be using the SHA256
hash algorithm, which is the one used in Bitcoin
.
Using SHA256
we’ll always result in 64 hexadecimal chars (256 bit) in length even if the input is less than 256 bits or much bigger than 256 bits:
Input | Hashed Results |
---|---|
VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEX VERY LONG TEXT VERY LONG VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT VERY LONG TEXT | cf49bbb21c8b7c078165919d7e57c145ccb7f398e7b58d9a3729de368d86294a |
Toptal | 2e4e500e20f1358224c08c7fb7d3e0e9a5e4ab7a013bfd6774dfa54d7684dd21 |
Toptal. | 12075307ce09a6859601ce9d451d385053be80238ea127c5df6e6611eed7c6f0 |
Note with the last example, that just adding a .
(dot) resulted in a dramatic change in the hash.
Therefore, in a blockchain, the chain is built by passing the block data into a hashing algorithm that would generate a hash, which is linked to the next block, henceforth, forming a series of blocks linked with the hashes of the previous blocks.
Building a Cryptocurreny in Crystal
Now let’s start creating our Crystal project and build our SHA256
encryption.
Assuming you have Crystal programming language installed, let’s create the skeleton of CrystalCoin
codebase by using Crystal’s built-in project tooling crystal init app [name]
:
% crystal init app crystal_coin
create crystal_coin/.gitignore
create crystal_coin/.editorconfig
create crystal_coin/LICENSE
create crystal_coin/README.md
create crystal_coin/.travis.yml
create crystal_coin/shard.yml
create crystal_coin/src/crystal_coin.cr
create crystal_coin/src/crystal_coin/version.cr
create crystal_coin/spec/spec_helper.cr
create crystal_coin/spec/crystal_coin_spec.cr
Initialized empty Git repository in /Users/eki/code/crystal_coin/.git/
This command will create the basic structure for the project, with an already initialized git repository, license and readme files. It also comes with stubs for tests, and the shard.yml
file for describing the project and managing dependencies, also known as shards.
Let’s add the openssl
shard, which is needed to build SHA256
algorithm:
# shard.yml
dependencies:
openssl:
github: datanoise/openssl.cr
Once that’s in, head back into your terminal and run crystal deps
. Doing this will pull down openssl
and its dependencies for us to utilize.
Now we have the required library installed in our code, let’s start by defining Block
class and then building the hash function.
# src/crystal_coin/block.cr
require "openssl"
module CrystalCoin
class Block
def initialize(data : String)
@data = data
end
def hash
hash = OpenSSL::Digest.new("SHA256")
hash.update(@data)
hash.hexdigest
end
end
end
puts CrystalCoin::Block.new("Hello, Cryptos!").hash
You can now test your application by running crystal run crystal src/crystal_coin/block.cr
from your terminal.
crystal_coin [master●] % crystal src/crystal_coin/block.cr
33eedea60b0662c66c289ceba71863a864cf84b00e10002ca1069bf58f9362d5
Designing our Blockchain
Each block is stored with a timestamp
and, optionally, an index
. In CrystalCoin
, we’re going to store both. To help ensure integrity throughout the blockchain, each block will have a self-identifying hash. Like Bitcoin, each block’s hash will be a cryptographic hash of the block’s (index
, timestamp
, data
, and the hash of the previous block’s hash previous_hash
). The data can be anything you want for now.
module CrystalCoin
class Block
property current_hash : String
def initialize(index = 0, data = "data", previous_hash = "hash")
@data = data
@index = index
@timestamp = Time.now
@previous_hash = previous_hash
@current_hash = hash_block
end
private def hash_block
hash = OpenSSL::Digest.new("SHA256")
hash.update("#{@index}#{@timestamp}#{@data}#{@previous_hash}")
hash.hexdigest
end
end
end
puts CrystalCoin::Block.new(data: "Same Data").current_hash
In Crystal lang, we replace Ruby’s attr_accessor
, attr_getter
and attr_setter
methods with new keywords:
Ruby Keyword | Crystal Keyword |
---|---|
attr_accessor | property |
attr_reader | getter |
attr_writer | setter |
Another thing you may have noticed in Crystal is that we want to hint the compiler about specific types through our code. Crystal infers the types, but whenever you have ambiguity you can explicitly declare types as well. That’s why we added String
types for current_hash
.
Now let’s run block.cr
twice and note that the same data will generate different hashes because of the different timestamp
:
crystal_coin [master●] % crystal src/crystal_coin/block.cr
361d0df74e28d37b71f6c5f579ee182dd3d41f73f174dc88c9f2536172d3bb66
crystal_coin [master●] % crystal src/crystal_coin/block.cr
b1fafd81ba13fc21598fb083d9429d1b8a7e9a7120dbdacc7e461791b96b9bf3
Now we have our block structure, but we’re creating a blockchain. We need to start adding blocks to form an actual chain. As I mentioned earlier, each block requires information from the previous block. But how does the first block in the blockchain get there? Well, the first block, or genesis
block, is a special block (a block with no predecessors). In many cases, it’s added manually or has unique logic allowing it to be added.
We’ll create a new function that returns a genesis block. This block is of index=0
, and it has an arbitrary data value and an arbitrary value in the previous_hash
parameter.
Let’s build or class method Block.first
that generates the genesis block:
module CrystalCoin
class Block
...
def self.first(data="Genesis Block")
Block.new(data: data, previous_hash: "0")
end
...
end
end
And let’s test it out using p CrystalCoin::Block.first
:
#<CrystalCoin::Block:0x10b33ac80 @current_hash="acb701a9b70cff5a0617d654e6b8a7155a8c712910d34df692db92455964d54e", @data="Genesis Block", @index=0, @timestamp=2018-05-13 17:54:02 +03:00, @previous_hash="0">
Now that we’re able to create a genesis block, we need a function that will generate succeeding blocks in the blockchain.
This function will take the previous block in the chain as a parameter, create the data for the block to be generated, and return the new block with the appropriate data. When new blocks hash information from previous blocks, the integrity of the blockchain increases with each new block.
An important consequence is that a block can’t be modified without changing the hash of every consecutive block. This is demonstrated in the example below. If the data in block 44 is changed from LOOP
to EAST
, all hashes of the consecutive blocks must be changed. This is because the hash of the block depends on the value of the previous_hash
(among other things).
If we didn’t do this, it would be easier for an outside party to change the data and replace our chain with an entirely new one of their own. This chain of hashes acts as cryptographic proof and helps ensure that once a block is added to the blockchain it cannot be replaced or removed. Let’s create the class method Block.next
:
module CrystalCoin
class Block
...
def self.next(previous_node, data = "Transaction Data")
Block.new(
data: "Transaction data number (#{previous_node.index + 1})",
index: previous_node.index + 1,
previous_hash: previous_hash.hash
)
end
...
end
end
To try it out all together, we’ll create a simple blockchain. The first element of the list is the genesis block. And of course, we need to add the succeeding blocks. We’ll create five new blocks to demonstrate CrystalCoin
:
blockchain = [ CrystalCoin::Block.first ]
previous_block = blockchain[0]
5.times do
new_block = CrystalCoin::Block.next(previous_block: previous_block)
blockchain << new_block
previous_block = new_block
end
p blockchain
[#<CrystalCoin::Block:0x108c57c80
@current_hash=
"df7f9d47bee95c9158e3043ddd17204e97ccd6e8958e4e41dacc7f0c6c0df610",
@index=0,
@previous_hash="0",
@timestamp=2018-06-04 12:13:21 +03:00,
@data="Genesis Block>,
#<CrystalCoin::Block:0x109c89740
@current_hash=
"d188fcddd056c044c783d558fd6904ceeb9b2366339af491a293d369de4a81f6",
@index=1,
@previous_hash=
"df7f9d47bee95c9158e3043ddd17204e97ccd6e8958e4e41dacc7f0c6c0df610",
@timestamp=2018-06-04 12:13:21 +03:00,
@data="Transaction data number (1)">,
#<CrystalCoin::Block:0x109cc8500
@current_hash=
"0b71b61118b9300b4fe8cdf4a7cbcc1dac4da7a8a3150aa97630994805354107",
@index=2,
@previous_hash=
"d188fcddd056c044c783d558fd6904ceeb9b2366339af491a293d369de4a81f6",
@timestamp=2018-06-04 12:13:21 +03:00,
@transactions="Transaction data number (2)">,
#<CrystalCoin::Block:0x109ccbe40
@current_hash=
"9111406deea4f07f807891405078a3f8537416b31ab03d78bda3f86d9ae4c584",
@index=3,
@previous_hash=
"0b71b61118b9300b4fe8cdf4a7cbcc1dac4da7a8a3150aa97630994805354107",
@timestamp=2018-06-04 12:13:21 +03:00,
@transactions="Transaction data number (3)">,
#<CrystalCoin::Block:0x109cd0980
@current_hash=
"0063bfc5695c0d49b291a8813c566b047323f1808a428e0eb1fca5c399547875",
@index=4,
@previous_hash=
"9111406deea4f07f807891405078a3f8537416b31ab03d78bda3f86d9ae4c584",
@timestamp=2018-06-04 12:13:21 +03:00,
@transactions="Transaction data number (4)">,
#<CrystalCoin::Block:0x109cd0100
@current_hash=
"00a0c70e5412edd7389a0360b48c407ce4ddc8f14a0bcf16df277daf3c1a00c7",
@index=5,
@previous_hash=
"0063bfc5695c0d49b291a8813c566b047323f1808a428e0eb1fca5c399547875",
@timestamp=2018-06-04 12:13:21 +03:00,
@transactions="Transaction data number (5)">
Proof-of-Work
A Proof of Work algorithm (PoW) is how new blocks are created or mined on the blockchain. The goal of PoW is to discover a number which solves a problem. The number must be difficult to find but easy to verify computationally by anyone on the network. This is the core idea behind Proof of Work.
Let’s demonstrate with an example to make sure everything is clear. We’ll assume that the hash of some integer x multiplied by another y must start with 00
. So:
hash(x * y) = 00ac23dc...
And for this simplified example, let’s fix x=5
and implement this in Crystal:
x = 5
y = 0
while hash((x*y).to_s)[0..1] != "00"
y += 1
end
puts "The solution is y = #{y}"
puts "Hash(#{x}*#{y}) = #{hash((x*y).to_s)}"
Let’s run the code:
crystal_coin [master●●] % time crystal src/crystal_coin/pow.cr
The solution is y = 530
Hash(5*530) = 00150bc11aeeaa3cdbdc1e27085b0f6c584c27e05f255e303898dcd12426f110
crystal src/crystal_coin/pow.cr 1.53s user 0.23s system 160% cpu 1.092 total
As you can see this number y=530
was hard to find (brute-force) but easy to verify using the hash function.
Why bother with this PoW algorithm? Why don’t we just create one hash per block and that’s it? A hash must be valid. In our case, a hash will be valid if the first two characters of our hash are 00
. If our hash starts with 00......
, it is considered valid. This is called the difficulty. The higher the difficulty, the longer it takes to get a valid hash.
But, if the hash is not valid the first time, something must change in the data we use. If we use the same data over and over, we will get the same hash over and over and our hash will never be valid. We use something called nonce
in our hash (in our previous example it’s the y
). It is simply a number that we increment each time the hash is not valid. We get our data (date, message, previous hash, index) and a nonce of 1. If the hash we get with these is not valid, we try with a nonce of 2. And we increment the nonce until we get a valid hash.
In Bitcoin, the Proof of Work algorithm is called Hashcash. Let’s add a proof-of-work to our Block class and start mining to find the nonce. We’ll use a hard-coded difficulty of two leading zeros ‘00’:
Now let’s redesign our Block class to support that. Our CrystalCoin
Block will contain the following attributes:
1) index: indicates the index of the block ex: 0,1
2) timestamp: timestamp in epoch, number of seconds since 1 Jan 1970
3) data: the actual data that needs to be stored on the blockchain.
4) previous_hash: the hash of the previous block, this is the chain/link between the blocks
5) nonce: this is the number that is to be mined/found.
6) current_hash: The hash value of the current block, this is generated by combining all the above attributes and passing it to a hashing algorithm
I’ll create a separate module to do the hashing and find the nonce
so we keep our code clean and modular. I’ll call it proof_of_work.cr
:
require "openssl"
module CrystalCoin
module ProofOfWork
private def proof_of_work(difficulty = "00")
nonce = 0
loop do
hash = calc_hash_with_nonce(nonce)
if hash[0..1] == difficulty
return nonce
else
nonce += 1
end
end
end
private def calc_hash_with_nonce(nonce = 0)
sha = OpenSSL::Digest.new("SHA256")
sha.update("#{nonce}#{@index}#{@timestamp}#{@data}#{@previous_hash}")
sha.hexdigest
end
end
end
Our Block
class would look something like:
require "./proof_of_work"
module CrystalCoin
class Block
include ProofOfWork
property current_hash : String
property index : Int32
property nonce : Int32
property previous_hash : String
def initialize(index = 0, data = "data", previous_hash = "hash")
@data = data
@index = index
@timestamp = Time.now
@previous_hash = previous_hash
@nonce = proof_of_work
@current_hash = calc_hash_with_nonce(@nonce)
end
def self.first(data = "Genesis Block")
Block.new(data: data, previous_hash: "0")
end
def self.next(previous_block, data = "Transaction Data")
Block.new(
data: "Transaction data number (#{previous_block.index + 1})",
index: previous_block.index + 1,
previous_hash: previous_block.current_hash
)
end
end
end
Few things to note about Crystal code and Crystal language examples in general. In Crystal, methods are public by default. Crystal requires each private method to be prefixed with the private keyword which could be confusing coming from Ruby.
You may have noticed that Crystal’s Integer types there are Int8
, Int16
, Int32
, Int64
, UInt8
, UInt16
, UInt32
, or UInt64
compared to Ruby’s Fixnum
. true
and false
are values in the Bool
class rather than values in classes TrueClass
or FalseClass
in Ruby.
Crystal has optional and named method arguments as core language features, and does not require writing special code for handling the arguments which is pretty cool. Check out Block#initialize(index = 0, data = "data", previous_hash = "hash")
and then calling it with something like Block.new(data: data, previous_hash: "0")
.
For a more detailed list of differences between Crystal and Ruby programming language check out Crystal for Rubyists.
Now, let’s try to create five transactions using:
blockchain = [ CrystalCoin::Block.first ]
puts blockchain.inspect
previous_block = blockchain[0]
5.times do |i|
new_block = CrystalCoin::Block.next(previous_block: previous_block)
blockchain << new_block
previous_block = new_block
puts new_block.inspect
end
[#<CrystalCoin::Block:0x108f8fea0 @current_hash="0088ca080a49334e1cb037ed4c42795d635515ef1742e6bcf439bf0f95711759", @index=0, @nonce=17, @timestamp=2018-05-14 17:20:46 +03:00, @data="Genesis Block", @previous_hash="0">]
#<CrystalCoin::Block:0x108f8f660 @current_hash="001bc2b04d7ad8ef25ada30e2bde19d7bbbbb3ad42348017036b0d4974d0ccb0", @index=1, @nonce=24, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (1)", @previous_hash="0088ca080a49334e1cb037ed4c42795d635515ef1742e6bcf439bf0f95711759">
#<CrystalCoin::Block:0x109fc5ba0 @current_hash="0019256c998028111838b872a437cd8adced53f5e0f8f43388a1dc4654844fe5", @index=2, @nonce=61, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (2)", @previous_hash="001bc2b04d7ad8ef25ada30e2bde19d7bbbbb3ad42348017036b0d4974d0ccb0">
#<CrystalCoin::Block:0x109fdc300 @current_hash="0080a30d0da33426a1d4f36d870d9eb709eaefb0fca62cc68e497169c5368b97", @index=3, @nonce=149, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (3)", @previous_hash="0019256c998028111838b872a437cd8adced53f5e0f8f43388a1dc4654844fe5">
#<CrystalCoin::Block:0x109ff58a0 @current_hash="00074399d51c700940e556673580a366a37dec16671430141f6013f04283a484", @index=4, @nonce=570, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (4)", @previous_hash="0080a30d0da33426a1d4f36d870d9eb709eaefb0fca62cc68e497169c5368b97">
#<CrystalCoin::Block:0x109fde120 @current_hash="00720bb6e562a25c19ecd2b277925057626edab8981ff08eb13773f9bb1cb842", @index=5, @nonce=475, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (5)", @previous_hash="00074399d51c700940e556673580a366a37dec16671430141f6013f04283a484">
See the difference? Now all hashes start with 00
. That’s the magic of proof-of-work. Using ProofOfWork
we found the nonce
and proof is the hash with the matching difficulty, that is, the two leading zeros 00
.
Note with the first block we created, we tried 17 nonces until finding the matching lucky number:
Block | Loops / Number of Hash Calculations |
---|---|
#0 | 17 |
#1 | 24 |
#2 | 61 |
#3 | 149 |
#4 | 570 |
#5 | 475 |
Now let’s try a difficulty of four leading zeros (difficulty="0000"
):
Block | Loops / Number of Hash Calculations |
---|---|
#1 | 26 762 |
#2 | 68 419 |
#3 | 23 416 |
#4 | 15 353 |
In the first block tried 26762 nonces (compare 17 nonces with difficulty ‘00’) until finding the matching lucky number.
Our Blockchain as an API
So far, so good. We created our simple blockchain and it was relatively easy to do. But the problem here is that CrystalCoin
can only run on one single machine (it’s not distributed/decentralized).
From now on, we’ll start using JSON data for CrystalCoin
. The data will be transactions, so each block’s data field will be a list of transactions.
Each transaction will be a JSON object detailing the sender
of the coin, the receiver
of the coin, and the amount
of CrystalCoin that is being transferred:
{
"from": "71238uqirbfh894-random-public-key-a-alkjdflakjfewn204ij",
"to": "93j4ivnqiopvh43-random-public-key-b-qjrgvnoeirbnferinfo",
"amount": 3
}
A few modifications to our Block
class to support the new transaction
format (previously called data
). So, just to avoid confusion and maintain consistency, we’ll be using the term transaction
to refer to data
posted in our example application from now on.
We’ll introduce a new simple class Transaction
:
module CrystalCoin
class Block
class Transaction
property from : String
property to : String
property amount : Int32
def initialize(@from, @to, @amount)
end
end
end
end
The transactions are packed into blocks, so a block can contain just one or numerous transactions. Blocks containing the transactions are generated frequently and added to the blockchain.
The blockchain is supposed to be a collection of blocks. We can store all of the blocks in the Crystal list, and that’s why we introduce the new class Blockchain
:
Blockchain
will have chain
and uncommitted_transactions
arrays. The chain
will include all the mined blocks in the blockchain, and uncommitted_transactions
will have all the transactions that have not been added to the blockchain (still not mined). Once we initialize Blockchain
, we create the genesis block using Block.first
and add it to chain
array, and we add an empty uncommitted_transactions
array.
We will create Blockchain#add_transaction
method to add transactions to uncommitted_transactions
array.
Let’s build our new Blockchain
class:
require "./block"
require "./transaction"
module CrystalCoin
class Blockchain
getter chain
getter uncommitted_transactions
def initialize
@chain = [ Block.first ]
@uncommitted_transactions = [] of Block::Transaction
end
def add_transaction(transaction)
@uncommitted_transactions << transaction
end
end
end
In Block
class we will start using transactions
instead of data
:
module CrystalCoin
class Block
include ProofOfWork
def initialize(index = 0, transactions = [] of Transaction, previous_hash = "hash")
@transactions = transactions
...
end
....
def self.next(previous_block, transactions = [] of Transaction)
Block.new(
transactions: transactions,
index: previous_block.index + 1,
previous_hash: previous_block.current_hash
)
end
end
end
Now that we know what our transactions will look like, we need a way of adding them to one of the computers in our blockchain network, called a node
. To do that, we’ll create a simple HTTP server.
We’ll create four endpoints:
- [POST]
/transactions/new
: to create a new transaction to a block - [GET]
/mine
: to tell our server to mine a new block. - [GET]
/chain
: to return the full blockchain inJSON
format. - [GET]
/pending
: to return the pending transactions (uncommitted_transactions
).
We’re going to use Kemal web framework. It’s a micro-framework that makes it easy to map endpoints to Crystal functions. Kemal is heavily influenced by Sinatra for Rubyists and works in a very similar way. If you are looking for Ruby on Rails equivalent then check out Amber.
Our server will form a single node in our blockchain network. Let’s first add Kemal
to the shard.yml
file as a and install the dependency:
dependencies:
kemal:
github: kemalcr/kemal
Now let’s build the skeleton of our HTTP server:
# src/server.cr
require "kemal"
require "./crystal_coin"
# Generate a globally unique address for this node
node_identifier = UUID.random.to_s
# Create our Blockchain
blockchain = Blockchain.new
get "/chain" do
"Send the blockchain as json objects"
end
get "/mine" do
"We'll mine a new Block"
end
get "/pending" do
"Send pending transactions as json objects"
end
post "/transactions/new" do
"We'll add a new transaction"
end
Kemal.run
And run the server:
crystal_coin [master●●] % crystal run src/server.cr
[development] Kemal is ready to lead at http://0.0.0.0:3000
Let’s make sure the server is working fine:
% curl http://0.0.0.0:3000/chain
Send the blockchain as json objects%
So far so good. Now, we can proceed with implementing each of the endpoints. Let’s start by implementing /transactions/new
and pending
end-points:
get "/pending" do
{ transactions: blockchain.uncommitted_transactions }.to_json
end
post "/transactions/new" do |env|
transaction = CrystalCoin::Block::Transaction.new(
from: env.params.json["from"].as(String),
to: env.params.json["to"].as(String),
amount: env.params.json["amount"].as(Int64)
)
blockchain.add_transaction(transaction)
"Transaction #{transaction} has been added to the node"
end
Straightforward implementation. We just create a CrystalCoin::Block::Transaction
object and add the transaction to the uncommitted_transactions
array using Blockchain#add_transaction
.
At the moment, the transactions are initially stored in a pool of uncommitted_transactions
. The process of putting the unconfirmed transactions in a block and computing Proof of Work (PoW) is known as the mining of blocks. Once the nonce
satisfying our constraints is figured out, we can say that a block has been mined, and the new block is put into the blockchain.
In CrystalCoin
, we’ll use the simple Proof-of-Work algorithm we created earlier. To create a new block, a miner’s computer will have to:
- Find the last block in the
chain
. - Find pending transactions (
uncommitted_transactions
). - Create a new block using
Block.next
. - Add the mined block to
chain
array. - Clean up
uncommitted_transactions
array.
So to implement /mine
end-point, let’s first implement the above steps in Blockchain#mine
:
module CrystalCoin
class Blockchain
include Consensus
BLOCK_SIZE = 25
...
def mine
raise "No transactions to be mined" if @uncommitted_transactions.empty?
new_block = Block.next(
previous_block: @chain.last,
transactions: @uncommitted_transactions.shift(BLOCK_SIZE)
)
@chain << new_block
end
end
end
We make sure first we have some pending transactions to mine. Then we get the last block using @chain.last
, and the first 25
transactions to be mined (we are using Array#shift(BLOCK_SIZE)
to return an array of the first 25 uncommitted_transactions
, and then remove the elements starting at index 0).
Now let’s implement /mine
end-point:
get "/mine" do
blockchain.mine
"Block with index=#{blockchain.chain.last.index} is mined."
end
And for /chain
end-point:
get "/chain" do
{ chain: blockchain.chain }.to_json
end
Interacting With our Blockchain
We’ll be using cURL
to interact with our API over a network.
First, let’s fire up the server:
crystal_coin [master] % crystal run src/server.cr
[development] Kemal is ready to lead at http://0.0.0.0:3000
Then let’s create two new transactions by making a POST
request to http://localhost:3000/transactions/new
with a body containing our transaction structure:
crystal_coin [master●] % curl -X POST http://0.0.0.0:3000/transactions/new -H "Content-Type: application/json" -d '{"from": "eki", "to":"iron_man", "amount": 1000}'
Transaction #<CrystalCoin::Block::Transaction:0x10c4159f0 @from="eki", @to="iron_man", @amount=1000_i64> has been added to the node%
crystal_coin [master●] % curl -X POST http://0.0.0.0:3000/transactions/new -H "Content-Type: application/json" -d '{"from": "eki", "to":"hulk", "amount": 700}'
Transaction #<CrystalCoin::Block::Transaction:0x10c415810 @from="eki", @to="hulk", @amount=700_i64> has been added to the node%
Now let’s list the pending transactions (i.e. transactions that have not been added to the block yet):
crystal_coin [master●] % curl http://0.0.0.0:3000/pendings
{
"transactions":[
{
"from":"ekis",
"to":"huslks",
"amount":7090
},
{
"from":"ekis",
"to":"huslks",
"amount":70900
}
]
}
As we can see, the two transactions we created earlier have been added to uncommitted_transactions
.
Now let’s mine the two transactions by making a GET
request to http://0.0.0.0:3000/mine
:
crystal_coin [master●] % curl http://0.0.0.0:3000/mine
Block with index=1 is mined.
Looks like we successfully mined the first block and added it to our chain
. Let’s double check our chain
and if it includes the mined block:
crystal_coin [master●] % curl http://0.0.0.0:3000/chain
{
"chain": [
{
"index": 0,
"current_hash": "00d469d383005b4303cfa7321c02478ce76182564af5d16e1a10d87e31e2cb30",
"nonce": 363,
"previous_hash": "0",
"transactions": [
],
"timestamp": "2018-05-23T01:59:52+0300"
},
{
"index": 1,
"current_hash": "003c05da32d3672670ba1e25ecb067b5bc407e1d5f8733b5e33d1039de1a9bf1",
"nonce": 320,
"previous_hash": "00d469d383005b4303cfa7321c02478ce76182564af5d16e1a10d87e31e2cb30",
"transactions": [
{
"from": "ekis",
"to": "huslks",
"amount": 7090
},
{
"from": "ekis",
"to": "huslks",
"amount": 70900
}
],
"timestamp": "2018-05-23T02:02:38+0300"
}
]
}
Consensus and Decentralization
This is cool. We got ourselves a basic blockchain that accepts transactions and allows us to mine new blocks. But the code that we’ve implemented till now is meant to run on a single computer, while the whole point of blockchains is that they should be decentralized. But if they’re decentralized, how we ensure that they all reflect the same chain?
This is the problem of Consensus
.
We’ll have to implement a consensus algorithm if we want more than one node in our network.
Registering New Nodes
To implement a consensus algorithm, we need a way to let a node know about neighboring nodes on the network. Each node on our network should keep a registry of other nodes on the network. Therefore, we’ll need more endpoints:
- [POST]
/nodes/register
: to accept a list of new nodes in the form of URLs. - [GET]
/nodes/resolve
: to implement our Consensus Algorithm, which resolves any conflicts—to ensure a node has the correct chain.
We need to modify our blockchain’s constructor and provide a method for registering nodes:
--- a/src/crystal_coin/blockchain.cr
+++ b/src/crystal_coin/blockchain.cr
@@ -7,10 +7,12 @@ module CrystalCoin
getter chain
getter uncommitted_transactions
+ getter nodes
def initialize
@chain = [ Block.first ]
@uncommitted_transactions = [] of Block::Transaction
+ @nodes = Set(String).new [] of String
end
def add_transaction(transaction)
Note that we’ve used a Set
data structure with String
type to hold the list of nodes. This is a cheap way of ensuring that the addition of new nodes is idempotent and that no matter how many times we add a specific node, it appears exactly once.
Now let’s add a new module to Consensus
and implement the first method register_node(address)
:
require "uri"
module CrystalCoin
module Consensus
def register_node(address : String)
uri = URI.parse(address)
node_address = "#{uri.scheme}:://#{uri.host}"
node_address = "#{node_address}:#{uri.port}" unless uri.port.nil?
@nodes.add(node_address)
rescue
raise "Invalid URL"
end
end
end
The register_node
function, will parse the URL of the node and format it.
And here let’s create /nodes/register
end-point:
post "/nodes/register" do |env|
nodes = env.params.json["nodes"].as(Array)
raise "Empty array" if nodes.empty?
nodes.each do |node|
blockchain.register_node(node.to_s)
end
"New nodes have been added: #{blockchain.nodes}"
end
Now with this implementation, we might face a problem with multiple nodes. The copy of chains of a few nodes can differ. In that case, we need to agree upon some version of the chain to maintain the integrity of the entire system. We need to achieve consensus.
To resolve this, we’ll make the rule that the longest valid chain is the one to be used. Using this algorithm, we reach consensus amongst the nodes in our network. The reason behind this approach is that the longest chain is a good estimate of the most amount of work done.
module CrystalCoin
module Consensus
...
def resolve
updated = false
@nodes.each do |node|
node_chain = parse_chain(node)
return unless node_chain.size > @chain.size
return unless valid_chain?(node_chain)
@chain = node_chain
updated = true
rescue IO::Timeout
puts "Timeout!"
end
updated
end
...
end
end
Bear in mind that resolve
is a method which loops through all our neighboring nodes, downloads their chains and verifies them using the valid_chain?
method. If a valid chain is found, whose length is greater than ours, we replace ours.
Now let’s implement parse_chain()
and valid_chain?()
private methods:
module CrystalCoin
module Consensus
...
private def parse_chain(node : String)
node_url = URI.parse("#{node}/chain")
node_chain = HTTP::Client.get(node_url)
node_chain = JSON.parse(node_chain.body)["chain"].to_json
Array(CrystalCoin::Block).from_json(node_chain)
end
private def valid_chain?(node_chain)
previous_hash = "0"
node_chain.each do |block|
current_block_hash = block.current_hash
block.recalculate_hash
return false if current_block_hash != block.current_hash
return false if previous_hash != block.previous_hash
return false if current_block_hash[0..1] != "00"
previous_hash = block.current_hash
end
return true
end
end
end
For parse_chain()
we:
- Issue a
GET
HTTP request usingHTTP::Client.get
to/chain
end-point. - Parse the
/chain
JSON response usingJSON.parse
. - Extract an array of
CrystalCoin::Block
objects from the JSON blob that was returned usingArray(CrystalCoin::Block).from_json(node_chain)
.
There is more than one way of parsing JSON in Crystal. The preferred method is to use Crystal’s super-handy JSON.mapping(key_name: Type)
functionality that gives us the following:
- A way to create an instance of that class from a JSON string by running
Class.from_json
. - A way to serialize an instance of that class into a JSON string by running
instance.to_json
. - Getters and setters for keys defined in that class.
In our case, we had to define JSON.mapping
in CrystalCoin::Block
object, and we removed property
usage in the class, like so:
module CrystalCoin
class Block
JSON.mapping(
index: Int32,
current_hash: String,
nonce: Int32,
previous_hash: String,
transactions: Array(Transaction),
timestamp: Time
)
...
end
end
Now for Blockchain#valid_chain?
, we iterate through all of the blocks, and for each we:
- Recalculate the hash for the block using
Block#recalculate_hash
and check that the hash of the block is correct:
module CrystalCoin
class Block
...
def recalculate_hash
@nonce = proof_of_work
@current_hash = calc_hash_with_nonce(@nonce)
end
end
end
- Check each of the blocks linked correctly with their previous hashes.
- Check the block’s hash is valid for the number of zeros (
difficulty
in our case00
).
And finally we implement /nodes/resolve
end-point:
get "/nodes/resolve" do
if blockchain.resolve
"Successfully updated the chain"
else
"Current chain is up-to-date"
end
end
It’s done! You can find the final code on GitHub.
The structure of our project should look like this:
crystal_coin [master●] % tree src/
src/
├── crystal_coin
│ ├── block.cr
│ ├── blockchain.cr
│ ├── consensus.cr
│ ├── proof_of_work.cr
│ ├── transaction.cr
│ └── version.cr
├── crystal_coin.cr
└── server.cr
Let’s Try it Out
- Grab a different machine, and run different nodes on your network. Or spin up processes using different ports on the same machine. In my case, I created two nodes on my machine, on a different port to have two nodes:
http://localhost:3000
andhttp://localhost:3001
. - Register the second node address to the first node using:
crystal_coin [master●●] % curl -X POST http://0.0.0.0:3000/nodes/register -H "Content-Type: application/json" -d '{"nodes": ["http://0.0.0.0:3001"]}'
New nodes have been added: Set{"http://0.0.0.0:3001"}%
- Let’s add a transaction to the second node:
crystal_coin [master●●] % curl -X POST http://0.0.0.0:3001/transactions/new -H "Content-Type: application/json" -d '{"from": "eqbal", "to":"spiderman", "amount": 100}'
Transaction #<CrystalCoin::Block::Transaction:0x1039c29c0> has been added to the node%
- Let’s mine transactions into a block at the second node:
crystal_coin [master●●] % curl http://0.0.0.0:3001/mine
Block with index=1 is mined.%
- At this point, the first node has only one block (genesis block), and the second node has two nodes (genesis and the mined block):
crystal_coin [master●●] % curl http://0.0.0.0:3000/chain
{"chain":[{"index":0,"current_hash":"00fe9b1014901e3a00f6d8adc6e9d9c1df03344dda84adaeddc8a1c2287fb062","nonce":157,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:45+0300"}]}%
crystal_coin [master●●] % curl http://0.0.0.0:3001/chain
{"chain":[{"index":0,"current_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","nonce":147,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:38+0300"},{"index":1,"current_hash":"00441a4d9a4dfbab0b07acd4c7639e53686944953fa3a6c64d2333a008627f7d","nonce":92,"previous_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","transactions":[{"from":"eqbal","to":"spiderman","amount":100}],"timestamp":"2018-05-24T00:23:57+0300"}]}%
- Our goal is to update the chain in the first node to include the newly generated block at the second one. So let’s resolve the first node:
crystal_coin [master●●] % curl http://0.0.0.0:3000/nodes/resolve
Successfully updated the chain%
Let’s check if the chain in the first node has updated:
crystal_coin [master●●] % curl http://0.0.0.0:3000/chain
{"chain":[{"index":0,"current_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","nonce":147,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:38+0300"},{"index":1,"current_hash":"00441a4d9a4dfbab0b07acd4c7639e53686944953fa3a6c64d2333a008627f7d","nonce":92,"previous_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","transactions":[{"from":"eqbal","to":"spiderman","amount":100}],"timestamp":"2018-05-24T00:23:57+0300"}]}%
Hooray! Our Crystal language example works like a charm, and I hope you found this lengthy tutorial crystal clear, pardon the pun.
Wrapping Up
This Crystal language tutorial covered the fundamentals of a public blockchain. If you followed along, you implemented a blockchain from scratch and built a simple application allowing users to share information on the blockchain.
We’ve made a fairly sized blockchain at this point. Now, CrystalCoin
can be launched on multiple machines to create a network, and real CrystalCoins
can be mined.
I hope that this has inspired you to create something new, or at the very least take a closer look at Crystal programming.
Note: The code in this tutorial is not ready for real-world use. Please refer to this as a general guide only.
Understanding the basics
What are the uses for Crystal programming language?
Crystal is a generic language. You can do whatever you can do in Ruby using Crystal, with better performance and less memory use. One selling point for Crystal is the ease with which you can interface with C libraries, as Crystal allows you to bind to existing C libraries without writing a single line in C.
Why should I learn Crystal programming language?
Crystal is a programming language that can be easily understood by humans and compiles to fast programs. It’s a statically-typed, compiled language, that achieves performance close to c/c++ while having a syntax as readable as Ruby.
What’s the difference between Ruby and Crystal?
Crystal has a Ruby-like syntax, but it’s a different language, not a Ruby implementation. Because it’s a compiled, statically typed language, the language has some differences when compared to Ruby. Coming from Ruby, Crystal has a low learning curve.
What about frameworks?
If you love the completeness of Rails, you’ll feel right at home with the Amber framework. If the simplicity and easy customization of Sinatra more your thing, you’ll find that simplicity with Kemal.
Is Crystal a promising programming language?
It is one of the more promising programming languages today. It appears to have shattered the barrier between syntactically sweet interpreted/dynamic languages like Ruby/Python, which are loved for their readability and ease-of-use, and the raw horsepower of C/C++ and low-level systems languages.
Eqbal Quran
Amman, Amman Governorate, Jordan
Member since June 13, 2014
About the author
Eqbal is a senior full-stack developer with more than a decade of experience working in web and mobile development.
Expertise
PREVIOUSLY AT