Use cases for the ThreeFold Peer-to-Peer Cloud

Any Docker Service

ThreeFold nodes natively run containers - therefore they can run any Docker service. But it does so in a more efficient way than the current Docker engine - the most used and known container technology platform in the world.

IAAS

The ThreeFold node technology enables hardware capabilities to be used for IAAS services in a very efficient way. Overhead is reduced to a minimum by stripping out unnecessary layers of software that have been implemented over the last decade to patch scaling and performance problems. Our engineers have always looked at the root cause of issues and by innovating at the core algorithm level, solved many issues. Examples are:

• Less is more: Scale down the operating system for a virtualization architecture to a minimum and boot it over the network - no local operating system files installed on local storage.
• Look at root causes, no painkiller approach: Step away from existing storage solutions that require proprietary acceleration hardware and use standard off the shelf, affordable and efficient components with a 100% software-based storage solution that can withstand hardware failures without requiring human intervention - creating a very efficient, reliable and performing storage solution that can operate standalone.
  
  The platform is secured by using a network boot mechanism that makes it virtually hack resistant as there are no operating system files installed locally. Infrastructure as a service traditionally runs in hyper-scale data centers, remote from the businesses using it. ThreeFold node technology allows you to install, operate and run IAAS services close to your business locations operated by your local IT team. It operates on a peer-to-peer network - only transferring bits of data that need to be transported to avoid network congestion and traffic bottlenecks.

Archive

Data generation is growing exponentially. More data has been generated in the last 2 years than ever before. This creates a big need to archive data. The ThreeFold node technology creates storage and archiving capabilities by using standard hardware and known interfaces (S3, FTP, WebDav, CIFS, and NFS) will be built by developers worldwide and made available. The low-level storage functionality supports version controlling of the archived data, which keeps all relevant changes without any limitation by using a small amount of actual storage capacity.

Security

Data security and reliability are key aspects of any storage system. The ThreeFold node allows all data to be compressed and encrypted if and when needed. The distributed character of the storage system enables site redundant storage algorithms to be deployed while keeping large data volumes from being stored in a single location. This removes the risk of data being stolen (physically) and prevents a site outage to result in data outage. The distributed character of the system is in line with the increasing spread of actual data generation. ThreeFold has sensory equipment everywhere these days, and the overhead of storing all that information in a few central places is enormous. Data collection and storage happens once - reading data to use it for a particular workload happens many times.

Data Mining

On traditional server architectures, every application has to set up its servers that run their code in isolated silos, making sharing of data hard. The distributed character of the ThreeFold node technology presents compute and storage capacity everywhere. This enables large data mining workloads to happen close to the source of the data. By storing data in a distributed manner, mining can be distributed as well. Distributed compute capacity next to storage capabilities creates the possibility to create data mining operations on local storage (which makes it very fast and efficient) by coding data mining algorithms in programming languages (such as Python, Lua, Javascript, and Golang). Data mining follows distributed data storage.

Never run into scalability problems again with the ThreeFold node. Data storage can expand horizontally using thousands of distributed nodes to create large storage volumes.

Smart Contracts

Smart contracts are applications that run on a decentralized platform, exactly as programmed without any possibility of downtime, censorship, fraud, or third-party interference. These applications run on a blockchain, a powerful shared global infrastructure that can move value around and represent the ownership of that value. Smart contracts enable developers to create markets, store registries of debts (or promises), move funds following past instructions (like a will or a futures contract), and many other things without a middleman or counterparty risk. ThreeFold nodes run based on smart-contracts.

Self-Learning and AI

Blockchains can be seen as databases. By current database standards, traditional blockchains like Bitcoin are terrible: low throughout, low capacity, high latency, poor query support, and so on. But even with these terrible characteristics, blockchains introduced three new and important characteristics:

• Decentralized data and shared data control leads to more data and better qualitative data which results in better models. Shortening the machine learning time also enables AI to progress at a faster pace.
• Immutable audit trails improve the trustworthiness of the data, and improves the models’ provenance on training and testing results.
• Native assets and assets exchanges make training and testing models intellectual property (IP) assets easier - enhancing decentralized data and model exchanges.
  
  Blockchain technology allows more and better quality data to be presented to machine learning algorithms. It has been proven (Microsoft researchers - Banko and Brill) that more data — not just a bit more data but way more data — while using the same algorithms results in a lot better machine learning. Meaning that the future of machine learning is not in trying to create more sophisticated algorithms (which will happen). So by moving AI learning into a blockchain environment we speed up machine learning and in the end create AI.

ThreeFold’s technology does not follow the traditional IT model

The Edge internet needs a different approach. So, we started from scratch and shaped a new approach. ThreeFold’s open-source technology addresses the scalability, security, storage, energy, and operating issues by simplifying architectures and not relying on third-party software. This drives down costs and improves efficiencies.

What is ThreeFold’s Technology Toolbox?

ThreeFold’s technology is based on three primary building blocks: Stateless Computing, Self-Healing Technology, and Blockchain technology - this is the core of our “Technology Toolbox“.

• Stateless Computing: a stateless operating system, Zero-OS, enabling distributed hardware to form an Edge Internet grid generating IT storage and compute capacity
• Self-Healing Technology: controls and operates a distributed cloud and reserves, procures, and executes workloads on the grid using our Jumpscale platform. It delivers autoscaling and automation frameworks that enable self-healing and can be adapted for any kind of IT workload.
• Blockchain Technology: Via Rivine - our secure, scalable “proof of stake” blockchain, along with our identity management platform, ItsYou.online. Our blockchain technology enables user authentication, access controls, secure workloads, and heightened security.

Enabling Stateless Computing

The global decentralized internet will be composed of thousands (millions?) of nodes distributed geographically. This means the decentralized internet must be powered by a stateless operating system that can be booted via the Internet (as opposed to living on the node locally). This removes the complexity of upgrades, truck rolls, etc.

Open-source components have been developed, Zero-OS, for the deployment of bare metal machines functioning as Internet nodes and application hosts. Our Zero-OS platform runs containerized applications and software code on secure and distributed IT infrastructure. Zero-OS can be deployed on off-the-shelf hardware by anyone willing to make server and storage capacity available in their markets and within our network.

The benefits of Stateless Computing are substantial, including:

• Performance: lightweight OS without overhead for running bare-metal machines.
• Free: finds the optimal performance of server & storage nodes at no cost.
• Private: no data footprint remains while workloads move from one node to another.
• Distributed: capacity is generated without the need for data centers: At The Edge.
• High-available: easy creation of clusters while applications run in fragmented segments.
• Portability: moving software from one node to another within the grid is instant.
• Secure: open-source distributed software with transparent code. No security backdoors
• Sustainable: energy-efficient OS resulting in very low node power consumption.

Technology that is self-healing & self-driving

ThreeFold’s Jumpscale framework and ZeroBot allow the creation of blueprints to auto-deploy applications within the compute environment. The bots are constantly checking the health and uptime of the infrastructure, and runtime environments, to guarantee the performance and availability of applications at all times.

The benefits of our self-healing technology protocols include the following:

• Self-Healing: the platform regenerates when health issues that cause downtime for infrastructure and applications are detected.
• Intelligence: agents capture information from the grid and make intelligent decisions to provide an ideal and optimal host environment for software code and containerized applications.
• Lifecycle Management: the status of the infrastructure and the lifecycle of running applications are constantly checked and recorded.
• Autoscaling: the platform automatically scales to meet application and infrastructure performance.

Blockchain Technology to execute workloads and applications via smart contracts

The engineers developed new blockchain algorithms based on proof of stake for validation purposes and turned this into our Rivine open-source blockchain software.

Unlike the “proof of work” blockchains that currently dominate the space, our Rivine blockchain is scalable, green, and secure - and does not rely on huge mining resources. We run our blockchain on approximately 200 dedicated nodes and do not rely on any third-party software.

In addition to our blockchain technology, we have also developed ItsYou.Online, an identity, and access management platform. Itsyou. online uses three-factor authentication to manage access on our network and facilitate execution.

The benefits of TF blockchain technology and identity management platform include:

• Sustainable: our algorithms to reach consensus use a minimal amount of energy and electricity compared to other blockchain software and networks.
• Lowest possible cost: of transactions make it an efficient and fast payment system.
• Neutral: state of the network and applications are recorded in a transparent and immutable manner.
• Payment fulfillment: transactions are automatically paid using ThreeFold Tokens.
• Smart Contracts: fully automated execution of contracts from customers buying Internet IT capacity.
  
  All of our technology is open-source and can be accessed in the following Github Repos:
  
  

• Zero-OS: https://github.com/threefoldtech/zos
• Jumpscale: https://github.com/threefoldtech/js-ng
• BCDB : https://github.com/threefoldtech/bcdb

Introduction

Blockchain technology has the power to change the world of IT. There are a huge number of new initiatives using blockchain technology around the world, with countless use cases.

In general, there are two types of blockchains: proof of work blockchains and proof of stake blockchains. They are consensus algorithms on how transactions are verified and added to the ledger (forever, as blockchains provide an immutable record of events). More on these two types of blockchains below.

Blockchain Basics

A blockchain is, as the word says, a chain of blocks of information. These “blocks” contain information on transactions. For instance transactions like the transfer of value of a digital currency (token) from wallet A to wallet B. The “chain” aspect comes when part of the information is stored in a block as a link to the previous block (a unique number). This is a very simple representation of what a blockchain is.

Private, Permissioned or Public Blockchains

The complexity around a blockchain comes from how the blockchain software is run. The software can run in two different ways: a small (or large) number of nodes controlled by one single entity (private blockchain) a person or a company, or a large (or small) number of nodes controlled by multiple entities (public) that do not know each other and where anyone interested to run a blockchain node can join.

By running blockchain software under the supervision of one entity (private, centralized) you can quite easily make sure the information stored in the blockchain (transactions) are not tempered with by only operating a small number of nodes and deploying them in very safe physical locations (private data-centers, vaults, basements of offices, etc). A private blockchain does not allow unknown entities to join and run the blockchain software.

A permissioned blockchain is operated by known entities such as stakeholders of a given industry. It is a mix of both private and public blockchains. In this type of blockchain network, a participant may not need permission to join the network but needs permission to transact with another network participant.

A public blockchain allows anyone to join the blockchain operation and create a new blockchain node. The blockchain is run by multiple entities (decentralized, public) that have no relationship nor knowledge about each other

For these three types of blockchains, there are different mechanisms to protect and guarantee the validity of transactions and make them tamper-proof.

Private Data and Anonymous or Pseudo-Anonymous Data?

For private blockchains, it is straightforward to see that all data that is stored on private blockchains is private and creates 100% anonymity for its users. The only entity or person that has access to information stored on the blockchain is the blockchain operator. This is comparable to how a bank operates. The bank knows everything about all of its customers and the customers only know about themselves - the non-bank customers do not know a thing.

On a public blockchain, people can join and operate a blockchain node on which all information of that blockchain is stored. While private blockchains get their security from putting their blockchain nodes in secure buildings operated by trusted people a public blockchain achieves its security from having a large number of nodes that have the same data replicated over and over. A consensus algorithm is required in order to accept new data (new blocks) to the chain.

Consensus Protocols for Public Blockchains

The main consensus protocols used today are Proof-of-Work and Proof-of-Stake.

Proof of Work (High Cost in Compute and Energy)

The proof of work consensus protocol is best explained as a race. The consensus is achieved by having all participating nodes solve a puzzle. The puzzle is a one-way translation of an amount of information consisting of the following items:

• All the data that present the transactions in the block - all the data is known by all participating nodes
• The number of the previous block. Remember a blockchain links blocks of information together.
• A (variable) number. This number is the part that can be changed.
  
  The “work” is done by changing the variable number and see what the translation of all data brings. The translation of all this data is a new number, and the requirement is that that number needs to be smaller than what is known as the “difficulty level”. The difficulty level number has a fixed amount of numbers but when the first digits of the number are zero the overall number is lower. So a higher difficulty is built by having a number that has more zeros at the start in the difficulty number.
  
  Once the block is completed, all nodes will get the complete block of transactional data, everyone knows the previous block number and will start to change the variable number to get output from the puzzle algorithms comparing it to the required difficulty level.
  
  The node that will find the number first will announce that it has found a solution matching the required difficulty level and the variable number of shared with all nodes in the network to verify that his solution is correct. If a large portion of the nodes has verified the solution to be valid the solution is accepted and the block is added to the chain because consensus is reached.

Proof of (Block) Stake (Low Cost in Compute and Energy)

With proof of stake consensus algorithms, the consensus algorithm does not solve puzzles in a race to be the first. The first step is to identify and allow a certain number of nodes to be part of the blockchain. This is called a permissioned blockchain because you need to have permission to partake. This permission is given by distributing stakes.

Proof of Stake consensus mechanism is solving a puzzle, but the puzzle contains only static elements - not variables - based on different characteristics:

• The block number
• The content (part) of blocks a long time ago (>2000 blocks)
• The transaction number (between the first and the last transaction) in the block of the stake transfer transaction all participating nodes needs to
• The current timestamp
  
  All these numbers are fixed but the time stamp and every node in the block stake blockchain will solve this puzzle every second until the output of this puzzle matches the required difficulty level.
  
  To make sure the one node that has solved the puzzle to match the difficulty level cannot introduce information in the block which allows him to tamper with the next blocks this node cannot use his used stake for a period amount of time for creating new blocks.
  
  The fact that the puzzle-solving includes historic block data and once a node is assigned to create the block and put it on a chain will not be able to use that block state for a period of time makes it very costly to try to tamper with the content of the block (and the next blocks).

The Blockchain Dilemma Problem

Public and private blockchains have their specific advantages and disadvantages and are very good for specific use cases - but not all. If we list the specifics of both we end up with a table like this:

| * | Public Blockchains | Private Blockchains | |—|—|—| | Security | Provided by having a large number of nodes running it | Provided by having them in secure controlled places | | People | Build, managed and controlled by unknown people | Build, managed, and controlled by known people |
| Speed | Slow by design - all data needs to be copied to all of them | Fast(er) and more efficient by design - a smaller number of copies of the blockchain data |


So what if we want to have a secure, performant, and easy to manage blockchain solution? This is not what can be delivered by either type of blockchains. We have to create a new type of blockchain that combines the best of both worlds and delivers.

The key element in finding a solution for this problem is to take the human element out of it. The fact that when you deploy blockchains solutions in locations without human intervention means that you need to create a different deployment mechanism that takes out people touching code, compiling code, deploying code, and in the end operating the machine in which the code runs.

A way to get around the trusted people problem is to select a platform on which software can be downloaded, compiled, installed, and operated without human intervention. Such a platform is a unique platform that has not been developed and launched. Such a platform would have to have the following characteristics. To make this system work and solve the blockchain dilemma we need to have three components working together in an orchestrated way. The three elements are:

• Zero-OS: A (simple) operating system that does not allow local and remote logins - a closed operating system that receives instructions to launch or kill applications. This operating system needs to be stateless to keep as simple as possible:
• 3Bot: A virtual system administrator that is able to perform system administration tasks to make the system operate, self-heal and execute instructions from authenticated and authorized sources.
• Zero-Chain: A ledger or database to store information with regards to authentication, operational and financial transactions.
  
  This system architecture allows for authenticated and trusted virtual system administrators to download, compile and deploy software on a grid of secure Zero-OS nodes with an immutable ledger that records all transactions, operational and financial. This presents a platform on which a public (permissioned) blockchain can be run securely without requiring trusted people to operate the nodes.
  
  In such architecture, we can build and deploy a high-performance secure blockchain which are the combined benefits of a public and private blockchain.

“This article was originally published by Andreas Hartl, a former member of ThreeFold Foundation.”

Hyperscale realities are very different from what most people think

Myth 1: Bigger data centers are more efficient

There are advantages in building larger-scale data centers, like economies of scale and sheer bulk buying power, but these are not as significant as what people think. The average cost per rack in a hyper-scale data center is $20-35K USD including all energy requirements and safety systems. The cost of hardware per rack is around the $200-300K USD mark.

What is often forgotten is that anything that is of enormous scale and highly concentrated comes with complexity and specific problems to deal with. For example. resource requirements such as investment, operational costs, knowledge, and people increase significantly with size.

In reality, keeping things simple, small, and distributed is much more cost-effective than large complex environments.

Myth 2: Big data centers can be sustainable

The carbon footprint of a big data center is enormous. To improve the power usage effectiveness (PUE) of most data center farmers have adopted wind, hydro, and/or solar power technologies, which indeed helps drop their PUE by an estimated 20%. But is this leading to more sustainability?

This 20% looks great on paper and in the farmers’ corporate social responsibility reports. However, it just represents an improvement in the cooling technology and sourcing of energy. It doesn’t actually impact the energy consumed by the equipment that runs in their data centers (servers, storage chassis, physical disks, etc.) which is the equipment responsible for the carbon footprint. PUE only refers to overhead power consumption, i.e. cooling the facility, opening, and closing doors, maintaining power security systems, etc.

Real improvement lies in deploying technologies that actually consume less power to deliver the actual Internet capacity to run workloads, real CPU chassis, physical disks, and storage cabinets. Improving on how hardware is being more effectively used can have an impact of up to a thousand percent and lead to ten times more power-efficiency.

Myth 3: Redundant systems have better uptime

A lot of us believe that systems need redundancy mechanisms to improve their operational uptime and reliability. While it may make sense to IT experts, let’s translate this for the non-IT world?

To make a car more reliable we add redundancy (as we do in IT). So for the risk of having a puncture, we add one extra tire for all the tires we use continuously. This adds 4 extra tires to the car. Then a decision needs to be made: Do we put those tires in a structure where they are always running along with the primary tires or do we choose not to have them “online” all the time, wearing and tearing in the same way as the primary tires?

Building such a system would take a large number of engineers to come up with a solution, and would change cars as we know them. Wouldn’t it make more sense to think outside the box and solve the root of the problem by making tires un-deflatable?

The IT industry has gone overboard with the concept of redundancy, having forgotten to look at the root cause issues. This has spawned a whole new industry of itself, which has a financial interest in creating complicated and expensive redundant systems.

Myth 4: Big companies optimize better

Big companies with a certain track record will know better how to optimize as they have more people and resources.

At first glance, this sounds logical, but in reality, ninety percent of the innovation within the IT space comes from startups. The giant IT companies have a heritage they hardly can overcome. They are locked in old infrastructure designs, and building outside of that infrastructure would be costly, timely, and probably put the breaks on their businesses.

Real innovation gives way to fix the core symptoms rather than taking the problem pain-killer approach.