🔗Compute & Storage

What it is:

Compute is responsible for processing data. It runs programs and applications by leveraging computational resources like CPUs (Central Processing Units) and GPUs (Graphics Processing Units). It handles the calculations, logic, and running of software that users depend on. Compute resources are provided by servers located in centralized data centers and cloud platforms.

Storage is about saving and retrieving data. It provides a persistent way to store data so that it can be accessed in the future. This encompasses various types of storage media, such as hard drives (HDDs), solid-state drives (SSDs), and distributed storage systems that span across multiple locations. Storage ensures that data is kept safe and available for when it’s needed, either for immediate use by applications or for long-term retention.

The problem with the current model:

A key problem with the current compute & storage layer lies in its centralized structure, which creates vulnerabilities, scalability challenges, and performance issues. Relying on centralized data centers and cloud platforms leads to single points of failure, which increases the risk of outages and cyberattacks, thus heightening concerns about service disruption and data security. Scaling this centralized infrastructure to meet increasing demand is costly and not straightforward, while latency issues hinder real-time data processing.

The concentration of the compute & storage within data centers primarily located in Europe, North America, or China raises significant concerns. This geographical bias in Internet infrastructure keeps data, jobs, and economic benefits centralized in certain regions, putting developing areas at a disadvantage and raising concerns around data sovereignty and autonomy.

For example, when two people in Zanzibar undertake a video call, their call data is sent to a data center in Europe via costly fiber optics, wasting resources and diminishing performance. This scenario represents a broader problem affecting various locations and use cases. This inefficient and inequitable system can be changed.

The ThreeFold model:

ThreeFold decentralizes compute & storage by distributing it across a global network of independent nodes rather than concentrating it in large data centers. Autonomous technology allows us to remove most human involvement from the equation. This approach allows anyone, regardless of technical know-how, to contribute compute & storage to the network or utilize it for their own applications. In essence, the compute & storage layer becomes co-owned, allowing anyone to become a cloud provider.

ThreeFold’s autonomous technology efficiently allocates compute resources, increasing performance and reliability while reducing energy consumption. The distributed storage system fragments data and stores it securely across multiple locations, ensuring that data is always recoverable. Data security is also enhanced as there is no central repository for attackers to target. In this decentralized model, individuals’ data remains within their country’s borders, safeguarding their information from being exploited as a commodity by external entities. Storing data closer to the end user reduces latency and improves access speeds.

Let’s revisit the scenario of a video call between two individuals in Zanzibar. With ThreeFold technology, call data will be sent directly between local ThreeFold nodes. These nodes are deployed locally by people and organizations, avoiding centralized ownership by major tech corporations. Instead of relying on data centers thousands of miles away, our decentralized infrastructure significantly shortens the connection distance. The pair in Zanzibar can now communicate peer-to-peer, reducing bandwidth needs by up to 10 times while boosting security.

🔗Network

What it is:

The purpose of the network layer of the Internet is to send data packets from their source (point A) to their destination (point B) across multiple networks. Data packets are small pieces of information which include the data being sent and details like where it’s coming from and where it’s going. They can be thought of as small parcels of data that travel through the Internet. These parcels can take different paths to reach their destination and once they arrive they are put back together to form the original data.

The network operates on protocols like IP (Internet Protocol), which assigns unique addresses to devices on the Internet to ensure data reaches the correct location. The network layer handles the path selection and directs data packets through various routers and networks to reach its destination. This layer enables communication between devices in different networks, ensuring efficient delivery across the vast expanse of the Internet.

The problem with the current model:

The current model of the network layer relies heavily on traditional routing protocols and infrastructure, which often lead to inefficiencies and vulnerabilities. Centralized routing mechanisms may result in suboptimal path selection, leading to longer latency and decreased network performance. As highlighted in the picture below, the path that these data packets take in the current system can be very complex and ineffective. The reliance on a centralized architecture makes the network susceptible to single points of failure and cyberattacks. Traditional networks lack inherent privacy and security measures, leaving data vulnerable to interception and unauthorized access during transmission. The lack of scalability, reliability, and security of the current network model highlights the need for a new approach.

Credit: Lazario Gamio, 2015, The Washington Post

The ThreeFold model:

ThreeFold is disrupting the network layer through its new Mycelium technology, a true peer-to-peer mesh network. It is the only overlay network in the world capable of taking locality into consideration, routing decisions based on proximity between individuals, and selecting the quickest route possible. This addresses the current system’s suboptimal path selection as visualized above. In this new model data packets take the quickest route from point A to point B. Mycelium also ensures end-to-end encryption for all traffic, enhancing data privacy and security. This new ‘always-on’ network offers continuous connectivity and seamless access to resources, even in the face of network disruptions or outages.

Credit: Haseeb Qureshi, 2019, Nakamoto

🔗Applications

What it is:

At the top of the Internet stack are applications — the user-facing software that provides the services and experiences we interact with daily. Applications range from web browsers and social media to more complex systems like blockchain platforms and decentralized apps (dApps). They are the interface through which users interact with the digital world.

The problem with the current model:

The problem with the current model is the lack of control over our digital identities. Currently, users are scattered across numerous platforms, each hosting their data separately. This means that individuals exist in multiple instances online, with their personal information spread across different servers and databases controlled by different applications (visualized in the picture below). This fragmentation not only leads to redundancy but also raises concerns about security, privacy, and control over personal data. Users are forced to trust multiple third-party applications with their sensitive information, often without transparency or control over how their data is used or shared. The centralized nature of these applications exposes users to risks such as data breaches, unauthorized access, and exploitation of personal information for targeted advertising or other purposes. In the existing system, users are at the mercy of application providers, who dictate the terms of service and privacy policies. This asymmetry in power dynamics leaves users with limited options to protect their privacy or exercise control over their data.

The ThreeFold model:

ThreeFold delivers a decentralized and user-centric approach to the application layer, empowering users and developers. By providing access to decentralized compute, storage, and networking resources, ThreeFold facilitates the development and deployment of decentralized applications (dApps) within its ecosystem. Unlike traditional centralized applications, which rely on proprietary infrastructure and are subject to the control of centralized entities, dApps built on ThreeFold operate within a decentralized environment increasing security and privacy. Users have greater autonomy and control over their digital identities and data. Instead of being scattered across numerous platforms with data hosted separately, users have one digital identity. This reduces fragmentation and eliminates the need to trust multiple third-party applications with sensitive information. In the ThreeFold model, applications are designed to serve the users rather than the other way round.

🔗Conclusion

Understanding the Internet’s three layers is key to grasping the current digital landscape and envisioning the transformative potential of ThreeFold’s technology. Just as tectonic forces reshape our physical world, ThreeFold is disrupting the layers of the digital world. By decentralizing compute & storage, ThreeFold eliminates the vulnerabilities in centralized models, fostering efficiency and security. Through Mycelium, ThreeFold is rewriting the rules of connectivity, ensuring resilience and security. At the application layer, ThreeFold empowers users, shifting the paradigm to user empowerment. It’s a new dawn for the Internet, one where connectivity is inclusive, efficient, and equitable, and ThreeFold is leading the charge. Let’s seize the opportunity to shape an Internet that thrives locally, serves globally, and empowers universally.

🔗The Two Main Flaws of Existing Data Centers

🔗Inadequate Power Provision

The advent of AI has dramatically escalated power needs. An AI-centric rack can consume up to 200 kW – a staggering 15 times more than the capacity of data centers built in the conventional model. This exponential increase poses significant challenges, particularly when dissipation exceeds 50 kW per rack. Traditional designs fall short in crucial aspects like cooling and power distribution, with conventional air cooling methods becoming increasingly unviable.

🔗The Shift in Market Needs: Beyond Co-location

The market dynamics have evolved; selling space in a data center, akin to ‘advanced real estate’, is a model that is rapidly losing relevance. The traditional co-location framework, which expects customers to manage their hardware and infrastructure within leased rack space, is becoming antiquated.

In the modern landscape, the role of a data center should transcend mere space rental. The emphasis should be on providing integrated solutions – selling compute, storage, and network capacity, not just rackspace and power. It’s about delivering a comprehensive service that aligns with the complex, ever-evolving needs of businesses immersed in the digital and AI revolution.

🔗The Data Center Landscape Today

Tier 3 and Tier 4 Data Centers are classifications outlined by the Uptime Institute, which provide standards for the design, construction, and operation of data centers. The Institute defines four tiers (1, 2, 3, & 4) to categorize data centers according to their design and infrastructure capabilities. For the purposes of this article, we will focus on the top two tiers.

🔗Tier 3

A Tier 3 Data Center is a type of data center facility that is designed to provide a high level of availability and redundancy for IT infrastructure and data storage.

🔗Key Characteristics

• Availability: Tier 3 Data Centers typically offer 99.982% availability, translating to around 1.6 hours of downtime per year.
• Redundancy: They feature N+1 redundancy for power and cooling systems, providing backup components to ensure continuous operation in case of failure.
• Concurrent Maintenance: Tier 3 Data Centers allow for maintenance and repairs to be performed on systems without disrupting operations.
• Cost: Tier 3 Data Centers are less expensive to build and operate compared to Tier 4 due to lower redundancy requirements.

It’s important to note that while Tier 3 Data Centers provide a high level of availability and redundancy, they are not as robust as Tier 4 Data Centers, which are designed to provide even higher levels of fault tolerance and uptime. The choice of data center tier depends on the specific needs and risk tolerance of the organization using the facility.

🔗Key Challenges

• Limited Uptime: Despite high reliability, Tier 3 Data Centers may still experience noticeable downtime, which could impact critical operations.
• Single Points of Failure: While designed for redundancy, Tier 3 Data Centers may still have single points of failure, potentially leading to downtime during component failures.
• Scalability: They may have limitations in scalability compared to higher tier data centers, potentially requiring more significant upgrades for expansion.

When a business is considering colocation or hosting in a Tier 3 Data Center, it’s important to assess its power requirements carefully. For operations with high power demands, it might be necessary to discuss with the data center provider the possibilities of allocating additional power resources or choosing a high-density zone within the data center. Data center providers typically measure power in kilowatts (kW) per rack and will have different pricing models based on the power and cooling requirements of the equipment hosted.

🔗Tier 4

A Tier 4 Data Center is the most advanced type of data center tier, as classified by the Uptime Institute. The Tier 4 Data Center is the most robust and less prone to failures. Its design ensures redundancy and reliability.

🔗Key Characteristics

• Availability: Tier 4 Data Centers offer the highest level of availability, typically guaranteeing 99.995% uptime or just around 26.3 minutes of downtime per year.
• Fault Tolerance: They feature a fault-tolerant design with 2N+1 redundancy for critical systems, ensuring uninterrupted operation even during component failures.
• Redundancy: Tier 4 Data Centers have redundant power and cooling distribution paths, eliminating single points of failure.
• Security: They often have advanced security measures in place, including biometric access controls and surveillance systems.

🔗Key Challenges

• Complexity: Tier 4 Data Centers are more complex and expensive to design, build, and maintain due to their fault-tolerant architecture.
• High Cost: The increased redundancy and fault tolerance come with higher costs, making Tier 4 Data Centers more expensive to operate compared to lower tiers.
• Overkill for Some Applications: For organizations with less critical operations or smaller budgets, Tier 4 Data Centers may be overkill and not cost-effective.

🔗Choosing Between Tier 3 and Tier 4

• Cost vs. Need: Tier 4 Data Centers are significantly more expensive to build and operate. Organizations should assess whether they need the highest level of uptime and fault tolerance that Tier 4 provides.
• Business Requirements: Businesses that can afford occasional downtimes for maintenance or those that don’t have extremely critical data operations may find Tier 3 facilities adequately sufficient.
• Industry Compliance: Certain industries may require or favor Tier 4 facilities due to regulatory requirements.

In summary, while both Tier 3 and Tier 4 Data Centers are designed to ensure high levels of uptime and reliability, Tier 4 takes this a step further with increased redundancy, fault tolerance, and a correspondingly higher price point.

🔗Introducing the Tier S Data Center

The Tier S model is designed to exceed the capabilities of traditional Tier 3 and 4 Data Centers, addressing the increasing demand for infrastructure that can support emerging IT workloads with greater efficiency and resilience. Where “S” stands for “Self-Healing,” Tier S Data Centers present a new way forward, while remaining backwards compatible.

The Tier S “circular approach” allows us to build the most reliable and redundant data centers in the world. A larger data center (typically 2-10 mW capacity) is extended with at least nine edge data centers. Key features include advanced liquid cooling technology and cybersecurity mechanisms, enhancing the center’s ability to handle complex IT workloads like AI and ML.

Flexibility is central to the design, with each edge data center capable of housing 3-30 pods (subsections), and a versatile main data center. The innovative self-healing technology ensures continuous, secure operations, reducing downtime and maintenance needs. The infrastructure, integrating state-of-the-art hydro coolant distribution units, is designed for high-performance computing, meeting emerging technological demands.

🔗Key Benefits

• Superior Reliability: Data storage is highly secure and can’t be lost, would even be safe if quantum computing would be used for hacking.
• 100% Redundancy: Fully redundant power and network paths. All hardware is equipped with liquid cooling for enhanced reliability and energy-efficiency..
• Capacity and Expansion: A main data center with a 2-10 MW capacity, supplemented by at least 9 edge data centers.
• Flexible Configuration: Each edge data center can include 3 to 30 pods (more information below), with the main data centers configurable as pod, rack, or container-based.
• Disaster Resilience: The circular design, with a main data center and edge data centers, offers robust protection against natural disasters, war, or unrest.
• Self-Healing: Equipped with self-healing properties, Tier S Data Centers ensure that workloads can operate safely with minimal need for human intervention.

🔗In Conclusion

The existing data center model is simply insufficient for emerging demands of the IT world. Billions of dollars will be mis-spent if we continue down the existing path. This is why ThreeFold is presenting a new approach. While the Tier S Data Center is still in its conceptual phase, we are actively working with partners on implementing the first of its kind in the near future.

🔗Technical Redundancy

Our quantum safe storage ystem, coupled with replication, and web gateways, presents a pathway to achieve technical redundancy already. While these capabilities exist, we admit that at this time it is too complex for a starting user. Making this easier through refined tooling and a strong user experience will be imperative in order to allow a wide audience to take advantage here.

🔗ThreeFold Cloud

Our upcoming commercial offering, TF Cloud, will bolster reliability and trustworthiness. Through TF Cloud, we will need to provide comprehensive service, robust support, service level agreements (SLAs), and streamlined payment methods, including credit card options. The development of user-friendly tooling will significantly simplify and easen the process as well, enabling users to deploy and maintain reliable workloads effortlessly.

🔗Farming Pools

The introduction of farming pools in TF Grid 4.x will play an important role in fostering reliability as well. These pools not only contribute to resource pooling but also offer visibility into the network’s participants. This transparency enhances trust and aids in identifying reliable contributors within a decentralized framework.

🔗Establishing Reputation and Transparency

Building a system that emphasizes reputation and complete visibility regarding service levels and uptimes is pivotal. By offering insights into each participant’s performance metrics, Grid users will be able to more effectively choose a trusted provider.

In essence, through a combination of technical and strategic advancements, our journey towards a dependable and trustworthy decentralized Internet infrastructure is underway, setting the stage for the next phase of ThreeFold.

🔗Breaking the Distance Barrier

In many regions, the internet economy remains anchored in distant lands due to reliance on long cables to Europe, North America, or China, where colossal data centers reside. This scenario keeps data, jobs, and economic benefits outside developing regions. But what if connectivity was local, decentralized, and efficient?

🔗The East Africa Example

Picture our technology in action. Instead of relying on data centers thousands of miles away, our decentralized infrastructure brings the connection closer. Those friends in East Africa can communicate peer-to-peer, reducing bandwidth needs by up to 10 times while boosting security. This isn’t just about video calls; it’s about revolutionizing every digital interaction.

🔗Streamlining Complexity

The tech world often gets tangled in layers of abstraction, favoring integration over innovation. While layers aid compatibility and incremental progress, they hinder true change. Excessive abstraction results in inefficiencies, performance lags, increased costs, and scalability woes.

🔗The Education Gap

Innovation takes a back seat when education leans heavily on commercial technologies. The push for quick-fix solutions neglects long-term innovation. It’s like opting for a painkiller instead of addressing the root cause of the problem.

🔗Local, Secure, Reliable

Our vision? A decentralized internet where compute, storage, and network are local, sovereign, and secure. Storage becomes more reliable with less overhead, doubling up as a global content delivery network. The network stays resilient, routes around issues, encrypts end-to-end, and knows precisely where data originates and ends.

🔗Tackling the Root Issues

By constructing a decentralized, autonomous internet infrastructure from scratch, we’re tackling these issues head-on. Minimizing complexity while solving problems at their core is our mantra. It’s not just about a tech revolution; it’s about reshaping connectivity to be inclusive, efficient, and empowering for everyone, everywhere.

The digital world stands at a crossroads. We can either perpetuate the status quo, or we can pave a new path—one that’s decentralized, efficient, and equitable. The time for a decentralized internet that solves problems at their roots is now. It’s time for connectivity that knows no boundaries.

The power to transform connectivity is in our hands. Let’s reimagine the internet not as a distant entity but as a network that thrives locally, serves globally, and empowers universally. Together, let’s build a decentralized internet that propels us into a future where connectivity is a right, not a privilege.

At ThreeFold, we believe that everything we do should benefit people and our planet. The emergence of a decentralized Internet grid not only reshapes the digital landscape but helps to pioneer a more sustainable future. At the core of this evolution lies a series of innovative strategies poised to revolutionize the way we perceive and utilize digital infrastructure.

🔗Zero OS

Traditionally, operating systems have been laden with an array of features, consuming substantial hardware resources. However, with Zero OS, our paradigm shifts towards a lightweight, autonomous design. By getting rid of unnecessary complexities, this design not only optimizes hardware space but also fosters an environment conducive to enhanced IT workloads.

🔗Quantum-Safe Storage

One of the most staggering inefficiencies within the industry has been the substantial overhead of storage systems. Our quantum-safe storage system marks a monumental departure from the norm, slashing the overhead from a staggering 400% to a mere fraction, 20%. This unprecedented leap in efficiency heralds a new era of streamlined storage practices, ensuring optimal utilization of resources.

🔗Efficient Node Management

Energy conservation takes center stage with our node management tool, Farmerbot. Embracing the principle of ‘on-demand’ functionality, nodes remain dormant when idle, conserving energy resources. Through the implementation of Farmerbot, these nodes seamlessly power on and off as required, minimizing unnecessary consumption without compromising accessibility.

🔗Automated Autonomy

We’re saying goodbye to human intermediaries and centralized control centers. Our system operates autonomously, governed by smart contracts that orchestrate every facet of its functionality. This automated ecosystem not only eradicates the need for constant human intervention but also ensures streamlined operations, optimizing efficiency at every turn.

🔗Peer-to-Peer Advancement

The conventional route from sender to recipient often travels extensive distances, bouncing through data centers and intermediaries. Our peer-to-peer network revolutionizes this process by forging direct connections, seeking the shortest path between sender and recipient. By circumventing unnecessary detours, this approach minimizes energy expenditure and data latency, elevating both speed and efficiency.

🔗Final Thoughts

The dawn of ThreeFold’s decentralized Internet grid beckons a transformative era in digital infrastructure. Through the combination of a lightweight operating system, quantum-safe storage, energy-conscious node management, automated autonomy, and a peer-to-peer network, the paradigm of efficiency in the digital realm has been redefined. The aggregated result is up-to 90% energy savings depending on workloads. This evolution not only champions sustainability but also propels us towards a future where connectivity thrives in harmony with resource conservation.

🔗Yggdrasil’s Limitations

To start, it should be noted that Yggdrasil is an experimental routing protocol (based on DHT’s), which is currently still in an alpha stage. We very much admire the fact that they have been able to create a standalone network with worldwide participants from scratch. Over time, the network has grown, in part due to the grid, but also because a lot of individual participants joined and started running a node.

After investigating, we reached the conclusion that the current architecture has some scalability issues. While we have no doubt that these will eventually be solved, the importance of the planetary network in the grid is unfortunately too big to be able to wait for this. Additionally, Yggdrasil uses a technique called crypto key routing, and uses a tree structure for its routing info. This means data follows the “shortest path” in the tree, rather than the physical shortest path, which might deviate from this. As such, we have decided to implement our own Ipv6 overlay network, Mycelium.

🔗Implementing Mycelium

As previously pointed out, the main goal of our network is to be efficient, and end-to-end encrypted. Every node in the network is identified by a key pair. The address of a node is derived from this key pair, and data traffic between 2 nodes is encrypted using a secret derived from the key pair of both these nodes. As a result, every 2 nodes in the system use a different secret for encryption, and we can verify that the address used by a node is indeed owned by it (i.e. it has the corresponding private key). We have based our initial implementation (loosely) on the babel routing protocol.

Therefore, sending data to a remote node will follow the shortest path in terms of latency (i.e. the fastest route). The network adjusts itself in case nodes disappear or join, and multiple paths to a given destination are possible.

We are also considering implementing more advanced use cases in the future, though this remains to be seen at this stage. We are gearing up for some larger scale testing on the grid in the near future (after all, we do have multiple thousand nodes at our disposal to verify scalability), and will adjust as needed once these are done.