Sharing Space for Fun and Profit—Part 2

It’s been over two years since the first version of Sharing Storage Space for Fun and Profit was published and we covered the details on the incentive model for Storage Nodes. We’ve been operating the Storj DCS network for over two years now. In that time, we’ve grown the capacity of the network from 0 to over 30 PB. While the initial growth was fielded by Storage Nodes transitioning from our second generation network to the current network, we’ve seen steady growth to over 13,000 Storage Nodes operating today in more than 90 countries. Our current network is very stable, with only a small percentage of Nodes failing or leaving the network every month. In this blog, we’re going to discuss what’s worked about the network incentive model, and how it’s likely to evolve over time.

But first, a bit of context‍

Storj makes open-source software that anyone can run—individuals with a Network-Attached Storage Device (NAS), those with desktop computers that are always on, businesses, or data centers. This allows these devices to share unused disk drive space and bandwidth with the network. Our software aggregates all of that storage capacity and bandwidth to create an extremely secure, private, and durable cloud object storage service. Storj makes that storage and bandwidth available as a distributed cloud object storage service for developers, with an enterprise-grade 99.95% availability, eleven 9s of durability, and S3 compatibility under the Storj DCS brand. 

The devices that share storage space and bandwidth with the network are called Storage Nodes. The over 13,000 participating Storage Nodes are run by third parties who are paid in STORJ token for the actual use of the storage capacity and bandwidth provided to the network.  

Can you trust 13,000 Nodes to do the right thing?

Before I talk about the specifics of the Storj incentive model, it’s important to understand how these models work in decentralized systems in general. Relevant to this discussion, decentralized systems have two constructs that work in concert:

• Incentive model - A system of rewards and penalties that motivates Node Operators to provide reliable resources to the network that are sufficient for the network to provide a service to users who will consume those services in exchange for value
• Consensus model - A system that enables all participants in the network to determine that the services provided and value exchanged are actually being provided including trustless validation via technical enforcement points
  

You can read about the peer classes in the Storj network in our white paper or learn more in our documentation. 

A few words on the consensus model

I want to touch briefly on the consensus model for the Storj network because it is atypical of decentralized systems. Many decentralized systems use a blockchain for their core function and a distributed ledger for the consensus mechanism. Essentially all Nodes store all of a sufficient subset of transactions and participate in confirming transactions. This approach is great for a range of use cases from cryptocurrency to DeFi services, but distributed ledgers require a significant amount of coordination which also means a significant amount of latency, two things that are to be avoided at all costs in data storage services.

Instead of a distributed ledger, we use a number of services that work together, using many of the primitives from blockchain for an efficient system that is based on statistical sampling, audits, and cryptographically verifiable transactions. Ultimately, this set of services is designed to ensure that all peer classes are doing what they claim to be doing. For Storage Nodes, that means they are online almost all the time (99.5% uptime), actually storing the encrypted and erasure-coded pieces of data uploaded to them, allowing applications to retrieve those pieces on demand, responding to audits and uptime checks, and they are reporting accurate resource utilization data. You can read more about this in the white paper or on the blog, but the key takeaway is that there is a system of checks and balances in place designed for coordination avoidance and low latency that allows us to ensure data durability and meet (and often exceed) our SLA.

The Byzantine, Altruistic & Rational (BAR) model

The BAR model is a useful rubric to describe the participants in a decentralized system. The BAR model describes three categories of network participants:

• Byzantine - Nodes will actively try to disrupt the network
• Altruistic - Nodes that will behave in the way most healthy for the network, regardless of the outcome or reward
• Rational - Nodes that will optimize behavior for the highest possible reward

In this context, a goal of the consensus model is to identify and mitigate byzantine behavior. The primary objective of the incentive model is to make the rational behavior mirror the altruistic behavior. 

It’s a delicate balance

The big idea is that if you get your incentive and consensus models working correctly, the decentralized network and service will work well and perform well, but:

• If your consensus model is too strict - Performance will suffer and it will be more expensive to join and participate in the network
• If your consensus model is not strict enough - The network will be susceptible to byzantine attacks, rational Nodes will be able to exploit your incentive structure and quality of service will suffer
• If your incentives are misaligned - The behavior that earns the highest reward for rational Nodes will cause the quality and reliability of the service to be poor regardless of your consensus model
  

The consensus model is essential, but, even if it’s perfect, rational Nodes will burn the network to the ground if the incentive structure is wrong. 

The ideal, altruistic Node is online almost all the time, actually storing the encrypted and erasure-coded pieces of data uploaded to it, allowing applications to retrieve those pieces on demand, responding to audits and uptime checks, is reporting accurate resource utilization data, stays on the network for a long time, and, if the Node decides to leave the network, it does so in a way that is healthy for the network. Here’s how our incentive model achieves that result. 

The incentive model

We pay Storage Nodes to participate in the network with STORJ token, an ERC20 compatible utility token for conveying value for services provided on the network. Node Operators can use the STORJ token they receive to purchase storage space on the network or convert to other tokens or fiat currency using any one of the available cryptocurrency exchanges supporting the token. Developers can also use the STORJ token to purchase storage space and bandwidth on the network via the Storj DCS service, at a discount. Node Operators are paid for:

• Storage - Storage Nodes are paid for storing data on the hard drive space they share with the network. Payments are made in STORJ tokens at $1.50 per TB. Storage payments are calculated based on GB-hours per month, inclusive of the expansion factor resulting from erasure coding the data. So, if you share 4 TB of data on the network, and 2 TBs are utilized by the network for an entire month, that would earn you $3 in STORJ tokens.
• Egress bandwidth - Storage Nodes are paid for the egress bandwidth used when customer applications download data from the Storage Nodes. Payments will be made to Node Operators at $20 per TB. These bandwidth payments are for all data downloaded by those applications as part of the normal retrieval of uploaded data.
• Audit & repair bandwidth - Storage Nodes are paid for the egress bandwidth used when Satellites download pieces of files from Storage Nodes as part of the repair process. Payments will be made to Node Operators at $10 per TB. These bandwidth payments are for all data downloaded by Satellites as part of the audit process to confirm data stored on the Nodes is retrievable and to repair data that is lost due to Node churn. 
  

Node vetting & Held Amount

The rates described above are paid on a monthly basis for established Nodes. When new Nodes join the network however, there are two parallel processes that run- Node vetting and held amount - that do impact initial earnings.  

It’s important to note that Node churn is bad for decentralized networks and that’s true for Storj as well. As Nodes leave the network and make the pieces of object segments they were storing unavailable to the network, some segments may need to be repaired. Repair is a resource-intensive process that incurs cost for the network. We have implemented Node vetting and Held Amount to reduce the amount of data repair related to Node churn.

Node vetting is designed for early identification of Nodes that are likely to churn shortly after joining the network. Immediately after joining the network, Nodes receive a reduced amount of data and an increased amount of audits and uptime checks to establish that the Node is stable and well run. Node vetting identifies Nodes that are likely to fail early before beginning to add a significant amount of data to the Node. Node vetting typically is completed in just the first few weeks of running a Node.

Held amount is a complementary component of the incentive structure that works in conjunction with the Graceful Exit feature. Nodes don’t need to provide any up-front stake to start earning STORJ tokens as a Node Operator. Rather, during the first nine months of operation, a percentage of earnings are placed in a holding account. These funds are held until a Node Operator chooses to leave the network. After 15 months, a portion of the balance is returned, while the remainder is held indefinitely until the Node leaves the network via Graceful Exit.

If the Node Operator uses the Graceful Exit function (described more fully below) when leaving the network, the funds will be returned to them after the exit is complete. If the Node Operator exits the network abruptly without completing the Graceful Exit, the funds will be forfeited to offset the cost of data repair caused by the Storage Node exit.

The Held Amount function is structured with a tiered reduction in held amount as the amount of time the Node is on the network increases. The held amount model is as follows:

• Months 1-3: 75% of revenue is withheld, 25% is paid to the Node Operator
• Months 4-6: 50% of revenue is withheld, 50% is paid to the Node Operator
• Months 7-9: 25% of revenue is withheld, 75% is paid to the Node Operator
• Months 10-15: 100% of Storage Node revenue is paid to the Node Operator
• After Month 15: 50% of total withholdings are returned, with the remaining 50% held until the Node gracefully exits the network
  

The held amount model is designed to incentivize and reward both-long term reliable Storage Nodes as well as Nodes that, when they do choose to leave the network, exit in a way that is least damaging to the network. 

Graceful Exit

If a Node Operator unexpectedly shuts down their Node when they want to leave the network, the unavailability of the data on the network will contribute to repair costs. Graceful Exit is a function in which a Node Operator who wants to exit the network can trigger a process to return the data the Storage Node is holding to the network, instead of exiting the network and deleting the data stored by the Node.

The sequence of events following the triggering of the Graceful Exit function is as follows:

1. Graceful Exit is triggered by the Node Operator and the Node contacts any Satellites for which it is storing data
2. Each Satellite returns a list of new Storage Nodes to which the Node should upload the pieces of data stored on the Storage Node
3. The exiting Storage Node uploads the pieces to the appropriate Nodes and updates the appropriate Satellites when complete
4. Satellites disqualify the Node ID associated with the gracefully exited Storage Node in order to signal that the business relationship between the Node associated with this Node ID and the Satellite has concluded
5. Satellites pay out the balance of STORJ withheld during the next pay cycle
6. Graceful Exit is designed for Nodes that intend to permanently exit the network
   

At this point, we’ve covered an overview of our consensus model and technical enforcement points as well as the incentive model. Ultimately, Storage Nodes will maximize potential earnings through consistent, reliable, long-term operation.

So, get to the point already—how much can Node Operators earn?‍

This is everybody’s favorite part of the blog post, but I think the above context is both interesting and important. Hopefully you read all that and found it interesting. In the first edition of this blog post, all the way back in 2019, we provided an online calculator and provided a lot of detail on what Node earnings could be based on different scenarios. The community provided feedback that they politely declined to use the calculator and produced one of their own. 

So, rather than go down that same path, I’d encourage you to join the community and get the perspective of long term, successful Node Operators. Something I can provide that might also help is that since the launch of the Storj network back in the beginning of 2017, Node Operators have been paid nearly $3.5 million dollars worth of STORJ tokens for providing storage and bandwidth.

Running a Storage Node requires some technical ability and knowledge of port forwarding and either Docker or Linux command line. We have individual prosumers, businesses and data centers running Storage Nodes. The most important success factors for Storage Nodes are:

• Bandwidth - While it might seem counter-intuitive, the amount of bandwidth you have available and whether you have a bandwidth cap is the most important factor in operating a Storage Node.
• Uptime - Your Node has to be online to receive data and respond to requests to download pieces, audits and uptime checks. NAS or SAN devices, servers, dedicated workstations, or even a Raspberry Pi with an HDD attached all make good Storage Nodes. Laptops or home computers that get turned off frequently will be quickly disqualified.
• Space - You should have between 1-16 TB of space available per Storage Node. The network is optimized for a large number of heterogeneous Storage Nodes as opposed to a few giant Nodes. We recommend that you repurpose old hardware vs buying new and generally try to stick to a model of one processor core and 1 disk per Node. Complex configurations like RAID or mirrors are unnecessary as the network is self-healing and handles redundancy within the protocol. 
  

Recent changes and the future of payments

Since we launched the current generation of the network, we’ve made very few changes to the incentive design and consensus model/technical enforcements. The most significant changes have been:

• Suspension and containment modes - We introduced suspension and containment modes to reduce the probability that an otherwise healthy Storage Node might be inadvertently disqualified by providing Nodes with the opportunity to recover from minor interruptions in service.
• Minimum payouts - With the spike in ETH transaction fees on Layer 1 (L1), we introduced a minimum payout threshold as a way to reduce the cost impact of processing Storage Node payments. Storage Nodes with payouts disproportionately low compared to transaction fees are rolled into the next month's payment until they reach the threshold.
• L2 payout solution - As an additional alternative to minimum payouts on L1, we offered the options to use the zkSync Rollup from Matter Labs with no minimum for monthly payouts. We expect to continue to invest development resources in L2-based payouts in the future.
  

With the early success that we’ve had with Layer 2, we will begin the transition from offering an L2 solution as an option for storage node payouts to making it the default. The combination of the flexibility, the speed at which we can complete payouts, the reduction in ETH transaction fees and the ability to avoid the high minimum payout associated with ETH transactions fees make this the best option for the short and medium term.

As the Ethereum network continues to innovate with broader adoption of L2 solutions, with the shift to proof of stake and the upcoming change to gas fees, the scalability of the Ethereum ecosystem will continue to become more efficient at the same time that our network continues to scale. In the future, we also anticipate increasing support for new DeFi services and models to enable new mechanisms to fund storage and bandwidth utilization related to emerging use cases such as perpetual storage of NFTs.

Sustainability & pricing

Storj DCS is a new and disruptive alternative to established centralized cloud storage providers and we are aggressively working to drive use and adoption of the service. We’ve introduced new, disruptive pricing and will continue to test different pricing models as we identify new use cases. As we grow our market and test new pricing and packaging, it’s extremely important that we continue to maintain the existing network of loyal and reliable Node Operators and grow the network with new Nodes. For that reason, we are maintaining the status quo on payouts to ensure that it remains economically viable and rewarding to be a Node Operator.

As we see increasing adoption of the service, especially with higher bandwidth use cases, and we see usage patterns that support a long-term, sustainable model for the network, we may make additional changes to the incentive design. Ultimately, we need a healthy network where the operation of any peer class is financially viable and rewarding. 

What might that look like? Today we use the Held Amount to offset the cost of repair related to Node churn, but we expect to restructure the Held Amount as a bonus for long term, healthy Storage Nodes. Today we pay a premium for bandwidth to incentivize Storage Node participation in the network, but as we see use cases with higher bandwidth utilization, we may reduce the premium with the expectation that Storage Nodes will earn more based on volume. Our current pricing incentive is designed for that type of use case.

We’ll continue to be open and transparent with our community of Storage Nodes and solicit input and feedback from them as we look to future changes.

Viability of the network in the future

The Storj network represents a disruptive approach to the traditional way of providing cloud object storage. In the current paradigm, only a small handful of hyperscalers have the resources to compete in that market. Our approach allows anyone to be the cloud and participate in that market. We’re at the very beginning of our journey and while the new S3-compatible gateway and disruptive pricing have been well received and are driving increased adoption, we have a long way to go! We are committed to ensuring the long term success of the network, with a low cost of entry to operate different peer classes in the decentralized network leveraging existing low cost and sustainable infrastructure.

If you're interested in becoming a Node Operator, sign up today to get started.