Today’s cloud storage has become a significant problem. For companies like video streaming and distribution platforms, the cloud is holding back their growth. Yet their growth is causing problems for the earth. At the heart of this issue is the fundamental truth that the current model for cloud storage is not working effectively for video and is creating high costs for our environment.
How bad is it?
Today’s cloud storage model is built around data centers. As demand grows, cloud providers build more data centers. Data centers are expected to account for more than 3% of global carbon output by 2025. That doesn’t sound too bad. But by 2040, storing digital data is estimated to create 14 percent of the world’s emissions, around the same proportion as all carbon the US emits today.
Video streaming demand is going to continue its growth trajectory, so our only option is to consider greener and higher performing alternatives to the way we do cloud storage today. In this article, I aim to provide evidence on why today’s cloud storage model is not viable for the continued growth of video production and distribution and introduce a new model that creates a sustainable future for video storage.
Rapid growth in media size and volume is adding to the problem
We all are enjoying crisper viewing experiences thanks to continued improvements in video capture and display resolutions. Of course, as video resolution keeps going up, so do the file sizes, which means more cloud storage. For instance, if you compare a video shot in full HD to one using 4K resolution, the file size jumps by a factor of 173. That same video in 8K is 277 times larger than the full HD file.
On top of video files getting larger, more video is moving to the cloud out of necessity for video production and distribution. The recent M&E digital storage report from Coughlin Associates provides a visual illustration of the incredible growth projection of cloud storage for media assets with backups, post production, and video on demand (VOD) all significant contributors to the cloud storage growth. In fact, the report predicts that video storage in the cloud will grow from around $1.25B in 2022 to $3.25B in 2026. That is more than doubling the demand on cloud storage in just 4 years.
MovieLabs, a non-profit research and development venture founded by the major Hollywood studios, has a vision for video production through to distribution to be 100% in the cloud by 2030. The report states, “For this principle to be realized, the industry needs hardware, software, and cloud vendors to work together to design cloud-integrated systems that can securely create, encrypt, validate and store captured assets in cloud object storage over advanced data communications networks.” This requires taking a critical look at existing cloud storage and considering what needs to change to make this happen. Because media creators and distributors can’t sacrifice their productivity, the security of their video, or their budgets in order to get to 100% cloud production. Let alone the PR hit Hollywood will take when it comes out that their videos are a significant part of the increase in carbon emissions.
Creators and distributors are feeling the “gotchas” of cloud storage
The problems with today’s cloud storage isn’t just about the earth. At a basic level, cloud storage was not built to transfer large files quickly to anywhere in the world. Netflix realized this early on and built their own distributed cloud network as large providers like AWS, Microsoft and Google would not be able to give them the performance and accessibility they needed.
The problem can be explained simply with a ‘pick two dilemma’. The three major factors of cloud storage are cost, performance, and security. With today’s cloud storage, customers can only get two out of three of those things in their favor. So, take the example of a video studio that needs to upload large video files every night from their California location for their counterparts in Asia to download and keep editing going 24x7. Except the upload takes 4 hours and the download takes another 4 hours—essentially killing 8 hours of possible productivity daily.
Similarly, video distributors want to move into new, underserved markets, but they are underserved because they are located far from datacenters and streaming performance is lacking. And the prohibitive cost of replicating their content into datacenters to serve additional regions is further limiting their growth. For those platforms looking to create ad supported video on demand, streaming performance is a key factor in the revenue model. This means that video platforms have to pay premium prices for fast replay and often have to keep a narrow market focus due to this high cost.
Fundamentally, the video industry is feeling the limitations of today’s cloud storage.
The only sustainable way forward is distributed storage
If the video industry wants to capitalize on the growth opportunity without doubling global carbon emissions every 5 years, it needs to consider alternatives like distributed cloud storage. Instead of creating honeypots of data stored in isolated datacenters, distributed cloud storage spreads data globally across different hardware operated by third parties. Sounds less secure, right? Actually, the opposite is true. Distributed cloud storage had to be built with zero-trust principles so it is fully end-to-end encrypted and erasure coding is used to break up the files into around 80 segments where only a third of those segments are needed to reconstitute the file. All this makes it the most secure cloud storage available.
Relative to performance and accessibility, distributed cloud storage is built for large data files using parallelism and optimizing long tail latency to ensure fast transfer. And replication is not needed for accessing files globally since the system pulls the file segments needed from the closest regional storage nodes for the faster performance to any location.
Of course, because the storage in the network comes from existing unused capacity, this makes distributed cloud storage much less expensive and better for the environment than cloud storage using data centers. With today’s cloud storage the video industry is paying extra for multi-region replication and sometimes extra hidden fees for egress. Together, it makes for monthly storage cost reduction of 80%-90%. And perhaps more importantly, it eliminates the hardware churn and energy cost of data centers. Because what company wants to be associated with increasing carbon emissions?
Distributed cloud storage is the only way forward for the video industry so they can capitalize on growth without significantly damaging the environment.
