I recently found the peertube and its technology is very nice. You could deliver video using p2p torrent like streaming if there are more than 1 user watching it. For me it looks like the hosting provider gets a huge savings in terms of bandwidth. But I don't see many companies making use of this technology.
I think Bitchute is using p2p to deliver videos, or is it, really?
Good summary here:
https://techcrunch.com/2014/04/17/spotify-removes-peer-to-pe...
"This is when the company took advantage of cached songs. Before today’s change, if you streamed a popular song for the first time, the client would download the song from other users, using peer-to-peer. All of this happened in the background, but it greatly contributed to making the overall user experience snappier."
"Yet, now that the company has many servers, using peer-to-peer in addition to direct downloads actually adds a bit of overhead. Moreover, the company has to maintain the peer-to-peer code base, and update it with each new version."
Really, that's (part of) their excuse? A bit lame, that. :|
Time to first byte is pretty good, https://webtorrent.io/
To the OP, you should research some other options, are you sure that you are asking the right question?
So a FHD stream for a video could be super popular while a 480p version might not have any viewers. When some 480p viewers load the stream they don't get any benefit from the P2P/cache popularity of the FHD stream.
This is compounded by the fact that for video time to first byte doesn't matter very much. A decoder can't do anything with a single byte or even a partial stream segment. It's the time to locally buffer a decodable stream segment and a steady reception of segments as the play head reaches their time stamp.
If a segment gets held up (as happens with P2P) it can't be decoded so you either pause playback or drop frames (if you've got later segments available). You want the media segments to cover at minimum a single GOP (Group of Pictures) which is a relatively large I-frame and the following smaller P and B frames. If you were to tune the video segmentation such that each segment was some tiny time slice it would balloon the bitrate since you need way more I-frames. Super long GOPs don't help either because segment drops end up ruining playback until a new I-frame is received.
Streaming video is hard enough with a reliable CDN, adding P2P makes much harder and orders of magnitude less reliable.
No, it'll be genuinely be worse, as you can't really have efficient P2P today.
Let's start with the points; I'm not talking about Europe or US today. P2P or CDN is a coin flip there, and I'll agree that P2P is as reliable as a CDN. I'll also agree that theoretically P2P and CDN will be a coin flip (performance-wise, I'm leaving monetary considerations in this discussion).
Those points break down quickly in Latin America, Asia and Africa. I'll talk about Asia as it is where I'm more familiar of, but the points apply also in Africa and Latin America. Outside of Jio and actual hosters/colos (meaning nearly all of residential and mobile connections), IPv6 in Asia is actually more unreliable than you think (it should not be, but that's peering fights for you) and IPv4 is CGNATTed heavily so P2P is no dice. Even if somehow there's a unique IPv4 for every device, internet connections there is very top-down, unlike with Europe's mesh-like connectivity.
How bad? It'll be better to route Telkom Indonesia's connections to America than to another hoster located in Indonesia or Singapore (nearest regional hub), unless you're Akamai which have agreed to buy colocation space inside Telin just to have good connectivity. And before you comment, OVH in Singapore has the precise issues we're discussing (https://lowendtalk.com/discussion/172659/ovh-routing-issues-...). Thus, it is miles better to use your time to build good CDNs with good connections (or spend money to Akamai) than bothering with the issues you'll face with P2P. Moreover, you can concentrate your money into buying a better data link (either by renting/IRU a dedicated wavelength or even fiber pairs) than relying on a best-effort service.
Funnily, even bittorent downloads in practice seems to be concentrated to a few seedboxens - even for purely legal downloads like GIMP.
Postscript: technically P2P and CDN would be the best for reliability, but you're spending the time to do it (instead dealing with a single thing) and deal with the possible fallout (like Windows Updates before Microsoft decided to limit P2P to local networks due to bandwidth and privacy concerns). Speaking of bandwidth concerns, I promised myself to limit monetary concerns just to point out that's it's not good in actual deployment, but I forgot data caps (not just a problem in Africa but in the US too)! It'll be a disaster when P2P is not voluntary because you just wasted their precious money!
I always wondered why not combine the ways the media is served into a hybrid of P2P with the classic way by just using P2P (e.g. BitTorrent) and also seeding yourself (as the hoster). Just run a seedbox of the capacity you want/can and let the audience help you by handling a share of the load they can. Wouldn't this save you some bandwidth while keeping the same service quality?
So why should we help these for-profit corporations? We're supposed to just donate our bandwidth to them? Are we getting a discount in exchange or are they just gonna pocket their bandwidth cost savings at our expense?
Seeding internet archive data is one thing. Letting corporations invisibly take advantage of resources we pay for is a completely different matter.
Frankly, I never saw any other seeds on Archive's torrents—but one of those I snatched has a ratio of 3 now, so presumably it does work as intended.
Maybe bandwidth and storage caps would work better. You seed X MB, even if you don't need the full X yourself.
I wonder if this might be improved by an appointment system, to stream popular titles at pre-planned times so that people could benefit from others watching at the same time.
It would be like watching a broadcast television program during prime time.
I guess this has been tried before but also never heard of it
Unfortunately P2P projects always try to be replacements for centralized rather than supplements, and almost none of them aside from BitTorrent are what I'd call great.
Live streaming p2p is the easy one. Live streaming content is very skewed: top 1% streams can cover majority of the bandwidth, the top room can easily have tens of thousands of users, so p2p helps a lot.
Video-on-demand is harder. But now there's also many "seed boxes" out in the market: it's basically a custom home router with a big disk, users buy it and put it in their home as a regular wireless router, but in the background it would automatically connect to server, cache videos and serve the videos to other peers. The user may get some bonus from it (mostly digital points). Essentially, these companies are buying users' home internet as CDN edge nodes.
But in either way, P2P is used to save some bandwidth (cost), but the performance would almost always be worse. There would always be traditional CDN as a fallback.
Some possible reasons why it's more popular in china: 1. there's lots of people here; 2. users don't care about their privacy much
(I worked on this area in one of the largest video hosting company in china)
It drives me absolutely up the wall that some torrent networks insist that you have a 1-1 ratio because it is close to impossible for me and there is no point because often the network has so many seeds…. The fact that I can complete a transfer uploading just 5% of what I download proves my upload isn’t needed.
IIRC the total usable bandwidth is about 10Tbps
Case Study: Bit Torrent [0]
Bit torrent is a brilliant idea - allows everyone who has part(s) of a file to contribute to the pool of availability so that any given central server/mirror isn't overwhelmed
And it has its place (eg in file sharing)
But for streaming? Not so much
Say you're getting the "next chunk" (whatever 'chunksize' is in this context) from me, and I go offline (it's the end of my day, need to reboot for updates, any of myriad reasons). Where does the next bit of the video come from in a way that is seamless to the viewer?
That is the fundamental problem of shared/p2p streaming protocols - everytime the host of the current/next blob o' data goes offline, you need to waste time finding a replacement
Even if the replacement can be found "quickly", how do you ensure they don't go offline in the middle of streaming? How do you ensure "enough" copies of every chunk are distributed that it, effectively, 'doesn't matter' how many go offline [at once], it will still stream?
--------------
[0] https://en.wikipedia.org/wiki/BitTorrent
You still have a good chance of buffer underruns when you start a video, but that would likely get mitigated by the fact that the data for the start of the video would likely end up highly duplicated on the network.
The bigger issue is seeking. Jumping to the middle of a video could take many seconds before it played. The UX would be awful.
That sounds horrible - now you've got to have something buffering [potentially] 100s of MBs (depending on quality) over lousy residential upload speeds?
Ther is an algorithm in place where the rarest is downloaded. Also a priority list of older peers is implemented. Some protocol also request the chunk from muliple peers (kademlia).
You don't need to _ensure_ all that stuff, in the strictest sense of the word. Network hiccups happen, nodes go down. As long as the frequency of interruptions scales inversely with the number of nodes in the network, once you're on a big enough network everything will work smoothly.
That being said, most residential internet packages have low upload speeds, which is an issue
It all makes for a shitty user experience.
Essentially no one wants to host and run the infrastructure for these things, only the "diehards" do. This is akin to how only the "diehards" produce content on Wikipedia, or YouTube. Remember, this is relative to the volume of people who only consume that content.
It's the same in P2P e.g. Bittorrent. The count of seeders (those who have the content and are uploading it to peers who need to download it) is almost always lower than leechers (those who are only downloading).
This is exacerbated with Peertube because video files are big, require a lot of bandwidth, and someone has to pay for that. If there is a set of diehards who love running Matrix or IRC (think other smaller networks not just Libera) or what have you then how many of that already small set can meaningfully afford to run a Peertube instance capable of serving a momentus volume of video traffic?
I wish it wasn't so but it is.
[1]: https://en.wikipedia.org/wiki/1%25_rule_(Internet_culture)
That's (at least partially) solved with incentives. Plenty of private torrent sites require you to keep a good ratio of download/upload to use them. As a kid I was definitely not a die-hard but still seeded to have access.
Free-of-charge video might seem like a better fit, but that market was totally cornered by Google and YouTube; who basically have as much bandwidth as they could ever want. Yes, you can get around that with PeerTube, but there are so many other entirely unrelated advantages YouTube has over everyone else that this doesn't really confer much of a benefit.
[0] Interestingly, the FTP topsite scene would stick their nose up at even private BitTorrent trackers as not closed enough.
These people were _the_ tracker for the content in question so they were seeding a lot of content themselves hence the donations were a proxy.
The constraint on wide adoption is literally the quantity of "peers" in both Peertube and peer-2-peer.
The underlying human incentives are not there for most people to host a peer node. This limitation applies to all types of digital domains including videos, or files (IPFS or bittorrent), or crypto (Bitcoin/Ethereum peer node).
Let's follow the trail of incentives for one Peertube node:
- see list of Peertube instances: https://joinpeertube.org/instances#instances-list
- I pick the 2nd one on the list: https://the.jokertv.eu/about/instance
- in that instance's "About" page, it says: How we will pay for keeping our instance running -- personal funds. If you feel like donating, put up your own instance instead and host some creators you find interesting.
Companies can't build a Youtube/TikTok competitor based on examples like that. Same forces of economic incentives that limits quantity of IPFS peers which means businesses can't use it to replace Cloudflare CDN or AWS S3 buckets.
It should be understandable why most people aren't willing to spend personal funds on hosting home nodes so businesses can freeload off of p2p and save money on bandwidth.
I don't really see how the fact that there is someone who hosts videos using personal funds proves that it isn't possible for a professional company to host videos using the same company. It proves Peertube does work at small scale, and says "not much" about how it would handle large scale with a real budget.
From a business perspective, a big problem is trust and safety. Torrents were also notorious for having mislabeled content or malware, and users are not going to use a system which serves arbitrary content from their IPs, so you have challenges keeping the experience good while building a large enough peer network to provide a competitive experience.
While peertube saves one big pile of money (the streaming costs), it makes it significantly harder to make any money. Videos are expensive to make, so generally creators want a wide audience, and ways to monetise that audience.
Of course, because of network effects and YouTube ad revenue there is little incentive to do so except for idealists and people who get banned on YouTube.
A video hosting company needs to hold all the videos hostage so they can make money somehow, like via ads. Plus the ability to control content: the videos themselves (e.g. be able to censor or delete them), and other content like user accounts, comments and whatnot.
This is like asking, why doesn't Reddit just operate as a Usenet gateway with a Web UI?
Like a CDN? :)
The main difference being, of course, CDNs provide a delivery and storage service which you pay for, meaning the content remains yours, the circumstances under which it is viewed is under your control, and you're simply paying for them to deliver it for you. With peer-based systems, those two key points are no longer true.
Then the benefits of distribution are largely only realized by the owner.
For the same reason we don't see many video hosting companies, period: YouTube. It eats the entire space for reasons that are entirely unrelated to the pros and cons of p2p. For hypothetical competitors that means solving their biggest issue – gaining users over YouTube – can also not be done by going p2p (but it might of course still be part of their hypothetical tech stack for other reasons).