Beyond BitTorrent - A Decentralized Internet Filesystem

Tagged: decentralized & filesystem & bittorrent & ipfs

Remember BitTorrent?

Yeah, the pirate protocol. Actually, it's not specifically designed for breaking laws, but rather to be an efficient way to distribute files to many people - making files available to others in a way that takes advantage of each person downloading to speed up the downloads of other people thus reducing the required bandwidth and resources of the uploader (usually resource-constricted relative to the many downloaders) while then removing the uploader or any other single point of failure - it's decentralized! So, unlike normal file download sites, the original uploader can go offline and a file can still be accessible, all without relying on a server. Compare: If a non-torrent file download website goes down, that file can no longer be downloaded. BitTorrent solves this.

This makes the BitTorrent network a decentralized file store, something other projects like InterPlanetary File System (IPFS) are also building and running to support a decentralized web. At least, in theory, BitTorrent could be a storage layer component of a decentralized web. Does it work though?

For something to be our centerless networked filesystem, it would also need to work independently of any one entity / party / server. And BitTorrent has become more resistant to single points of failure over time. In the beginning, the protocol heavily relied on centralized servers to be "trackers" which were weak against legal attacks (think Bay of Pirates) but today web-based trackers are no longer strictly necessary - file search and peer tracking can be done directly over the network of BitTorrent nodes.

This is accomplished in one way through the implementation of Distributed Hash Tables (DHT) to track peers instead of a tracker. DHTs are also seen in the concept of Content Addressable Networks.

Also, it's no longer necessary to rely on websites to download .torrent files to start sharing a file. With Magnet links (URIs), files on BitTorrent are content-addressed (see content addressable storage) rather than location-addressed. Essentially, you refer to a file by its cryptographic hash, a sort of unique description of the content, rather than by the server who hosts it and the path to the file on that server. Then all you need to access a file is its magnet link. This is similar to how IPFS files are addressed as a hash within a base url. BT further uses cryptographic hashing to ensure that each section of a file is correct and not corrupt, and so chunks of a file can be safely transferred independently of the rest of the file. This allows peers who don't have the full file or all the files to still participate.

But BitTorrent, Inc. is also a company, which Bram Cohen, BitTorrent's creator and the company's founder, recently sold to the Chinese Ethereum (blockchain) clone very originally called Tron (from Justin Sun, infamous weaver of stories, now CEO of BitTorrent, Inc.), who then issued a token (BTT) to get money and promote this new project to the crypto community, and now the company's media presence has turned into a Tron (TRX) shill. But BitTorrent is a protocol, and no company can force all the other software implementations of the protocol to do anything, and neither can they force users to use any one official BitTorrent software. Assuming the protocol is open, well-specified and documented, and there's a robust and vibrant community of developers writing clients, so that users have a real choice. The source of a protocol's software needs to remain decentralized as well.

The company recently announced / launched "BitTorrent Filesystem" (BTFS). But, actually, BitTorrent as a filesystem is an idea that's been around for years (at least 2014 with a proof of concept) and has been implemented as a FUSE-mounted filesystem (thus it works on Linux and macOS) since 2016 (GitHub/johang/btfs but his main site is blocking Vietnamese IP address space), although Tron is claiming to be the one to launch it (PR announcement) in 2019. Many different and probably incompatible implementations, all confusingly using the same name.

BTFS would have a lot of limitations. It would be optimized for reading and not writing files. Opening a file would usually take a really long time. And data accessibility is not guaranteed. But it also improves certain use cases where you want to randomly access files and even just parts of files, and Popcorn Time (streaming video torrents) showed that this can be useful.

But, it does mean that decentralized file storage is possible today, practical concerns aside. Files can be stored and accessed in a sufficiently decentralized way. But one issue remains, and is an issue in common with IPFS, which is that there must be peers online when someone requests the file. BTFS isn't a magical infinite and free storage bucket. Til now, there wasn't any mechanism to financially incentivize hosts to be online in order to serve a specific file (BitTorrent works, but sometimes files can't be downloaded due to lack of peers, although new versions allow a web location to serve as a backup peer). Filecoin is the proposed solution for this problem on IPFS. And seeing their success in raising money, Justin Sun is probably thinking to replicate it with BTT / BitTorrent / BTFS.