BitTorrent (BTFS) vs IPFS, brothers from another mother

Tagged: decentralized & filesystem & bittorrent & ipfs

Following up from the last post about BitTorrent which mentioned IPFS (Interplanetary File System). There I framed BitTorrent as a distributed or decentralized filesystem or storage layer of the internet, and not just a pirate's den. That would make it a lot like IPFS.

BitTorrent, the protocol and apps, are well known and have been widely used for many years (since 2001). It came after many previous generations of incrementally improved file sharing applications. Originally, these were primarily for sharing music (Napster, Gnutella, Kazaa, etc.) as the technology and idea of compressed digital music (mp3) spread on the nascent consumer internet in the late 90s. BitTorrent arrived in an age when bandwidth and hard disk space could also start to support downloading movie files, orders of magnitude larger than music files. BitTorrent's innovations were that people could theoretically discover media to share outside of any particular centralized server, and then simultaneously download a file from many other people, and also upload that file to more people at the same time before even finishing the download. This was a much smarter and more efficient way to transfer a popular file to a lot of people compared to everyone hitting a central server for the same file simultaneously, creating bottlenecks and overloading servers.

IPFS has also been around for a few years but not nearly as long (2015). It can also essentially be used to get a file from one person to another, and then another, and another, increasing bandwidth and download efficiency along the way as more people are interested enough in a file to download it, thereby making that file available to their peers. IPFS also doesn't require a central server to coordinate this activity, although discovering media on the network is a separate concern and probably does happen on a normal service outside of IPFS. Both BitTorrent and IPFS suffer from the problem where a file which isn't "popular" (having many nodes sharing it) will be slow to transfer and will often simply disappear, becoming inaccessible.

Neither network is a free and unlimited online hard drive. You can't just put a file on IPFS and have it magically be available to everyone forever. Storage resources are provided by peers for altruistic but, more commonly, for selfish reasons.

Both BT and IPFS let you target and only download a subset of files (IPFS people might say otherwise but BitTorrent clients let you select a subset of files to download out of a torrent).

I mention BTFS in parentheses because it's the form of BitTorrent that is more similar to IPFS. As mentioned in the previous post, BTFS has been invented independently with the same name several times.. Since then, I've discovered that Tron's BTFS software simply cloned IPFS software on GitHub without proper attribution. Shady.

So what makes IPFS different or better than BitTorrent (including BTFS)? The main advantage proponents of IPFS will claim over BitTorrent is how it can handle duplicate files. Since IPFS is one system ("swarm") and one filesystem where each file is hashed and that becomes its "filename", if two different people add the same file, which could be in a collection of files they're adding, the hashes of files that are the same will collide. This happens even if the people are uploading other different files which don't match, only applying when there's an exact match for a particular file. In contrast, BitTorrent matches torrents and not individual files. But this is only beneficial when people are uploading large files that happen to be the same. BitTorrent networks often do see large files (like movies) that are very similar but at the byte level quite different due to different levels of compression, thus smaller and larger file sizes. And IPFS wouldn't catch these files as being similar either. If a file is changed by just 1 bit, the hash will be completely different.

BitTorrent nodes or peers need to find each other only when they have a torrent in common, whereas in IPFS everything is in the same global namespace, so to speak. So BitTorrent is more amenable to having a secret cabal of nodes who are aware of certain private torrents invisible to the wider network, whereas on IPFS everything is equal and public.

Quick note on "filesystems". IPFS and BTFS aren't filesystems in the way NTFS (Windows) or btrfs (Linux) or HFS+ (macOS) are.

BitTorrent is much older than IPFS, so it's had more time to disseminate to users, and it is indeed very popular, taking up a significant chunk of all internet bandwidth. But BitTorrent grew quickly after release. IPFS just hasn't been able to get used by normal internet users the way BitTorrent has. The time since launch is not the main factor in BT's popularity over IPFS. BT is immediately useful to end users in its current state. IPFS is not user friendly and it's not apparent to normal users why they would use it. It's attracting attention from other internet infrastructure developers instead.

Both BitTorrent and IPFS could be fast, but it depends on who is sharing a file.

As it stands, BTFS isn't how BitTorrent is primarily used. For the file hosting paradigm (vs file sharing), IPFS has standardized workflows with products built on that. IPFS is commonly used by Ethereum dapps to provide storage, which is what Tron hopes to build with their BTFS for Tron dapps.

In conclusion, BitTorrent and IPFS are technically similar with IPFS being slightly better for an edge case of uncoordinated sharing of subsets of similar files. While BitTorrent has the lead when it comes to normal users, IPFS is doing best at attracting developers needing decentralized online storage.