Bitcoin Files Protocol - Storage on Blockchain

Tagged: data & files & storage & decentralized & bitcoin & blockchain & p2p

A decentralized internet needs decentralized file storage. IPFS (especially IPFS as opposed to BitTorrent) is the leading solution and previously we discussed BitTorrent as decentralized file storage ) too. What other decentralized networks can we piggy back on?

Genesis

Storing data on blockchains is not a new idea. Even Bitcoin's Genesis Block (the first block) contained this free-form text in the coinbase:

The Times 03/Jan/2009 Chancellor on brink of second bailout for banks.

But the coinbase transaction only happens once per block, thus publishing one piece of information approximately every 10 minutes, and is controlled by the lucky but random miner who mines that block. So it's not at all a reliable way to put data onto the blockchain in a timely manner. For doing that and still using bitcoin, initially, people just stuffed data into bitcoin transactions and then all the bitcoin nodes had to keep track of the addresses involved in that transaction forever and still to this day, when block-space is much more expensive. Then the community wised up and changed the code of the core node software to start storing data in an OP_RETURN transaction, because OP_RETURN would signal that this balance (transaction output) could not be spent and therefore nodes did not need to keep track of it any longer as it was no longer usable. A slight improvement for node resources. This was a kludge, a way to let people do what they were doing anyways (store data) but cause slightly less damage when doing it.

Bitcoin Files Protocol

Now there's an even more formal way to put data, as files and directories, onto a blockchain. It's called the Bitcoin Files Protocol (BFP) and was originally designed for the Bitcoin Cash blockchain although I believe it would work on the Bitcoin (Core) chain although the fees would make it impractibly expensive. The people associated with the bitcoinfiles website later switched to the Bitcoin SV blockchain and now it's a confusing mess. Just know that Bitcoin Files shouldn't be used on Bitcoin and the infrastructure isn't working with Bitcoin Cash even.

To make things even more confusing, there's also a Chinese thing called BitcoinFile, which you might find by googling "bitcoin files". And this isn't even a clone of Bitcoin Files but rather some other scheme which just uses similar naming. The blockchain space as a whole is rife with scams and so many of them target a Chinese community but attract victims worldwide. All the things discussed in this article are about technology and not at all about finance or investing or tokens. Also note, BitcoinFiles has nothing to do with Filecoin.

Bitcoin Files claims to store files and even directories on the blockchain. How does this work?

The spec is in the Simple Ledger Protocol repository on GitHub.

So it appears that a file uploaded to the blockchain with BFP is represented as a series of linked transactions that look like (as a series of bytes of bitcoin script opcodes and variable data):

OP_RETURN
<chunk_X_data_bytes>

Then the final transaction in this linked list looks like:

OP_RETURN
<lokad_id_int = 'BFP\x00'>
<bfp_msg_type = 0x01>
<chunk_count_int> 
<filename_utf8*> 
<file_extension_utf8*>
<file_byte_count_int*>
<file_sha256_bytes*>
<previous_file_sha256_bytes*>
<file_uri_utf8*>
<chunk_X_data_bytes*>

There's a more succinct format for small files fitting in a single transaction.

Transactions are linked by "passing the baton", the next transaction spending the UTXO dust of the previous transaction.

When the system was set up, they were testing with files up to 5K in size. You can imagine saving a JSON document or a very tiny web page, but not any kind of media (audio, video, even images). There's a second way to represent files which is meant for larger files (something like 100K and up). This uses the pay-to-script-hash (P2SH) transaction type.

Bitcoin Files on IPFS

The other documented and supported method of "uploading" is to actually host the file first on Interplanetary File System (IPFS) and then record a link (uri) to the file's id into a BSV transaction in accordance with BFP. IPFS supports as much data as you want to host, but a key difference between the networks is in what the nodes store. IPFS nodes only store data as they use or transport it or otherwise when they specifically "pin" a file. Bitcoin nodes, and the same goes for forks like BSV, store the entire historical blockchain (pruning aside). While putting data (via a transaction) will permanently place it on all nodes, the same is far from true for IPFS, and someone needs to run an IPFS node and share the file forever otherwise it is not available. So using the IPFS option with BFP does not provide any of the same guarantees as the main BFP method, and the BFP (protocol) itself doesn't provide any support for getting files onto IPFS nodes or incentivizing them to pin a file.

There's a website, BitcoinFiles.com, that purports to facilitate uploading a file to the Bitcoin SV blockchain after making payment with BSV, but the website is unavailable. Not a very resilient system.

Can we see it in action somewhere? There's this example file.

$ curl https://bitcoinfilesmatter.s3.amazonaws.com/mytesth.falsee38ee790-8f3a-11ea-bb89-2bd752d34d1a.false
<h1>Hello</h1>

This file is cached on AWS S3 and not actually fetching anything from any blockchain.

Conclusion

While it seems technically possible and there's code (a JavaScript library) to compose the transactions and pay to put files of varying sizes onto the BSV blockchain, there is a dearth of information about Bitcoin Files, the main website is down, and it's otherwise hard to see it in action. While I'm not saying it's been completely abandoned, it's not very encouraging for any user unless they're already a diehard BSV/CSW (Craig Wright) follower. But there doesn't seem to be many. Even finding working BSV block explorers is a hassle.

At this point, Bitcoin Files is a working proof of concept but limited to a minor blockchain that many skeptics, nay, nullifidians, would stay away from while Bitcoin (Core) has enough troubles attracting mainstream adoption as is. Blockchains are an extreme medium for file storage and what still needs to be shown is that this is scalable, will remain cheap and affordable, and that miners (nodes) will continue to have the incentive to store all that (from their perspective) worthless data far into the future.