Cannot transfer files over 10M by file sharing

[Lebei2046]

Hi, all,

I tested file sharing of examples, found that the file size is below 10M.
It's possible to transfer large files?

[drHuangMHT]

The limit is set by request-response. For the codec used bu file-sharing example(which is cbor):

rust-libp2p/protocols/request-response/src/cbor.rs

Lines 77 to 81 in 61d436a

	Codec {
	request_size_maximum: 1024 * 1024,
	response_size_maximum: 10 * 1024 * 1024,
	phantom: PhantomData,
	}

For JSON codec:

rust-libp2p/protocols/request-response/src/json.rs

Lines 76 to 80 in 61d436a

	Codec {
	request_size_maximum: 1024 * 1024,
	response_size_maximum: 10 * 1024 * 1024,
	phantom: PhantomData,
	}

You can increase response_size_maximum to allow bigger transfers. But the example is just an example. It also prints the contents to stdout, which is obviously not for real-world scenarios. You should implement it yourself.

To improve on top of the example, I would do the following:

• Save the received file somewhere on the disk instead of printing its contents.
• Send a pre-flight request to get the size of the file first, and then send multiple requests to get each chunk of the file if the file is larger than response_size_maximum. Since request-response strictly pairs one request with one response, you will not able to get by with one request for multiple responses.

You can also build a custom protocol dedicated to large transfers, either from scratch or from libp2p-stream.