[tracking-issue] CBF overhaul

[Davidson-Souza]

Our Compact Block Filters implementation currently have several problems:

• It is slow and brittle, not reliable at all.
• It stores all filters in a range, when most clients actually only keeps the last N filters and all headers.
• It pours most of the filters logic inside wire, which is a domain violation.
• It forces users to re-implement the rescan logic every time

For that reason, we should have a new approach, that:

• Prefer keeping headers, not full filters (modulo a small cache for recent filters). This is usually how other implementations does it, and saves disk space.
• Don't make floresta-wire coordinate anything. It is a wire library, it shouldn't know about filter's business logic.
• Encapsulate all rescan logic under a common interface and avoid code repetition

The approach I've came up with (see #1079), introduces a new filters manager that takes care of everything. This will be a separated service that runs on a loop and responds to requests made through a handle. It uses NodeInterface to gather data such as blocks and filters.

Rescanning is an async process, as it might take a considerable amount of time to run. For that reason, I've implemented a polling-based system to fetch rescan blocks as we find them. Since a wallet might have too many transactions, that could
DoS our node in case we get all those blocks in-memory. To solve that, there's a pagination system where you need to consume available blocks before we request more. Once we finish with all blocks, the rescan is over.

This task is meant to run from the get-go, and will start downloading filter right after headers-sync is finished, so our node is "usable" in a few minutes. Even if IBD takes hours.

Roadmap

Some things will be required during this project, I'll break them into a few PRs to help with review

• Implement GetCFilterHeaders for node_handle (feat(node): Add GetCFHeaders to the node handle #1088)
• Implement GetCFilter for node_handle
• Make the node handle into a trait, so we can test our code (refactor(wire): make node interface a trait #1035)
• Move try_and_* to common, so we can use them here as well (refactor: move try_and_* to common #1096)
• Build a more rich state machine for IBD status for chain (feat(chain)!: Use a more granular state machine for IBD #1095)
• Implement the filter header manager with sync ([WIPWIPWIP] filters: rework how filters work #1079)
• Build filters locally
• Allow getting filters from the manager
• Implement rescan
• Implement caching
• Implement checkpointing

Some of these might be a follow up to reduce complexity here.

[luisschwab]

Build a more rich state machine for IBD status for chain

Yes please!

Perhaps wire, chain and compact-filters should all implement FSMs tracking their current state, which can be exposed by either wire's NodeHandle, and/or individually by each crate. Then consumers can monitor the current state in one of two ways:

• Poll using a get_state() function
• Subscribe to a channel that emits a new event every time a state transition happens

I think the latter is more elegant and cleaner.

I implemented this in bdk_floresta so that the Client blocks scan requests until the Node has finished downloading filters:

• https://github.com/luisschwab/bdk-floresta/blob/master/src/node/mod.rs#L128-L129
• https://github.com/luisschwab/bdk-floresta/blob/master/src/client/mod.rs#L192-L212

The FSM there is not perfect, since I can only do a best-effort inference of the actual state using information that is made available by this library. Having the actual FSMs implemented here will be much better.