Topic: SPV wallets which lets me verify blocks myself (e.g. BIP-157/Neutrino)? (Read 393 times)

I just remember there are another implementation of BIP 157 such as bcoin[1] (not sure whether you can just use it's wallet feature without run full node), neutrino (Go library)[2]. Zap desktop version also support BIP 157[3]. But it's SPV LN wallet where it's on-chain feature is very limited.


Thank you for additional information, although what i said isn't exactly wrong since i said "still on works". And that's why i doubt Electrum will support BIP 157 anytime soon.

[1] https://github.com/bcoin-org/bcoin/pull/1067
[2] https://github.com/lightninglabs/neutrino
[3] https://docs.zaphq.io/docs-desktop-neutrino-connect

Yeah, Electrum is mainly "supporting" Taproot through hardware wallets that have the option. More specifically Ledger and Trezor. Other than that it can only send to Bech32m addresses (witness version 1 address encoding used by Taproot) otherwise it can't create nor sign such transactions spending from such addresses.

Electrum wallet software supports Taproot but not fully support it.
Bech32 adoption


The development for is still in todo list. Last reply on Github is two years ago.

It seems actually that no really decentralized end-user wallet solution exists for the technologies I'm looking for, so it may be indeed the best solution to code a minimal app with an independent database. As for the needed data, maybe it is enough with only store block data for blocks where a prior query found a transaction to/from the wallet addresses.

I was able to install the official Neutrino implementation by Lightning Labs now but there seems to be no way to run it as a standalone app. For a short moment I even thought of "abusing" lnd (which does run using neutrino as a kind of "lib"). But then I found the Bitcoin-s project, it seems to be a little bit more advanced in the direction I'm looking for, although its Neutrino support seems to be experimental and doesn't have an active mempool. I'll try to install this one, and write here about my experiences

No offense taken. At a quick read it looked like you were looking for something different.

Thinking about it this is where an independent database solution would be ideal. You can get rid of a bunch of stuff that you don't need a just stuff the data into a DB that you are querying.
What I can see being an issue is that what I want in the DB YOU might not want in the DB, so then by default the app is going to try to put everything into it.

-Dave

To be clear: I'm not looking exclusively for a solution for me or an use case of mine. I'm simply interested in the current "offers" which exist of a certain category of technological solution, which is what BIP-157-based software would offer, but for desktop and not for mobile. The idea is that also other people which are looking for a better SPV solution could "profit" from this thread. Maybe I should clarify that in the OP.

Thank you for all suggestions of the type "you should do this (e.g. use a full node) instead of that", but that's not the topic of this thread (please don't feel offended though).   (Of course I understand that it's also important for Bitcoin to count with a good number of full nodes, but again, this discussion is off topic here.)

I hate to just put this out there, and it really might be a US based thing, BUT 1 TB SSD are now under $50

https://www.microcenter.com/search/search_results.aspx?N=4294945779+4294818519+667&NTK=all&sortby=pricelow

so it's not like you can't have the entire blockchain and an OS and everything else you need on a decent drive. Older (6th gen) PCs are overkill for doing this and are also dirt cheap since they are for the most part 7+ years old.

There really is no cost involved with this the right way anymore.

Sorry if this seems grumpy, but you will spend more time trying to figure out a way not to spend a few dollars then the extra few dollars is worth.
And then pull your hair out dealing with issues because you tried to cut corners.

-Dave

I doubt it'll happen anytime soon since,
1. Electrum have it's own protocol, which usually referred as Electrum/Electrum server protocol.
2. Electrum currently lack of developer/contributor, which makes Taproot support still on works.
3. Sparrow already support 3 different method, which are Electrum protocol, Bitcoin Core RPC-JSON and BWT (with purpose connect to Bitcoin Core version below 24.0).

It's actually doable but it's not recommended since your client wont be verifying the downloaded blocks, so you'll need to trust the integrity of your pruned blockchain.
That's basically defeating the purpose of using a Bitcoin Node.

Download at your own risk:
There's a user in GitHub who is hosting pruned blockchain which you can directly load in your newly installed Bitcoin Core.
Here's the link: https://github.com/Blockchains-Download/Bitcoin/releases (file names have the wrong year in the name)
Just follow the provided instructions by the owner, specifically those for "local Bitcoin client".

The data seem legit since it starts without errors and continue to sync up to the tip but as mentioned, it's not verified by my Bitcoin Core.
Also, you can't load an existing wallet.dat file or import old keys with txns past the pruned blocks, you can only create new wallets.

I've looked into some of the wallets you recommended me:

- Unstoppable: On the website there is unfortunately no mention of the way the balances/accounts are synced. I think in any case I would have to look into the code.
- Breez Wallet and Blixt Wallet seem to be simply graphical frontends for Neutrino SPV (and also lnd) on mobile devices.

Maybe the best option I found until now is using the Bitcoin Dev Kit (BDK) for a very simple wallet implementation, which also supports Neutrino and thus BIP 157/158.

It would be however interesting too if Electrum or Sparrow devs plan to implement Neutrino eventually. I have found no related feature request though, but maybe someone knows

You can actually do this for far less than 550GB (and without a node) if someone scrapes all the gz.blockchair.com dumps for the Bitcoin chain and distributes it in a usable form, because it contains all the inputs/outputs, transactions, blocks, etc. and even address data for each individual day. Of course, you still have to rely on this dataset to be correct, but the worst that can happen if someone screws up is simply that broadcasted transactions get rejected.

That's possible, but i never seen anyone attempt to fork Wasabi wallet or port specific feature on Wasabi wallet.


Don't forget to check getblockfilter RPC command.

Neutrino seems to be the closest project until now for my requirements, thanks! I'll investigate if I can install it as a desktop client. The Neutrino client at Github (by Lightning Labs) seems to be more a "base technology" than a full-fledged desktop client (meant probably as a base to build mobile wallets like those your mentioned), and I had problems compiling/installing it due to dependencies, but that may be solvable. I've still however not 100% understood the trust model, it seems to be a client-server architecture - but is the Github program the client or the server or both? Have to investigate more.

On mobile there is Breez Wallet an Blixt Wallet which use BIP157 and Neutrino for client-side block filtering. They will use more storage than a normal light wallet but it is still a fraction compared to running a full node.

Thanks! I've read through the BIP and it seems to be indeed what I'm looking for (so I put the BIP in the thread title as maybe it attracts someone who knows lesser known implementations). As Wasabi seems to be open source, maybe the relevant part can be "ported" into completely open/decentralized solutions like Electrum ...

@pooya87: Thank you; yes I've only recently looked in that file and I think I understand the way net_processing.cpp works approximately even not being fluent in C++. I still hope to not have to code that thing myself, but it seems I have to (or at least parts), so its possible I've to implement your idea

Well at least Feather (afaik!) works quite simple: You set a start date for your wallet (when it was created or when the first transaction was done) and then all blocks after that one are verified (i.e. downloaded and validated) but only the important data is kept on disk. I would like to have a little bit finer-grained control, for example if I know that in some period in the middle I didn't receive nor sent transactions, I wouldn't have to download these blocks, so I don't waste bandwidth I don't need. BIP 157 seems to provide that. I think it wouldn't even be necessary to store entire blocks (those with the transactions), the relevant part of the merkle tree may be enough.

In Feather everything works smooth even downloading all blocks from the start date, something you have to wait a bit but in general it is of little issue. However, one has to take obviously into account that Bitcoin blocks are larger than Monero blocks (the difference is abysmal, XMR blocks seem to have only 15-400 kB, while BTC's are ~2-4 MB), so the verification time everytime you launch the app would be much higher.

Exactly. And having your own Electrum server you may use it not only with electrum, but also with Blue Wallet on your mobile, Trezor Suite etc.

By the way, how it would work, you expect to have a wallet connected to the node which keeps only blocks which contain data about your address? How about incoming transactions, do not you want to keep blocks which confirms validity of your inputs? How about the current state, if you want to create outgoing transaction you plan to keep the block and then download/ignore blocks which does not have information important for you?

What you're looking if wallet which support BIP 157[1]. AFAIK the only desktop wallet which support it is Wasabi Wallet which is controversial mainly due to surveillance on it's CoinJoin feature.

[1] https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki

The problem with your idea is its practicality otherwise it is pretty trivial to implement.
Normally you don't know when exactly you've received bitcoin and when exactly you've spent those coins. Even knowing a range is not going to help much because you'd have to download roughly 6 blocks per hour or 144 blocks per day which is high enough for a light client.

There is no trust needed, just knowing that you are on the correct chain which is easy to do. You basically have to play around in the net_processing part of bitcoin core:
Step 1:
Initial connection is the same as a full node, fetching a list from seed nodes and building a listening node IP address database for future usages using the getaddr and addr messages.

Step 2:
Exactly as a full node, start by downloading the blockheader list from the hard-coded Genesis block to the tip from multiple different nodes with enough distance that makes it safe. You can also add hard-coded checkpoints to add an additional verification for the "map" of the chain you are downloading.
You can also do full verification on these headers just like a full node does (SPV clients should also do this). The verification includes version verification, PoW verification using the header hash and the shortened target in the header, difficulty verification with the target and finally time verifications.

The network messages involved are: version+verack (handshake), sendheaders, getheaders, and ping pong messages.

Step 3:
Now that you have a "map" of the whole blockchain use the time field in the headers to select the block(s) you want to download then send a getdata message (with the inventory set to block type and the hash(es) of the block(s) you want to download) to any full node from your connection list to download the full block from that node.
Then you can perform a limited verification on the block you received to make sure it is "correct". The important ones are deserializing and merkle root hash. This way you make sure all the transactions in the block you requested are received correctly and are associated with the header you requested. Here we rely on PoW and SHA256 both being secure.

Step 4:
Now that you have a block that you are sure is "correct" with the trust you put in the PoW, you can go through the inputs and outputs of each tx to see which one belongs to you and update your balance accordingly.

If you have those types of skills I assume you're skilled enough to set up a Linux machine with a full core node and your own SPV server.  Even if your home network is behind CGNAT you'll be able to connect to it while you're on the go by using tor (even on mobile devices.)  I have a set up like that using Romanz Electrs, and it's been running great.  Romanz Electrs is super easy to set up and doesn't require a ton of additional recourses.  It gives you so many options, it's quite worth the effort and hardware.  No trust is required other than trusting the software itself, but that can be verified of course.

I'm looking for a solution for a low-spec desktop computer/notebook.

I know a pruned Bitcoin Core node would also be an option, but the problem isn't so much the 500 MB-few GB of storage of this option, but the time until I'd have synced the blockchain. I'm looking for a solution which would be there instantly, to be used flexibly, like a SPV wallet.

If I don't find such a solution I'm perhaps even able to code it myself as an Electrum extension as I know a little bit Python.


Thank you. I'm looking at it and if it has the feature I'm looking for, I'll report it in this thread

PS: seems to be mobile-only Anyway. I'll take a look.

Not exactly. That could still be used for chain analysis, unless you really add tens of thousands of addresses (and then I think it would become a laggy monster). I think the approach to download and verify blocks myself is more private.

Of course, and that is also my current solution However, I'm looking for something a bit better Basically some intermediate step between "classic SPV" and "(pruned) Core".