Pages:
Author

Topic: looking for coinkit or pybitcoin for running on python3 (Read 314 times)

legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
Also thanks for the snippet you provided. This is a really helpful for me as a beginner. How did you know which module is the correct one for this task? Did you just run something similar like
Code:
grep -R -i 'get_balance'

No, i found out the correct module by checking pycoin documentation[1]. Since you mentioned get_balance function, i use that as keyword on search bar[2]. One of the search result lead to module called BlockcypherProvider[3]. Afterwards, i simply try initialize module BlockcypherProvider and then use get_balance function.

[1] https://pycoin.readthedocs.io/en/latest/
[2] https://pycoin.readthedocs.io/en/latest/search.html?q=get_balance&check_keywords=yes&area=default
[3] https://pycoin.readthedocs.io/en/latest/source/pycoin.services.html?highlight=get_balance#pycoin.services.blockcypher.BlockcypherProvider.get_balance

If documentation isn't exist, usually i'd look for example on their GitHub repository or read the source code directly.

Please look over it and tell me where the errors are or how you suggest to optimize for better performance.

Code:
#!/usr/bin/python3
import sys
from pycoin.services.blockcypher import BlockcypherProvider

args = sys.argv
filepath = args[1]
bp = BlockcypherProvider()

with open(filepath, 'r') as f:
  addresslist = f.readlines()
  addresslist = [a.strip('\n') for a in addresslist]

# do something with address list here
print(addresslist)

# retrieve and output balance and tx info of each address listed in provided address.lst
# with a line separator (horizontal line or similar)
#result = bp.get_balance(addresslist)
for details in addresslist:
   result = bp.get_balance(details)
   print(result)
   print('―' * 40)  # U+2015, Horizontal Bar, 40 times

At a glance, i didn't see any error. But personally i'd replcae for details in addresslist: with for address in addresslist: for better readability. I don't know much about performance (since i use Python for small/light stuff), but people usually advice to use Python Async feature.
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Of course, I would be limited by the hardware used for running both, the full-node and the blockexplorer underneath. Thanks for pointing out to take this into consideration.

Also thanks for the snippet you provided. This is a really helpful for me as a beginner. How did you know which module is the correct one for this task? Did you just run something similar like
Code:
grep -R -i 'get_balance'
in pycoins' root folder? On my computer I get this result...
Quote
~/pycoin$ grep -R -i 'get_balance'
pycoin/wallet/SQLite3Wallet.py:            while confirmations > 0 and self.get_balance(confirmations=confirmations) < amount + estimated_fee:
pycoin/wallet/SQLite3Wallet.py:    def get_balance(self, confirmations=1):
pycoin/services/blockcypher.py:    def get_balance(self, address):
So you know that the particular file we're looking for is pycoin/services/blockcypher.py and the class is called "BlockcypherProvider" where it is contained. Is that correct? My goal ist to learn, that's why I am asking this stupid questions, I don't want to simply copy&paste&use  Roll Eyes

currently my script and the provided address.lst looks like that:
Code:
$ cat address.lst

157dPuCytnX1tTDcviaqxxTogRzTJUSbc4
3HvgFC6VSeT3DZthW4CH6QK9VYmS2KDwA8
bc1qaqn5ay35jm3whdj2z8y76n9q4d7wlx6vemrp0k
1EhpfhPtnRFUi1B8Z8hSEHrnNv6qDC5abp

Code:
#!/usr/bin/python3
import sys
from pycoin.services.blockcypher import BlockcypherProvider

args = sys.argv
filepath = args[1]
bp = BlockcypherProvider()

with open(filepath, 'r') as f:
  addresslist = f.readlines()
  addresslist = [a.strip('\n') for a in addresslist]

# do something with address list here
print(addresslist)

# retrieve and print balance and tx info of each address listed in provided address.lst
# with a line separator (horizontal line or or a line feed \n or similar)
result = bp.get_balance(addresslist)
print(result)

I execute the script with
Code:
./myscript.py address.lst

but I get the error
Quote
['157dPuCytnX1tTDcviaqxxTogRzTJUSbc4', '3HvgFC6VSeT3DZthW4CH6QK9VYmS2KDwA8', 'bc1qaqn5ay35jm3whdj2z8y76n9q4d7wlx6vemrp0k', '1EhpfhPtnRFUi1B8Z8hSEHrnNv6qDC5abp']
Traceback (most recent call last):
  File "/home/citb0in/myscript.py", line 18, in
    result = bp.get_balance(addresslist)
  File "/home/citb0in/.local/lib/python3.10/site-packages/pycoin/services/blockcypher.py", line 63, in get_balance
    url = self.base_url("addrs/%s" % (address + url_append))
TypeError: can only concatenate list (not "str") to list

I am pretty sure this error occurs due to addresslist containing several addresses. Is the variable "addresses" a string type here in this example or is it an array? What is the best approach to run the get_balance correctly against a bunch of addresses? Thank you so much for your patience and assistance.

EDIT: I think that was the culprit. The variable is kind of an array, at least I am pretty sure it is not a string or a concatenated string but rather a list. I've made some progress and it seems, that it works. Here my actual snippet. Please look over it and tell me where the errors are or how you suggest to optimize for better performance.

Code:
#!/usr/bin/python3
import sys
from pycoin.services.blockcypher import BlockcypherProvider

args = sys.argv
filepath = args[1]
bp = BlockcypherProvider()

with open(filepath, 'r') as f:
  addresslist = f.readlines()
  addresslist = [a.strip('\n') for a in addresslist]

# do something with address list here
print(addresslist)

# retrieve and output balance and tx info of each address listed in provided address.lst
# with a line separator (horizontal line or similar)
#result = bp.get_balance(addresslist)
for details in addresslist:
   result = bp.get_balance(details)
   print(result)
   print('―' * 40)  # U+2015, Horizontal Bar, 40 times
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
Thanks for your thoughts. I intend to set up and run my own full node in the future. I think I'll have an option (3) then, speaking: my Python application would simply do the queries against my own full-node. Advantages: Third parties don't know which addresses are being queried and I'm not restricted or limited in the amount and throughput of queries.

Take note you're limited by your hardware instead.

Am I seeing this correctly or do I have a thinking error? Or is "running an own full-node NOT equal runngin an own blockexplorer" ?

Running full node and block explorer are 2 different thing. Block explorer depends on full node to get all block and transaction. So you'll need to run both full node and block explorer.

However, I'm still facing the problem and question of how to access pycoins' .get_balance function to query one or more addresses online on the blockchain using the URL defined in pycoin. This is my first concernt at the moment, I need to build up the pyton commands. Am very grateful for any hint and suggestion as always.

I have trouble understanding your question, but here's example of working script.

Code:
from pycoin.services.blockcypher import BlockcypherProvider
bp = BlockcypherProvider()
result = bp.get_balance(address="157dPuCytnX1tTDcviaqxxTogRzTJUSbc4")
print(result)

Here's the output.

Quote
{'address': '157dPuCytnX1tTDcviaqxxTogRzTJUSbc4', 'total_received': 15011490, 'total_sentalance': 14518229, 'unconfirmed_balance': 0, 'final_balance': 14518229, 'n_tx': 4, 'uncon 0, 'final_n_tx': 4}
hero member
Activity: 630
Merit: 731
Bitcoin g33k
I'm not sure what the issue with the Python code is right now; I can take another look later.
Simply said: I have a file with valid Bitcoin addresses, each per line. The idea was somehow using function get_balance of pycoin library to query all these adresses contained in the file. I would be glad an it would be enough for the first step to get the result and redirect it to a new file result.txt which I could run several "grep" and "awk" filtering to it. I could then filter eg. to show only those adresses containing a positive balance, or to show those "greater than x.xx BTC" etc. However it would be awesome to have the option to retrieve not only the balance but also the state of transactions. I'd like to know if the adresses with balance > 0 had any historical outputs so I could run a query like "Show me all the addresses that either have a positive balance or had incoming/outgoing transactions  in the past.
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!
Thanks for the links n0nce, I really appreciate your help. I am running Linux only so I'll need to go for the Linux tuts, but at a later time because I'm really busy with various projects currently. Can you give me any hints how to proceed with the python code to reach my goal?
My guide is for Linux, and the second guide is for WSL, so just Ubuntu running on Windows.
I'd probably just follow the OpenSUSE install guide though (you can skip the whole Lightning part, just need Bitcoin and Electrs) and then do the mempool Docker install as described here:
https://github.com/mempool/mempool/tree/master/docker#configure-with-bitcoin-core--electrum-server

I'm not sure what the issue with the Python code is right now; I can take another look later.
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Thanks for the links n0nce, I really appreciate your help. I am running Linux only so I'll need to go for the Linux tuts, but at a later time because I'm really busy with various projects currently. Can you give me any hints how to proceed with the python code to reach my goal?
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!
Thanks for your thoughts. I intend to set up and run my own full node in the future. I think I'll have an option (3) then, speaking: my Python application would simply do the queries against my own full-node. Advantages: Third parties don't know which addresses are being queried and I'm not restricted or limited in the amount and throughput of queries. Am I seeing this correctly or do I have a thinking error? Or is "running an own full-node NOT equal runngin an own blockexplorer" ?
You need to run a full node to run a block explorer on top of it. It's an extra software package you need to install which then queries your local blockchain.
If you query the blockchain a lot and value privacy, running a full node is absolutely the way to go.

I have a complete full node install guide here (though no instructions for installing a block explorer like https://github.com/mempool/mempool).
https://bitcointalksearch.org/topic/guide-full-node-opensuse-153-bitcoind-electrs-c-lightning-rtl-5366854

There's a Windows install guide (WSL) including block explorer here:
https://bitcointalksearch.org/topic/bitcoin-electrum-server-block-explorer-under-windows-with-wsl-and-debian-5367296
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Thanks for your thoughts. I intend to set up and run my own full node in the future. I think I'll have an option (3) then, speaking: my Python application would simply do the queries against my own full-node. Advantages: Third parties don't know which addresses are being queried and I'm not restricted or limited in the amount and throughput of queries. Am I seeing this correctly or do I have a thinking error? Or is "running an own full-node NOT equal runngin an own blockexplorer" ?

But until I have familiarized myself with the procedures and how-tos for installing and operating a full-node, I will probably have to stick with the offline variant (1GB file from LoyceV) as suggested.

However, I'm still facing the problem and question of how to access pycoins' .get_balance function to query one or more addresses online on the blockchain using the URL defined in pycoin. This is my first concernt at the moment, I need to build up the python commands. Am very grateful for any hint and suggestion as always.
hero member
Activity: 630
Merit: 731
Bitcoin g33k
thanks for the hint, didn't know that you can download the complete blockchain in this human-readable way. Wink

However, I would then have to download the 1GB file daily and that at a transfer rate of < 100 kB/S, which is a pain in the a$$ and not acceptable for my project. In addition, I would have an offset of at least 24h, so my queries would not be completely up to date. What's wrong with doing my query directly at the blockchain, i.e. online via pycoin library as initially planned? I thought that I could somehow get pycoin to perform the key.get_balance query online using a self-made Python snippet. How do I do that, how can I access pycoins' routine of get_balance from my own script?

On the other hand, the question then arises again, how can I most efficiently perform a query of, say, 1 million addresses with the highest possible throughput. I guess the answer therefore is "locally, through LoyceV's list" but is there an alternative to do it online so I can query up-to-date data from the blockchain? And... if I would do it with pycoin library as I actually wanted to do, would the server where pycoin queries the get_balance just block and ban me because they see that I am doing thousands of queries in a short time? Is there a way to circumvent this?
legendary
Activity: 3346
Merit: 3125
Hi ETFbitcoin and thanks for assistance. Luckily I already figured out how to pass arguments but your example however was very helpful, too. Is it better to use sys.argv or would you suggest to go with the getopt.getopd method instead ?

My main problem is how to pass the address.lst to pycoin lib for processing the get_balance part. Unfortunately the provided binary program tool "ku" is only for manipulating keys, I didn't find anything related to get_balance there. In that case I would have piped the example.py results to ku, something like that....

To get the balance from an address you need to call a third party, You can call any block explorer api, you can call your bitcoin core with RPC, or you can directly download the file with all the address containing balance.

https://gz.blockchair.com/bitcoin/addresses/

That list gets updated daily, i hope it helps, and if you want to manipulate that list i will recommend the topic created by LoyceV for that topic:

https://bitcointalksearch.org/topic/list-of-all-bitcoin-addresses-with-a-balance-5254914
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Hi ETFbitcoin and thanks for assistance. Luckily I already figured out how to pass arguments but your example however was very helpful, too. Is it better to use sys.argv or would you suggest to go with the getopt.getopd method instead ?

My main problem is how to pass the address.lst to pycoin lib for processing the get_balance part. Unfortunately the provided binary program tool "ku" is only for manipulating keys, I didn't find anything related to get_balance there. In that case I would have piped the example.py results to ku, something like that
Code:
./example.py | ku get_balance
that would have been convenient and very easy for me as beginner. So I don't understand, how can I pass the address list to pycoins python class get_balance ? And my next question I have is: what is best practice for maximum performance if I -lets say- wanna check millions of addresses ? Should the particular python code read the address.lst into memory first and then process the get_balance part in a for loop ? Sorry if being a dumb question, I am still a beginner looking for some helpful advice and hints.

Thank you so much
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
EDIT: Meanwhile, can anyone show me a small python snippet which I could pipe a private key (hex) into or a bitcoin address and which in advance will run the get_balance function for the address? My goal is following: I'd like to have a foo.py and run it as a bash command like
Code:
~/foo.py bc1qmyaddressXXXXfoobar

or

~/foo.py address.lst (where this file contains a bitcoin address in each line)

or

~/foo.py privkeyhey.lst (where this file contains private hex key in each line)
I'd like to show/save the results only for those addresses that either have a balance > 0.0 or had any transaction in the past.

Any help appreciated.

Here's a short example. #!/usr/bin/python3 indicate location of program to run the script. sys.argv is used to obtain passed argument.

Code:
#!/usr/bin/python3
import sys

args = sys.argv
print('args:', args)
filepath = args[1]

with open(filepath, 'r') as f:
  address = f.readlines()
  address = [a.strip('\n') for a in address]

# do something with address here
print(address)

Here's content of address.lst i used as example.

Code:
3HvgFC6VSeT3DZthW4CH6QK9VYmS2KDwA8
bc1qaqn5ay35jm3whdj2z8y76n9q4d7wlx6vemrp0k
1EhpfhPtnRFUi1B8Z8hSEHrnNv6qDC5abp

After creating the script and file which contain private key (or other content), you need to make it executable with command chmod +x example.py.

Code:
$ chmod +x example.py
$ ./example.py address.lst
args: ['./example.py', 'address.lst']
['3HvgFC6VSeT3DZthW4CH6QK9VYmS2KDwA8', 'bc1qaqn5ay35jm3whdj2z8y76n9q4d7wlx6vemrp0k', '1EhpfhPtnRFUi1B8Z8hSEHrnNv6qDC5abp']
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Hello all and sorry for late reply.

It's not hard to see that coinkit is abandonware - Pavol has probably moved on to other things.

I'd suggest to check different Python library which is still actively developed. Usually i'd recommend these library,
https://github.com/richardkiss/pycoin
https://github.com/1200wd/bitcoinlib
https://github.com/ofek/bit

There is also no reason to port stuff yourself; there are enough working, maintained Python 3 Bitcon libraries. The ones ETFbitcoin mentioned should all work just fine.

Thanks for clarification guys! I fully agree, no reason to go with abandoned initially named coinkit. I simply installed pycoin with
Code:
pip3 install pycoin
and the provided ku tool already gave me all the information I needed for my 1st steps. For instance I run
Code:
ku 
and get a bunch of information like private/public keys in various formats (compress/uncompressed) and the addresses etc... very nice and not complicated. Thanks again to you for putting me to the right way Wink

Now I need to find out an easy way to query the blockchain for the balance of one or more address/es. If understood correctly pycoin is not capable of querying and thus interacting with the blockchain, is that correct or wrong? I looked forward to "btctools" Just like pycoin it is a package within python which can be easily installed
Code:
pip3 install bitcoinlib

with w.scan() I can query the blockchain and sync my wallet and with w.info() I can display all the wallet info. This outputs also the balance, honestly said I'd like a more convenient way to scan and display only the balance. Let's go one step further and try "bit" which is also within pythons package system and can easily be installed
Code:
pip3 install bit
Although it seems is not frequently maintained as pycoin or bitcoinlib it claims to be the fastest lib (100x faster than the closest lib) out there. Sounds promising if you ask me  Roll Eyes well, tried it out and I was able to fulfill all the needed tasks with the key.commands, so far GREAT impression. The next page called Networking in the documentation explains how we can interact with the block chain. So my previous question about how to easily get the actual balance of my address, simply answered:
Code:
key.get_balance('btc')
et-voila!  Cool

I'm impressed and I guess this is the way to go. Now I just need to learn coding Python Tongue but thanks to your grateful suggestions certainly I will succeed some day. Will dig further into Codecademy, W3schools and learnpython

EDIT: Meanwhile, can anyone show me a small python snippet which I could pipe a private key (hex) into or a bitcoin address and which in advance will run the get_balance function for the address? My goal is following: I'd like to have a foo.py and run it as a bash command like
Code:
~/foo.py bc1qmyaddressXXXXfoobar

or

~/foo.py address.lst (where this file contains a bitcoin address in each line)

or

~/foo.py privkeyhey.lst (where this file contains private hex key in each line)
I'd like to show/save the results only for those addresses that either have a balance > 0.0 or had any transaction in the past.

Any help appreciated.

Thanks a bunch to all of you, already sent some merits to you. Have a nice weekend everybody, cheers!
citb0in
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
I tried already, see my last post (converting it with the python tool 2to3). However I guess I need to dig further in python to get some basic understanding of the modules and how to use small snippets of all these mentioned libs. Can you guys suggest any good learning platform for Python? I am asking because there are dozens of them and I have no clue which one to choose. I know here on forum are very skilled devs and certainly one of you can give me a good hint. Thanks so much!

Codecademy and W3schools both have good Python tutorials and are free. Some of the others are more data-science oriented and show you how to use Jupyter instead of actual Python.
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!
I tried already, see my last post (converting it with the python tool 2to3). However I guess I need to dig further in python to get some basic understanding of the modules and how to use small snippets of all these mentioned libs. Can you guys suggest any good learning platform for Python? I am asking because there are dozens of them and I have no clue which one to choose. I know here on forum are very skilled devs and certainly one of you can give me a good hint. Thanks so much!
Just don't use Python 2 libraries. All those errors you got (like the following one) were from running Python 2 programs in Python 3.
Code:
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

There is also no reason to port stuff yourself; there are enough working, maintained Python 3 Bitcon libraries.
The ones ETFbitcoin mentioned should all work just fine.



If you want to learn Python, just go through this website. https://www.learnpython.org/
hero member
Activity: 630
Merit: 731
Bitcoin g33k
I tried already, see my last post (converting it with the python tool 2to3). However I guess I need to dig further in python to get some basic understanding of the modules and how to use small snippets of all these mentioned libs. Can you guys suggest any good learning platform for Python? I am asking because there are dozens of them and I have no clue which one to choose. I know here on forum are very skilled devs and certainly one of you can give me a good hint. Thanks so much!
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
It's not hard to see that coinkit is abandonware - Pavol has probably moved on to other things.

If you want, you're able to manually modify the source code yourself to make it python3-compatible.
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
Which way do you suggest to go?

I'd suggest to check different Python library which is still actively developed. Usually i'd recommend these library,
https://github.com/richardkiss/pycoin
https://github.com/1200wd/bitcoinlib
https://github.com/ofek/bit
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Originally I was looking for a Bitcoin toolkit to perform different tasks, like:
- transforming private key to public key (compressed and uncompressed)
- generating BTC addresses from the pubkey
- generating and validating BIP39 mnemonics
- generating random seeds or addresses
- creating new wallets, or restoring existed ones with mnemonic or private key
- receiving wallet info from the blockchain (balance, transactions, fees) either by connecting to a local or remote full-node or by using any blockexplorer on the web

In doing so I came across "coinkit" by the developer Pavol Rusnak through some Google research. The mentioned tool coinkit contains a python script called vagabond which is what I am looking for and I guess it might suit my needs for the 1st steps.

Unfortunately I can't find anything about coinkit on his GIT repository. Then I searched in GIT and found this. I am not sure but I think this is from him (Pavol Rusnak). Unfortunately very old (last commit happened 10 years ago). However I cloned that git repo and tried to run "vagabond.py" but it fails with error code:
Code:
citb0in@blockbuster:~/coinkit$ ./vagabond
Quote
bash: ./vagabond: /usr/bin/python: bad interpreter: No such file or directory
that's true, because
Code:
$ which python; echo $?
will output error level 1. I have no python executable, I am on Ubuntu Jammy Jellyfish with python3 installed
Code:
python -V
Python 3.10.4

I installed python 2to3 and ran
Code:
2to3 coinkit -n -w -o coinkit-new
but the result does not work. It will remove completely ./coinkit-new/vagabond and also some other python scripts that was in the original ./coinkit folder. No clue what I missed on the 2to3 command line, maybe I need some switch like "-f all" or similar? I gave up and went back to the GIT repository SimpleStickyWallet and clicked on the Forks link. Whatever of these forks I click on seems looking same to me, I tried the same stuff with one of these forks but unfortunately no luck either, same results. I found another coinkit GIT repo here which sounds also promising to me but unfortunately this is written in SWIFT which I'm absolutely not familiar with. I would prefer python instead. Here is another coinkit on git, but also outdated. Which way do you suggest to go?

Any help appreciated. Thanks to all!
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
pybtctool is different than pybitcoin.

If yo need pybtctools you can find them here:
https://pypi.org/project/pybtctools/
https://github.com/danvergara/pybtctools

And the error you get is because you are looking to run the python package like it it was a bash command.

The pybitcoin tools should be on:
Code:
~/.local/lib/python3.10/site-packages/pybitcoin/*
and not under
Code:
/usr/local/bin/...

It would be better if you tell us what do you want to use that tool for, maybe we can find another way to it.


It shouldn't be too difficult to make a basic Python3 port, usually all that needs to be done is remove the references to "print", xrange(), and unicode().
Pages:
Jump to: