agran

hero member

Activity: 910

Merit: 1000

Our coin Fuguecoin is very necessary Block Explorer. I now pronounce the bounty for fist Block Explorer.
(Must be able to find the balance of addresses.)

Bounty: 1000 FC (0.05 BTC)

unick

sr. member

Activity: 504

Merit: 254

Quote from: bergi on March 20, 2014, 11:59:14 AM

Code:

#!/usr/bin/python
import subprocess, sys, os
command=["/home/berg/exeblockexp/abe", str(os.getpid())]
subprocess.Popen(command, stdin=sys.stdin).wait()

this is sudo log, if it useful

Code:

Mar 20 02:00:21 noc sudo:   apache : TTY=unknown ; PWD=/var/www/localhost/htdocs/execoin ; USER=berg ; TSID=000028 ; COMMAND=/home/berg/exeblockexp/abe 13410

i can't find something erroneous in this

Well you didn't post your abe-exe.conf, here's mine for reference

Code:

default-loader=no-serve
dbtype MySQLdb
connect-args {"user":"abe","db":"abe","passwd":"PASSWORD"}
upgrade
datadir=[]

you will notice that I don't want abe to load in "server" mode and read the block chain, just want it to connect to the database and read it's content. I used another Abe instance to read the blockchain and update the database.

If you followed the guide, it is mentioned how to add your "abe" user (the one that will run the FastCGI script) to the sudoers file so your apache user can access the FastCGI script without password and be able to run it.

Make sure you edit your sudoers file with visudo and append to it

Code:

# This allows the Apache account (www-data) to run Abe as USER.
www-data ALL=(USER) NOPASSWD: /home/USER/cgi-bin/abe

unick

sr. member

Activity: 504

Merit: 254

Quote from: unick on March 15, 2014, 05:29:10 AM

Check this quote for reference of original issue

So this is a bump and an update at the same time. Can't read the raw data from GrowthCoin's blockchain still.

here's the error again

Code:

root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --no-serve
Opened /root/.GrowthCoin/blk0001.dat
Chain 11 genesis tx: 020000001d0bfa51010000000000000000000000000000000000000000000000000000000000000000ffffffffb904ffff001d020f274caf4a756c792033312c2032303133203132706d204544543a2054686520552e532e2065636f6e6f6d79206772657720312e372520696e20746865207365636f6e6420717561727465722c20616964656420627920736f6c696420636f6e73756d6572207370656e64696e6720616e6420612072616d702d757020696e20627573696e65737320696e766573746d656e742c2074686520676f7665726e6d656e742073616964205765646e65736461792effffffff0100000000000000000000000000
Exception at 368
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': None, 'loader': None, 'conf': None, 'dirname': '/root/.GrowthCoin', 'id': 3L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2789, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 3053, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3185, in import_blkdat
    store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1804, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=d5e4fc54c53a9a20bbe01e70d8b47ce66f15c1bd3504f7e80390bf5601150ded

So I went to see what happens at byte 368.

Code:

root@grw:~/bitcoin-abe# hexdump -n 368 -C ~/.GrowthCoin/blk0001.dat
00000000  a5 ef db fd 69 01 00 00  01 00 00 00 00 00 00 00  |....i...........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 bf f2 79 9b  |..............y.|
00000030  bf ec 52 41 32 ea 94 54  5f 53 83 7c 5c eb 46 26  |..RA2..T_S.|\.F&|
00000040  b6 3d 18 70 76 01 0c cb  96 10 53 14 31 0b fa 51  |.=.pv.....S.1..Q|
00000050  ff ff 0f 1e ba 6e 15 01  01 02 00 00 00 1d 0b fa  |.....n..........|
00000060  51 01 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 ff ff ff ff b9 04  ff ff 00 1d 02 0f 27 4c  |..............'L|
00000090  af 4a 75 6c 79 20 33 31  2c 20 32 30 31 33 20 31  |.July 31, 2013 1|
000000a0  32 70 6d 20 45 44 54 3a  20 54 68 65 20 55 2e 53  |2pm EDT: The U.S|
000000b0  2e 20 65 63 6f 6e 6f 6d  79 20 67 72 65 77 20 31  |. economy grew 1|
000000c0  2e 37 25 20 69 6e 20 74  68 65 20 73 65 63 6f 6e  |.7% in the secon|
000000d0  64 20 71 75 61 72 74 65  72 2c 20 61 69 64 65 64  |d quarter, aided|
000000e0  20 62 79 20 73 6f 6c 69  64 20 63 6f 6e 73 75 6d  | by solid consum|
000000f0  65 72 20 73 70 65 6e 64  69 6e 67 20 61 6e 64 20  |er spending and |
00000100  61 20 72 61 6d 70 2d 75  70 20 69 6e 20 62 75 73  |a ramp-up in bus|
00000110  69 6e 65 73 73 20 69 6e  76 65 73 74 6d 65 6e 74  |iness investment|
00000120  2c 20 74 68 65 20 67 6f  76 65 72 6e 6d 65 6e 74  |, the government|
00000130  20 73 61 69 64 20 57 65  64 6e 65 73 64 61 79 2e  | said Wednesday.|
00000140  ff ff ff ff 01 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 1d 74 65 78 74 3a  47 72 6f 77 74 68 43 6f  |...text:GrowthCo|
00000160  69 6e 20 67 65 6e 65 73  69 73 20 62 6c 6f 63 6b  |in genesis block|
00000170

at byte 369

Code:

root@grw:~/bitcoin-abe# hexdump -n 369 -C ~/.GrowthCoin/blk0001.dat
00000000  a5 ef db fd 69 01 00 00  01 00 00 00 00 00 00 00  |....i...........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 bf f2 79 9b  |..............y.|
00000030  bf ec 52 41 32 ea 94 54  5f 53 83 7c 5c eb 46 26  |..RA2..T_S.|\.F&|
00000040  b6 3d 18 70 76 01 0c cb  96 10 53 14 31 0b fa 51  |.=.pv.....S.1..Q|
00000050  ff ff 0f 1e ba 6e 15 01  01 02 00 00 00 1d 0b fa  |.....n..........|
00000060  51 01 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 ff ff ff ff b9 04  ff ff 00 1d 02 0f 27 4c  |..............'L|
00000090  af 4a 75 6c 79 20 33 31  2c 20 32 30 31 33 20 31  |.July 31, 2013 1|
000000a0  32 70 6d 20 45 44 54 3a  20 54 68 65 20 55 2e 53  |2pm EDT: The U.S|
000000b0  2e 20 65 63 6f 6e 6f 6d  79 20 67 72 65 77 20 31  |. economy grew 1|
000000c0  2e 37 25 20 69 6e 20 74  68 65 20 73 65 63 6f 6e  |.7% in the secon|
000000d0  64 20 71 75 61 72 74 65  72 2c 20 61 69 64 65 64  |d quarter, aided|
000000e0  20 62 79 20 73 6f 6c 69  64 20 63 6f 6e 73 75 6d  | by solid consum|
000000f0  65 72 20 73 70 65 6e 64  69 6e 67 20 61 6e 64 20  |er spending and |
00000100  61 20 72 61 6d 70 2d 75  70 20 69 6e 20 62 75 73  |a ramp-up in bus|
00000110  69 6e 65 73 73 20 69 6e  76 65 73 74 6d 65 6e 74  |iness investment|
00000120  2c 20 74 68 65 20 67 6f  76 65 72 6e 6d 65 6e 74  |, the government|
00000130  20 73 61 69 64 20 57 65  64 6e 65 73 64 61 79 2e  | said Wednesday.|
00000140  ff ff ff ff 01 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 1d 74 65 78 74 3a  47 72 6f 77 74 68 43 6f  |...text:GrowthCo|
00000160  69 6e 20 67 65 6e 65 73  69 73 20 62 6c 6f 63 6b  |in genesis block|
00000170  00                                                |.|
00000171

and byte 374 --- @ byte 370 we get the first byte of the magic number

Code:

root@grw:~/bitcoin-abe# hexdump -n 374 -C ~/.GrowthCoin/blk0001.dat
00000000  a5 ef db fd 69 01 00 00  01 00 00 00 00 00 00 00  |....i...........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 bf f2 79 9b  |..............y.|
00000030  bf ec 52 41 32 ea 94 54  5f 53 83 7c 5c eb 46 26  |..RA2..T_S.|\.F&|
00000040  b6 3d 18 70 76 01 0c cb  96 10 53 14 31 0b fa 51  |.=.pv.....S.1..Q|
00000050  ff ff 0f 1e ba 6e 15 01  01 02 00 00 00 1d 0b fa  |.....n..........|
00000060  51 01 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 ff ff ff ff b9 04  ff ff 00 1d 02 0f 27 4c  |..............'L|
00000090  af 4a 75 6c 79 20 33 31  2c 20 32 30 31 33 20 31  |.July 31, 2013 1|
000000a0  32 70 6d 20 45 44 54 3a  20 54 68 65 20 55 2e 53  |2pm EDT: The U.S|
000000b0  2e 20 65 63 6f 6e 6f 6d  79 20 67 72 65 77 20 31  |. economy grew 1|
000000c0  2e 37 25 20 69 6e 20 74  68 65 20 73 65 63 6f 6e  |.7% in the secon|
000000d0  64 20 71 75 61 72 74 65  72 2c 20 61 69 64 65 64  |d quarter, aided|
000000e0  20 62 79 20 73 6f 6c 69  64 20 63 6f 6e 73 75 6d  | by solid consum|
000000f0  65 72 20 73 70 65 6e 64  69 6e 67 20 61 6e 64 20  |er spending and |
00000100  61 20 72 61 6d 70 2d 75  70 20 69 6e 20 62 75 73  |a ramp-up in bus|
00000110  69 6e 65 73 73 20 69 6e  76 65 73 74 6d 65 6e 74  |iness investment|
00000120  2c 20 74 68 65 20 67 6f  76 65 72 6e 6d 65 6e 74  |, the government|
00000130  20 73 61 69 64 20 57 65  64 6e 65 73 64 61 79 2e  | said Wednesday.|
00000140  ff ff ff ff 01 00 00 00  00 00 00 00 00 00 00 00  |................|
00000150  00 00 1d 74 65 78 74 3a  47 72 6f 77 74 68 43 6f  |...text:GrowthCo|
00000160  69 6e 20 67 65 6e 65 73  69 73 20 62 6c 6f 63 6b  |in genesis block|
00000170  00 a5 ef db fd 08                                 |......|
00000176

So my guess, and I'm no expert nor do I know how to fix this, is that Abe can't read the lenght of the block properly? or like the error says, the merkle root is off... I'm willing to help searching but I'm a bit over my head with this so a few pointers in the right direction would be appreciated.

Thanks

bergi

newbie

Activity: 8

Merit: 0

Quote from: unick on March 19, 2014, 06:21:01 PM

can you post your abe-exe.conf setings?

also you need to setup you apache server to run Abe.

Did you follow this guide ?
https://github.com/bitcoin-abe/bitcoin-abe/blob/master/README-FASTCGI.txt

Yep, read this and make everything as described.

I found, that sudo make some troubles, hard to find it. So, i bypass it, truncate abe.fcgi to

Code:

#!/usr/bin/python
import subprocess, sys, os
command=["/home/berg/exeblockexp/abe", str(os.getpid())]
subprocess.Popen(command, stdin=sys.stdin).wait()

this is sudo log, if it useful

Code:

Mar 20 02:00:21 noc sudo:   apache : TTY=unknown ; PWD=/var/www/localhost/htdocs/execoin ; USER=berg ; TSID=000028 ; COMMAND=/home/berg/exeblockexp/abe 13410

i can't find something erroneous in this

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: elcoco on March 19, 2014, 08:30:34 PM

Quote from: John Tobey on March 19, 2014, 10:52:59 AM

Quote from: agran on March 19, 2014, 05:47:41 AM

Shows only the first block. The thing I do not understand.

When Abe loads many blocks but shows only the genesis block, it means that Abe is using the wrong algorithm to calculate the block header hash as it appears in the next block's hashPrev field. By default, Abe uses double-SHA256, the same as Bitcoin. Many coins use scrypt. I would like to make this selectable by datadir=[{...,"hashPrev":"scrypt"}], but for now it requires a few lines of code. Look for "NovaCoin" in the sources for an example.

is there a way to get a raw block before abe splits it and puts it in the database, maybe there are new fields added in this coin, or the data got moved around, I can then change the abe code to make it work

See how ds_parse_block and ds_parse_block_header are overridden in Abe/Chain.py. However, note that this module is new and under heavy development.

elcoco

member

Activity: 111

Merit: 10

Quote from: John Tobey on March 19, 2014, 10:52:59 AM

Quote from: agran on March 19, 2014, 05:47:41 AM

Shows only the first block. The thing I do not understand.

When Abe loads many blocks but shows only the genesis block, it means that Abe is using the wrong algorithm to calculate the block header hash as it appears in the next block's hashPrev field. By default, Abe uses double-SHA256, the same as Bitcoin. Many coins use scrypt. I would like to make this selectable by datadir=[{...,"hashPrev":"scrypt"}], but for now it requires a few lines of code. Look for "NovaCoin" in the sources for an example.

is there a way to get a raw block before abe splits it and puts it in the database, maybe there are new fields added in this coin, or the data got moved around, I can then change the abe code to make it work

unick

sr. member

Activity: 504

Merit: 254

is it possible to get address balance from command line query?

I get the following error

Code:

python -m Abe.abe --query /chain/FairCoin/q/addressbalance/fc44xJawRecE9xrZwGHMUtAfxWeS3CJqtP
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2374, in 
    sys.exit(main(sys.argv[1:]))
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2370, in main
    serve(store)
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2236, in serve
    }, start_response)
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 276, in __call__
    handler(page)
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 415, in handle_chain
    chain = abe.chain_lookup_by_name(symbol)
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 397, in chain_lookup_by_name
    ret = abe.store.get_chain_by_name(symbol)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 819, in get_chain_by_name
    return store.chains_by.name.get(name, None)
AttributeError: 'DataStore' object has no attribute 'chains_by'

maybe I have the syntax wrong

unick

sr. member

Activity: 504

Merit: 254

bergi

newbie

Activity: 8

Merit: 0

Guys, help me, please.
Trying to up abe with fastcgi, but all output instead to go to browser
going to stderr, like this

Code:

2014-03-19 22:34:07,163 [8311:MainThread] Abe.DataStore INFO - block_tx 53355 138881
2014-03-19 22:34:07,165 [8311:MainThread] Abe.DataStore DEBUG - commit
2014-03-19 22:34:07,169 [8311:MainThread] Abe.DataStore INFO - block_tx 53356 138882
2014-03-19 22:34:07,171 [8311:MainThread] Abe.DataStore DEBUG - commit
2014-03-19 22:34:07,178 [8311:MainThread] Abe.DataStore INFO - block_tx 53357 138883
2014-03-19 22:34:07,178 [8311:MainThread] Abe.DataStore INFO - block_tx 53357 138884
2014-03-19 22:34:07,186 [8311:MainThread] Abe.DataStore DEBUG - commit
2014-03-19 22:34:07,192 [8311:MainThread] Abe.DataStore INFO - block_tx 53358 138885
2014-03-19 22:34:07,192 [8311:MainThread] Abe.DataStore INFO - block_tx 53358 138886
2014-03-19 22:34:07,196 [8311:MainThread] Abe.DataStore DEBUG - commit
2014-03-19 22:34:07,196 [8311:MainThread] __main__ INFO - Abe initialized.
WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI!
WSGIServer: missing FastCGI param SERVER_NAME required by WSGI!
WSGIServer: missing FastCGI param SERVER_PORT required by WSGI!
WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI!
2014-03-19 22:34:07,199 [8311:MainThread] Abe.DataStore INFO - Opened /home/berg/.execoin/blocks/blk00000.dat
Status: 200 OK
Content-type: text/html; charset=utf-8
Cache-Control: max-age=30

href="static/abe.css" />

Abe Search

Abe Search

Search by address, block number or hash, transaction or public key hash, or chain name:

Currency	Code	Block	Time	Started	Age (days)	Coins Created	Avg Coin Age	% CoinDD
Execoin	EXE	53344	2014-03-19 19:33:38	2014-01-29	49.7427	2667250	9.16489	26.5331%

API (machine-readable pages)

Powered by Abe

Tips appreciated!
BTC
NMC

Configuring like in README-FASTCGI.txt
Also fell in trouble with PATH_INFO

Code:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flup/server/fcgi_base.py", line 558, in run
    protocolStatus, appStatus = self.server.handler(self)
  File "/usr/lib/python2.7/site-packages/flup/server/fcgi_base.py", line 1118, in handler
    result = self.application(environ, start_response)
  File "/usr/lib/python2.7/site-packages/Abe/abe.py", line 226, in __call__
    if abe.fix_path_info(env):
  File "/usr/lib/python2.7/site-packages/Abe/abe.py", line 1940, in fix_path_info
    if pi[-1] != '/' and env['PATH_INFO'][-1] == '/':
IndexError: string index out of range

But fixed it with

Code:

#! /bin/sh
[b]PATH_INFO='/'[/b] PYTHONUNBUFFERED=1 exec python -m Abe.abe --config /home/berg/cgi-bin/abe-exe.conf --static-path static/ --watch-pid="$1"

Still dont sure how it correct...

agran

hero member

Activity: 910

Merit: 1000

Fuguecoin is very necessary Block Explorer. I now pronounce the bounty for fist Block Explorer:

https://bitcointalksearch.org/topic/m.5786707

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: agran on March 19, 2014, 05:47:41 AM

Shows only the first block. The thing I do not understand.

When Abe loads many blocks but shows only the genesis block, it means that Abe is using the wrong algorithm to calculate the block header hash as it appears in the next block's hashPrev field. By default, Abe uses double-SHA256, the same as Bitcoin. Many coins use scrypt. I would like to make this selectable by datadir=[{...,"hashPrev":"scrypt"}], but for now it requires a few lines of code. Look for "NovaCoin" in the sources for an example.

agran

hero member

Activity: 910

Merit: 1000

Coin: [FC] Fuguecoin
https://bitcointalksearch.org/topic/annfc-fuguecoin-fugue256-hash-launched-no-pre-m-cpugpu-minable-519138
https://github.com/fuguecoin/fuguecoin

http://agran.net:2750/chains

abe-my.conf

dbtype MySQLdb
connect-args {"user":"abe","db":"abe","passwd":"*****"}
upgrade
port 2750
host agran.net

datadir = [{
"dirname": "/root/.fuguecoin",
"chain": "Fuguecoin",
"code3": "FC",
"address_version": "\u0023",
"conf": "fuguecoin.conf"
}]

DataStore.py
CHAIN_CONFIG = [
{"chain":"Fuguecoin",
"code3":"FC", "address_version":"\x23", "magic":"\x0b\x11\x09\x07"}
]

python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
python -m Abe.abe --config abe-my.conf

http://agran.net:2750/chains

Shows only the first block. The thing I do not understand.

gdassori

hero member

Activity: 980

Merit: 1002

John, check the edit2, previous post: I included it. About the strange transaction: it's bread for gavin's teeths but since the block is old, I guess he already know about this.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: gdassori on March 18, 2014, 05:27:05 AM

Enabled log-sql:

Quote

EXEC:
   INSERT INTO tx (tx_id, tx_hash, tx_version, tx_lockTime, tx_size)
   VALUES (%s, %s, %s, %s, %s) (88379L, 'c333a53f0174166236e341af9cad795d21578fb87ad7a1b6d2cf8aa9c722083c', 1, 0, 998171)
EXEC: INSERT INTO txout_seq () VALUES () ()
EXEC: SELECT LAST_INSERT_ID() ()
FETCH: (115923L,)
EXEC:
   INSERT INTO txout (
   txout_id, tx_id, txout_pos, txout_value,
   txout_scriptPubKey, pubkey_id
   ) VALUES (%s, %s, %s, %s, %s, %s) (115923L, 88379L, 0, 0, 'fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff ffff
fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff fffff [...cut... many, many, MANY, 'f']

This is the query that screws everything up

Edit: WELL, found the problem: http://blockexplorer.com/testnet/tx/c333a53f0174166236e341af9cad795d21578fb87ad7a1b6d2cf8aa9c722083c
a crazy block in testnet blockchain. Now I have to figure out how to skip this block.

I don't know when I'll have time to fix this, but I'm glad we see it on testnet before main. Thanks.

gdassori

hero member

Activity: 980

Merit: 1002

Hi

Quote

commit
block_tx 54506 88356
block_tx 54507 88357
Exception at 22941369
Failed to catch up {'blkfile_offset': 21942607, 'blkfile_number': 100000, 'chain_id': 2, 'loader': None, 'dirname': u'/home/guido/testnet/testnet3', 'id': 1L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2689, in catch_up
   store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2948, in catch_up_dir
   store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3072, in import_blkdat
   store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1784, in import_block
   tx['tx_id'] = store.import_tx(tx, pos == 0)
  File "Abe/DataStore.py", line 2248, in import_tx
   store.binin(txout['scriptPubKey']), pubkey_id))
  File "Abe/DataStore.py", line 521, in sql
   store._execute(cached, params)
  File "Abe/DataStore.py", line 498, in _execute
   store.cursor.execute(stmt, params)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
   self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
   raise errorclass, errorvalue
OperationalError: (2006, 'MySQL server has gone away')
Reconnecting after rollback error: (2006, 'MySQL server has gone away')
Reconnecting to database.

happened twice, block_tx 54507 88357
blockchain is bitcoin testnet.
mysql server is still there.
ideas?

Enabled log-sql:

Quote

EXEC:
   INSERT INTO tx (tx_id, tx_hash, tx_version, tx_lockTime, tx_size)
   VALUES (%s, %s, %s, %s, %s) (88379L, 'c333a53f0174166236e341af9cad795d21578fb87ad7a1b6d2cf8aa9c722083c', 1, 0, 998171)
EXEC: INSERT INTO txout_seq () VALUES () ()
EXEC: SELECT LAST_INSERT_ID() ()
FETCH: (115923L,)
EXEC:
   INSERT INTO txout (
   txout_id, tx_id, txout_pos, txout_value,
   txout_scriptPubKey, pubkey_id
   ) VALUES (%s, %s, %s, %s, %s, %s) (115923L, 88379L, 0, 0, 'fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff ffff
fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff fffff [...cut... many, many, MANY, 'f']

This is the query that screws everything up

Edit: WELL, found the problem: http://blockexplorer.com/testnet/tx/c333a53f0174166236e341af9cad795d21578fb87ad7a1b6d2cf8aa9c722083c (warning, it could crash or hang the browser)
a crazy block in testnet blockchain. Now I have to figure out how to skip this block.

Edit2: Asked myself "why skipping the block if you can include it, also if its crazy?". Answer was setting max_allowed_packet in my.cnf from 1MB to 8MB. Now it works.

wing_hk

full member

Activity: 182

Merit: 100

Well I have a ABE working well for a particular Altcoin now.

Is there any existing data mining tool for ABE? (i.e. largest coin holder distribution etc. ) Huh

adversor

member

Activity: 112

Merit: 10

John Tobey, great job with Abe. Thanks for sharing this work with the community!

I'm trying to set up a website with several altcoin block explorers. The first one I tried was Klondikecoin, this works already like a charm.

Now I'm trying Noblecoin, but get same error, same as unick, just one post above.

Code:

$ python -m Abe.abe --config NobleCoin.conf --commit-bytes 100000 --no-serve
Opened /home/adversor/.noblecoin/blk0001.dat
Chain 2 genesis tx: 02000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4604ffff001d01043e36204a616e20323031353a20434e4e3a204d696c6c696f6e7320696e20552e532e20627261636520666f72207265636f7264206465657020667265657a65ffffffff010000000000000000434104dcba12349012341234900abcd12223abcd455abcd77788abcd000000aaaaabbbbbcccccdddddeeeeeff00ff00ff00ff001234567890abcdef0022446688abc11ac00000000
Exception at 286
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 2, 'loader': None, 'conf': u'noblecoin.conf', 'dirname': u'/home/adversor/.noblecoin', 'id': 3L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2686, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2950, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3082, in import_blkdat
    store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1804, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=dbf8ee8b91b5f03d7786cb1d58f2588538b2d48d8c23a7913db5aac7bb58c481

After I have commented out the Merkle root verification in DataStore.py, it reads 214 blocks from chain and this is the output:

Code:

.....
Skipped 1 bytes at block end
block_tx 212 212
Skipped 1 bytes at block end
block_tx 213 213
Skipped 1 bytes at block end
block_tx 214 214
Skipped 1 bytes at block end
Exception at 60825391604920033
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 2, 'loader': None, 'conf': u'noblecoin.conf', 'dirname': u'/home/adversor/.noblecoin', 'id': 4L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2686, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2950, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3072, in import_blkdat
    b = chain.ds_parse_block(ds)
  File "Abe/Chain.py", line 59, in ds_parse_block
    d['transactions'].append(chain.ds_parse_transaction(ds))
  File "Abe/Chain.py", line 52, in ds_parse_transaction
    return deserialize.parse_Transaction(ds)
  File "Abe/deserialize.py", line 90, in parse_Transaction
    d['txIn'].append(parse_TxIn(vds))
  File "Abe/deserialize.py", line 46, in parse_TxIn
    d['sequence'] = vds.read_uint32()
  File "Abe/BCDataStream.py", line 71, in read_uint32
    def read_uint32 (self): return self._read_num('  File "Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 4 bytes

I'm using

Code:

default-loader = blkfile

Someone has a solution?

unick

sr. member

Activity: 504

Merit: 254

Quote from: unick on March 14, 2014, 05:10:16 PM

Ok I'm still trying to figure this thing out... I've manage to get passed the invalid address error (not sure why but I tried putting the chain specifcation into the config file instead of the python file). But now moving forward on this I get yet another error!

Code:

root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/bitcoin-abe/Abe/abe.py", line 2276, in 
    sys.exit(main(sys.argv[1:]))
  File "/root/bitcoin-abe/Abe/abe.py", line 2270, in main
    store = make_store(args)
  File "/root/bitcoin-abe/Abe/abe.py", line 153, in make_store
    store = DataStore.new(args)
  File "Abe/DataStore.py", line 3446, in new
    return DataStore(args)
  File "Abe/DataStore.py", line 165, in __init__
    store.init_conn()
  File "Abe/DataStore.py", line 226, in init_conn
    store.conn = store.connect()
  File "Abe/DataStore.py", line 237, in connect
    conn = store._connect(cargs)
  File "Abe/DataStore.py", line 269, in _connect
    return store.module.connect(**cargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'abe'@'localhost' (using password: YES)")
root@tmp:~/bitcoin-abe# nano /home/abe/abe-mysql.conf
root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:536: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
Abe/DataStore.py:536: Warning: Converting column 'test_varbit' from VARCHAR to TEXT
  store.cursor.execute(stmt)
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
Abe/DataStore.py:536: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Assigned chain_id 1 to GrowthCoin
Opened /root/.GrowthCoin/blk0001.dat
Chain 1 genesis tx: 020000001d0bfa51010000000000000000000000000000000000000000000000000000000000000000ffffffffb904ffff001d020f274caf4a756c792033312c2032303133203132706d204544543a2054686520552e532e2065636f6e6f6d79206772657720312e372520696e20746865207365636f6e6420717561727465722c20616964656420627920736f6c696420636f6e73756d6572207370656e64696e6720616e6420612072616d702d757020696e20627573696e65737320696e766573746d656e742c2074686520676f7665726e6d656e742073616964205765646e65736461792effffffff0100000000000000000000000000
Exception at 368
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 1L, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 1L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2687, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2951, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3083, in import_blkdat
    store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1805, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=d5e4fc54c53a9a20bbe01e70d8b47ce66f15c1bd3504f7e80390bf5601150ded

I've read to comment out in DataStore.py

Code:

# Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)

But as you can image if I post this... it didn't work!

It gave me a few of these

Code:

Skipped 73 bytes at block end
block_tx 970 970
Skipped 71 bytes at block end
block_tx 971 971
Skipped 72 bytes at block end
block_tx 972 972
Skipped 72 bytes at block end
Exception at 1677721600398062
Failed to catch up {'blkfile_offset': 206488, 'blkfile_number': 1, 'chain_id': 1, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 2L}
Traceback (most recent call last):

Then

Code:

error: unpack_from requires a buffer of at least 4 bytes

Witch I had earlier in my attempts to get Abe read the blockchain. It looks to me as the magic bytes are causing troubles again.

Any inputs/help/solutions on this would be appreciated Wink

lunalauncher

newbie

Activity: 15

Merit: 0

Abe is giving me the error

Code:

block_tx 290732 34916394
block_tx 290732 34916395
block_tx 290732 34916396
block_tx 290732 34916397
block_tx 290732 34916398
block_tx 290732 34916399
commit
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat

Made bitcoind do a reindex but it's still the same. Any ideas?

crabel

newbie

Activity: 18

Merit: 0

I developed a method for analyzing the bitcoin blockchain using very simple analytical tools. While my initial work was with bitcoin it works with altcoins in general. I wrote a post describing the method and theory. In it I posted the MatLab files I used to perform the analysis.

I am not familiar with programing in general. I do not see any reason why these functions and methods can't be incorporated into abe's source code. They provide powerful description of these currencies and are very easily implemented.

I used abe to parse the bitcoin blockchain and greatly appreciate the effort that you put into developing the source code. Please take my work as a thank you for the work that you did.

http://statisticaleconomics.org/2014/03/13/quantifying-the-value-of-bitcoin-2/

Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff - page 18. (Read 221044 times)

src="static/logo32.png" alt="Abe logo" /> Abe Search