Pages:
Author

Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff - page 27. (Read 220986 times)

sr. member
Activity: 271
Merit: 250
Hey John, I scanned a coin in with the wrong address version, and now addresses appear incorrectly. I would reload te whole database, but there are 36 coins and it would take over 24 hours. Is there a way to drop a single chain and reload it?
hero member
Activity: 532
Merit: 500
Hello John w/b just need to ask you where would I get the api for the explorer
http://blockexplorer.coinworld.us/

http://blockexplorer.coinworld.us/chain/AlphaCoin/q
etc. replacing AlphaCoin with coin name.

Thanks John I hope that Is all they need
hero member
Activity: 481
Merit: 529
Hello John w/b just need to ask you where would I get the api for the explorer
http://blockexplorer.coinworld.us/

http://blockexplorer.coinworld.us/chain/AlphaCoin/q
etc. replacing AlphaCoin with coin name.
hero member
Activity: 532
Merit: 500
Hello John w/b just need to ask you where would I get the api for the explorer
http://blockexplorer.coinworld.us/
sr. member
Activity: 448
Merit: 250
Hi John,

good to hear from you Smiley

Ok, I see. But I dont see any problem then:

Copy'n Paste:

Quote
$ sudo python -m Abe.abe --config /usr/local/mysql/my-abe.cnf --commit-bytes 100000 --no-serve
Password:
Skipping datadir /Users/btcinvest/Desktop/bitcoin-datadir: [Errno 2] No such file or directory: u'/Users/btcinvest/Desktop/bitcoin-datadir/blk0001.dat'

Btw. whats that 'u' before the path begins? Typo in your source I guess?

Copy'n Paste:

Quote
blocks btcinvest$ ls
blk00000.dat   blk00013.dat   blk00026.dat   blk00039.dat   blk00052.dat   blk00065.dat   rev00005.dat   rev00018.dat   rev00031.dat   rev00044.dat   rev00057.dat   rev00070.dat
blk00001.dat   blk00014.dat   blk00027.dat   blk00040.dat   blk00053.dat   blk00066.dat   rev00006.dat   rev00019.dat   rev00032.dat   rev00045.dat   rev00058.dat   rev00071.dat
blk00002.dat   blk00015.dat   blk00028.dat   blk00041.dat   blk00054.dat   blk00067.dat   rev00007.dat   rev00020.dat   rev00033.dat   rev00046.dat   rev00059.dat
blk00003.dat   blk00016.dat   blk00029.dat   blk00042.dat   blk00055.dat   blk00068.dat   rev00008.dat   rev00021.dat   rev00034.dat   rev00047.dat   rev00060.dat
blk00004.dat   blk00017.dat   blk00030.dat   blk00043.dat   blk00056.dat   blk00069.dat   rev00009.dat   rev00022.dat   rev00035.dat   rev00048.dat   rev00061.dat
blk00005.dat   blk00018.dat   blk00031.dat   blk00044.dat   blk00057.dat   blk00070.dat   rev00010.dat   rev00023.dat   rev00036.dat   rev00049.dat   rev00062.dat
blk00006.dat   blk00019.dat   blk00032.dat   blk00045.dat   blk00058.dat   blk00071.dat   rev00011.dat   rev00024.dat   rev00037.dat   rev00050.dat   rev00063.dat
blk00007.dat   blk00020.dat   blk00033.dat   blk00046.dat   blk00059.dat   index      rev00012.dat   rev00025.dat   rev00038.dat   rev00051.dat   rev00064.dat
blk00008.dat   blk00021.dat   blk00034.dat   blk00047.dat   blk00060.dat   rev00000.dat   rev00013.dat   rev00026.dat   rev00039.dat   rev00052.dat   rev00065.dat
blk00009.dat   blk00022.dat   blk00035.dat   blk00048.dat   blk00061.dat   rev00001.dat   rev00014.dat   rev00027.dat   rev00040.dat   rev00053.dat   rev00066.dat
blk00010.dat   blk00023.dat   blk00036.dat   blk00049.dat   blk00062.dat   rev00002.dat   rev00015.dat   rev00028.dat   rev00041.dat   rev00054.dat   rev00067.dat
blk00011.dat   blk00024.dat   blk00037.dat   blk00050.dat   blk00063.dat   rev00003.dat   rev00016.dat   rev00029.dat   rev00042.dat   rev00055.dat   rev00068.dat
blk00012.dat   blk00025.dat   blk00038.dat   blk00051.dat   blk00064.dat   rev00004.dat   rev00017.dat   rev00030.dat   rev00043.dat   rev00056.dat   rev00069.dat

As you can see my conf-file is in the mysql folder, but I guess this doesnt matter.
Also the bitcoin-datadir was customized by me to get rid of the blank space in the path, who knows.

EDIT: Abe is installed in /Library/Python/2.7/site-packages/Abe
hero member
Activity: 481
Merit: 529
Hi, I've been away the last few days and will take a while to catch up.

Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin/blocks
Leave /blocks off.  This should be:
Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin


If I leave it off, the error message is "../Application Support/Bitcoin/blck0001.dat not found".
In case I use /blocks, the error message points correctly to "../Application Support/Bitcoin/blocks/blck0001.dat".
Now I am even more confused.

While ABE searches after blck0001.dat, blck00001.dat is existent, and would be perhaps the correct file?
Is this a misconfiguration?

Old versions of Bitcoin start with blk0001.dat, and new (0.8.x) ones start with blocks/blk00000.dat.  To accommodate all known versions, including users who upgrade, Abe first searches in datadir for the blk*.dat series, and then in the blocks subdirectory for blk00000.dat.  Since you use a new bitcoin-qt, it should not find blk0001.dat but should continue and load blocks/blk00000.dat.

The blockchain loading is still at the very beginning, I just ran bitcoin-qt once, as listed in the read me.

How many blk*.dat files are in  "../Application Support/Bitcoin/blocks" ?  Does Abe exit after the error message about blck0001.dat?  Is that a typo, and did you mean blk0001.dat?  Please paste the last several lines of output if Abe exits before it finishes loading.
sr. member
Activity: 455
Merit: 251
Yes, you're right, Elephantcoin does have transaction messages.  I had a feeling that might be something to do with the problems I've been having.  Is there any fix that might allow an Abe Explorer to be set up?
sr. member
Activity: 271
Merit: 250
If I remember correctly ElephantCoin incorporates transaction messages like Florincoin ... which results in some blocks which confuse Abe.
sr. member
Activity: 455
Merit: 251
Hello John, I'm trying to set up an Abe block explorer for ElephantCoin, but when trying to do the initial load of:
Code:
    python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
It fails with the error

Code:
error:
failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': Decimal('9'), 'loader': None, 'dirname': u'/root/.elephantcoin', 'id': 5L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2596, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2854, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 2982, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1750, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=13405c6ebfa325bf97847d3b998341b4f3db5805c55bd413543cccb74e262258

I've tried commenting out in datastore.py:
Code:

 # Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)

 
This allows the data to start loading but it stops after a short while

Code:

Skipped 1 bytes at block end
block_tx 641 644
Skipped 1 bytes at block end
block_tx 642 645
Skipped 1 bytes at block end
block_tx 643 646
Skipped 1 bytes at block end
block_tx 644 647
block_tx 644 648
Skipped 46 bytes at block end
Exception at 13428709007330245116
Failed to catch up {'blkfile_offset': 104283, 'blkfile_number': 1, 'chain_id': 8L, 'loader': None, 'dirname': u'/root/.elephantcoin', 'id': 1L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2637, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2895, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3022, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "Abe/DataStore.py", line 3053, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "Abe/deserialize.py", line 90, in parse_Transaction
    d['txOut'].append(parse_TxOut(vds))
  File "Abe/deserialize.py", line 65, in parse_TxOut
    d['value'] = vds.read_int64()
  File "Abe/BCDataStream.py", line 72, in read_int64
    def read_int64  (self): return self._read_num('  File "Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
OverflowError: Python int too large to convert to C long

The resulting block explorer (or part of one up to block 643 only, which is where it stops) can be seen on my test page at http://exploretheblocks.com:27700/chain/ElephantCoin?count=500.  (I've changed everything to port 27700 for testing purposes).  This part-block-explorer seems to have a number of very odd-looking blocks, for example block 513, 515 and 517 and the "final" block 643

My abe-my.conf:
Code:
    dbtype MySQLdb
    connect-args {"user":"abe","db":"abetest","passwd":"################################"}
    upgrade
    port 27700
    host exploretheblocks.com


datadir += [{
  "dirname" : "/root/.elephantcoin",
  "chain": "ElephantCoin",
  "code3": "ELP",
  "address_version": "\u005c"
}]

ignore-bit8-chains = ["Bitcoin", "Testnet", "ElephantCoin"]


If you can offer any suggestions I'd be really grateful as I really like the way Abe displays its output, and would really like to get an Abe Explorer set up for the Elephants! Smiley

hero member
Activity: 481
Merit: 529
Now after about 14 days of importing it's finally done Tongue Anyway everything seems to be working just fine I only got one small problem. It's only importing new blocks when I load a new webpage in the Abe interface. Should it really be like that? So if I for example stop using the web interface for about 1 hour then when I request one page load it will star importing the 6 missing blocks. Why doesn't Abe just import the block as soon my bitcoin-qt instance accepts the new block?

Yup, this is an underdocumented quirk.  There are many ways to keep Abe up to date, an no one fits every need.

You can use a script like abe_loader.  Actually, abe_loader is more complex than needed.  It watches bitcoind's debug.log for hints that a block has arrived.  Now that bitcoind supports blocknotify, we could just add something like this to bitcoin.conf:

Code:
blocknotify=python -m Abe.abe --config /etc/abe/abe.conf --no-serve

Another strategy is to run in a loop, like this:

Code:
while true; do python -m Abe.abe --config abe.conf --no-serve; sleep 10; done

I used to have a cron job that just loaded the homepage, serving as both a system monitor and a blockchain updater:

Code:
* * * * * /usr/bin/wget -q -O /dev/null http://john-edwin-tobey.org:2750/

One more question is there a way to check so the information in the mysql database is intact? (That does not take 14 days)

A complete check of all invariants does not exist.  It would be a nice feature.  Abe/verify.py does a small fraction of the work, calculating Merkle roots from stored transaction hashes.  I haven't tested it lately.
newbie
Activity: 24
Merit: 0
Now after about 14 days of importing it's finally done Tongue Anyway everything seems to be working just fine I only got one small problem. It's only importing new blocks when I load a new webpage in the Abe interface. Should it really be like that? So if I for example stop using the web interface for about 1 hour then when I request one page load it will star importing the 6 missing blocks. Why doesn't Abe just import the block as soon my bitcoin-qt instance accepts the new block?



My config:
Code:
dbtype MySQLdb
connect-args {"user":"****","db":"abe","port":3306,"host":"127.0.0.1"}
#datadir =C:\****
default-loader =rpc
datadir [{"dirname":"C:\\*****","chain":"Bitcoin","loader":"rpc"}]
upgrade
port 2750

My os:
Windows 7 64bit

Also note that I edited the database manually to get rpc to work. Like someone else explained in this thread.








One more question is there a way to check so the information in the mysql database is intact? (That does not take 14 days)
sr. member
Activity: 448
Merit: 250
Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin/blocks
Leave /blocks off.  This should be:
Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin


If I leave it off, the error message is "../Application Support/Bitcoin/blck0001.dat not found".
In case I use /blocks, the error message points correctly to "../Application Support/Bitcoin/blocks/blck0001.dat".
Now I am even more confused.

While ABE searches after blck0001.dat, blck00001.dat is existent, and would be perhaps the correct file?
Is this a misconfiguration?

The blockchain loading is still at the very beginning, I just ran bitcoin-qt once, as listed in the read me.
hero member
Activity: 481
Merit: 529
Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin/blocks
Leave /blocks off.  This should be:
Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin
sr. member
Activity: 448
Merit: 250
I got the libs running now. I had to reinstall everything in 64bit mode, I wasn't aware that 32bit won't work.

Now I've got another issue, it seems to be the last one.

Code:
$ python -m Abe.abe --config /usr/local/mysql/my-abe.cnf --commit-bytes 100000 --no-serve
Skipping datadir /Users/thisuser/Library/Application Support/Bitcoin/blocks: [Errno 2] No such file or directory: '/Users/thisuser/Library/Application Support/Bitcoin/blocks/blk0001.dat'

Path is correct and working if I try to cd in it.

Code:
datadir = /Users/thisuser/Library/Application Support/Bitcoin/blocks

Any idea?

Block files are located there for sure.

Code:
$ ls
blk00000.dat blk00002.dat index rev00001.dat rev00003.dat
blk00001.dat blk00003.dat rev00000.dat rev00002.dat
sr. member
Activity: 448
Merit: 250
Code:
ImportError: dlopen(/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): no suitable image found.  Did find:
/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so: mach-o, but wrong architecture

Sorry, this is an issue with Python modules on Mac, and I have no relevant experience.  Put this in a file and run it with "python FILE.py".  It will have to complete without error before you can use Abe:

Code:
import MySQLdb

Thank you John, also for your respond on my PN.
Indeed, this helps me as a non abe-experienced to locate the mac-based problem.

In case of news I'll post.
hero member
Activity: 481
Merit: 529
Code:
ImportError: dlopen(/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): no suitable image found.  Did find:
/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so: mach-o, but wrong architecture

Sorry, this is an issue with Python modules on Mac, and I have no relevant experience.  Put this in a file and run it with "python FILE.py".  It will have to complete without error before you can use Abe:

Code:
import MySQLdb
sr. member
Activity: 448
Merit: 250
hi guys,

I have some trouble with installing ABE on my macbook.

everything went fine so far, I..
* installed mysql 32bit, it's running, path configured
* installed python 2.7, it's running
* installed python-mysql in 32bit arch (i386) mode, seems to be running
* installed ABE, no problem so far

then when I want to run ABE...

Code:
$ ARCHFLAGS="-arch i386" python -m Abe.abe --config /usr/local/mysql/abe-my.cnf --commit-bytes 100000 --no-serve

this comes up:
Code:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Library/Python/2.7/site-packages/Abe/abe.py", line 2026, in
    sys.exit(main(sys.argv[1:]))
  File "/Library/Python/2.7/site-packages/Abe/abe.py", line 2020, in main
    store = make_store(args)
  File "/Library/Python/2.7/site-packages/Abe/abe.py", line 115, in make_store
    store = DataStore.new(args)
  File "/Library/Python/2.7/site-packages/Abe/DataStore.py", line 2843, in new
    return DataStore(args)
  File "/Library/Python/2.7/site-packages/Abe/DataStore.py", line 139, in __init__
    store.module = __import__(args.dbtype)
  File "build/bdist.macosx-10.7-intel/egg/MySQLdb/__init__.py", line 19, in
  File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 7, in
  File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): no suitable image found.  Did find:
/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so: mach-o, but wrong architecture

I ran "file " for mysql (bin file) and the given .so file above, it's both i386.
Can anyone help me out?
member
Activity: 79
Merit: 10
Also
blk00000.dat file, for WDC, it is not present, the default is blk0001.dat and blkindex.dat files.
Currently reading blk00000.dat file that I copied blk0001.dat file, rename came.
member
Activity: 79
Merit: 10
Yes
Operating in BTC state, there is no problem
I do not know LTC, Not tested
home/mapleshadow/.worldcoin/blocks/blk00000.dat have 500M Size
ABE version is GITHUB the MASTER
--------
My configuration as described above, for the WDC, then it is an error
hero member
Activity: 481
Merit: 529

That I can build wdc of abe services?

I don't know.  Can you get Abe working with BTC or NMC or LTC?  Those are known to work.

I need a little more information to understand your problem.  What is your python command line?  How big is /home/mapleshadow/.worldcoin/blocks/blk00000.dat ?  What exact version of Abe do you have?  (Post the output of "git rev-parse HEAD" and "git --no-pager diff")
Pages:
Jump to: