[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff - page 20.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: lunalauncher on March 03, 2014, 01:55:18 AM

Thank you for the quick update! I pulled the fix and reran Abe but the same error still appears.

If the error is still at

Code:

File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 247, in script_GetOp

then Python is using the old, installed version. Either reinstall Abe or adjust PYTHONPATH to use the new code.

lunalauncher

newbie

Activity: 15

Merit: 0

Quote from: John Tobey on March 02, 2014, 10:18:13 PM

I've just copied a fix from the master branch that I think handles this. Please pull and retry. Thanks.

Thank you for the quick update! I pulled the fix and reran Abe but the same error still appears.

Code:

remote: Counting objects: 3, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From https://github.com/bitcoin-abe/bitcoin-abe
   6049d56..3eb0032  no-statistics -> origin/no-statistics
Updating 6049d56..3eb0032
Fast-forward
 Abe/deserialize.py |   24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: lunalauncher on March 01, 2014, 11:32:39 AM

When no-statistics Abe starts working on Bitcoin blocks in late Oct 2013, it is showing a IndexError: string index out of range. Retrying does not help the problem. Any ideas how to solve this?

I've just copied a fix from the master branch that I think handles this. Please pull and retry. Thanks.

lunalauncher

newbie

Activity: 15

Merit: 0

When no-statistics Abe starts working on Bitcoin blocks in late Oct 2013, it is showing a IndexError: string index out of range. Retrying does not help the problem. Any ideas how to solve this?

Code:

# python -m Abe.abe --config abe-my.conf --commit-bytes 1000000 --no-serve;

no chain_id
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00088.dat
Exception at 38408358
Failed to catch up {'blkfile_offset': 38175818, 'blkfile_number': 100088, 'chain_id': None, 'loader': None, 'dirname': '/root/.bitcoin', 'id': Decimal('1')}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1786, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2044, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2174, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1009, in import_block
    tx['tx_id'] = store.import_tx(tx, pos == 0)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1351, in import_tx
    pubkey_id = store.script_to_pubkey_id(txout['scriptPubKey'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1723, in script_to_pubkey_id
    for opcode, data, i in deserialize.script_GetOp(script):
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 247, in script_GetOp
    nSize = ord(bytes[i])
IndexError: string index out of range

altcoin

member

Activity: 84

Merit: 10

altcoin

member

Activity: 84

Merit: 10

While running:

Code:

sudo python -m Abe.abe --config arc.conf --commit-bytes 100000 --no-serve

I got the following errors, every block has been skipped, can some one help me?

Code:

ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:459: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:459: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
1389784906
block_tx 1 1
Skipped 1 bytes at block end
1389816604
block_tx 2 2
Skipped 72 bytes at block end
1389816625
block_tx 3 3
Skipped 73 bytes at block end
1389816648
block_tx 4 4
Skipped 72 bytes at block end
1389816671
block_tx 5 5
Skipped 71 bytes at block end
1389816681
block_tx 6 6
Skipped 72 bytes at block end
1389816682
block_tx 7 7
Skipped 71 bytes at block end
1389816697
block_tx 8 8
Skipped 73 bytes at block end
1389816710
block_tx 9 9
Skipped 71 bytes at block end
1389816731
block_tx 10 10
Skipped 73 bytes at block end
1389816740
block_tx 11 11
Skipped 71 bytes at block end
1389816749
block_tx 12 12
Skipped 73 bytes at block end
1389816752
block_tx 13 13
Skipped 72 bytes at block end
1389816758
block_tx 14 14
Skipped 72 bytes at block end
1389816763
block_tx 15 15
Skipped 72 bytes at block end
1389816789
block_tx 16 16
Skipped 72 bytes at block end
1389816793
block_tx 17 17
Skipped 72 bytes at block end
1389816797
block_tx 18 18
Skipped 72 bytes at block end
1389816843
block_tx 19 19
Skipped 73 bytes at block end
1389816847
block_tx 20 20
Skipped 72 bytes at block end
1389816852
block_tx 21 21
Skipped 73 bytes at block end
1389816852
block_tx 22 22
Skipped 72 bytes at block end
1389816859
block_tx 23 23
Skipped 71 bytes at block end
1389816864
block_tx 24 24
Skipped 71 bytes at block end
1389816863
block_tx 25 25
Skipped 72 bytes at block end
1389816863
block_tx 26 26
Skipped 71 bytes at block end
1389816875
block_tx 27 27
Skipped 72 bytes at block end
1389816889
block_tx 28 28
Skipped 73 bytes at block end
1389816893
block_tx 29 29
Skipped 72 bytes at block end
1389816903
block_tx 30 30
Skipped 71 bytes at block end
1389816914
block_tx 31 31
Skipped 73 bytes at block end
1389816918
block_tx 32 32
Skipped 72 bytes at block end
1389816937
block_tx 33 33
Skipped 71 bytes at block end
1389816939
block_tx 34 34
Skipped 73 bytes at block end
1389816942
block_tx 35 35
Skipped 72 bytes at block end
1389816945
block_tx 36 36
Skipped 71 bytes at block end
1389816952
block_tx 37 37
Skipped 72 bytes at block end
1389816969
block_tx 38 38
Skipped 73 bytes at block end
1389816979
block_tx 39 39
Skipped 72 bytes at block end
1389816982
block_tx 40 40
Skipped 71 bytes at block end
1389816986
block_tx 41 41
Skipped 72 bytes at block end
1389817005
block_tx 42 42
Skipped 73 bytes at block end
1389817005
block_tx 43 43
Skipped 72 bytes at block end
1389817025
block_tx 44 44
Skipped 72 bytes at block end
1389817036
block_tx 45 45
Skipped 73 bytes at block end
1389817038
block_tx 46 46
Skipped 73 bytes at block end
1389817044
block_tx 47 47
Skipped 72 bytes at block end
1389817049
block_tx 48 48
Skipped 73 bytes at block end
1389817051
block_tx 49 49
Skipped 71 bytes at block end
1389817060
block_tx 50 50
Skipped 71 bytes at block end
1389817064
block_tx 51 51
Skipped 72 bytes at block end
1389817067
block_tx 52 52
Skipped 73 bytes at block end
1389817090
block_tx 53 53
Skipped 72 bytes at block end
1389817120
block_tx 54 54
Skipped 72 bytes at block end
1389817122
block_tx 55 55
Skipped 73 bytes at block end
1389817137
block_tx 56 56
Skipped 72 bytes at block end
1389817147
block_tx 57 57
Skipped 72 bytes at block end
1389817153
block_tx 58 58
Skipped 72 bytes at block end
1389817157
block_tx 59 59
Skipped 71 bytes at block end
1389817166
block_tx 60 60
Skipped 73 bytes at block end
1389817178
block_tx 61 61
Skipped 73 bytes at block end
1389817182
block_tx 62 62
Skipped 72 bytes at block end
1389817186
block_tx 63 63
Skipped 71 bytes at block end
1389817196
block_tx 64 64
Skipped 73 bytes at block end
1389817203
block_tx 65 65
Skipped 72 bytes at block end
1389817196
block_tx 66 66
Skipped 73 bytes at block end
1389817218
block_tx 67 67
Skipped 71 bytes at block end
1389817217
block_tx 68 68
Skipped 72 bytes at block end
1389817226
block_tx 69 69
Skipped 73 bytes at block end
1389817237
block_tx 70 70
Skipped 71 bytes at block end
1389817248
block_tx 71 71
Skipped 73 bytes at block end
1389817262
block_tx 72 72
Skipped 72 bytes at block end
1389817262
block_tx 73 73
Skipped 71 bytes at block end
1389817288
block_tx 74 74
Skipped 71 bytes at block end
1389817288
block_tx 75 75
Skipped 72 bytes at block end
1389817294
block_tx 76 76
Skipped 71 bytes at block end
1389817305
block_tx 77 77
Skipped 73 bytes at block end
1389817307
block_tx 78 78
Skipped 72 bytes at block end
1389817310
block_tx 79 79
Skipped 73 bytes at block end
1389817314
block_tx 80 80
Skipped 71 bytes at block end
1389817317
block_tx 81 81
Skipped 71 bytes at block end
1389817336
block_tx 82 82
Skipped 73 bytes at block end
1389817352
block_tx 83 83
Skipped 72 bytes at block end
1389817367
block_tx 84 84
Skipped 72 bytes at block end
1389817381
block_tx 85 85
Skipped 72 bytes at block end
1389817384
block_tx 86 86
Skipped 72 bytes at block end
1389817399
block_tx 87 87
Skipped 73 bytes at block end
1389817401
block_tx 88 88
Skipped 73 bytes at block end
1389817423
block_tx 89 89
Skipped 73 bytes at block end
1389817428
block_tx 90 90
Skipped 72 bytes at block end
1389817433
block_tx 91 91
Skipped 71 bytes at block end
1389817441
block_tx 92 92
Skipped 72 bytes at block end
1389817463
block_tx 93 93
Skipped 73 bytes at block end
1389817465
block_tx 94 94
Skipped 72 bytes at block end
1389817465
block_tx 95 95
Skipped 72 bytes at block end
1389817469
block_tx 96 96
Skipped 72 bytes at block end
1389817466
block_tx 97 97
Skipped 72 bytes at block end
1389817478
block_tx 98 98
Skipped 71 bytes at block end
1389817493
block_tx 99 99
Skipped 73 bytes at block end
1389817505
block_tx 100 100
Skipped 72 bytes at block end
1389817513
block_tx 101 101
Skipped 72 bytes at block end
1389817536
block_tx 102 102
Skipped 71 bytes at block end
1389817531
block_tx 103 103
Skipped 71 bytes at block end
1389817547
block_tx 104 104
Skipped 73 bytes at block end
1389817568
block_tx 105 105
Skipped 71 bytes at block end
1389817583
block_tx 106 106
Skipped 72 bytes at block end
1389817597

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: anush3070 on February 28, 2014, 12:27:20 PM

Hello,
Sorry for not Being Precise,
Here is the Output in the Command Line when I visit the Block Chain Explorer Website:

Quote

Traceback (most recent call last):
File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/root/bce/bitcoin-abe/Abe/abe.py", line 209, in __call__
handler(page)
File "/root/bce/bitcoin-abe/Abe/abe.py", line 307, in handle_chains
100.0 - (100.0 * (ss + more) / (total_ss + more)))
ZeroDivisionError: float division by zero
183.87.156.69 - - [28/Feb/2014 12:25:58] "GET / HTTP/1.1" 500 59

This looks like an older version of bitcoin-abe. Can you please try the latest from https://github.com/bitcoin-abe/bitcoin-abe/ ?

anush3070

member

Activity: 111

Merit: 10

Quote from: John Tobey on February 28, 2014, 12:01:59 PM

Quote from: anush3070 on February 28, 2014, 12:41:58 AM

Tried Everything But all I get is:

Quote

A server error occurred. Please contact the administrator.

Help.

Block Chain Explorer Link: http://107.170.63.233:2750/

What do you see at the command window where you started python? See: http://www.catb.org/~esr/faqs/smart-questions.html#beprecise

Hello,
Sorry for not Being Precise,
Here is the Output in the Command Line when I visit the Block Chain Explorer Website:

Quote

Traceback (most recent call last):
File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/root/bce/bitcoin-abe/Abe/abe.py", line 209, in __call__
handler(page)
File "/root/bce/bitcoin-abe/Abe/abe.py", line 307, in handle_chains
100.0 - (100.0 * (ss + more) / (total_ss + more)))
ZeroDivisionError: float division by zero
183.87.156.69 - - [28/Feb/2014 12:25:58] "GET / HTTP/1.1" 500 59

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: anush3070 on February 28, 2014, 12:41:58 AM

Tried Everything But all I get is:

Quote

A server error occurred. Please contact the administrator.

Help.

Block Chain Explorer Link: http://107.170.63.233:2750/

What do you see at the command window where you started python? See: http://www.catb.org/~esr/faqs/smart-questions.html#beprecise

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: StakeHunter on February 28, 2014, 11:56:04 AM

Quote from: John Tobey on February 28, 2014, 11:32:24 AM

Please verify the line that you added to /etc/sudoers.

No longer getting tty, still getting the fastcgi errors...

Code:

[Fri Feb 28 11:53:21 2014] [warn] [client 108.162.237.97] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 11:53:21 2014] [error] [client 108.162.237.97] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

Sorry, but I can not troubleshoot the FastCGI setup here. Does Abe work in non-FCGI mode with --port=2750? Are your scripts executable? Can you find the output of abe.fcgi? It may include a useful error message.

StakeHunter

full member

Activity: 155

Merit: 100

Quote from: John Tobey on February 28, 2014, 11:32:24 AM

Please verify the line that you added to /etc/sudoers.

Code:

www-data ALL=(nathan) NOPASSWD: /home/nathan/cgi-bin/abe

No longer getting tty, still getting the fastcgi errors...

Code:

[Fri Feb 28 11:53:21 2014] [warn] [client 108.162.237.97] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 11:53:21 2014] [error] [client 108.162.237.97] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: lunalauncher on February 27, 2014, 11:02:46 PM

Quote from: artiface on February 27, 2014, 10:58:37 PM

Quote from: lunalauncher on February 27, 2014, 09:46:33 PM

Somehow the current block height displayed on Abe's main page no longer changes as new blocks are processed by Abe.

Right now the current block displayed on main page is 521941. However I am able to access a newer block 521948 by going to http://localhost:2750/block/a4436cbcf8986ab38e64fc88983c973a76cd938157dbb2f0d3a20748842dc8d5. Abe's title also shows Block a443...48842dc8d5 instead of Block 521948

q/getblockcount also shows the wrong block height of 521941.

Is it possible to do like a rescan to update the Abe's current block height on the front page?

The chain was probably forked. Once the current chain catches up to the previous highest block it will sort itself out. I've seen this a few times.

Quote

Abe keeps orphan blocks and side chains and reuses them if they join the main chain.

Thanks for the quick reply! The chain I'm using is LTC, is this just a temp fork that affects some people, and also the reason why we wait for several confirmations for a transaction?

Well, this looks to me like Abe failing to find block 521941 by its hash in block 521942. This can occur because of bugs, especially when two or more processes simultaneously load blocks. This can also occur because of block format changes, such as changes to the hash algorithm.

Can you browse 521942 by its hash? You might have to delete 521941 and/or 521942 from the database and rescan the blockfile that contains them. I don't have a tool to do exactly that, but you may be able to avoid a full reload with this, assuming "Litecoin" is what Abe calls the chain:

Stop all processes that might load blocks.
Run: python -m Abe.admin delete-chain-blocks Litecoin. This should delete all block header information but not the transactions.
Run Abe with --no-serve --rescan options. This should reinsert the block information.
Restart stopped processes.

Alternatively, a full reload using a single process will work, if anything will. Try to separate loading from serving web pages, if you haven't already. Only a single process should load blocks until we fix the bugs.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: StakeHunter on February 28, 2014, 09:39:33 AM

Code:

sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: 3 incorrect password attempts
[Fri Feb 28 09:37:59 2014] [warn] [client 108.162.221.55] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 09:37:59 2014] [error] [client 108.162.221.55] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

Please verify the line that you added to /etc/sudoers.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: DigitalNexus on February 27, 2014, 05:40:18 PM

Quote from: DigitalNexus on February 27, 2014, 04:54:40 PM

Another question, is it possible to set the magic address in the config and not DataStore.py? I would like to parse multiple blocks but have separate databases for them. It seems I can only do this by running separate instances of Abe.

Looks like I might be able to specify the magic when doing 'datadir+=' in the .conf but it wont insert it into the database.

You don't need the magic number if you supply the chain name. Magic number is for datadir=/path so Abe can find the chain name given only the magic number found in the block files.

When Abe sees a datadir with a previously unknown chain name, it copies the magic number and some other fields to the chain table. On subsequent runs, it does not overwrite these values based on datadir. You would have to update the row or rebuild the database with the modified datadir configuration.

Hope this helps.

StakeHunter

full member

Activity: 155

Merit: 100

Code:

sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: 3 incorrect password attempts
[Fri Feb 28 09:37:59 2014] [warn] [client 108.162.221.55] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 09:37:59 2014] [error] [client 108.162.221.55] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

anush3070

member

Activity: 111

Merit: 10

Tried Everything But all I get is:

Quote

A server error occurred. Please contact the administrator.

Help.

Block Chain Explorer Link: http://107.170.63.233:2750/

lunalauncher

newbie

Activity: 15

Merit: 0

Quote from: artiface on February 27, 2014, 10:58:37 PM

Quote from: lunalauncher on February 27, 2014, 09:46:33 PM

Somehow the current block height displayed on Abe's main page no longer changes as new blocks are processed by Abe.

Right now the current block displayed on main page is 521941. However I am able to access a newer block 521948 by going to http://localhost:2750/block/a4436cbcf8986ab38e64fc88983c973a76cd938157dbb2f0d3a20748842dc8d5. Abe's title also shows Block a443...48842dc8d5 instead of Block 521948

q/getblockcount also shows the wrong block height of 521941.

Is it possible to do like a rescan to update the Abe's current block height on the front page?

The chain was probably forked. Once the current chain catches up to the previous highest block it will sort itself out. I've seen this a few times.

Quote

Abe keeps orphan blocks and side chains and reuses them if they join the main chain.

Thanks for the quick reply! The chain I'm using is LTC, is this just a temp fork that affects some people, and also the reason why we wait for several confirmations for a transaction?

artiface

full member

Activity: 238

Merit: 100

Quote from: lunalauncher on February 27, 2014, 09:46:33 PM

Somehow the current block height displayed on Abe's main page no longer changes as new blocks are processed by Abe.

Right now the current block displayed on main page is 521941. However I am able to access a newer block 521948 by going to http://localhost:2750/block/a4436cbcf8986ab38e64fc88983c973a76cd938157dbb2f0d3a20748842dc8d5. Abe's title also shows Block a443...48842dc8d5 instead of Block 521948

q/getblockcount also shows the wrong block height of 521941.

Is it possible to do like a rescan to update the Abe's current block height on the front page?

The chain was probably forked. Once the current chain catches up to the previous highest block it will sort itself out. I've seen this a few times.

Quote

Abe keeps orphan blocks and side chains and reuses them if they join the main chain.

lunalauncher

newbie

Activity: 15

Merit: 0

Somehow the current block height displayed on Abe's main page no longer changes as new blocks are processed by Abe.

Right now the current block displayed on main page is 521941. However I am able to access a newer block 521948 by going to http://localhost:2750/block/a4436cbcf8986ab38e64fc88983c973a76cd938157dbb2f0d3a20748842dc8d5. Abe's title also shows Block a443...48842dc8d5 instead of Block 521948

q/getblockcount also shows the wrong block height of 521941.

Is it possible to do like a rescan to update the Abe's current block height on the front page?

DigitalNexus

newbie

Activity: 7

Merit: 0

Quote from: DigitalNexus on February 27, 2014, 04:54:40 PM

Another question, is it possible to set the magic address in the config and not DataStore.py? I would like to parse multiple blocks but have separate databases for them. It seems I can only do this by running separate instances of Abe.

Looks like I might be able to specify the magic when doing 'datadir+=' in the .conf but it wont insert it into the database.

Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff - page 20. (Read 220986 times)