Pages:
Author

Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff - page 33. (Read 220986 times)

legendary
Activity: 1400
Merit: 1005
Is there a list somewhere of all of the publicly-available servers/websites running Abe?  When blockchain.info was down, I went looking for one, but couldn't find any that were still running.
hero member
Activity: 481
Merit: 529
i stoped supporting I0Coin due to its memhogging, how do i cleanly delete the I0Coin chain from the MySQL DB? (already uncommented it in abe.conf)
Sorry, I haven't written this feature.  You can remove the currency from the main page by finding its chain_id (in the chain table) and deleting all matching rows from chain_candidate and chain, I think.  It's been a while since I did this.

Then reclaiming the block and tx space is a matter of identifying rows in a half dozen tables and deleting them in dependency order, being careful not to delete other chains' data.  Or you could disable keys, delete in any order, and enable keys again, but this is not advisable while other chains may be loading.  I seem to remember "clean" chain removal taking me 20-30 minutes.  In 1-2 hours I could develop a utility that does it all, but it's still a to-do.

Quote from: TODO.txt
* Admin interface to delete chains, etc.
legendary
Activity: 1792
Merit: 1008
/dev/null
i stoped supporting I0Coin due to its memhogging, how do i cleanly delete the I0Coin chain from the MySQL DB? (already uncommented it in abe.conf)
hero member
Activity: 481
Merit: 529
I am not receiving email at [email protected] since three days ago.  If you sent email there in the last 72 hours, it did not reach me, and GMail is not sending delivery failure messages.  I've filled out the forms and await a solution in the next couple of days, or I'll be an ex-GMail user.  I'll check my personal messages here.

Sorry for any inconvenience.
sr. member
Activity: 426
Merit: 250
Wow, that is seriously awesome! I am going to set up a public server so everyone is able to use this great piece of software.
donator
Activity: 2772
Merit: 1019
it's awesome to see you keep improving.

I don't have use for the newer features, but let me say I frequently use bitcoin-abe for various analyses on the blockchain.

A big thanks!
hero member
Activity: 481
Merit: 529
I've added a branch to the code repository: no-statistics.

This branch can now TRIM SPENT OUTPUTS, transactions, and addresses from the database.  It is not polished code, but it basically seems to work.  Loading the blockchain may be a little slower than before, since it does all the same SQL work, plus some deleting.  The space reduction appears to be "only" around 50% as of Block 134000 (under PostgreSQL 8.4).  This may be due in part to all those early unspent coinbase outputs.

To use the new feature, pass --default-trim-depth=6 (EDIT: make that --default-trim-depth=40 thanks to the March 12 fork) when creating the database.  Replace "6" with a higher number for defence against corruption by long side chains, at the cost of slightly more space.  See "default-trim-depth" in abe.conf for details and caveats.
hero member
Activity: 481
Merit: 529
I've added a branch to the code repository: no-statistics.

no-statistics is an experimental branch of Abe that lacks certain features present in the master branch.  The features are Percent Coin-Days Destroyed and Chain Age.  Removing these features simplifies the code and reduces Abe's storage and processing requirements.

I have only lightly tested it, but I consider it the direction of future development.  I would like to reimplement the original statistics and others, but their inclusion should be a configurable option so as not to burden applications that do not need them.

There is not (yet) an option to convert a database in the original format to one that uses the no-statistics branch or vice versa.
hero member
Activity: 481
Merit: 529
Oh nice!

In readme.txt: "The Abe.reconfigure module turns firstbits on and off once you have upgraded Abe's schema."

How do I updrage Abe's schema?

You already have a schema supporting firstbits, so just ignore that step.  Sorry it's unclear, but upgrading is for databases created by earlier versions of Abe, before firstbits support.  Anyway, to upgrade, just use the new version of bitcoin-abe and include "--upgrade" after "python -m Abe.abe".
sr. member
Activity: 426
Merit: 250
Am I correct that firstbit is not yet available trough api-calls?

No, there is /q/fb/ADDRESS and /q/addr/FIRSTBITS.  However, firstbits is disabled by default, since it requires a big table.  (Something like 99% of all addresses are single-use "change addresses" but must be remembered for firstbits.)  Also, please note that Abe's algorithm probably differs slightly from firstbits.com and blockchain.info's.  See elsewhere in this thread for details.

More info, including how to turn on Abe's firstbits support, is in README-FIRSTBITS.txt.

Oh nice!

In readme.txt: "The Abe.reconfigure module turns firstbits on and off once you have upgraded Abe's schema."

How do I updrage Abe's schema?
hero member
Activity: 481
Merit: 529
Am I correct that firstbit is not yet available trough api-calls?

No, there is /q/fb/ADDRESS and /q/addr/FIRSTBITS.  However, firstbits is disabled by default, since it requires a big table.  (Something like 99% of all addresses are single-use "change addresses" but must be remembered for firstbits.)  Also, please note that Abe's algorithm probably differs slightly from firstbits.com and blockchain.info's.  See elsewhere in this thread for details.

More info, including how to turn on Abe's firstbits support, is in README-FIRSTBITS.txt.
sr. member
Activity: 426
Merit: 250
Am I correct that firstbit is not yet available trough api-calls?
sr. member
Activity: 426
Merit: 250
I've reimplemented /unspent so it should outperform /address. 

Let me know if this helps.

It works great Cheesy
sr. member
Activity: 426
Merit: 250
I was expecting 50*210000 + 25*(height-209999) as well, that was why I was wondering if it added fees or something.

chain/Bitcoin/q/totalbc/100000 : 5000049.98
chain/Bitcoin/q/totalbc/200000 : 10000037.06486171
chain/Bitcoin/q/totalbc/220998 : 10774961.89978955

The numbers are actually below what the formula gives.  After block 100000, there had been 100001 blocks (counting block 0), so the expected total was 5000050, 0.02 more than actual.  After block 200000, expected 10000050, 12.93513829 more than found.  After 220998, expected 10774975, 13.10021045 more than found.  It looks okay to me, although we could take off another 100 for at least two cases of duplicate 50BTC coinbase transactions, if I recall correctly.


Interesting to see those numbers Smiley

/tx/2c637592a4b4a95cf4b19260730c66de540d7d3b14d8d352de591c5ee6eac0fc indeed has bitcoins destroyed Smiley
hero member
Activity: 481
Merit: 529
I was expecting 50*210000 + 25*(height-209999) as well, that was why I was wondering if it added fees or something.

chain/Bitcoin/q/totalbc/100000 : 5000049.98
chain/Bitcoin/q/totalbc/200000 : 10000037.06486171
chain/Bitcoin/q/totalbc/220998 : 10774961.89978955

The numbers are actually below what the formula gives.  After block 100000, there had been 100001 blocks (counting block 0), so the expected total was 5000050, 0.02 more than actual.  After block 200000, expected 10000050, 12.93513829 more than found.  After 220998, expected 10774975, 13.10021045 more than found.  It looks okay to me, although we could take off another 100 for at least two cases of duplicate 50BTC coinbase transactions, if I recall correctly.
sr. member
Activity: 426
Merit: 250
Question regarding this: https://bitcointalksearch.org/topic/m.741984
An Abe instance that is started with --no-serve automatically keeps the blockchain up to date?

A loop that runs it with --no-serve would keep it up to date.  It brings it up to date and then exits.

And what does /q/totalbc show? Bitcoins created, or bitcoins included in coinbase transaction? Because
Local Abe: 10774286.89978955
Blockexplorer: 10736300.00000000
Blockchain.info: 10761275.00000000

Not that it really matters, I don't use it, but just wondering Smiley

It gives a statistical estimate of amount in circulation.  I am curious which block number you got the Abe value from?  You can add the number to the URL like /q/totalbc/220962 to be sure.

The number is normally 50*210000 + 25*(height-209999) but is lower due to the following rare events:
  • coinbase transactions outputting less than they are entitled to,
  • coins sent to known dead addresses (Abe recognizes the hash consisting of all 0 bits),
  • duplicate coinbase transactions (I don't think Abe accounts for them, but the others may)

and there may be bugs.


I was expecting 50*210000 + 25*(height-209999) as well, that was why I was wondering if it added fees or something.

chain/Bitcoin/q/totalbc/100000 : 5000049.98
chain/Bitcoin/q/totalbc/200000 : 10000037.06486171
chain/Bitcoin/q/totalbc/220998 : 10774961.89978955
hero member
Activity: 481
Merit: 529
Question regarding this: https://bitcointalksearch.org/topic/m.741984
An Abe instance that is started with --no-serve automatically keeps the blockchain up to date?

A loop that runs it with --no-serve would keep it up to date.  It brings it up to date and then exits.

And what does /q/totalbc show? Bitcoins created, or bitcoins included in coinbase transaction? Because
Local Abe: 10774286.89978955
Blockexplorer: 10736300.00000000
Blockchain.info: 10761275.00000000

Not that it really matters, I don't use it, but just wondering Smiley

It gives a statistical estimate of amount in circulation.  I am curious which block number you got the Abe value from?  You can add the number to the URL like /q/totalbc/220962 to be sure.

The number is normally 50*210000 + 25*(height-209999) but is lower due to the following rare events:
  • coinbase transactions outputting less than they are entitled to,
  • coins sent to known dead addresses (Abe recognizes the hash consisting of all 0 bits),
  • duplicate coinbase transactions (I don't think Abe accounts for them, but the others may)

and there may be bugs.
sr. member
Activity: 426
Merit: 250
I've reimplemented /unspent so it should outperform /address.  A couple of things to note.  The denial-of-service limit on history rows applies to the total across all addresses given to /unspent, so in some cases, /unspent will return "too many records" when iterated /address calls would work.  This limit applies to both spent and unspent coins, unlike in the previous implementation, so you may see "too many records" where my first /unspent worked.  In abe.conf, address-history-rows-max controls this limit.  For now, I've left the first implementation available by adding "?impl=1" to the URL.  It should yield the same results apart from "too many records" errors.

Let me know if this helps.


Excellent! I'll look into it this week.

Question regarding this: https://bitcointalksearch.org/topic/m.741984
An Abe instance that is started with --no-serve automatically keeps the blockchain up to date?


And what does /q/totalbc show? Bitcoins created, or bitcoins included in coinbase transaction? Because
Local Abe: 10774286.89978955
Blockexplorer: 10736300.00000000
Blockchain.info: 10761275.00000000

Not that it really matters, I don't use it, but just wondering Smiley
hero member
Activity: 481
Merit: 529
I've reimplemented /unspent so it should outperform /address.  A couple of things to note.  The denial-of-service limit on history rows applies to the total across all addresses given to /unspent, so in some cases, /unspent will return "too many records" when iterated /address calls would work.  This limit applies to both spent and unspent coins, unlike in the previous implementation, so you may see "too many records" where my first /unspent worked.  In abe.conf, address-history-rows-max controls this limit.  For now, I've left the first implementation available by adding "?impl=1" to the URL.  It should yield the same results apart from "too many records" errors.

Let me know if this helps.
hero member
Activity: 481
Merit: 529
@Jouke,

Thanks for the generous tip and the report of slow performance.  It should be simple to change /unspent to match the speed of repeated /address calls.  Please feel free to remind me if I haven't got to it in a week. Smiley

Yes, 1PWC7PNHL1SgvZaN7xEtygenKjWobWsCuf is still under my control.
Pages:
Jump to: