CryptoCoin Explorer - Coin Block Explorers. - page 18.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Yes I updated the daemon last night ..... Yet again!?!?!?... Roll Eyes

Code:

101642	2013-04-09 16:29:09	1	20	1.0	2032890	30.0376	165.304	64.2496%
101641	2013-04-09 16:25:09	1	20	1.0	2032870	30.0351	165.301	64.2517%
101640	2013-04-09 16:25:09	1	20	1.0	2032850	30.0354	165.301	64.2517%
101639	2013-04-09 16:23:09	1	20	1.0	2032830	30.0343	165.3	64.2528%
101638	2013-04-09 16:22:00	1	20	2.113	2032810	30.0338	165.299	64.2534%
101637	2013-04-09 16:22:00	1	20	8.452	2032790	30.0341	165.299	64.2534%
101636	2013-04-09 16:22:00	4	5343.18432958	33.809	2032770	30.0344	165.299	64.2534%
101635	2013-04-09 16:22:00	1	20	135.239	2032750	30.0397	165.299	64.2474%
101634	2013-04-09 16:01:04	1	20	1352.383	2032730	30.0255	165.285	64.2585%
101633	2013-04-09 15:40:30	1	20	13523.835	2032710	30.0115	165.27	64.2695%
101632	2013-04-09 15:20:17	1	20	135238.105	2032690	29.9977	165.256	64.2802%
101631	2013-04-09 15:00:00	1	20	135238.105	2032670	29.9839	165.242	64.291%
101630	2013-04-09 15:00:00	1	20	135238.105	2032650	29.9842	165.242	64.291%

It appears the actual drop started on block 101633. Depending on the alog it may have taken a couple of blocks for the difficulty to actually drop, even though the version cutoff may have been on 101631.

For example noted in the code:

Code:

If the new block's timestamp is more than 20 minutes
    // have to be greater than the max accepted time delta ; 15mins.
    // this way, if one would artificially increase block nTime to its max value,
    // we'd still take the 5mins periods without block before allowing a one-shot
    // diff decrase, later keeping the block time used for ema computation.

Walter Rothbard

sr. member

Activity: 476

Merit: 250

Bytecoin: 8VofSsbQvTd8YwAcxiCcxrqZ9MnGPjaAQm

Hey, dreamwatcher, have you updated to the new Terracoin emergency build from yesterday? Block 101631 should be showing a difficulty drop (I think), and is not.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Poll bump...... Cheesy

dreamwatcher

legendary

Activity: 1064

Merit: 1000

So far so good, I have not had to unlock an explorer since the framework change. I only reset the web server once as I had forgotten to set the ppcexplore.org address in the server alias config.

I want to gauge the interest in seeing Novacoin and/or Bytecoin on CCE.

Both these coins already have at least one public explorer, so the decision to add them is going to be based mostly on the interest of CCE users through the poll.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

This evening I finished switching the entire CryptoCoinExplorer site to use the FastCGI server framework.

Since the sudden up shoot of Alt-coins this weekend, I felt I did not have the luxury of time to completely switch over to a Cherrypy framework.

Now it is observe and tweak as needed......and chase down bothersome IP's that just make it slower for everybody... Cheesy

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Just thought I would give an update as to what is going on with the explorer and related projects.

1. dermoths' ABE pull request that I applied to the TRC explorer does much in the way of reducing explorer lock up. However, it does not completely solve the issue and the TRC explorer has locked a couple of times, but that is a big improvement. Hopefully this is no longer an issue.

2. I have decided the only way to correctly fix the issue is to run the explorers with a different server framework. I have tried to get it to run on the FastCGI framework, but I cannot get it to work correctly and it is probably an issue on my part. Moved to FastCGI.

3. I have been working hard on the new pool. I have gained enough experience with Pushpool, to use it for all the alt-coins I plan to support. I have been looking at Eloipool and I should be able to adapt it relatively quickly once the alt-coins move to a 0.8 BTC daemon base. Eloipool is written in python which I find much easier to code with then C. After experimenting with integrating front end components into Pushpool, I have found I really dislike C vs Python. Cheesy

4. Through writing the front end of the pool from scratch, I have come to the decision to use Cherrypy as the server framework for the pool. I am currently working on the payout system, once that is finished I will most likely open the pool up with Bytecoin. This is not because of any kind of "coin loyalty" , but for the simple reason that it is probably the least mined of the current coins because of its newness. Besides, my "coin loyalty" lies mostly with PPC, as it was the first coin I started to do projects with. Wink

TRC and FRC will be the second pools as they really needs some pool choices. PPC will come last only because of its popularity, I need to shake the bugs out and make sure of server capacity before opening.

~~5. My experience with Cherrypy has also led me to the decision to use it for the server framework replacement in the explorers.~~ Moved to FastCGI instead

6. I apologize for the slow speed at which improvements and other projects are coming out. I am doing these projects alone, along with a wife, school and keeping enough cash flow through the business to keep everything going. I cannot remember ever pulling as many all nighters in my life as I have recently Tongue

I am a bit sad to see doublec shutdown his projects in the alt-coin world. I can see where he is coming from as the production and upkeep of projects in this realm can be much more work then most people realize.

I wish him the best of luck in his future endeavors.

I will try and get as much done as I can this weekend, although I have some "Honeydo" projects around the house to get done. Cheesy

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Everything is back up.

I will get the test net daemons up in a few.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

All the explorers and webpages are down.

It appears the issue is on the hosting end as I cannot SSH into either the Cryptocoinexplorer server or the new pool server I am developing.

I will post any updates here.

Sunny King

legendary

Activity: 1205

Merit: 1010

Nice!

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Finally some good news...It looks like dermoths' commit is working well. i just noticed the first bout of connection resets and broken pipes, and ABE gracefully handled them without getting stuck or crashing.

Code:

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run
    self.finish_response()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 127, in finish_response
    self.write(data)
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 210, in write
    self.send_headers()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 268, in send_headers
    self.send_preamble()
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 189, in send_preamble
    self._write('HTTP/%s %s\r\n' % (self.http_version,self.status))
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 389, in _write
    self.stdout.write(data)
  File "/usr/lib/python2.7/socket.py", line 324, in write
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 104] Connection reset by peer
----------------------------------------
Exception happened during processing of request from ('xxx.xxx.xxx.xxx', 52683)
Traceback (most recent call last):
  File "/usr/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
    self.finish()
  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
    self.wfile.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
----------------------------------------
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:37:55] "GET /logo32.png HTTP/1.1" 200 13245
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:37:55] "GET /abe.css HTTP/1.1" 200 222
xxx.xxx.xxx.xxx- - [01/Apr/2013 01:38:05] "GET /chain/Terracoin HTTP/1.1" 200 8023
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:05] "GET /chain/Terracoin/q/getblockcount HTTP/1.0" 200 5
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:05] "GET /chain/Terracoin/q/getdifficulty HTTP/1.0" 200 8
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:05] "GET /chain/Terracoin/q/getblockcount HTTP/1.0" 200 5
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:05] "GET /chain/Terracoin/q/getdifficulty HTTP/1.0" 200 8
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:05] "GET /abe.css HTTP/1.1" 200 222
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:05] "GET /logo32.png HTTP/1.1" 200 13245
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:06] "GET / HTTP/1.1" 200 2492
xxx.xxx.xxx.xxx- - [01/Apr/2013 01:38:07] "GET /abe.css HTTP/1.1" 200 222
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:07] "GET /logo32.png HTTP/1.1" 200 13245
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:07] "GET /favicon.ico HTTP/1.1" 200 20222
xxx.xxx.xxx.xxx - - [01/Apr/2013 01:38:07] "GET /favicon.ico HTTP/1.1" 200 20222

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Update:

Some may have noticed the explorers going up and down today, that was myself working to solve the major issues.

Some things behind the scenes have changed.

1. The database refresh is now completely separate from the explorers web serving. I have set one instance of ABE for each coin to serve without updating (using the datadir = [] trick I read about in the ABE github. Thanks John Tobey).

2. I am using the blocknotify feature of the daemons to run a no server config of ABE to update the database whenever there is a new block on the network. The no server config exits once the database is refreshed.

3. I am now testing on the TRC explorer a pull request by dermoth for ABE, that allows ABE to recover gracefully from broken pipes. I had tried a variation of this pull request when I first put the explorers on github. Those changes are what caused the problems with generating a new database that I had to revert the first day. I am sure it was some change I did to dermoths' commit, so after some more testing on the live server, I will try the modified TRC explorer with creating a new database.

I am hoping by limiting the public facing explorers to reading the database, it will solve some of the deadlocking problems. I also believe by precisely updating the database only when the blocks actually change and then stopping the refresh process until the next block, will reduce the thread load and help keep things running smooth.

After more testing I will issue a commit on github with the changes and documentation to make it all work.

If these changes work out, the last major bug to squash is the TRC difficulty (0 nBits) bug. It appears that the difficulty issue is only a symptom and the real issue is ABE is putting a null block on the latest database block entry.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

I have seen a couple of posts and I have also had some issues with test nets on some of the alt-coins. I will put up some test net daemons on my sites as I see requests or as I see are needed.

The addnode command is flexible in the sense it will work for both live and test net. Addnode will attempt a connection and still try to find other nodes, unlike the "connection=" which is an explicit command to only connect to that address.

First up TRC:

I have started a test net daemon for TRC on the cryptocoinexplorer.com server.

add to your config file:

addnode=84.200.84.74

This will have your daemon automatically try the cryptocoin explorer server for a connection.I will keep the daemon up as long as it does not start to effect the explorer site itself.

You can also add:

addnode=84.200.84.76

That is the IP for my future alt-coin mining site (cryptocoinmine.com)I will run a TRC test net daemon as much as possible, at least until the mining site goes live. Wink

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Quote from: John Tobey on March 28, 2013, 02:45:33 PM

Quote from: dreamwatcher on March 23, 2013, 08:16:59 PM

After some research I found that Chrome has a tendency to create a rather large amount of parallel connections and then proceeds to communicate on the second connection it created.

Well, that's pretty perverse, but all right, we should be multithreaded.

Quote from: dreamwatcher on March 23, 2013, 08:16:59 PM

I will start working on using a different server mechanism for the explorers, possibly Twisted or Fastcgi.

If anybody has suggestions or would like to help, it would be greatly appreciated. The source for the PPC and FRC explorers are on github.

If you have any trouble with README-FASTCGI.txt, I'd like to know.

Easier, though not as well tested: define a server class in abe.py:

Code:

from wsgiref.simple_server import WSGIServer
import SocketServer
class HttpServer(SocketServer.ThreadingMixIn, WSGIServer):
    pass

Then add the text in bold: httpd = make_server(args.host, port, abe, server_class=HttpServer)

There is also SocketServer.ForkingMixIn in case ThreadingMixIn gives weird errors.

Thank you.

Hopefully I can get to it this weekend.

I have also been looking at cherry.py.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: dreamwatcher on March 23, 2013, 08:16:59 PM

After some research I found that Chrome has a tendency to create a rather large amount of parallel connections and then proceeds to communicate on the second connection it created.

Well, that's pretty perverse, but all right, we should be multithreaded.

Quote from: dreamwatcher on March 23, 2013, 08:16:59 PM

I will start working on using a different server mechanism for the explorers, possibly Twisted or Fastcgi.

If anybody has suggestions or would like to help, it would be greatly appreciated. The source for the PPC and FRC explorers are on github.

If you have any trouble with README-FASTCGI.txt, I'd like to know.

Easier, though not as well tested: define a server class in abe.py:

Code:

from wsgiref.simple_server import WSGIServer
import SocketServer
class HttpServer(SocketServer.ThreadingMixIn, WSGIServer):
    pass

Then add the text in bold: httpd = make_server(args.host, port, abe, server_class=HttpServer)

There is also SocketServer.ForkingMixIn in case ThreadingMixIn gives weird errors.

Sunny King

legendary

Activity: 1205

Merit: 1010

PPC explorer seems down to me.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Just thought I would give an update on that is happening with the explorers.

I am no longer having issues with the databases, I believe I have narrowed down the major issues to an compatibility problem between Chrome and the web server built into python.

Over the last month or so, I noticed that requests from the same IP addresses or address pools, seemed to be crashing the explorers.

After some research I found that Chrome has a tendency to create a rather large amount of parallel connections and then proceeds to communicate on the second connection it created.

The python server gets stuck waiting for a response from the first connection, connections time out..pipes get broken...and so forth.

I will start working on using a different server mechanism for the explorers, possibly Twisted or Fastcgi.

If anybody has suggestions or would like to help, it would be greatly appreciated. The source for the PPC and FRC explorers are on github.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Dammit,

Day ONE issue. Shocked

I have released a commit on both PPC and FRC explorers addressing an issue I just learned about.

I have not had to create new databases in quite some time, so I did not realize there was a DDL implicit issue when creating new databases.

I discovered the issue when setting up a fresh TRC explorer on the test server to work on the current difficulty bug in the TRC explorer.

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: dreamwatcher on March 18, 2013, 11:04:55 AM

I was never fully happy with my changes.

I sympathize, but if complete satisfaction were a criterion, there would be no Abe. See: Release Early, Release Often.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Quote from: John Tobey on March 18, 2013, 10:17:52 AM

Quote from: dreamwatcher on March 17, 2013, 06:57:32 PM

I have decided to publish the source code to the explorers on Github. I do this in the spirit of open source and to encourage others to make suggestions and improve upon them.

Excellent, I was just about to request the same!

When I have time (if no one else does it first) I'd like to organize these source trees as clones or branches of jtobey/bitcoin-abe (or a new Abe repo under a shared account). I think that will facilitate merging the code and relegating the differences to config options.

Thanks for all your work!

I was never fully happy with my changes. I always thought they could be cleaner, some minor bugs fixed or a feature added. I am still not completely satisfied with the coding, but I have delayed publishing long enough.

This is the reason I have not released the TRC explorer yet, one bug I still want to squash before I publish. Cheesy

John Tobey

hero member

Activity: 481

Merit: 529

Quote from: dreamwatcher on March 17, 2013, 06:57:32 PM

I have decided to publish the source code to the explorers on Github. I do this in the spirit of open source and to encourage others to make suggestions and improve upon them.

Excellent, I was just about to request the same!

When I have time (if no one else does it first) I'd like to organize these source trees as clones or branches of jtobey/bitcoin-abe (or a new Abe repo under a shared account). I think that will facilitate merging the code and relegating the differences to config options.

Thanks for all your work!

Topic: CryptoCoin Explorer - Coin Block Explorers. - page 18. (Read 30758 times)