Open Source CCE 4.0 Block Explorer | Bitcointalksearch.org

oldcoinguru

full member

Activity: 176

Merit: 100

Nice work! Wink

Using it now: http://45.32.67.42

https://bitcointalksearch.org/topic/m.13046478

Quote from: oldcoinguru on November 23, 2015, 04:23:03 AM

Quote from: MickyCOB on November 23, 2015, 04:06:31 AM

Quote from: oldcoinguru on November 23, 2015, 03:38:20 AM

Block explorer

http://45.32.67.42

Nice work mate block explorer up and running

It's not my work. I used some open source software provided by dreamwatcher for the block explorer https://bitcointalksearch.org/topic/open-source-cce-40-block-explorer-1088291

MorGan-id

sr. member

Activity: 372

Merit: 250

Quote from: cinnamon_carter on October 31, 2015, 02:08:53 AM

Your request has zero /nothing to do with this topic.

This thread is for Dreamwatchers open source CCE "State of the Art" Block Crawler

not for the coin you mention,

btw I did not report the post to a mod but it really does not belong here,

get in touch with r3wt , it was his coin , before he turned it over to a developer who cannot even compile hello world............... I doubt anyone here wants anything to do with that project.

Please don't spam this thread.

Quote from: BitcoinNational on October 30, 2015, 10:42:21 AM

can you responsible adults pls take over the git on OSC? (open source coin)

thank

cinnamon_carter

legendary

Activity: 1148

Merit: 1018

It's about time -- All merrit accepted !!!

Your request has zero /nothing to do with this topic.

This thread is for Dreamwatchers open source CCE "State of the Art" Block Crawler

not for the coin you mention,

btw I did not report the post to a mod but it really does not belong here,

get in touch with r3wt , it was his coin , before he turned it over to a developer who cannot even compile hello world............... I doubt anyone here wants anything to do with that project.

Please don't spam this thread.

Quote from: BitcoinNational on October 30, 2015, 10:42:21 AM

can you responsible adults pls take over the git on OSC? (open source coin)

MorGan-id

sr. member

Activity: 372

Merit: 250

Quote from: BitcoinNational on October 30, 2015, 10:42:21 AM

can you responsible adults pls take over the git on OSC? (open source coin)

folder Open-Source-CCE-4.0-master\docs there are 2 files dbload.rst and web_server.rst
I edit them under their own coin ?
On the server you want to install Apache and to put in the directory WWW all files from html ?
And before that, you need to start a coin purse ?

Only after that I do not understand what to do ?

BitcoinNational

legendary

Activity: 1470

Merit: 1010

Join The Blockchain Revolution In Logistics

can you responsible adults pls take over the git on OSC? (open source coin)

MorGan-id

sr. member

Activity: 372

Merit: 250

Hi,
Tell me how to connect to the coin Open Source CCE 4.0 Block Explorer ?
There are detailed instructions?

cinnamon_carter

legendary

Activity: 1148

Merit: 1018

It's about time -- All merrit accepted !!!

dreamwatcher has been around a long while, i am not sure what you are asking of him but I trust he will return

vampirus

hero member

Activity: 982

Merit: 517

Nature decays, but Latinum lasts forever. RoA:102

Hi, dreamwatcher. You not answer any email or PM.
What status of scifi exchange and KED, GPL, UFC?
If you abandon it, why not return our coins from exchange?

lahlor

member

Activity: 98

Merit: 10

Hi,

try your script but currently getting "Main Loop label empty or too long" after running ./dbload.py -n -v

Any ideas?

Hoshimaru

sr. member

Activity: 411

Merit: 252

It's up & running dreamwatcher \(^_^)/ ^(*)

At first I had some trouble with it because I'm already running a vhost on that server for bitcoin-abe.
The configuration of this virtual host is slightly different to what's written in the docs on github:

.morningminerals.com:80> ServerName .morningminerals.com ServerAlias .morningminerals.com ProxyPreserveHost On ProxyErrorOverride On DocumentRoot /home/akira/www/cce Order allow,deny Allow from all ProxyPass /images ! ProxyPass /robots.txt ! ProxyPass /css ! ProxyPass /js ! .morningminerals.com> Order allow,deny Allow from all ProxyPass / http://localhost:8222/ ProxyPassReverse / http://localhost:8222/
Without the added green directive it won't load the bootstrap files located in documentroot.
Also, the domain/host must specified in each individual vhost's conf or only one of them works ;-)

Hoshimaru

sr. member

Activity: 411

Merit: 252

Thank for your reply. I got over exited today because of passing a screening for a job I'd really like to do, hence the stupid questions.
The client I use doesn't have a hashrate field to be returned and that's indeed what went wrong. Peers and richlist are now populated as well.

I'll do the webserver part sometime tomorrow or the following days. So far I love OS CCE v4.0. You've created a fantastic piece of software m(._.)m

Concerning InnoDB vs MariaDB, there is not much I can do about it, except install a private instance of mysql on the VPS. The offloaded DB is running as:

mysql Ver 14.14 Distrib 5.5.43, for debian-linux-gnu (x86_64) using readline 6.3 Connection id: 25126661 Current database: Current user: [email protected] SSL: Not in use Current pager: stdout Using outfile: '' Using delimiter: ; Server version: 5.5.5-10.0.14-MariaDB-1~precise-log mariadb.org binary distribution Protocol version: 10 Connection: daldb.vpsdime.com via TCP/IP Server characterset: latin1 Db characterset: latin1 Client characterset: utf8 Conn. characterset: utf8 TCP port: 3306 Uptime: 5 days 8 hours 42 min 20 sec

I thought it was InnoDB due to the engine being set to default:

+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+ | Engine | Support | Comment | Transactions | XA | Savepoints | +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+ | CSV | YES | CSV storage engine | NO | NO | NO | | MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO | | MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO | | BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO | | MyISAM | YES | MyISAM storage engine | NO | NO | NO | | InnoDB | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES | YES | YES | | ARCHIVE | YES | Archive storage engine | NO | NO | NO | | FEDERATED | YES | FederatedX pluggable storage engine | YES | NO | YES | | PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO | | Aria | YES | Crash-safe tables with MyISAM heritage | NO | NO | NO | +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+

The reported SQL error ... will it break something in the fuctionality or stability while running your block explorer?

dreamwatcher

legendary

Activity: 1064

Merit: 1000

I also wanted to add that CherryPy is configured to be run through a proxy. If you want to directly access it , most likely you will need to change the server.conf file.

tools.encode.on: True
tools.gzip.on: True
tools.gzip.mime_types: ['text/html', 'text/plain', 'application/json', 'text/javascript', 'application/javascript']
tools.proxy.on: True

dreamwatcher

legendary

Activity: 1064

Merit: 1000

The stats module runs quickly as it is using MySql itself to sort most of the results. Hash rate, difficulty and peer info are directly queried from the coin daemon.

Except for the sever log, it is normal for there not to be logs as they are only written to when warnings/errors occur.

The SQL error, I have not seen. However it appears you are using MariaDB, CCE and the SQL file I have included are written with InnoDB.

Quote

ERROR:root:06-29 16:25:05 'networkhashps' : General stat module error

Hash rate can only be obtained if the coin daemon provides it. As outlined in the instructions:

Code:

hashrate:

        Boolean indicating if the network hash is available by the daemon.
        Network hash rate can only be obtained from the coin daemon.

hashfield: (Suggested: networkhashps)

        Label of the field network hash rate is returned by the coin daemon.

ratelabel: (Suggested: MH or GH)

        Label to use for the hash rate on the index web page.

hashmult:

        Hash rate multiplier to use for storing hash rate at desired level.
        Example: Daemon output is 5678 H/s. Use the multipler 0.001 to store as 5.678.

The CCE web server uses Apache for http proxy and to serve static files.

It also requires bootstrap 3 be served statically or hosted (In which case change the header in base.html)

Code:

Install Bootstrap3 css and js files to their appropriate css and js directory in document root.

        Alternately, find a host for bootstrap3 and change the base.html header to reflect.

As outlined in the instructions, you can change the port the CherryPy server runs on in the server.conf file

Code:

[global]
server.socket_port:8222

If one wishes to use another web server then Apache, one will need to configure according to the server. It needs to proxy/reverse proxy and serve static files.( Bootstrap 3 and any images)

I believe the issues with the text based browser may be related to not having or using bootstrap 3.

Hoshimaru

sr. member

Activity: 411

Merit: 252

Hello \(^_^)/
The whole blockchain got loaded

I've got some questions concerning how to continue now:
After it finished, it calls the statistics module:

Code:

akira@Mineralscoin:~/CCE$ python dbload.py -v -l
Processing Block:  1749237  of  1749236 
Calling Statistics Module
Database load complete

This statistics steps in incredibly fast, so I was kind of suspicious that it did it's job.
Unlike the other modules, it didn't create a log file and I only have these:

comm.log
loader.log
server.log

When I launch python websrv.py, I can only connect to it using lynx localhost:8222 (I did not setup Apache yet, as it's currently serving Abe)
Shouldn't I be able to connect from a remote client as well even without Apache involved? Opening this port 8222 in iptables doesn't change its behaviour.

server.log contains:

Code:

[29/Jun/2015:11:18:35] ENGINE Daemonized to PID: 11612
[29/Jun/2015:11:18:35] ENGINE Started monitor thread '_TimeoutMonitor'.
[29/Jun/2015:11:18:35] ENGINE PID 11612 written to '/home/akira/CCE/cherrypy.pid'.
[29/Jun/2015:11:18:35] ENGINE Serving on 127.0.0.1:8222
[29/Jun/2015:11:18:35] ENGINE Bus STARTED

Using this text based browser, I noticed that the peers and richlist pages are not populated.

I believe the last lines in comm.log are from this statistics module:

Code:

ERROR:root:06-29 16:25:05 (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ': '74.15099844', u'proof-of-stake': '0.00024414'}' at line 1") : query no return
ERROR:root:06-29 16:25:05 UPDATE stats SET curr_diff = %s : query no return
ERROR:root:06-29 16:25:05 'networkhashps' : General stat module error

I can query the table address and it looks completely filled as one would expect.

cinnamon_carter

legendary

Activity: 1148

Merit: 1018

It's about time -- All merrit accepted !!!

thanks , will have another go at it this week i think

dreamwatcher

legendary

Activity: 1064

Merit: 1000

New commit pushed.

Added:

Configuration options to set MySQL server IP address and port
Configuration option to set RPC timeout
Instructions on how to set up an empty database from the command line

Fixed:

Type errors when logging or sending messages to stderr. Exception descriptors are now always cast to string
The database loader will now throw an exception when a block height of "-1" is sent to the block parser instead of attempting to process it.

dreamwatcher

legendary

Activity: 1064

Merit: 1000

Quote from: Hoshimaru on June 26, 2015, 01:14:17 AM

How much space does the DB require?
Guess I'll just have to test in on the VPS this weekend. My test VM with 16GB storage ran out of diskspace after ~800k blocks.

Also, I can't find a way to specity the mysql host in the configuration file. Does it absolutely need a local mysql server to function? I'd prefer using a remote mysqldb that could handle all the queries more easily. ~~Can I change the host address in comm.py without breaking it?~~

Code:

conn = pymysql.connect(db=CONFIG["database"]["dbname"], host='127.0.0.1', port=3306, user=CONFIG["database"]["dbuser"],passwd=CONFIG["database"]["dbpassword"])

Changed the host and it works with a remote server

Lol... with abe it was a little harder to get this working...

Lookin' forward to a completely loaded DB \(^_ ^)/

The size of the database depends greatly on the number of transactions in the block chain.
Using HoboNickels as an example:
~2.3 Million blocks - ~6.5 million TX_IN and ~6.5 million TX_OUT
Size of the active database: 6.7 GB

I will add an option to set the host address for the database in the next commit. Personally, I have never had need for it so I never thought to put it in.

Quote from: oniromancia on June 25, 2015, 09:03:57 AM

This is happening on my server after every few weeks . We have a getblock call to altcoin daemon for each blocks. After few block numbers bitcoind RPC hangs (I have tried waiting for 30 min, but doesn't help, have to kill daemon and restart). I'm also using a node to make rpc calls.

This is why there are two timers in the database loader. The main loop timer is set at 5 minutes while individual RPC calls are set at 10 seconds. Since adding the timers I have not had to reset the loader or the daemons. If the loader gets interpreted/timeout, the next time it is called it re-parses the last 5 blocks including transactions, this helps guarantee information integrity if the loader is interrupted in the middle of parsing blocks/transactions. I have not noticed the coin daemons RPC completely locking up where it will not answer subsequent calls, they appear only to ignore/hang on one call.

Quote from: Hoshimaru on June 24, 2015, 12:43:19 PM

The things like the rich list and largest tx... is that processed after all the blocks are loaded or when invoking stats.py? I imagine it will take some time to process it on 1.7M blocks.

The stats module is called when the database loader has completed its cycle. However, the stats module can be run independently, but the information will only be accurate to the point the database is loaded. Largest TX is populated on the fly as the loader is parsing transactions, again only will be acurate to the point where the database is loaded.

Added - Running the stats.py module while the loader is running might interrupt the loader due to the way Python handles module imports. If one really wants to run the stats module while the loader is running I would suggest making a copy of comm.py named something like comm2.py and change the import in stats.py to import the copied file instead of comm.py.

Hoshimaru

sr. member

Activity: 411

Merit: 252

How much space does the DB require?
Guess I'll just have to test in on the VPS this weekend. My test VM with 16GB storage ran out of diskspace after ~800k blocks.

Also, I can't find a way to specity the mysql host in the configuration file. Does it absolutely need a local mysql server to function? I'd prefer using a remote mysqldb that could handle all the queries more easily. ~~Can I change the host address in comm.py without breaking it?~~

Code:

conn = pymysql.connect(db=CONFIG["database"]["dbname"], host='127.0.0.1', port=3306, user=CONFIG["database"]["dbuser"],passwd=CONFIG["database"]["dbpassword"])

Changed the host and it works with a remote server

Lol... with abe it was a little harder to get this working...

Lookin' forward to a completely loaded DB \(^_ ^)/

cinnamon_carter

legendary

Activity: 1148

Merit: 1018

It's about time -- All merrit accepted !!!

i spent a few hours the other night looking it over in detail setting up the environment, may have time this weekend to fire it up

oniromancia

newbie

Activity: 56

Merit: 0

Quote from: Hoshimaru on June 24, 2015, 12:43:19 PM

Thank you for replying dreamwatcher

I've doubled the amount of RAM for the VM and it did accelerate the process. It loaded over 200000 blocks the past half hour.
I must agree with you that the daemon and qt wallet is very slow. Changing the rpc timeout to 60 seconds in comm.py and the deamon's config also seems to have a positive effect.

We discovered that our coin was forked from Hackcoin and have managed to more or less clean the code and take control, but that never solved the performance issues. In fact, it has been like this since the beginning and staking puts a considerable amount of stress on the system, which shouldn't happen. I'm working with a coin that has been abandoned by the original developer and the two of us trying to keep it afloat, are no developers.

The VM is basically running on top of a computer that I used for mining altcoins last year. I extended the memory to 16GB, but the CPU is still the old one, an Intel Celeron G1610, which probably acts as a bottleneck for quite demanding processes. I intend on replacing it soon.

The VPS that runs bitcoin-abe atm is a 4 vCPU VM with 6GB RAM and 30GB SSD storage and has an offloaded MySQL DB running on Dual E5 2620 CPUs, 384GB RAM and 8 x 480GB SSD on RAID10. I don't worry about its performance in running CCE.

It's just that before deploying something, I tend to test it first and see what I can expect

The things like the rich list and largest tx... is that processed after all the blocks are loaded or when invoking stats.py? I imagine it will take some time to process it on 1.7M blocks.

This is happening on my server after every few weeks . We have a getblock call to altcoin daemon for each blocks. After few block numbers bitcoind RPC hangs (I have tried waiting for 30 min, but doesn't help, have to kill daemon and restart). I'm also using a node to make rpc calls.

Topic: Open Source CCE 4.0 Block Explorer (Read 3555 times)