Pages:
Author

Topic: [ANN] Datacoin - Censorship-Free Data Storage - page 9. (Read 66642 times)

legendary
Activity: 2424
Merit: 1147
With a big input from gjhiggins Grin we've updated the http://www.datacoininfo.org site!

Any further input welcome! Thanks
legendary
Activity: 2254
Merit: 1278
Datacoin 0.16.3 client under OSX

Mac users of the latest OSX (Big Sur) will need to option-click on the Datacoin app when first running it else Gatekeeper will only offer Trash it/Cancel. Option-clicking opens an extended dialog box informing the user that Apple can't validate the security of the app (i.e. it's not been signed by an Apple Developer who has paid Apple $100 p.a. to enable the code-signing facility) but does allow Gatekeeper to be bypassed and the app to be Opened and run thereafter.

Relevant blog post: https://lapcatsoftware.com/articles/unsigned.html

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
BIP39 tool working with Datacoin extended keys.

The 0.16.3 client uses the Bitcoin "Extended Key" protocol and, as an educational resource, I have cloned Ian Coleman's standalone BIP39 web UI for converting BIP39 mnemonic phrases to addresses and private keys and have added the Datacoin extended keys: https://github.com/gjhiggins/bip39.

To use with Datacoin keys, clone the repos, open src/index.html in your browser, click the "Generate" button to generate a mnemonic phrase, select the desired Datacoin network from the drop-down menu and use the BIP tabs to navigate the Hierarchical Deterministic key "tree".

Keys generated by the above procedure need to be imported into the 0.16.3 client wallet. To work with the keys available in the 0.16.3 wallet, dump the wallet (use dumpwallet in the RPC console), inspect the (text) dump, copy the "extended private masterkey" and, in the bip39 tool with "Datacoin" selected from the drop-down, paste it into the "BIP32 Root Key" field. This will give you a GUI which will enable you to work with the key hierarchy currently in the wallet. (Unfortunately, pasting a private key won't result in the tool displaying a mnemonic - it can only go from mnemonic to private key, not the other way round.)

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
State of the Datacoin network as of Monday 11th Jan 2021 midday GMT.

The subscription to the chainz.io DTC explorer expired and, as no-one renewed it, there's no externally-hosted dedicated block explorer for Datacoin atm. Edit: Ah, yes there is: https://www2.bytestamp.net/blocks/index but it doesn't provide the list of recently-seen nodes that the chainz explorer used to.

As far as I can ascertain from my Hetzner-hosted node, the below represents a snapshot of the current Datacoin network in terms of IP address and client version with the IP of listening nodes¹ in bold and the IP of nodes that I own in italic:

[2a01:4f8:191:7330::2]:57252/Datoshi:0.16.3/
[2001:19f0:6801:3b9:5400:1ff:fe35:be4f]:4777??
144.76.118.44/Datoshi:0.16.3/
144.76.64.49:36046/Datoshi:0.16.3/
51.148.146.204:59696/Datoshi:0.16.3/
89.201.185.16:30604/Datoshi:0.16.3/
140.186.218.230:59638/DatacoinCore.Veter:0.15.99.8/
140.186.218.230:57980/DatacoinCore.Veter:0.15.99.8/
31.131.65.221:49687/DatacoinCore.Veter:0.15.99.8/
102.165.224.64:56402/DatacoinCore.Veter:0.15.99.8/
45.33.238.99:12234/DatacoinCore.Veter:0.15.99.8/
151.80.96.105/Satoshi:0.8.3/
85.19.25.38:4842/Datacoin:0.8.6/Datacoin:0.1.2(v0.8.6.0-dirty)/

¹ Listening nodes are those to which clients can connect in order to sync up to the last block minted.

Edit: My local 0.16.3 client is seeing a partially-connected node on 45.63.115.238 (apparently somewhere in FR) which is the same IP address as www.datacoin.tk, the site of (the other advertised) Datacoin explorer which is struggling to serve and appears to be stalled since 29 August 2020.

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
Transactions and Segwit with the alpha 0.16.3 client
I forgot a small but crucial detail. In order to be able to broadcast transactions using the 0.16.3 client on testnet, you need to add prematurewitness=1 to the config.file. (I've made an example 0.16.3 testnet client config file available as part of the alpha prerelease on https://github.com/gjhiggins/datacoin)

The reason for this requirement is that the alpha-release 0.16.3 client is configured by default to use segwit addresses src/wallet/wallet.h::Loc#112
Code:
enum OutputType : int
{
    OUTPUT_TYPE_NONE,
    OUTPUT_TYPE_LEGACY,
    OUTPUT_TYPE_P2SH_SEGWIT,
    OUTPUT_TYPE_BECH32,

    OUTPUT_TYPE_DEFAULT = OUTPUT_TYPE_P2SH_SEGWIT
};
The broadcast/acceptance of segwit-address transactions is enabled by the prematurewitness assertion in the config.

Addresses in the client are the Dxxx ones familiar to users and this is configured via the config assertion addresstype=legacy. This latter assertion is local to one's client and can be trivially removed later, after an upgrade to segwit takes place.

Segwit and Bech32 addresses are a new feature of later varsions of Bitcoin Core. I recommend that when you import privkeys from 0.8 clients into the 0.16.3 client, you ensure that you provide a label, otherwise it can get a bit confusing because importing a legacy privkey results in three addresses appearing in the "Receiving addresses" list - the legacy address, the segwit address and the bech32 address.

You can check this out for yourself with the new makekeypair and listkeypair RPC/console call I've included. The results include the three addresses associated with the privey as well as the actual ECC private and public keys and, ofc the wallet privkey that you'll be familiar with. (Yes, they're all different, I recommend that you see for yourself.)

The aim of having the client default to segwit-enabled addresses is basically to enable the Datacoin chain to function in a pre-segwit state, pending a migration to segwit. The 0.16.3 client contains an activating protocol which enables a seamless soft-fork to segwit, as explained by Chicago:
The development agenda does include activating protocol which was never enabled for us during the upgrade from 0.8 to 0.16.

I've spoken briefly about it in the past.

We can simply build clients with updated chain parameters (in source) and require miners to eventually create blocks in the new format (the regular upgrade process we skipped which uses IsSuperMajority and BIP 9 deployment).

The activation protocol (technical name: "versionbits") is configurably sensitive to the proportion of blocks mined during a time-limited window (https://github.com/gjhiggins/datacoin/blob/bf9d6ae2a2c8ec27ca1ec0fac801f483e9f9db5e/src/chainparams.cpp#L118). If the proportion exceeds a given (configured) value during the window, the new protocol is permanently activated. Although the 0.16.3 client isn't yet configured to do this, if it were, then if the majority of miners were to use the 0.16.3 client, the chain would roll over to the new protocol and segwit would be permanently enabled for the Datacoin chain.

For the Bitcoin cloneparent, this activation mechanism has fulfilled its role, the Bitcoin blockchain successfully transitioned to segwit over three years ago (on August 24, 2017 at 2:57 AM). Subsequently, the redundant code implementing the activation protocol was removed from the Bitcoin codebase, being replaced by a simple variable assignment (https://github.com/bitcoin/bitcoin/blob/094402430925ec5aac6edbbf52d74f10c665da43/src/chainparams.cpp#L76). The effect of this change means that an upgrade of Datacoin to a more recent clone of Bitcoin (say 0.20) would necessitate a hard fork and all the disruption, organisational effort and risk of failure that would entail. That's why the current "upgrade" is to a fast-ageing 0.16.3 - once the Datacoin chain has soft-forked to segwit, a subsequent upgrade to a more recent clone of Bitcoin becomes far less onerous to arrange.

REST interface to stored data on the alpha 0.16.3 client
One advantage of the 0.16.3 (and later) client is the addition of a REST interface. HTTP requests aimed at the standard RPC port of a host running the 0.16.3 client configured with rest=1 return results specific to the data included in the URL, e.g. http://localhost:11776/rest/tx/02d46971b8cba3fd1b3b4469f40676005dcb2313a24d13c36bb387a93b785bf6

The REST interface is not extensive as the somewhat self-explanatory code suggests:
Code:
uri_prefixes[] = {
  {"/rest/tx/", rest_tx},
  {"/rest/block/notxdetails/", rest_block_notxdetails},
  {"/rest/block/", rest_block_extended},
  {"/rest/chaininfo", rest_chaininfo},
  {"/rest/mempool/info", rest_mempool_info},
  {"/rest/mempool/contents", rest_mempool_contents},
  {"/rest/headers/", rest_headers},
  {"/rest/getutxos", rest_getutxos},

The implementation respects an "extension" protol, i.e. appending .hex to the above URL would return the transaction in hex and appending .json would return the results in the familiar JSON format.

I have extended the Datacoin REST interface with a txdata addition to the protocol which returns only the data of a data-storing transaction. In the designed primary use case, it accepts a .html extension - which is, of course, only meaningful for data stored as plaintext HTML (i.e. unlike the QDesktopServices "View locally", it does NOT handle gzipped/zipped stored HTML).

You can still use the GUI data storage facility directly to store binary data such as JPEGs and, because you will know the mimetype of what you stored, you can access the data via the REST interface by providing the appropriate extension.

So, if you are running the 0.16.3 client on testnet on the standard port (and configured with rest=1), the following URL will work in your browser: http://localhost:11776/rest/txdata/769864e01f7eff3d49a5381983f8f6078513d50789704eeea75363de1c90ec22.html.

The difference between the "View locally" QDesktopServices approach and the REST approach is that the REST approach allows you to use Datacoin host txdata URLs to retrieve images instead of having to include them statically as base64-encoded strings.

i.e. the "hero" image inclusion tag in my blog post has chenged from:
Code:

to
Code:

The storage saving is substantial. (BTW, you can try the above in your browser, it'll show the hero image.)

It's slightly cheaper to store images as binary than it is to include them statically as base64-encoded, I found a net saving of 1 DTC over the "View locally" QDesktopServices approach.

Users of the Linux/Windows/OSX binaries I made available will need to d/l the latest version from https://github.com/gjhiggins/datacoin/releases/tag/alpha (where the example config file is also located). Self-compilers will need to git pull and recompile.

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
It ok now, i forget to delete old blockchain.
Compiled ok and i running main wallet and testnet.

Phew!

Thanks for reporting back.

Cheers

Graham
jr. member
Activity: 44
Merit: 12
It ok now, i forget to delete old blockchain.
Compiled ok and i running main wallet and testnet.
legendary
Activity: 2254
Merit: 1278
Hay,
I have this error.

EXCEPTION: NSt8ios_base7failureB5cxx11E      
CDataStream::read(): end of data: iostream error      
datacoin in Runaway exception      

datacoin-qt: /usr/include/boost/signals2/detail/lwm_pthreads.hpp:60: void boost::signals2::mutex::lock(): Ubacivanje „pthread_mutex_lock(&m_) == 0”

On  0.16.3 client
Cloned from https://github.com/gjhiggins/datacoin or Linux binary from same? Testnet? Architecture? Is it the result of doing specific?

It's a large and complex codebase and it's been running without issue on several of my Linux machines for a few weeks now so the task is to try and work out why this error is occurring on your system.

Cheers

Graham

jr. member
Activity: 44
Merit: 12
Hay,
I have this error.

EXCEPTION: NSt8ios_base7failureB5cxx11E      
CDataStream::read(): end of data: iostream error      
datacoin in Runaway exception      

datacoin-qt: /usr/include/boost/signals2/detail/lwm_pthreads.hpp:60: void boost::signals2::mutex::lock(): Ubacivanje „pthread_mutex_lock(&m_) == 0”

On  0.16.3 client
legendary
Activity: 2254
Merit: 1278
It only works with uncompressed data right now but I may be able to develop the code so that it handles compressed data.
It now uncompresses gzipped/zipped data, detects the mimetype of the uncompressed content and saves that uncompressed content as a file in the OS' temporary file area with the relevant file extension to allow QDesktopservices to call the appropriate viewer. So now you can zip/gzip your standalone web pages, saving a few DTC.

I've also fixed a small interface bug introduced in the 0.15.99 client here src/wallet/rpcwallet.cpp#L532. The 0.15.99 implementation improperly checks the size of the base64-encoded data against MAX_TX_DATA_SIZE and I'm afraid I transcribed the error into the 0.16.3 client (src/wallet/rpcwallet.cpp#L560).

A correct implementation would check the length of the data after base64-decoding it - the base64 encoding is just a transport mechanism for ferrying the binary data to be stored through the RPC interface - although in this instance, the data length check is actually performed later, at the stage where the transaction is being generated, so the improper check is actually redundant and I have removed it: src/wallet/rpcwallet.cpp#L564

I've refreshed the Windows, OS X and Linux binaries for the alpha release https://github.com/gjhiggins/datacoin/releases/tag/alpha.

If you're on Linux and want to compile your own binaries, git clone https://github.com/gjhiggins/datacoin.git will get you this latest source.

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
Playtime.

The 0.16.3 client seems to be behaving quite well on testnet, I've not experienced any issues of late.

I've removed the madpool code introduced by Veronium/Crossverse, restored the internal miner and added a GUI mining page with which you can configure the mining parameters and see the effects on your hashrate via a hashrate graph. Removing the madpool code also removes the protobuf issues people were encountering when compiling the client.



I've also added a tab to the client which implements a basic GUI interface to the data storage facility.



You can use the tab to store files of data, selecting the file via your OS' filesystem navigator (via "Select File") and then create the transaction by simply clicking the "Store in the chain" button. If successful, a confirmation dialog box will be shown and the transaction hash will appear in the "Transaction Hash" field.

You can also use the tab to view stored data by entering the relevant transaction hash in the "Transaction Hash" field. Behind the scenes, the data is retrieved and written to a temporary file in your OS's temporary file storage location, the mime type is detected and the file renamed with the appropriate extension to allow the file type to be recognised by the OS and then Qt's QDesktopServices function is called with the temporary filename as "file:///..." protcol URL.

Detecting the mimetype of stored data is quite primitive at the moment. It only works with uncompressed data right now but I may be able to develop the code so that it handles compressed data. As it stands at the moment, the facility is mainly intended to show standalone web pages.

I used the GUI interface to store a standalone page of HTML on the testnet chain. It can be viewed using the transaction hash: f84ee63441ff28322edb0157d33159815f29bd52398a0b82c6ddd58f93f74f5f, clicking on "View stored data locally" will open the file in your browser.

Whilst I don't recommend you use it in as your standard mainnet client, it does sync mainnet from 0 in a couple of hours. When synced, you can view details of mainnet transaction records on the Bytestamp service by entering a transaction hash and clicking on the "View Bytestamp record" button which will open the relevant Bytestamp web page in your browser.

I encourage you to have a play on testnet. The alpha release on https://github.com/gjhiggins/datacoin/releases/tag/alpha offers binaries for Linux, OS X and Windows 64-bit. The repos can be checked out and builds out-of-the-box on my Mint Ulyana (Ubuntu 0.20).

Cheers

Graham
legendary
Activity: 2424
Merit: 1147
Renewed http://www.datacoininfo.org for another year  Grin Grin

If anyone would like to go through the site with me to remove dead links / update content and add new links then PM me and we will get it sorted.
newbie
Activity: 4
Merit: 0
sampei7777's last complaint was that the "block chain slowed down every day", which obviously would just be a result of mining habits.  Their file storage service registers only hashes of files when the chain is designed to store full files.  It looks like sampei7777 is trolling this community.  I wouldn't prioritise their concern.  EDIT: apologies for this.  Obviously this community needs every person it can get.  I talked with gjhiggins on github and discord and I think sampei7777's concern around the 0.8 client has been resolved.  I think solving it involved removing a check regarding peer behavior.
legendary
Activity: 2254
Merit: 1278
I understand that there are some issues with the modern datacoin clients.
See posts by sampei7777 one page upthread.

Cheers

Graham
newbie
Activity: 4
Merit: 0
Hey all,

I understand that there are some issues with the modern datacoin clients.  Is anybody able to add issues to the project page at https://github.com/gjhiggins/datacoin/issues for the bugs, to make them easier for visiting developers to find and fix?  Additional specifics such as transaction hashes, data, client version, and logs are also incredibly helpful.
member
Activity: 92
Merit: 18
How to register file to blockchain with Ubuntu 20.04


For a deeper insight: How to register file into blockchain with Ubuntu
member
Activity: 92
Merit: 18
How to register file to blockchain with Ubuntu 20.04

1)  Install prerequisites:
   
Code:
sudo apt-get install libcryptx-perl jq

2)  install Bitcoin core
   
Code:
sudo snap install bitcoin-core

3) clone the ByteStamp repo
   
Code:
git clone https://github.com/bytestamp/bt.git

4) make the script executable
   
Code:
chmod +x bt/bt

5) depending on your environment, put the script in your path
   
Code:
 sudo cp bt/bt /usr/bin

Done!

Now you can do

Code:
bt FILEs

This will list BlockChain information about FILEs

If no file is registered to BlockChain then register FILEs

The script is self-explained and can be used to understand how to connect to ByteStamp webservices with any programming language.

legendary
Activity: 2254
Merit: 1278
Quote
But from what I understand, the 0.15 version is the unique that can broadcast txs  to both 0.8 and 0.16 nodes (and 0.15, of course).
I don't think that's the case but I will check.
I checked with my own version of the 0.16.3 client (available here) and it does successfully broadcast senddata txs: https://www2.bytestamp.net/blocks/qtx/en/c58ecabe71ffa403b9c962a509ff3ffda3659b895ffbca23db714c06c4a4d51f

I've cut an alpha release of this reference client if people want to d/l it and play with it on testnet https://github.com/gjhiggins/datacoin/releases/tag/alpha (I've made Windows and OS X binaries available) and I have a node running the Datacoin testnet chain on 144.76.118.44, generating blocks for 2mins out of every 10, just to keep the tip from getting stale. Feel free to have a play, difficulty is low. Unless configured otherwise, the client will sync to the Datacoin mainnet chain (although harmless, it's not what you want atm), so an appropriate conf file would be:

Code:
rpcuser=
rpcpassword=
testnet=1
server=1
listen=0
txindex=1
gen=0
genproclimit=1
listenonion=no
addnode=144.76.118.44
(edit to suit your preferences), or just use -testnet on the command line if you prefer.

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
This is the second part of the response

What I was trying to do with

Code:
 addresstype=legacy

instead, was the opposite thing, i.e. sending a not-segwit tx from 0.16 client.

But your suggestion could work. I don't thing broadcasting segwit-enabled txs to a mixed network will cause a fork across versions.

In fact old clients see segwit txs as normal tx.

So the blockchain should not fork and maybe segwit txs could also be mined from old client.

BTW, it is a dangerous thing to do. We all don't want to risk a fork of Datacoin.
Thank you, the pointer to "old clients see segwit txs as normal tx" was most useful. The key fact is contained in the post following the one you linked to:

So I have this situation:

1) If I broadcast a tx from 0.8 client  it is not propagated across 0.16 nodes in the network and so the transaction don't get confirmed for days

2) If I broadcast a tx from 0.16 client the tx is never broadcasted nor confirmed because it causes the error

Code:
CommitTransaction(): Transaction cannot be broadcast immediately, no-witness-yet

and the wallet loses all its datacoins because it never receives from the network the tx of the unspent ones

Long story short, I ended with installing a 0.15 version and I deactivated all my 0.8 an 0.16 clients.
Given that experience, I can understand why. Smiley But there may be some other factor causing the problem - I've just successfully received some DTC from freiexchange with my 0.16.3 client (assuming that freiexchange are using an 0.8 client which may not be the case). After some "fettling", I now have a compiled 0.15.99 Crossverse reference client and am working to get the (rather elderly) 0.8 client to compile on my Mint/Ubuntu 0.20 laptop. Then I can replicate your experiment and maybe have a chance to see where things go astray.

Quote
I also suspect that running different node versions from the same public IP will cause network issues, you could confirm this.
No, that should work without problems, they'd each have a different port/rpcport assignment.

Quote
But from what I understand, the 0.15 version is the unique that can broadcast txs  to both 0.8 and 0.16 nodes (and 0.15, of course).
I don't think that's the case but I will check.

Quote
Each day between 6:00 UTC and 7:00 UTC the blockchain slows down.

You can see this  on the Datacoin block explorer selecting 3710777 as starting block.

This is happening every day.

Any idea?
No at the moment, there's nothing in the code (that I know of) which could cause that. It'll be worth looking at the hash rate and the tx record for that time of day.

Cheers

Graham
legendary
Activity: 2254
Merit: 1278
Thank You for your reply.
...
Now I am facing another issue. Each day between 6:00 UTC and 7:00 UTC the blockchain slows down.

This is part one of a two-part response.

Providing even limited support via bitcointalk is both time-consuming and awkward, so I've set up a Datacoin Discord server for this express purpose, here's the invitation https://discord.gg/evxVsb5FXB. I'm usually online mid-morning to late at night GMT if anyone wants specific help or advice.

fwiw, I'm working on the Datacoin 0.16.3 reference client at the moment so I'm interested in how these issues may be resolved.

Cheers

Graham
Pages:
Jump to: