Topic: [SKY] Skycoin Launch Announcement - page 103. (Read 381579 times)

I keep on getting "Requesting DHT peers"

Anyone experiencing the same?

I sent my IPO request long time ago but still havent received a reply, i already got "acknowledgement of message received".

Update:

There was a "disappearing coins" bug on windows.
- You download all ~120 blocks, you see 100 coins
- You close client, there is not sig signal for clear program exit triggered on windows, so you lose all the blocks (they are not saved to disc, like they are on exit, on linux)
- You open client and the coins are "missing", until you redownload the blockchain to current head

So coins were still there and appears to just be bug with blockchain not being saved to disc on exit on windows. So coins dont appear until the blockchain has been redownloaded.

This needs to be fixed.

Other bugs/Things to do
- connecting to network takes a bit of time. Some users cannot connect at all.
- PEX stores and exchanges peer lists. It is exchanging peers that cannot be connected to (only allow outgoing connections). PEX should only store peers that allow incoming connections.
- PEX should keep time you last connected to peer and keep list sorted. So you immediately connect to old peers on restart.
- PEX should be moved into the skycoin repo, so we can update it
- PEX needs interface in GUI, to allow you to see peers you are connected to and to add peers by hand, as default connections
- PEX needs option to only connect to selected list of peers and not publicly (in case you need hide application)
- Networking is still TCP/IP. We need to upgrade to UDP with hole punching. Until then, it is very difficult to connect to people behind firewall. The connection interface is abstracted, so adding new types of connection type (HTTPS, UDP) should be easy

Transaction Queue TODO
- the /transactions does not show pending transactions, but instead shows outputs. Need to fix this
- need to show pending transactions in and out, for your wallet, in the gui
- need to queue up transactions when multiple transactions are outgoing from same wallet

There are a few hundred improvements, like this that need to be made until its perfect.

Hello Skycoin,

I am not able to get this new skycoin download to work. It starts then just shuts down straight away.

The old download (skycoin-win) does work and the http://127.0.0.1:6420/ link shows my wallets, but there are no initial test coins showing in the wallet.

Can you please let me know why this download doesn't work and explain what to do in a simple and easy to follow manner.

Thanks.

2500 Skycoin per Bitcoin. Which is about $0.10 / Skycoin. Bitcoin to USD is too volatile so just set it to that.

I am going through the bitmessage requests by hand.

I think we started getting bitmessage requests for IPO in about two months ago and are just sending out coins now. However, there have been a lot of changes and fixes since then, so steady progress. We have to fix two more API calls and then can have Skycoin exchange traded.

For IPO. I am trying to do 2 to 24 hour max turn around time. between receiving bitcoin and sending out coins. I wrote a digital cryptographically signed receipt system, but am not using it. I am realizing that money is just monkeys trading sea shells and shiny pieces of metal they found on the ground. I probably should not be using elliptic curves for this.

There are about fifty people in the IPO so far. The average amount is $2000. About half the coins in the allotment are already accounted for. Very good so far.

New Bugs:

- If you try to send and the send fails because of insufficient coins, it did not give a popup and only printed error in terminal. That is fixed now.
- There was a bug discovered yesterday, where you can do multiple transactions from the same inputs (only one of the transactions will succeed). So if you do two sends form same wallet and transaction propagation is not successful, then some of the transactions wont go through. Fixing this now.
- transaction sync on connect may not be working. It works sometimes, but I think problem was somewhere else. I think network was experiencing a netsplit. Connecting to random nodes does not guarantee a robust network topology. We need to allow user to create persistent list of nodes you should always try to connect to. To ensure the core-nodes are fully connected.
- Two people reported disappearing and reappearing coins. there may be a bug in coin balance calculation. The coins never moved, but it is just bug in GUI, looking into this.

Network Status:

We will be out of things to code soon and then it will just be debugging and polish.

There are cyber warfare drills and border gateway protocol redirect attacks in middle east that are deteriorating VPN service and dropping people from IRC in Europe. I think we were seeing internet kill switch tests yesterday. We were seeing protocol specific deterioration of VPS traffic between hosts, only for traffic passing between particular countries and only for VPN but not for HTTPS or SSH traffic. Skycoin block and transaction peer-to-peer propagation seem to be holding up and were only intermediately affected for a few hours, even through the vpn was out.

We still have a lot to do in order to harden the network.

Once Skywire is working, we might want to put in some networking instrumentation for transaction/block/consensus propagation and node connectivity and then have nodes publish/aggregate that. Quantitative data on block propagation and network state could be helpful. Then we can verify Sybil attacks, diagnose outages, detect net-splits and identify disruptive nodes, instead of just guessing.

One of our site installations was using a consumer wifi router. The router was hacked and the DNS on the router was changed, so that it inserts popup ads into webpages. It could also hijack your computer if you downloaded an exe or and pdf, or potentially steal Bitcoin or other nasty things. We recommend routers with open source firmware like openwrt. Ubiquity hardware tends to be very and the security is much better than Linksys or other consumer routers.

It is worth $80 to get a router and networking equipment, that is not going to get hijacked and get your stuff stolen. If a website has 3rd party http tracking stuff in it and the router dns hijacks the page, I think they can keylog whatever you type through javascript even if the webpage is https.

Diagnostics:

This is the crypto-logic level data. This is canonical and is more reliable than the gui.

http://skycoin-chompyz.c9.io/blockchain/blocks?start=0&end=500
http://skycoin-chompyz.c9.io/outputs
http://skycoin-chompyz.c9.io/blockchain

This is
- the blockchain (the ledger of transactions and blocks)
- the unspent output set (the current account balances)
- the state of the client (the head block the client has)

If you are running a local skycoin client you can access these locally using

http://127.0.0.1:6420/blockchain/blocks?start=0&end=500
http://127.0.0.1:6420/outputs
http://127.0.0.1:6420/blockchain

This is very clean. This is what I think a second gen interface should look like.

The second generation is just cleaning up Bitcoin and making it usable. The third generation is where we begin to go beyond Bitcoin.

Ok, I managed to communicate through bitmessage and got some test SKY coins.


I invested BTC to the specified BTC address inside the bitmessage. When will I get the real SKY coins and see them in my wallet balance?

UPDATE: I can now see the SKY in my wallet balance

Update:

Fixed a lot of bugs.

The top levels, the gui and json are not as polished as lower levels (blockchain/crypto). I have not even looked at some of this code and discovering new things. There were things implemented, I didnt know we had working and things that needed to be cleaned up.

- there was a coinhour calculate bug that prevented transactions in strange cases. I hardcoded coinhour fee for transactions to 1, to get it working, but then after receiving coins, some people were unable to send them out.
- there was bug where sometimes send failed if balance for wallet was split among multiple inputs. Should be fixed now.
- I am removing code, simplifying the top level. We are at point for several months, where most of work is removing code, simplifying.
- Most users are behind firewall, they can connect out but not in. Two servers are running with open ports. We need more servers with open ports, running the client
- we need indicator for how many clients you are connected to, what block number is and time since last new block
- we need button for connecting to server by hand
- we need better configuration for how many outgoing connections to maintain, how many incoming connections and a list of server that you should always be connected to. Once Skywire is working, you will be able to just put in address/pubkey hash and then have authenticated communication to end point and that will speed up network a lot. Right now, it is possible to flood network with dead nodes or bad nodes and prevent honest nodes from connecting.
- we need json function for returning the pending unconfirmed transactions. This is six or twelve lines of code and almost done. (see below)
- we need the pending transactions for your wallet (incoming and outgoing) to show in the interface
- we need the unspent output set at http://127.0.0.1:6420/outputs to be sorted, either by time, block # or output id. We store a metadata uint64, that indexes each output, so you can say "output 15" created by "transaction 3" and each output and transactions will have a unique sequentially numbered int, for a given chain. However this value is not in the hash and may change if the transactions are migrated to another ledger, but the transactions/output hash will be same.
- If there are N-transactions from same wallet, we need to freeze later transactions (return error) or queue up transactions. The outputs from the wallet will be spend, so if the first transaction clears the second will be invalid (it needs to use outputs of the wallets after transaction 1 clears, not outputs that existed at time transaction was created).
- Base58 encodes addresses as integers. Some private keys generate addresses that decode to less than length of what addresses should be. I tested 128,000 random addresses during testing and decoding never failed for any of them. If someone has a private key, where people are unable to send to that address, because it is invalid please go into http://127.0.0.1:6420/wallets and send me the wallet seed to BM-2cXFat4fHmeRe8EFJjY3Dzo6RoifcgiKgp . No one will have coins in this address, because send fails because address is invalid, but it seems to happen to a few person. I cannot tell if people are copy/pasting addresses incorrectly or if its a base58 issue
- When a new block is minted, if you are connected to three people, you will request block as soon as you get notification. So you will end up making 3 download requests and getting block three times. This speeds up block propagation, so may not be a bug.
- Skycoin still kills someone's wifi for unknown reasons (one user has reported). This may be DHT (which is same as bitorrent uses). The issue is probably the router firmware.  
- PEX may need to be modified, so that it only keeps list of clients that allow incoming connections. If you cannot connect to the peer, knowing about them does not help the client.

People have reported that port 6000 is X11. Skycoin to prevent blocking by ISPs is not bound to particular port. DHT assumes port 6000. So disable DHT and then make sure you have people in your peer list, or manually connect to server. Once you are finding peers through PEX, you dont need DHT and port does not matter.

Other:

There are API commands I did not know about, that I just found in the code. Someone should document the API/json urls

You can test these on http://skycoin-chompyz.c9.io/blockchain/block?seq=5

Get blockchain head, header
- http://127.0.0.1:6420/blockchain
Get block as JSON
- http://127.0.0.1:6420/blockchain/block?seq=5
Get range of blocks as JSON
- http://127.0.0.1:6420/blockchain/blocks?start=2&end=30

Unspent Outputs:
http://127.0.0.1:6420/outputs

Get Network Connections:
http://127.0.0.1:6420/network/connections (renamed from /api/network/connections )

Make a network connection to peer:
http://127.0.0.1:6420/network/connection?addr=127.0.0.1:6000 (renamed from /api/network/connections )

Blockchain Progress:
http://127.0.0.1:6420/blockchain/progress

This is the highest block you know about. If you are not connected to anyone it will be zero, I think. There appears to be off by one error.

Documentation:

https://github.com/skycoin/skycoin/blob/master/src/gui/wallet.go

In /src/gui/wallet.go there is a URL handler that returns the JSON serialization of the unspent output set. Let us look at how that is implemented.

We define a function, that handles the URL request. Grabs the outputs from Visor and then returns the JSON or 404 error.

// Returns the outputs for a wallet
func getOutputsHandler(gateway *daemon.Gateway) http.HandlerFunc {
   return func(w http.ResponseWriter, r *http.Request) {
      ret := gateway.Visor.GetUnspentOutputReadables(gateway.V)
      SendOr404(w, ret)
   }
}

Then we add the handler to
func RegisterWalletHandlers(mux *http.ServeMux, gateway *daemon.Gateway) {}...

//get set of unspent outputs
mux.HandleFunc("/outputs", getOutputsHandler(gateway))

So it is about 7 lines of code, to expose the outputs as JSON. The "readable" for converting internal Skycoin objects to json is in /src/visor/readable.go https://github.com/skycoin/skycoin/blob/master/src/visor/readable.go

Looking at the godoc for Visor, we can see the functions it exposes

https://godoc.org/github.com/skycoin/skycoin/src/visor#Visor

Visor is one of the dirty libraries that need to be refactored and cleaned up. However it is very clean already. Some cleanup steps could be
- moving blocks into a blockstore library that visor pulls in and removing block serialization/saving to disc from visor
- removing the remaining functions in "spend.go" and moving into wallet handler
- fixing the unit tests
- exposing the full blockchain client API over network with Golang/RPC (partially implemented?)
- moving the readable to its own library or subfolder
- deleting json_rpc.go (which is empty)
- decide if the JSON api web server should be in visor, instead of gui or if it should be its own library or should stay where it is
- move the functionality out of /src/daemon into /src/visor and deprecate the existing daemon for the skywire daemon, making visor top level instead of daemon being top level

There is no transaction history. You can look at the blocks and the transactions using json API. You can check balances by looking for your address in the unspent output set ( http://skycoin-chompyz.c9.io/outputs ).

The coin history and Skycoin blockchain exporer, should
- go through block by block, creating index of addresses, transactions, blocks
- be its own library
- should run in the skycoin web wallet, locally, not on third party site
- should have a JSON api

IPO misc

You can now do Skycoin transactions. Some people are having trouble getting peer connections however. The network will be spotty while we clean things up. We recomend updating the client to latest version every two days.

Some test coins have been sent out.
- You have to find someone who has them
- If you put in request on bitmessage, check your wallet and you may have test coins. These are still being sent out.

Some people are having trouble connecting/syncing blockchain. Most users running client cannot receive incoming connections, so connecting to first peer on network is difficult. We need more people to run skycoin client on vps server, that has open port.

It should be possible to blocksync over HTTPS also. We should be added for reliability and for countries with protocol level censorship.

The blockchain and coin balances are being backed up daily, in multiple locations. This ensures coin ownership is maintained if there is an unforseen emergency or attack situation. If a coin has been in a output for over 24 hours, we can ensure with 100% certainty, that ownership stakes will be rolled over regardless of future events. The current chain of transactions/balance is being preserved, even if they are migrated to a new ledger in future upgrades.

Warning: the send function is a bit buggy. If you send, make sure that the balance in wallet has changed before doing another send. Sometimes the transaction does not proprogate. This is because the client is not connected to any peers or because half the nodes on network are using an older version of the client with a bug. Working on this. Adding pending transaction queue and intergrating it into the gui and other measures to make transaction injection fail safe.

IPO:

Processing the IPO receipts over bitmessage has started.

Your can check transactions here
http://skycoin-chompyz.c9.io/blockchain/blocks?start=0&end=500

Can check balances here
http://skycoin-chompyz.c9.io/outputs

@chompyZ: Thank you for taking time to answer my questions. I will try it the following days


EDIT: I sent the bitmessage, it says "Message sent. Waiting for acknowledgement."

How long do I have to wait to get the ACK?

AFAIK-

A: yes. Both addresses are legit, but I suggest you use an address that YOU generated. That way, you can remember your seed and generate it easily in the future, or load it from seed in the GUI...

A: in Linux, it is under /.skycoin/wallets/wallet-name.wlt

A: maximum cap has not been mentioned anywhere.

A: afaik- yes... the BTC/USD exchange rate for this calculation will probably be finalized at the last minute based on market price.

A: no idea. Imho it is not going to be a rolling IPO that extends for long...

Avast and MalwareBytes don't let the program run properly, but it runs when they're turned off.
I'd like to try and invest some, just because I value and appreciate innovation.

Help me to clarify things a bit:

By running the "address_gen" I get an address, a public and private key.
By running "skycoin" I open a browser to http://127.0.0.1:6420/ and I see a wallet and it's address but I don't see the public/private key of the address.


Please answer the following, when you'll find the time:

1) How do I participate? I install BitMessage and then send a message to as specified here (https://bitcointalksearch.org/topic/m.10555070) with the address I got from running "skycoin" or "address_gen" I guess it's the same, but I want to be sure. Private keys from running "skycoin" are stored inside ".wlt" files, correct?

2) Is there a maximum cap per user?

3) Is the price still $0.10 / Skycoin?

4) Which is the exact date that IPO ends?

Thank you in advance,
wizzardTim

How much metadata is embeddable for smart contracts?

It's great to see the network being boostraped, congrats!


So is the consesus module already pluged in ? Or is this minting process centralized right now ?

Update:

The network is running. Fixed a bunch of bugs.

Structure of Skycoin Networking:

This is overview of wire protocol.

https://github.com/skycoin/skycoin/blob/master/src/daemon/messages.go#L40

NewMessageConfig("INTR", IntroductionMessage{}),
NewMessageConfig("GETP", GetPeersMessage{}),
NewMessageConfig("GIVP", GivePeersMessage{}),
NewMessageConfig("PING", PingMessage{}),
NewMessageConfig("PONG", PongMessage{}),
NewMessageConfig("GETB", GetBlocksMessage{}),
NewMessageConfig("GIVB", GiveBlocksMessage{}),
NewMessageConfig("ANNB", AnnounceBlocksMessage{}),
NewMessageConfig("GETT", GetTxnsMessage{}),
NewMessageConfig("GIVT", GiveTxnsMessage{}),
NewMessageConfig("ANNT", AnnounceTxnsMessage{}),


- Each message is a struct.
- You put values in struct, then do a "Send" and say who you want to send it to
- It packs up the struct in canonical binary format
- the struct has a .Handle() method. When the data is received, this method gets called on the receiver. The struct is already filled in, with its values


PEX: Peer Exchange

This is "give me peers" and "request peers". This is for finding other peers running skycoin (and later skywire). These get saved into  ~/.skycoin/peers.txt . Once you have enough peers, you can disable dht with -disable-dht flag.


NewMessageConfig("GETP", GetPeersMessage{}),
NewMessageConfig("GIVP", GivePeersMessage{}),

This is good example

type GetPeersMessage struct {
   c *gnet.MessageContext `enc:"-"`
}

// Notifies the Pex instance that peers were requested
func (self *GetPeersMessage) Process(d *Daemon) {
   if d.Peers.Config.Disabled {
      return
   }
   peers := d.Peers.Peers.Peerlist.RandomPublic(d.Peers.Config.ReplyCount)
   if len(peers) == 0 {
      logger.Debug("We have no peers to send in reply")
      return
   }
   m := NewGivePeersMessage(peers)
   d.Pool.Pool.SendMessage(self.c.Conn, m)
}


Transactions:

This is how transactions are replicated peer to peer. This is before transactions are placed into blocks.

NewMessageConfig("GETT", GetTxnsMessage{}),
NewMessageConfig("GIVT", GiveTxnsMessage{}),
NewMessageConfig("ANNT", AnnounceTxnsMessage{}),

When you connect to someone:
- you request list of hashes for the transactions they have.
- if they have something you dont have, you request the data for it

Gossip Protocol: When you learn about a new transaction
- when you get a new transaction, you broadcast the hash to everyone you are connected to
- if they dont have the transaction already, they request it from you

Blocks:

Very similar to transactions

NewMessageConfig("GETB", GetBlocksMessage{}),
NewMessageConfig("GIVB", GiveBlocksMessage{}),

Overview:

This is much cleaner than Bitcoin's wire protocol. There are no bloom filters. There is minimum of code.

Skycoin uses a paradigm of source independent networking, similar to urbit. It is also similar to rsync.

- You have a list of objects in your local store
- each object is "named" by the hash of its binary serialization
- You connect to remote peer
- You check list of hashes for objects you have, against list of hashes for objects they have
- you do exchange, so you get any objects they have which you dont
- you apply a filter (discarding invalid transactions or executed transactions from your store or whatever policy is)

So if person A injects a new transaction to the network, it will spread throughout the network. Eventually a block minting server will shove it into a block and it will executed.

In Skycoin there are three type of data channels. These are peer-to-peer exchanges required for network operation
- transactions (things that users create to spend coins, that replicate between nodes and then are inserted into blocks)
- blocks (lists of transactions, with some header information, such as hash of previous block in the chain)
- consensus information. This is how a server decides, which chain of multiple competing chains is valid.

Skycoin Survivability

It is very important that this information is implemented/replicated peer-to-peer.

In a war scenario, Russia, China, Brazil and even the United States will balkanize the internet. There will be communication restrictions. You will not be able to connect in the global namespace, from a server in Brazil to a server in China.

As long as there is a single satellite phone link, or clandestine cross-border radio link between countries, the peer-to-peer replication will succeed. Data replicates independently within the balkanized zones. So the cross border bandwidth may only need to be a few KB/s.

We are designing, specifically for a balkinized internet, with intermediate node connectivity. Especially in war scenario where global communications is sporadic and the electromagnetic environment is unreliable or hostile.

The internet came out of DoD ARPANET. Tor came out of Navy being able to access their secure network web-mail email servers from a public network without NSA snooping in (secure access over public networks). Skywire came out of research on store-and-forward networks for disruption-tolerant networking, mobile ad-hoc networking and diversity routing in mesh networks.



Store and forward is useful for emergency communication networks. In theory it can tolerate minutes, hours or day of latency as long as the protocol is designed to minimize round trips. It is already used for downloading files and data from space craft, where there are sporadic communication windows.

Current drones require constant network connectivity or are fully autonomous. Future drones with be semi-autonomous and roam between network access points, uploading telemetry data and receiving commands, while operating autonomously when communication networks are unavailable. Store and forward, source independent networking works very well for this application. There are similar applications for asynchronous sensor networks, where a drone or vehicle may drive by a site and connect locally to the sensors and download the data in batches, without requirement for real time network access. Sensor and vehicle networks that have been cut off from global internet can continue to function in the local network without interruption and exchange data when possible.

A drone may have satellite uplink, but is also able to use stationary ground links it encounters. This allows continued operation if the satellite is jammed or shot down or equipment becomes damaged (redundancy). A drone may wander drifting between sources of power and communication and opportunistically uploading/exchanging data about resource locations with other elements of the sensor/vehicle swarm.

Meshnets, emergency communication networks, sensor/vehicle swarms require a new type of networking technology. TCP/IP end-to-end re-transmission no longer works when you have 10 hops and even 5% loss on each hop. The source independent networking and store-and-forward method is "scale invariant" in that it handles latencies of milliseconds the same as latencies of hours and days. However, it is a complete redefinition of the existing network stack.

It is ironic that the network of the future, is going back to Unix-to-Unix Copy.

Consensus Data

In Bitcoin, PoW is baked into the block headers. You cannot change Bitcoin consensus method without invalidating the whole blockchain. In Skycoin, consensus is flexible, can be changed later. Skycoin will use an open network of nodes, but a bank may fork it and decide to run customer account balances in Euro on a Skycoin chain, but will only use a closed set of bank servers for the consensus process.

Outputs/Transactions are designed to be independent of the blockchain. This means that a ledger of transactions can be migrated from one ledger, to a new ledger when needed. This is to ensure that Skycoin ownership and balances are preserved, while enabling upgrades in the future.

how much longer for the IPO?

Here to download wallet client: http://128.199.188.22/

Update:

Block replication worked. It just fixed itself. Transaction replication needs to be fixed.

I just want to get IPO over... get back to writing new code instead of debugging.

Security Link:

In the long term, we need open source CPUs.

Some of the CPUs have microcode backdoors that do not have any authentication on the microcode patches. The most recent ones use an RSA key, but anyone with the RSA key can exploit the CPU. There also appear to be buffer overflow attacks on the microcode update process itself. The security appears to be horrible. The Intel Processors were easier to exploit than AMD. There are more microcode example.

It is impossible to read out the ram of the microcode on the processor, impossible to audit and impossible to verify. CPUs can be clandestinely backdoored before export and it would not be detected.
 
https://www.dcddcc.com/pubs/paper_microcode.pdf

Intel vPro is very worrisome.

Can you post the links to the info you have on hardware? There is a group of us focused on the hardware end of things. Glad you're working on the software, since that is where the deficiency lies to make things really work.

It works, I have seen nodes connected for the first time on http://127.0.0.1:6420/api/network/connections .
I have three established connections at the moment.