Topic: [Stratum] Overlay network protocol over Bitcoin - page 3. (Read 37909 times)

Very nice, thank you :-).

I didn't commited into git for few days, because I'm finishing one project in my job, but then I'll work more actively on the Stratum again. I'm missing it!

Check out my answer here on why The Stratum protocol is secure.

Fixed, thanks

It's actually hosted by BitVPS.COM not .Net....

Hmm, I'll fix and retry ... possibly next weekend.

get_history() works now, however the result data may be slightly changed in the future. I'm writing API for standard services, then will be much cleaner what method expect what parameters and what will be the result object.

Hi, your payload is wrong, you must use json-encoded data. So

{"id":1, "method":"blockchain.address.get_history", "params": ["14obksEpeUTAKsdKsw3vTSUoiYeK9S3thA"]}

instead of url-encoded data.

Any documentation on what is and is not implemented right now?
ETAs?

This also happens when I try a prodnet address:

POST / HTTP/1.1
Host: chicago.stratum.bitcoin.cz:8000
Content-Type: application/stratum
Connection: keep-alive
Accept: */*
User-Agent: NING/1.0
Content-Length: 84

id=1&method=blockchain.address.get_history¶ms=14obksEpeUTAKsdKsw3vTSUoiYeK9S3thA

What's the expected time that requests are supposed to take?

I sent a request to http://chicago.stratum.bitcoin.cz:8000, with a timeout of 60 seconds ... and the timeout expired.

content-type: application/stratum
id: 1
method: blockchain.address.get_history
params: mw5h6BDh77GUPc4DCmzjoU7ry2R4exoaYr   (this is a testnet address, so I guess it wouldn't work ... but I should get an error, not a timeout)

Another question - I haven't seen any reference to testnet in this thread ... are there testnet servers? Does the protocol support it?

Assuming that such services are already implemented on the Stratum nodes (which isn't yet) - yes, it will work. But you're doing exactly the case b) from my previous post, so you don't need to use get_history at all, just subscribe the address in the time when you send it to the user.

But I understand you're mostly playing with the stuff so I'm ok with it...

For context, this is the application I'm developing. I wanted to implement a polling client at first, that polls every minute or so, just as an initial proof of concept - I'd like to see the protocol works. I intend to move to a more efficient implementation soon after that.

Let me try to rephrase. Assuming servers don't rate-limit just yet - if I don't send any cookies, and just poll every minute on address.get_history, and once in a while do transaction.broadcast ... will this work?

I really would like to get all transactions on these addresses. The addresses should in theory only have 1-2 transactions on each of them, but I'd still like to know all the transactions history.

This is not the final implementation, just a temporary POC.

Yes, some better tools should be better in the future, but at this moment the forum thread is doing well. There is only few people so the discussion is pretty clean now. Especially I'll try to move that google docs to some wiki which allow me to separate whole document into more parts and include some additional info (as an examples).


What's your purpose of such calls? You probably don't want to download whole address history on every request (say 5 seconds), right? I'm not sure, because I don't know your application, but you probably want to download address history on the startup and then subscribe your client for changes on given address, to receive only new transactions. In this way you save server resources and bandwidth. It's also possible that server may block your requests when you start overflooding server with "get_history" requests every few seconds, because there's no reason to do that so frequently (rate limiting isn't implemented yet, but it is pretty good feature for the future).

There are mostly two main usecases:
a) The address you're interested in is already existing, so you need to ask for history + subscribe for new transactions.
b) The address is freshly generated by you, so you know that there's no history and you want ONLY to subsribe for new transactions. This is the case of some merchant tool, which created fresh address for receiving coins for the order...

case a)
first http call (one HTTP request can contain more RPC calls, divided by newline character):
blockchain.address.get_history('1SomeAddress') => list of transactions
blockchain.address.subscribe('1SomeAddress') => returns only True on a success, but server will send you data for incoming transactions in following HTTP responses.

following http calls: blank HTTP request (with the session) to the server

case b)
first HTTP call:
blockchain.address.subscribe('1SomeAddress') (no get_history() command is needed!)

following HTTP calls: blank HTTP request (with the session) to the server

I think there's big misunderstanding about the "polling" thing. Polling does not mean that the connection is stateless and you need to ask for everyting on every request. Polling mechanism is only the workaround for the situation when only the client can initiate the communication and is giving a chance to counterparty to send pending data back in the polling response (like information about incoming transaction)...

So yes, you need to "care" about the session even in your use case. But almost every http client library can do cookie management internally, when you perform more calls ("http polling requests") in the row..

slush,

I'm finally starting to work on this, and have some questions.
I think I'll fire them one by one on this thread, instead of accumulating them.

The first one is a meta question - how about using Shapado as a Q&A forum? It's "setup in 8 seconds", and better than a forum thread. It might be better than just firing these questions away at the Stack Exchange. The advantage of a dedicate forum is that you (and other Stratum devs) could follow that website and quickly respond to questions, and the answers would be findable later via Google.

Another question is about a sessions while polling:

From the Google Doc:


The APIs that I need right now appear to be only address.get_history and transaction.broadcast (I'll start by implementing the simple & naive polling protocol). For these purposes, I'm not sure why I need to maintain a session. As a matter of fact ... can you give an example when a session is useful?

Of course. HTTP is just a transport carrying abstract Stratum payload. There is TCP transport already and there can be a lot more transports in the future. Stratum protocol is designed to be bi-directional, so server can send some payload to client (information about new block, new transaction) at any time. It's not the part of application layer itself to decide HOW transport will deliver such payload to the client. So server just tell "send to the client" and don't care about it anymore.

HTTP Poll transport is just the alternative for the environment where real bi-directional communication (like TCP socket or websocket) is not possible. So every poll request just give a chance to send pending messages from the server to the client in the HTTP response, simulating bi-directional communication, just with some delay (=polling interval).

From the view of this idea, creating RPC command for polling is going against the whole conception, because RPC commands are handled by different layer than HTTP transport mechanism. And there's nothing like 'poll' RPC command for standard bi-directional transports anyway.

Quote

Also, how would that work with TCP?

In the TCP transport, which is bi-directional, polling is not needed at all. Server will send the payload when it's ready, in real time. It's the beauty of pub-sub mechanism.

Can you explain a bit more your decisions?
Performing blank HTTP requests is an interesting idea, but I would like to know why you decided that polling should be performed that way. is this in order to minimize traffic?
Also, how would that work with TCP?

What exactly are you missing? I see only three session-related calls in server.py (new_session, update_session, poll_session) which are useless in Stratum protocol.

Edit: To be more verbose:

new_session: You just don't send session cookie to the server and it will create new session for the client automatically. No RPC command needed.

update_session: Replaced with blockchain.address.subscribe()

poll_session: Perform just blank HTTP request to Stratum server, it will put pending messages to the response. No RPC command needed.

Thomas, I proposed the protocol and asked you for the opinion. You told me that you don't understand and/or don't see any point in it and you want reference implementation of the protocol. So I created Stratum server implementation. What's unclear in this?


It's still Stratum protocol + server reference implementation + I'm writing some "default" services into it. Protocol implementation is already here and you know that I decided to separate Stratum from Electrum project, because I see some potential for lightweight clients generally, not only for Electrum.

But of course I'm open to help you with integrating Stratum into Electrum client, if you want. We already had the discussion about Twisted on the IRC and I accept that you don't like Twisted in the client, so I'll provide you clean-python implementation of Stratum client.

I do not know. The initial idea was to propose a new name for the communication protocol, so that it can be distinguished from the software.
Now it seems that slush has decided to rewrite client and server from scratch, and that "Stratum" is being used to designate his software, not the protocol.
So I guess we might want to find another name for the protocol...