Topic: electrum server/protocol vs. stratum server/protocol - please clarify confusion (Read 5680 times)

For anyone who might be interested, I figured out the protocol between electrum (i.e. client) and electrum-server.

OT: For the sake of colored bitcoins, my teammates and me are changing vision by the hour. My current vision is that we should adjust the electrum server (and extend the below protocol) and use a js client.

The google doc mentioned in this thread mentions other methods as well, such as 'blockchain.address.get_balance'. These methods actually don't exist in electrum nor in electrum-server.

Here's my document:
----------------------------------
Electrum server/client message specs

The information below was gathered by looking at the source code that generates the messages. This may have lead to errors.

protocol uses json format.


CLIENT TO SERVER (requests)

general format:

[
  {'method':"method",'id':"id",'params':[...]},
  {'method':"method",'id':"id",'params':[...]},
  {'method':"method",'id':"id",'params':[...]},
  {'method':"method",'id':"id",'params':[...]},
  ...
]

(edit: Note that the above is a single http / scp / whatever request, but I also use the term request for any one entry inside it. Same for response.)

where "id" is a message_id which is just a counter that's increased after each message.

example:

[{'method':'blockchain.transaction.get','id':5,'params':['aec13bHASH43',3700000]}]

used values for 'method': see below.


SERVER TO CLIENT (responses)

general format:

[
  {'id': "id", 'result': "result"},
  {'id': "id", 'result': "result"},
  {'id': "id", 'error': "error"},
  {'id': "id", 'result': "result"},
  ...
]

where an entry with id=5 is a response to a request with id=5


REQUEST / RESPONSE SCENARIO'S

below are the possible values for 'method' (in a request from client) and the corresponding (simplified) statement for the value of 'result' in the response. (method -> result).
Note that instead of the 'result'-key we could have the 'error'-key. Those cases are not specified here.
Note that params is part of the request.
Note that the filenames (.py) are files in the client where the calls can be found
Note that "/" (single) stands for "or" (as in "either .. or" but not both). No division is used below.

wallet.py:
'blockchain.transaction.broadcast' * -> urlopen(bitcoind_url, {'method': 'sendrawtransaction', 'params': params, 'id': 'jsonrpc'}).get('result')
//* note: blockchain.transaction.broadcast will not return key 'error'. Instead, the error message would be in the result
'blockchain.transaction.get' -> urlopen(bitcoind_url, {'method': 'getrawtransaction', 'params': [tx_hash=params[0], 0, height=params[1]], 'id': 'jsonrpc'}).get('result')
'blockchain.address.get_history' -> BlockchainProcessor.get_history(addr=params[0], FALSE/true)
'blockchain.address.subscribe' -> BlockchainProcessor.get_status(addr=params[0], FALSE/true)

verifier.py:
'blockchain.headers.subscribe' -> BlockchainProcessor.header
'blockchain.block.get_chunk' -> -1 / BlockchainProcessor.get_chunk(index=params[0])
'blockchain.transaction.get_merkle' -> BlockchainProcessor.get_merkle(tx_hash=params[0], tx_height=params[1])
'blockchain.block.get_header' -> -1 / BlockchainProcessor.get_header(height=params[0])

merchant.py:
'blockchain.numblocks.subscribe' -> BlockchainProcessor.height

unused server-accepted values for 'method':
'server.version' -> version.VERSION  (also sets the version)
'blockchain.address.unsubscribe' -> 'ok' / 'authentication error'

As simple as I can state it;

- Stratum is the network layer (built on top of bitcoin p2p network) that uses the stratum protocol

- Electrum clients and Electrum servers are the reference implementation of the stratum layer (i.e. they communicate using the stratum protocol)

- other client, server or miner implementations (e.g. call them aurum or plumbum) when they come along and use stratum protocol will become part of Stratum (network overlay)

I just fixed the issue on Electrum server, so california.stratum.bitcoin.cz is operating again.

That stratum node running on california.stratum.bitcoin.cz is mostly my playground for testing new things and it is likely misconfigured. I'll try to repair Electrum services there.

Well, I'll try to explain it.

Stratum is just a protocol, like HTTP/FTP/Jabber/whatever. Stratum has been designed for Electrum to replace it's original wire protocol which had some drawbacks and current Electrum client/servers uses Stratum protocol to talk each other.

Stratum is based on JSON-RPC and you can read the basic concept on http://stratum.bitcoin.cz - that Google document there is probably the best summary to start with.

Stratum protocol allows servers to expose services. For example, Electrum servers are exposing blockchain-related services, so Electrum clients can ask for address history and so on.

Now about confusion with Electrum and Stratum mining protocol; Stratum mining is just a service exposed by some mining pools, giving the opportunity to connected clients to mine bitcoins over very lightweight protocol. Usage of Stratum wire format is the only relation between Electrum and Stratum mining.

Thanks to this concept, it's quite possible that some Electrum backend servers will expose Stratum mining service as well, so lightweight clients using Stratum protocol will be able to request address balance and mine bitcoins over the same connection at the same time...

But why are the stratum servers not listed in the electrum servers list and also trying to manually add the only existing stratum server I could find only results in error messages? Or are we talking about a different stratum or a different electrum here?

Stratrum is being used both by Electrum and some mining software.

Hopefully this link will answer some of your questions:
 https://mining.bitcoin.cz/stratum-mining#stratum

There IS official documentation for the protocol but I'm not sure of its current location.
EDIT: here's a  technical draft I've been able to find - but I don't know if it's the most recent one.

I can't elaborate much further... To my knowledge, the top Stratum experts are
slush (author of the Stratum protocol), ThomasV (author of Electrum) and genjix (who is on a
well-deserved break following the London conference.)

I'm sure they will help clarify a lot of things.

There is this: https://bitcointalksearch.org/topic/stratum-overlay-network-protocol-over-bitcoin-55842
that somehow suggests that "electrum" and "stratum" are actually the same thing or "stratum" is the new name for "electrum"

Then there quite a lot of postings or answers along the lines of this (for example): https://bitcointalksearch.org/topic/m.1013281
that come up when googling for "electrum protocol" or "electrum stratum" that claim that electrum is actually using the stratum protocol or that there is no need to reverse engineer the electrum protocol because there is the stratum protocol documentation.

There is this: http://stratum.bitcoin.cz/
not very informative and mentioning exactly one running stratum server (and nothing about electrum)

    california.stratum.bitcoin.cz (Ports 3333 TCP, 8000 HTTP, 8001 HTTPS, 8002 Websocket, 8003 Websocket SSL, California US, hosted by BitVPS.com).

and there is this: http://electrum-desktop.com/
mentioning nothing about the used protocol, the servers and especially no reference to stratum at all. The client when started present a short list of servers:

ecdsa.org
uncle-enzo.info
electrum.bytesized-hosting.com
electrum.novit.ro

and when trying to use the california.stratum.bitcoin.cz in electrum I get only error messages like [-3, u"Method 'subscribe' not found for service 'server.peers'", None] and a few more which suggest that the electrum and the stratum protocols are not the same thing.

Now here come the questions:
* how are stratum and electrum related?
* if not related then does there exist an electrum-like client for stratum?
* how many electrum servers are there?
* how many stratum servers are there?
* which *existing* protocol should be chosen when implementing an electrum-like client?
* if the answer to the last question is electrum then where is the protocol documentation (if such a thing exists at all)?

Maybe it would also be nice if both (the stratum and the electrum) websites clearly state that they are *not* related (or if they are related somehow then maybe explain in which way they are related or if there exist other "stratums" or "electrums" with the same names that are not related, etc).