Excellent Gavin. We should start a list of proposed config file options in this thread.
I, for one, suggest a noirc (off by default) option and a minimizetotray option, both of which would do the same thing as the command-line switches by the same name. That'll stop the hackery of launching my Bitcoin with -minimizetotray each time. Also, maybe -server and -daemon should get similar options?
Topic: JSON-RPC password - page 3. (Read 36685 times)
I volunteered to implement this, and made good progress today. Satoshi: I should have patches for you tomorrow.
Done: teach Bitcoin to read settings from {BITCOIN_DIR}/bitcoin.conf file, and added -conf=path_to_config_file.conf command-line option.
Done: teach Bitcoin RPC to require HTTP Basic authentication, and reject requests with the wrong username/password.
TODO: teach Bitcoin command-line RPC to add the Authorization: header. You won't have to give the username/password when controlling bitcoin from the command line, it'll read them from the bitcoin.conf file and Do the Right Thing.
TODO: dialog box or debug.log warning if no rpc.user/rpc.password is set, explaining how to set.
TODO: limit password guessing attempts if the rpc.password is < 15 characters long.
TODO: update the JSON-RPC wiki page
After all that is done and I've sent patches to Satoshi, I'm going to add a couple more things to bitcoin.conf :
port= # to set the listen port (override default 8333)
rpc.port= # to set the JSON-RPC port (override default 8332)
With the existing -datadir option, that'll make it easier for me to run multiple bitcoins on one box.
Done: teach Bitcoin to read settings from {BITCOIN_DIR}/bitcoin.conf file, and added -conf=path_to_config_file.conf command-line option.
Done: teach Bitcoin RPC to require HTTP Basic authentication, and reject requests with the wrong username/password.
TODO: teach Bitcoin command-line RPC to add the Authorization: header. You won't have to give the username/password when controlling bitcoin from the command line, it'll read them from the bitcoin.conf file and Do the Right Thing.
TODO: dialog box or debug.log warning if no rpc.user/rpc.password is set, explaining how to set.
TODO: limit password guessing attempts if the rpc.password is < 15 characters long.
TODO: update the JSON-RPC wiki page
After all that is done and I've sent patches to Satoshi, I'm going to add a couple more things to bitcoin.conf :
port= # to set the listen port (override default 8333)
rpc.port= # to set the JSON-RPC port (override default 8332)
With the existing -datadir option, that'll make it easier for me to run multiple bitcoins on one box.
boost::program_options has the same "key=value" format. Gavin pointed out we can use it in a simple way as a parser without getting into all the esoteric c++ syntax like typed value extraction. We can use more features if we want later.
Lets go ahead with HTTP basic authentication instead of password as a parameter.
Lets go ahead with HTTP basic authentication instead of password as a parameter.
I just did a quick survey of 20 .conf files in /etc on my debian system, and found:
1 file used "key value"
5 used "key=value"
Thanks for that survey!1 file used "key value"
5 used "key=value"
I find "key value" a little unnatural. There ought to be a more definite separator between key and value that suggests assignment. The space people may just be getting lazy using their language's split function.
key=some full sentence with spaces in it. # seems more clear
key some full sentence with spaces in it. # than this
Allright then, lets go with self-parsed mapConfig, syntax:
# comment
key=value
file extension .conf. What's the filename, is it ~/.bitcoin/settings.conf or ~/.bitcoin/bitcoin.conf or what?
I think we better strip whitespace at the beginning and end of the key and the value.
# user who likes column formatted
k = value
key = value
longerkey = this sentence would be this # "this sentence would be this"
key = value # guess this is ok too
nextkey = value
right = justified
The normal syntax should be "key=value", but you can't blame people for the occasional "key = value".
I think there's no such thing a a "typical" settings file on Linux!
I just did a quick survey of 20 .conf files in /etc on my debian system, and found:
1 file used "key value"
5 used "key=value" (actually, a couple were "key = value", allowing whitespace around the "=")
14 did their own thing.
The 14 that did their own thing were all over the map; from one-value-per-line to "key:value" to full-blown XML. # is
the universal comment character in the Linux world.
My vote would be for:
# comment
key1=value1
I just did a quick survey of 20 .conf files in /etc on my debian system, and found:
1 file used "key value"
5 used "key=value" (actually, a couple were "key = value", allowing whitespace around the "=")
14 did their own thing.
The 14 that did their own thing were all over the map; from one-value-per-line to "key:value" to full-blown XML. # is
the universal comment character in the Linux world.
My vote would be for:
# comment
key1=value1
Out of those options Satoshi presented, I'd go with the self parsed file with the key/value separator character being a space like lachesis suggested, especially if that's the typical format for at least one OS.
The simplest Linux config files typically have formats that look something like this:
and C-style comments crop up too.
I think a good, human-manipulatable config file that ISN'T part of the wallet would be a big step forward. There are a lot of options that are currently specified on the command line (noirc, for example, or -minimizetotray) which might be better specified in a config file.
Code:
# how often (in minutes) data is saved when all interface are offline
OfflineSaveInterval 30
# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 1
# file used for logging if UseLogging is set to 1
LogFile "/var/log/vnstat.log"
# file used as daemon pid / lock file
PidFile "/var/run/vnstat.pid"
The standard extension, if there is one, is .conf. Hash (#) is the most common comment character by far. Semicolon (OfflineSaveInterval 30
# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 1
# file used for logging if UseLogging is set to 1
LogFile "/var/log/vnstat.log"
# file used as daemon pid / lock file
PidFile "/var/run/vnstat.pid"
and C-style comments crop up too.I think a good, human-manipulatable config file that ISN'T part of the wallet would be a big step forward. There are a lot of options that are currently specified on the command line (noirc, for example, or -minimizetotray) which might be better specified in a config file.
I was researching config file formats, here's a comparison.
YAML is massive. I'm not sure there's a lightweight easy to build library we can integrate into our project. Seems overkill.
JSON is tempting and I'm inclined to like it, but two main sticking points:
1) No comments! How can you have a config file where you can't comment out a line to disable it?
2) Not very user friendly to have to "quote" all the strings, including the keys, and also have to remember the comma at the end of lines.
{
"key" : "value",
}
I suppose we could easily preprocess JSON reading the config file one line at a time, truncate the lines at any # character (and/or "//"?), concatenate them into a string and pass it to JSON, so you could go:
# comment
"key" : "value", # still have to remember the comma
"key2" : "value", // comment like this or both
Boost has boost::program_options.
We could read lines ourselves and feed them into a map mapConfig.
while (!eof)
read line
if '#' found, truncate line
split line at first ':' -> key, value
mapConfig.insert(key, value)
If we use the syntax:
# comment
key : value
...and don't allow whitespace indenting before the keys, I guess we would be a subset of YAML and could switch to YAML someday if we need more complexity.
If we go with self parsed, that doesn't mean we can't use JSON on particular parameter values as needed. If an option needs a list or more structured data, it could always parse its value as json:
key : ["item1", "item2", "item3"]
Although it has to be all on one line then.
I guess I'm leaning towards self parsed mapConfig:
# comment
key : value
YAML is massive. I'm not sure there's a lightweight easy to build library we can integrate into our project. Seems overkill.
JSON is tempting and I'm inclined to like it, but two main sticking points:
1) No comments! How can you have a config file where you can't comment out a line to disable it?
2) Not very user friendly to have to "quote" all the strings, including the keys, and also have to remember the comma at the end of lines.
{
"key" : "value",
}
I suppose we could easily preprocess JSON reading the config file one line at a time, truncate the lines at any # character (and/or "//"?), concatenate them into a string and pass it to JSON, so you could go:
# comment
"key" : "value", # still have to remember the comma
"key2" : "value", // comment like this or both
Boost has boost::program_options.
We could read lines ourselves and feed them into a map
while (!eof)
read line
if '#' found, truncate line
split line at first ':' -> key, value
mapConfig.insert(key, value)
If we use the syntax:
# comment
key : value
...and don't allow whitespace indenting before the keys, I guess we would be a subset of YAML and could switch to YAML someday if we need more complexity.
If we go with self parsed, that doesn't mean we can't use JSON on particular parameter values as needed. If an option needs a list or more structured data, it could always parse its value as json:
key : ["item1", "item2", "item3"]
Although it has to be all on one line then.
I guess I'm leaning towards self parsed mapConfig:
# comment
key : value
You could do worse than using yaml for the settings
Still need to know what's the most typical settings file format on Linux. Is there a standard file extension? I've never seen a settings file using JSON, and it doesn't look very human friendly with everything required to be in quotes. I think what I usually see is like:
# comment
setting=value
Is there a settings file thing in Boost?
When you're using bitcoind to issue commands from the command line as a client, can we have it get the password from the settings file then too?
Gavin pointed out I forgot to increment the column of numbers in CommandLineRPC, so the current -rpcpw= implementation doesn't work right from the command line with non-string parameters. (JSON-RPC is fine) Still under construction.
# comment
setting=value
Is there a settings file thing in Boost?
When you're using bitcoind to issue commands from the command line as a client, can we have it get the password from the settings file then too?
Gavin pointed out I forgot to increment the column of numbers in CommandLineRPC, so the current -rpcpw= implementation doesn't work right from the command line with non-string parameters. (JSON-RPC is fine) Still under construction.
If you're using another JSON-RPC client that you wrote you can take care to protect the password, but using the bitcoin binary as the client and passing the password on the command line has the same issue as starting the daemon with it. It's still visible to every user that way.
So both the server and the client mode invocation need to use the file and not accept the password on the command line. Generally programs like this refuse to start if the mode on the file isn't 600 or something like that, because that means other users can read it.
So both the server and the client mode invocation need to use the file and not accept the password on the command line. Generally programs like this refuse to start if the mode on the file isn't 600 or something like that, because that means other users can read it.
Should bitcoind be more of a system-wide daemon? With PAM authentication and support for all system users, not just the one running it?
Like scanning /home/*/.bitcoin folders for a wallet file and provide access by both username and system password...
Like scanning /home/*/.bitcoin folders for a wallet file and provide access by both username and system password...
So you drop a settings file in the ~/.bitcoin directory, that sounds better. In the "no password is set" warning, it could tell you where the file is and what to do.
What is the most popular and common settings file format?
You ask hard questions! Most common: probably Windows INI files, because Windows is most common OS.What is the most popular and common settings file format?
I'd lobby for using JSON; it's (mostly) a subset of YAML (which is a common choice for config files), so any JSON or YAML parser will read it.
I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list. I agree that reading the command-line password from a file would be more convenient and more secure.
You're also confusing me, what do you mean? Did I do something unintended?Code:
> bitcoind help
error: First parameter must be the password.
> bitcoind help
error: unknown command:
>bitcoind help
... that works.
error: First parameter must be the password.
> bitcoind
error: unknown command:
>bitcoind help
... that works.
So you drop a settings file in the ~/.bitcoin directory, that sounds better. In the "no password is set" warning, it could tell you where the file is and what to do.
What is the most popular and common settings file format?
HTTP basic authentication should be considered. In actual practice though, it's more work for web developers to figure out how to specify the password through some extra parameter in the HTTP or JSON-RPC wrapper than to just stick an extra parameter at the beginning of the parameter list. What do you think? Does HTTP basic authentication get us any additional benefits? Moving it off the parameter list but then you still have to specific it in a more esoteric place I'm not sure is a net win.
What is the most popular and common settings file format?
HTTP basic authentication should be considered. In actual practice though, it's more work for web developers to figure out how to specify the password through some extra parameter in the HTTP or JSON-RPC wrapper than to just stick an extra parameter at the beginning of the parameter list. What do you think? Does HTTP basic authentication get us any additional benefits? Moving it off the parameter list but then you still have to specific it in a more esoteric place I'm not sure is a net win.
I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list. I agree that reading the command-line password from a file would be more convenient and more secure.
You're also confusing me, what do you mean? Did I do something unintended?
After further research... I think the Transmission approach, combined with the existing "only allow connections from 127.0.0.1" is a good short/medium-term solution.
Putting the username:password in a settings.json file in the Bitcoin directory aught to work nicely (since Bitcoin can already parse JSON). And keeping the authentication stuff off the command line and in the HTTP headers instead of the JSON request params is nice and clean.
Long term, the "right" way to do authenticated, secure JSON-RPC is with client-side certificates and https. But that looks like it would be a lot of work to implement and a big learning curve for users to figure out how to generate client-side certificates and how to get both sides of the JSON-RPC connection using them. And I'm not even certain a full-blown client certificate solution would solve the problem of malicious Javascript making JSON-RPC requests via XMLHttpRequests to localhost; if the user installed the client certificate in the browser (because maybe there was a nifty JSON-RPC-powered web front-end to controlling Bitcoin), would the browser automatically send the client certificate if a malicious website made requests?
Putting the username:password in a settings.json file in the Bitcoin directory aught to work nicely (since Bitcoin can already parse JSON). And keeping the authentication stuff off the command line and in the HTTP headers instead of the JSON request params is nice and clean.
Long term, the "right" way to do authenticated, secure JSON-RPC is with client-side certificates and https. But that looks like it would be a lot of work to implement and a big learning curve for users to figure out how to generate client-side certificates and how to get both sides of the JSON-RPC connection using them. And I'm not even certain a full-blown client certificate solution would solve the problem of malicious Javascript making JSON-RPC requests via XMLHttpRequests to localhost; if the user installed the client certificate in the browser (because maybe there was a nifty JSON-RPC-powered web front-end to controlling Bitcoin), would the browser automatically send the client certificate if a malicious website made requests?
The Transmission BitTorrent client does authenticated JSON-RPC; see "Remote Control" section of:
https://trac.transmissionbt.com/wiki/ConfigurationParameters
E.g. setting.json file might look like:
It uses HTTP 'basic' authentication (Authorization: basic base64(username:password) in the HTTP headers).
https://trac.transmissionbt.com/wiki/ConfigurationParameters
E.g. setting.json file might look like:
Code:
{
"rpc-enabled":1
"rpc-authentication-required": 1,
"rpc-password": "xxxxxxxxxx",
"rpc-port": 9091,
"rpc-username": "xxxxxxxxxx",
"rpc-whitelist-enabled":1
"rpc-whitelist":"127.0.0.1,192.168.*.*"
}
"rpc-enabled":1
"rpc-authentication-required": 1,
"rpc-password": "xxxxxxxxxx",
"rpc-port": 9091,
"rpc-username": "xxxxxxxxxx",
"rpc-whitelist-enabled":1
"rpc-whitelist":"127.0.0.1,192.168.*.*"
}
It uses HTTP 'basic' authentication (Authorization: basic base64(username:password) in the HTTP headers).
The Bitcoin Faucets (production and TEST) are now running with this change.
I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list. I agree that reading the command-line password from a file would be more convenient and more secure.
I'll try to do some research on how other projects tackle JSON-RPC authentication.
I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list. I agree that reading the command-line password from a file would be more convenient and more secure.
I'll try to do some research on how other projects tackle JSON-RPC authentication.
Right, that is quite a bit better.
Can you give me any examples of other stuff that does it that way? (and what the command line looks like)
The main change you're talking about here is instead of -rpcpw= when you start bitcoind, you'd use a switch that specifies a text file to go and read it from, right? (any ideas what I should name the switch?)
Can you give me any examples of other stuff that does it that way? (and what the command line looks like)
The main change you're talking about here is instead of -rpcpw= when you start bitcoind, you'd use a switch that specifies a text file to go and read it from, right? (any ideas what I should name the switch?)
I'm afraid I have to agree with laszlo here, using a certificate/keyfile would be far more secure. Saying that, thanks for adding some security to the JSON api 
I guess it's ok for remotely doing it but if your concern is that someone else on the same unix machine can steal your bitcoins this still doesn't help because they can see your command line in /proc, top, ps etc. It could read the password on stdin or use readline or something, to guard against that particular thing at least. Allowing it to be passed on the command line is not good, in my opinion.
Even better might be to use a key file, then you can use unix permissions to make it readable to only that user, kind of like ssh does.. then the bitcoind could have an 'authorized_keys' file with the public keys. Anyway I don't mean to be an ass but the command line thing is just a false sense of security.
Even better might be to use a key file, then you can use unix permissions to make it readable to only that user, kind of like ssh does.. then the bitcoind could have an 'authorized_keys' file with the public keys. Anyway I don't mean to be an ass but the command line thing is just a false sense of security.
Jump to: