Topic: Vulnerability bounties proposal (Read 2335 times)

Hey Sergio,

I have been in the vuln business for 6-7 years.

encouraging people to test and get their results is exactly what we are after. I do believe that there are some very talented hackers already in this community and they are working hard on the future security of bitcoin - albeiet in disparate groups and not seeking reconition. bitcoin has survived some turbulent times already.


no, they wont do anything, this is one of the issues. they will take up to  2 weeks to review a submission.

there are so many issues in selling vulns let alone buying them. but I will put more stuff up in the testing site. just because I am not sure about the idea, it doesnt mean we shouldnt investigate the possiblity of doing it.  - I have been wrong many times before

I only picked on the ZDI because they will buy more or less anything (and I have submitted through them, not very often I work more with pentest companies.). see the thinks below for much more detail and other companies and what they pay.

I do think we need some kind of bounty/reward scheme, and we very much need discussion as to what sort any where/why it gets paid.  I am really glad that you have signed up to the testing website, so we can work what sort of incentives will work and how to get people testing

You are very much the person the project needs and wants to attract.

this is exactly the kind of discussions that need to happen for bitcoin to be able to progress to the next stage.

here is some more info on the world of vuln sales. I hope you enjoy the read.

http://weis2007.econinfosec.org/papers/29.pdf - The hazardous path of vuln sales (i was involved with that paper)

here are some links about selling exploits and who buys them for what money. The market has changed since this was done, but it is still valid. (the market never changes that much)

http://unsecurityresearch.com/index.php?option=com_content&view=article&id=52&Itemid=57

This is a presentation written quite a while ago by Pedram Amini one of the founding members of iDefense (owned by VeriSign) and TippingPoint/ZDI (owned by 3com)

It is well worth the time to read it.

http://docs.google.com/present/view?id=dcc6wpsd_20ghbpjxcr

Steve, you seem to know better the security vuln research/advisory job than me. If you say that ZDI (*) will always pay more than Bitcoin, then we must encourage white hat hackers to either become testers or go through ZDI (if they want to).
As I never submitted  to ZDI, I don't know how much they value the vulns. But I will contact them and ask them how much would they pay for a vuln in Bitcoin (I don't know if that's possible without a real vuln)

Still (and this is my last thought) offering bounties transmits the message to the public that Bitcoin is in a position of strength.

(*) ZDI is the acronym of www.zerodayinitiative.com

PS: "Audit" was picked erroneously (English is not my native language)

Generally a source review (i dont like the word audit, it has spesific legal implications - i only just remembered this will generate quite a few questions.  It will normally generate some bugs, maybe of low severity, or maybe look high severity but there is another function somewhere sanitising the inputs

from these reports you get a good feel for the persons skill.

also finding a bug gives you no clue as to what else has been tested or how secure it is (it is the same problem and the one you posed to me)

just because someone has looked over the code for one type of issue it does not mean that the testcase is gone, someone else can do the same testcase and you should hopefully get a better spread of reviewers. also someone might look for format strings and someone else off by ones.

I just realised that i missed a bit off my last post. in step 4 all freeform testcases have a time attached like 4 hours free form security testing.

I am still wading through the vids on youtube about bettermeans, it looks like this will be simple to implement.

large bounties will not attract the talent and people you expect.  they would get paid more elsewhere. what advantage is it to bitcoin to buy a vuln? other people will pay more and report it to bitcoin for free...

you have really got me thinking on this. I keep changing my mind.

cheers for the brain food!

Why only may? we will be able to find vulns.

I still dont see how offering a bounty will help.  if we offer $10k per vuln and zdi will give you $50k will report it to bitcoin for free then that is the path that makes the most money. i do not think we will ever be able to compete with the likes of the zdi, their business model is completely different and vulns are worth _a lot_ more to them and thier customers. I do not think we can be competitive with big bugs anymore, bitcoin is worth too much.

I still think paying people for work and reports is a decent alternative which still rewards the community. and can be integrated with normal testing.

Earlier you mentioned that the price to be paid would be set by the community.  This is asking for trouble.  What does the general populace know about competitively pricing vuln bids?

one last point, you also implied that it would be trivial to verify.  I can attest that it is never simple.  I have worked for the REACT team at microsoft (the people that verify vulns submitted by the public) it takes at least a week and a lot of back and forth (and that is for the reproducible ones).  This would be a nightmare for people reporting their $10k dos.

I understand and agree that it is very important to incentivise and reward the testers (both sec and non sec) however I do not feel bounties are the right way for bitcoin at the moment (the business model only really works for people that sell something and need zero days to fund IPS2.0(r) ).  there is a lot more work involved with it than i think you realise.

The more I think about it, the more I think it should work something like this:
1 - you go to the security test section of https://secure.bettermeans.com/boards/4240
2 - find the section that you fancy having a look at, in this case it is network (for arguments sake lets say https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp)
3 - you then think you will see how the ipv6 stuff was played with when it was merged
4 - when you drill down to net.cpp testcases there will be a number of security ones, ranging from things like check source for off by ones to check source for double frees, check for races or something more free form like design new testcases for attack vector, investigate attack vector xyz, general review of source.
5 - then their will be a difficulty rating and a suggested reward for completion of the testcase

This is so much more useful, we get recorded coverage, the same tests will be looked at by many people, people are encouraged (by being rewarded) to share testcases, it still allows people to do free form testing and get rewarded (as long as their is a report and the testcases are annotated)

The software on betterminds seems to support this out of the box.  It is now just working out how all this should hang together to make it usable and consistent.

This is still a few weeks off yet though.

Watch the vid when you click on dashboard, it is well worth it.

Thanks for starting this thread, it is a very good point.

cheers,

steve

The problem I see with remote source code audits for security vulnerabilities is that is difficult (for the auditor) to prove that in fact the code was audited if no bugs are found. How much would you pay for the audit? How can we evaluate the performance of the auditor? It works if the auditor is a well known security firm that everybody trusts. On the contrary, bounties should allow the best talented researchers to audit the code, and in fact multiple audits will take place for the price of one.

This is just my opinion.

Best regards, Sergio.

I guess most Bitcoin developers do care about Bitcoin a lot AND also about money. They probably work part time in other things.
If we don´t offer bounties, we still may be able to find vulnerabilities, but maybe not before the attackers.
We need to be one step ahead.

Hi, please see inline comments.


I understand that.  I do the same but with the assembler.

I was more saying that I would like to see someone get a bounty for reviewing the code even if they dont find anything because there might not be anything to find. I think that is a fairer method of compensation.  I think that would allow us to get some confidence in the code. 

I disagree with it only being the new code that needs auditing, what happens if the new bit of code doesnt know about some back end rules but and thinks there should be a different outcome? there is so many intergration issues and potential to open a new or reintroduce a bug in a completely unrelated bit of code.

I was in no way saying or implying that yourself or the bitcoin developers lacked talent, I apologise if that is what you took from my statement from the posts I have seen you make I have nothing but respect for you, it is obvious you have a talent and passion for security.

What I was trying to say was the bitcoin developers are already doing this for love, so they will report bugs even if there is no reward, the idea behind a bug bounty is to attract in new talent, talent that might not care about bitcoin but does care about money.

This is the same logic as I use to workout what programs to look for vulns in - i could care a lot less about office, however it pays me well to find vulns there.


I was not saying DoS is not a valid attack vector, I was trying to say that I believe that if a half decent bounty is offered for DoS bugs there will be so many supurious bug reports by well meaning but mistaken people that most of testing would actually be customer service/tech support, I have seen this happen at a workplace before.

i would like to clearly state that if anyone reports a significant bug in bitcoin or the protocol they should be rewarded if that reward exists.  I do not however feel that this reward should be priorotised above or below the rewards of the person who reviews the code and does not find a bug. This is a very tricky subject.

to sum up, I like the idea, but i am not sure what is to be gained and where it would be gained. I think a better use of resources might be to give bounties for reports of code reviews, etc rather than bug. I dont know though.

I am glad you are also thinking how we can get more talent in, and how to reward the people that already do the work. just out of interest, what would your bounty rewards roughly be for type 1, 2 and 3 bugs? I do not think we would be able to compete with the likes of zdi, idefense or  pentest companie securiteam, ngs, etc. (all of them will be $500k+)

I am still really undecided on this topic.

cheers,

steve

Check the article http://erratasec.blogspot.com.ar/2012/05/15-years-ago-when-we-started-industry.html about the cgi-php vulnerability and why open source does not mean better security unless there is a incentive for white hat hackers to look at the code.
 

I'm talking about "the undocumented vulnerability". And I will talk about the undocumented vulnerability until it becomes a documented vulnerability. It has been fixed.

It's not clear that it's TYPE 1.  We think it might have that risk but the nature of it is so hard to characterize that we don't know for sure.   This is one issue for bounties— it may change the incentive for how bugs are reported to users.

If there is a memory corruption bug that I can't prove that it's not remotely triggerable or can't overwrite something important, I'm inclined to call those remotely exploitable— because I can't show that they aren't and it's better that users treat at as potentially too serious than not serious enough.  But if thats tied to paying someone the incentives aren't quite the same.


Which bug are you talking about here?

Security researchers don't need to continuously read the code one time after the other.
Personally I read the code once or twice, and I "smell" problems.
Afterwards, only the new code of new releases has to be audited.


Well, Bitcoin developers seems to be very talented, as there were so few security vulnerabilities.
I cannot rate myself.

Nevertheless there are at least three documented vulnerabilities: see https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures.
And one undocumented (silently patched) vulnerability.

CVE-2011-4447    wxBitcoin and bitcoind    Wallet (non-)encryption   -> Not categorized (not remotely explotable)
CVE-2012-1909    Bitcoin protocol    Transaction overwriting    -> Not categorized (requires a lot of hasing power)
CVE-2012-1910    Bitcoin-Qt for Windows    MingW non-multithreading    -> TYPE 1

The undocumented vulnerability is of type 3.


Probably. Now that traders are offering put/call options, a DoS vulnerability can be directly converted into money.
The attacker just buy an option to sell 10K bitcoins at 5 USD/BTC, attack the network, and them execute the contract when the market price is 2.5 USD/BTC.
Then the vulnerability is worth 25K BTC (or 625K USD).
I think that still the market of put/call options is not big enought to handle this options, but it will be.


If we find more that a few vulnerabilities of type 3, then we are in serious problems, since each vulnerability on the wild could cost a btc price drop of 50% (this is only my estimation). We'll have a run on the currency.
Categories are can be narrowed, but I think they are narrow enough.
All types of bounties require that the attack can be carried with limited resources (e.g. a couple of PCs).


Best regards,
 Sergio.

This is a nice idea,

but as someone that participates in real world software bounties (from office, to firefox, to qnx and cashmachines) i can see a couple of initial issues,

I am not sure how this would work, you would need to hire people are retain them even if they are not doing productive work.  an exploit for bitcoin would be worth far more than any dev bounty. 

So you will only get bugs that people report because they love the project.  dont get me wrong i am not against the idea, i just think it probably wont attract the kind of people and talent that you are expecting.

a new unpatched remote bug in the client, or in spending someone elses coins bug in a _production_ version of bitcoin would be worth well into the $500k range, probably more. 

Maybe bounties for security bugs in the latest test branch... it would be workable and make sense.

I am in two minds about paying for bug types 3 and 4 i think the system will be swamped with type 3 that are not really type 3 or cant be fixed, or wont be fixed, i am nor sure.  the catagories are so broad.

i will keep thinking about this, it is an interesting topic.

I crossed-linked to this thread before seeing you had already posted about it! Great job!

I've been procrastinating creating a "Bitcoin Testing Project" to fund testing work, and I was actually thinking a few days ago that vulnerability bounties would fit in nicely as one of the things a Bitcoin Testing organization would tackle.

I decided to stop procrastinating today; see my announcement here for a Bitcoin Testing Project:
   https://bitcointalksearch.org/topic/announce-bitcoin-testing-project-80019

Creating a Bitcoin Foundation is a great long-term goal. Creating an entity/Bitcoin address to  accept donations for vulnerability bounties can be a short-term project. A single person can handle the work of verifying the submitted vulnerability reports in his spare time.  Each bounty amount could be set up by consensus in this forum. I would do it myself, but since I wanted to research and probably submit my own vulnerability reports, there would be a conflict of interests. Maybe Gavin wants to do it.

We could setup up a scheme so that the vulnerability verifier gets 5% of the bounty and the programmer who writes the patch gets a 10%. Obviously a vulnerability report should describe a new, unpatched vulnerability existent in a production version of the Bitcoin client.

Related ... Bitcoin Foundation thread:
 - https://bitcointalksearch.org/topic/bitcoin-foundation-49841

I think it's a good idea.

I´ve dedicated some time to research Bitcoin security and resiliency and I´m investigating some possible attacks and corresponding patches. The problem is that I cannot use more of my work time for the project, since I must earn my living. Since I really would like to go forward with this research, It would be great if the community (the developers, the exchanges, all of us) could donate bitcoins to create vulnerability bounties. This would give an incentive for researchers like me to leave out other tasks and focus on Bitcoin. Also bounties would reduce the risk that vulnerabilities are sold in black markets.

For example, we could give bounties (sorted by severity) for:

1. Remote code execution
2. Stealing money by exploiting bugs using specialty crafted transactions/blocks.
3. Low cost Denial of Service of the whole Bitcoin network
4. Lost of privacy / pseudo-anonymity.

In the first 3 cases people should immediately download a new client version to allow the network to keep running.

I think we won´t find many vulnerabilities of type 1-2 but we might find many vulnerabilities of type 3-4. A vulnerability of type 3 may render Bitcoin out of reach for days, and this would cost exchangers (and most of us) a lot of money.

What do you think?

Best regards,
 Sergio.