Topic: [Crowdfund] User-issued assets, p2p exchange, off-chain transactions, and more - page 2. (Read 10382 times)

I have to think more about this, but I think this can work. At least I don't see any flaw so far.
#3 is the less important part and can be simulated with several smaller transactions, just like regular granularity.
Even all or nothing options would be very good for colored coins. Let me know if you write a more detailed design to review, it definitely looks promising.


Ok, I got it.
Not a fatal flaw I think.


Yes, if it doesn't break the contract, it would be definitely better.


I agree. But I don't think the more advanced features add that much complexity to the protocol validation itself.
As Mark explained, the big barrier that exists between 1 and 2 (a hard-fork) doesn't exist between 2 and 3 if you do it at once, so...why not?

Well, who can attack depends on the exact protocol. I was assuming
that Alice published the offer and also sent the partial transaction
to Carol for her to confirm.
If in your protocol is the counterparty and not the publisher of the
offer who sends the partial transaction, then the publisher is the
attacker: Alice would publish an offer and then collect a ton of
partial transactions that she never completes.
And by the way, including the txins with the non-binding offer
doesn't help.
You may need reputation, external PoW or something else to solve this DoS.


Double-spending the order is at all lights equivalent to a cancel
order button on a centralized exchange.
Are centralized exchanges that allow users to cancel their opened
orders flawed? I wouldn't say so.
Are centralized exchange that allow to place fake orders which produce
many fails when another user tries to fill them flawed?
Yes, I would say so.

If my fulfillment of Alice's order gets into the chain before Alice's
cancel, she cannot do anything. For some reason you're assuming that
Alice has greater capacity than me to broadcast her transaction to
miners.
Even if that's the case, you're not halting me from trading, that's
just how this system works: it takes some time to confirm
transactions.
I could even fulfill Alice's offer and another similar offer at the
same time with the same inputs. I would broadcast it and I know only
one of them can get into the chain, but maybe I don't care.
In that case, if Alice cancels her order and it arrives to miners
earlier (or with a higher fee) than my fulfillment would become
invalid and only my other fulfillment can get into the chain.
Similarly, Alice's order cancellation becomes invalid if my
fulfillment gets into the chain earlier, as she probably was
re-claiming all the funds in the input and now it only has what I
haven't buy.
If you donate to 2 different nonprofits with the same funds, only one
of them will get into the chain: that's not an attack to yourself, is
just how the system works.


That's what I'm saying maybe we're not predicting accurately. Maybe
we're wrong.
But you missed my analogy. Why do you think people don't run their
own mail servers on their smartphones despite "having them online all
the time"?


2PC Ripple is almost equivalent to Freimarket's private server in
which Ripple IOUs users can:

1) Run their own private server, like their own mail server, and have
   it always online.

2) Trust another private server.

Much of the value of doing the IOU accounting in-chain is precisely
to be able to go off-line without trusting anyone.
That's my main point rather than it being critical for
old-Ripple-like uses.
But as said sub-tx have many other uses.


Cool, thanks. Having binding orders, even with some limitations, is
my eyes a big improvement.


I will assume this is a joke.


Hmm, interesting... Well, you can partially address granularity problem by publishing many offers of different size which use same inputs, e.g. 100, 50, 25, 10, etc. This will reduce blockchain bloat, but isn't same thing as proper granularity, I guess.
[/quote]

Then if someone buys 10 out of the 100, you need to publish another
offer with 90, after another trade you publish another offer, etc.
This introduces sequencing in the execution.
To prevent this, you would prefer to publish 10 offers of 10 each,
just because is more convenient for you.
But someone who wants to buy 50 will complete 5 of those similar
orders instead of a single one with nQuantity=5

It is a minor optimization, so we would probably remove sub-txs if we
didn't need them for the rest of the cases.


We've tried hard to find new vulnerabilities in the protocol
introduces by our changes and we encourage everyone else to do the
same.
We haven't found fundamental flaws, but of course concrete
implementations could have their own flaws.
Should we sacrifice, for example, BIP32 because some people could
implement it wrong? I don't think so.

Stupid programmers can also have their user's bitcoins stolen by
using a flawed random number generation function in their clients,
but that's not a fundamental design flaw of the bitcoin protocol,
just a flawed implementation.

Well, I just wanted to outline the general idea, not to describe a concrete recipe. Here's a protocol which fixes problems you mentioned:

1. txAB has inputs from both Alice and Bob, sends coins to 2-of-2 multisig, additionally pays premium to Alice
2. tx-call executes the call option, is signed only by Alice
3. tx-timeout returns coins to original owners after a timeout, is signed by both parties
4. tx-cancel returns coins to original owners, is signed only by Alice

Parties sign tx-call, tx-timeout and tx-cancel first, and after that both of them sign txAB. At that point Alice receives the premium.

If Bob cancels option early, Alice also receives her coins early, but she cannot do it on her own.

It is possible to provide partial execution/partial cancellation by signing many different versions of transactions, essentially a tree. For example, suppose Alice agreed to sell 100 AAA for 100 BTC. Bob might want to get 10 BTC out of this, so he publishes tx-cancel-10, and then other branch of transactions is in effect: tx-call-90, tx-timeout-90, tx-cancel-90.

This tree can be pretty big, but it is hardly a problem, as only part of it is included into blockchain, the rest is just shared between Bob and Alice. People routinely send each other 10 MB photos, and a lot of different options can fit into 10 MB of data.


I think I've addressed all issues, although #3 can be a bit of a problem.


Well, this kind of call contract I described requires concrete coins, so Bob has to provide some concrete coins, and avoid spending them until contract becomes irrelevant. He can always spend his coins, but in that case contract is canceled.

If contract is a partial transaction (SIGHASH_ANYONECANPAY), then there is no need to reference concrete coins of Bob, which is definitely better.

So my conclusion is:

1. we can do something useful with colored coins
2. Freimarkets-lite (i.e. Freimarkets without sub-transactions and other advanced features) can be significantly more useful
3. full implementation of Freimarkets is definitely even better, and it removes some complexity from the client-side implementation, but it makes cryptocurrency protocol more complex

First of all, let's note that the problem is actually not with fake exchange proposals, but with fake offers: we could simplify the protocol and require Alice to include her txins with the offer. In this case Carol would post a partially signed transaction, and will be left at mercy of Alice who can either make a fully-signed transaction, or ignore it.

Freimarkets kind of eliminate the problem with fake offers, as offers are sub-transactions and are already signed.

But Alice can still disrupt the exchange with double-spending: when she sees that somebody made a complete transaction out of her sub-transaction, she will double-spend her inputs, thus making exchange impossible. The effect is the same as with fake offers DoS attack described above: offers are seen, but cannot be executed. (I assume that Alice can perform this double-spending attack with high probability of success via network-level shenanigans.)

Thus if you want an exchange which has more stable behavior, you still might want some kind of reputation system or double-spend-prevention service even if you have sub-transactions.


I don't think that that it is a same thing... Smartphones are, essentially, online all the time, they can receive push notifications and such. Somehow it doesn't inconvenience people.

Although, more realistically, trader will keep his laptop open for 8+ hours a day (because he uses it), and execution of orders is possible during that time. 8 hours a day is good enough. In fact, that's how exchanges like NYSE work, they are not available 24h a day.

Having an ability to get your offers filled 24h a day might be better, but not always: some catastrophe might happen over night, when you're sleeping, and you'll buy some worthless securities.

So my point is, offline execution is a useful feature, but not critical.


I see, it might be really important for some use cases.


We haven't decided yet, I'll post more information about these two approaches on the bitcoinX mailing list in a couple of days.


Yes. I guess making Bitcoin a special kind of an asset is good for people who invested in Bitcoin, so this limitation might be seen as a feature.


Hmm, interesting... Well, you can partially address granularity problem by publishing many offers of different size which use same inputs, e.g. 100, 50, 25, 10, etc. This will reduce blockchain bloat, but isn't same thing as proper granularity, I guess.

By the way, did you consider the fact that extra complexity brought by advanced features like sub-transactions and such might lead to higher chances of having critical bugs in code? Many feature in Freimarkets proposal are far from trivial, so it is a serious concern, in my opinion.

Yes, if we consider only simple trading (without considering any of the other [more than 10] use cases described in the doc), the main difference is that sub-transactions open the door to orders that can be executed without you being online.
That also closes the door to potential (not unsolvable) DoS attacks like the one Carol does in the previous example: she does it once, but she could had done it ad nauseam as well.
But maybe being online all the time stops being the inconvenience we predict and people also start to run their own mail servers.
Not having to be online was, in any case, the main advantage of ripplecoin over 2PC distributed Ripple (at least for "IOU-trade-chain" use cases). Maybe that's why I give it that importance.
Another basic thing is granularity vs all or nothing.


I read something about an alternative approach to order-based
coloring with some advantages at bitcoinX mailing list. And although
it sounded interesting I didn't went into too much detail.
Where can I read more about this?
Is chromawallet opting for this approach or order-based coloring?


I guess you mean an "ask" in AAA/BTC, which is the same as a bid in
BTC/AAA.
Please, don't use that terminology as it is confusing.
What's the ask in AAA/BBB ?

If the trade were AAA/BBB, anyone could get the AAA for uncolored
satoshis instead of the wanted BBB.
So what you really want to say is that "it is limited to offers in
which colored coins are offered in exchange of the host currency (BTC)".

But back to the "basic freimarket without sub-txs" discussion, an
approach similar to this one could work generically without being
online and without sub-txs. The main difference being the absence of
granularity, which can be simulated by bloating the chain with N
smaller offers instead of a single sub-tx.
So I correct my answer to the first question: without considering off-chain interoperability, options or any of the other use cases, if we only consider only the most basic p2p trade, the main advantage of sub-txs is granularity in the orders.



The premium is a critical part of the option. There's no reason for
Alice to give that trade privilege to Bob if she's not getting a
premium in return.
Making the trade between the premium and the right to trade atomic is
the hardest part of the problem, not just a nice addition you can skip.
Early exit for Alice just means the option is worthless.


Again, a CALL in AAA/BBB is a put in BBB/AAA. So although I'm not
sure I understand this limitation, it seems to me you're not being
generic enough.


So the first transaction signed by both parties is txA-Out.
After that Alice can sign txA more or less safely (if we forget about
Bob "DoS-ing" Alice by always making her wait for the lock in vain).
txC can in fact be signed safely from the beginning (not in step 7)
since it depends on txA and txB, which don't exist in the chain yet.
I miss why txB has to be sent to a 2-of-2.
Once txA is in the chain, Bob can sign and broadcast txB at any moment,
followed by txC.
So I don't see the need for Bob to lock his funds.


There's the 4th option mentioned that Bob is not interested in
actually exercising the option, but only limiting Alice's capacity to
trade properly (and make her spend fees) by continuously making Alice
wait for the 3rd option. Until Alice gets bored and stops to try to
trade options.

So in summary, comparing my approach to yours:

1) It makes sense for Alice because she receives a premium in
   exchange of the trade privilege for Bob.
2) It is secure for Alice, who can't be "DoSed"
3) It allows the option contract to be in the form of "up to 100 AAA
   for 100 BBB" instead of "100 AAA for 100 BBB, all or nothing".
   That is, freimarket's options are more flexible.

This is forgetting your "Bob locks his funds" limitation, which as
said I don't really understand. Bob should never sign txB before
having txC signed txA in the chain unless he wants some extortion
from Alice, since there's no txB-Out.

Thank you, Jorge. So, if we consider only simple trading (without options and such), difference is largely in ability to execute orders offline, right?

Well, I don't know whether this feature is important. It is certainly might be convenient to be able to execute orders offline, but I think decentralized exchange is usable (and useful) even if it functions only as long as all participants are online.

BTW some limited form of offline order execution is possible with colored coins. For example, using bitfield-tagging scheme proposed by Peter Todd: destination output(s) are specified in inputs. Thus it is possible to create a partial transaction which sends colored coin from input_0 to output_1, while output_0 has the requested payment. Input_0 is signed with "SIGHASH_SINGLE | SIGHASH_ANYONECANPAY" hashtype specified.

Buyer creates a complete transactions buy adding payment as input_1 (which he signs) and output_1 (which receives the colored coin).

Note that it is only possible to create sell offers ('asks') through this method.

As for options, it is trued that you can't do them with colored coins alone, but it is possible to do it using contracts ( https://en.bitcoin.it/wiki/Contracts ). Let's consider call option, for example:

1. We assume that Alice and Bob know each other's pubkeys and can communicate indefinitely long.
2. Alice creates a transaction which sends her colored coin to 2-of-2 multisig script, but doesn't sign this transaction. (Let's call it txA)
3. Bob creates a transaction which sends his bitcoins to 2-of-2 multisig script, doesn't sign it either. (Let's call it txB.)
4. Alice and Bob together create a call option contract transaction: it spends colored coin output from txA, sending it to Bob's address, it also spends txB's output, sending it to Alice. (Let's call it txC.)
5. Alice creates a transaction which spends output from txA to Alice's another address, and has nLockTime in future (e.g. a month from now). Let's call this transaction txA-out. Both Alice and Bob sign it.
7. Now Alice can sign txC.
8. Alice signs txA.

Three scenarios are possible:

1. Bob wants to exercise call option. To do this, he signs and published both txB and txC.
2. Bob wants to spend his money, he just double-spends txB's inputs.
3. When option expires, Alice can get her coins back by publishing txA-out.

I've left several niceties (premiums, early exit for Alice) as an exercise to the reader.

This is less than ideal because Bob's money is locked for as long as he wants an ability to exercise an option. It's possible to use the trick with SIGHASH_SINGLE I described above to avoid this problem, but it works only for CALL options.

Thank you for your contribution, Alex. It's also good to hear that we will have regular colored coins soon.
Even if we got all the funding from day 1 of the crowdfunding, our development estimates are around 1 year, so having simpler colored coins schemes working first was completely expected and nothing bad in my opinion.
Hopefully this create some usage and more understanding on the more generalized p2p accounting that colored coins represent.
Maybe some infrastructure or at least clients parts are reusable by freimarkets.

Now let me try to solve your doubts on the additional counterparty risks Mark means.
Say Alice wants to sell 10 AAA for 5 BTC, only in complete AAA units (doesn't make sells smaller than 1 AAA).
In Freimarkets, she would create a sub-transaction with 10 AAA as input, 5 BTC as output to an address under her control and granularity set to 10.
She broadcasts it to the network, turns her computer off, and goes to sleep.
Bob want's to buy 3 AAA for 1.5 BTC. If he has Alice's sub-transaction, he can just complete the transaction properly and broadcast it to be included in the chain.
Even if he hasn't seen Alice's sub-tx, Bob can create his own with 1.5 BTC as input, 3 AAA as output to an address under his control granularity to 1 (all or nothing), turn his computer off and go to bed too.
Any miner that has both transactions can form a complete transaction with both to claim any fees they have included or the difference between prices if there were any.
All this process was completely trust-less and Alice and Bob didn't needed to communicate directly with each other at any moment.

Now let's compare with the same trade with colored coins.
Alice publishes her offer (without signing anything) in some network created for this purpose.
She can't turn the computer off, because she's waiting for someone to connect with her to perform the trade.
Carol connects Alice sending "I will buy the 10 AAA for 5 BTC as you're requesting". Maybe she adds "I proof that I control and address containing 5 BTC" so that Alice can trust her more.
Alice says, "ok, this is the full transaction with my signature, when you add yours, the transaction will be valid and can be broadcasted to be included in the chain"
Carol disconnects.
Now Alice doesn't know whether Carol has received it and has intention to sign her part and broadcast or this was just a simple DoS attack.
After 3 blocks Alice is tired of waiting and decides she will look for another buyer. Maybe she double-spends the 10 AAA first to make the previous transaction invalid.
Maybe she downvotes Carol in a reputation system of some sort (maybe Carol actually lost her connection, who knows).
So Alice publishes a similar advertisement again.
Now Bob connects to her with his offer: "I want 3 AAA for 1.5 BTC".
Alice prepares the transaction, signs it and sends it to Bob.
Bob signs his part, broadcasts and the trade is done.

Carol's attack is not the end of the world at all, but there's more inconvenient differences.
If Alice was really tired and wanted to go to bed, some software could have taken these decisions for her, but she can't turn the computer off.
That won't be a problem for many people, there's even individuals that run their own mail servers at home.
But others will want to leave opened orders for when they are off-line, delegating that to an specialized service.
Alice would need to give them complete control of her 10 AAA to that service, that is, "deposit her 10 AAA in the service's web".
Bob could trust a different company and still make the trade, but trusting a service like this greatly reduces the "trustlessness" of the whole thing, as these services could just run with the money just like mail servers can read your unencrypted mails.
Of course holding your balances implies knowing your balances as well.

But this is only the most basic case and as stated earlier 10 of the 10 example use cases in the doc make use of sub-transactions.
Take, for example, the option contract. It may seem a little bit confusing, but 100% trust-less options isn't an easy problem.
As far as I know, there's no solution for them using order-based colored coins and I think all proposals I've seen for trust-less options required a new special option contract message with special structure and validation rules in the chain. Instead of just sub-tx and validation scripts (both generic primitives widely used by other use cases in the docs) like freimarkets.

Well, I hope this helps you see the advantages with sub-transactions, but don't hesitate to ask any further questions.

Two weeks(tm). No, really, I believe we'll release a usable version in January.

I'd like to see more information about this: practical difference between Freimarkets and a hypothetical system without partial transactions (or however you call it). In what situations do we see this difference? What kind of guarantees Freimarkets will allow which alternative system can't?

I hope you understand that I'm not just wasting your time, I genuinely want to learn more about this. FYI I've just donated 5000 FRC to this project.


You're in control of your coins only after transaction is included in the blockchain. Before that, you can be subject to double-spends.

I see no difference in case with decentralized exchange: there is no certainty before transaction is included into the blockchain.

So, philosophically, it isn't different from Bitcoin.

Prognosis not good then.  :-(

Killerstorm,  what is the ETA for colored coins?

.
First, I am unconvinced that reputation-based systems can offer the same feature set or scale as well. Second, it would be a fundamental step backward philosophically. Bitcoin puts you in control of your coins; the type of system you are developing would leave you at the mercy of counterparty risk. I firmly believe that there is value in making contracts directly enforceable, on the other hand.

I do not know of any other proposal at this time which offers, for example, binding signatures on partial transactions.


We are not working on Freimarkets right now due to a lack of resources.

It isn't a problem. There are other ways to enable decentralized exchange.

It looks like your plan is to provide it all in a monolithic package, while we aim solve it outside of the scope of the core protocol, which gives a lots of room for different approaches.

If you implemented only "color" validation without the rest of changes, Freimarkets could be a drop-in upgrade to colored coins: if we'll be able to provide decentralized exchange capabilities for colored coins (and I'm confident we'll provide them in near future), one could simply replace transaction encoding without changing the rest of the protocol, and thus benefit from fast validation.

But if you're going to implement the whole thing or nothing, it might be late at the party... There are many developments in this area, so it is possible that in a year or two Freimarkets won't offer significant advantages over other approaches.

Given the current funding situation,  what is the ETA for a prototype of the Freimarkets proposal?

Are we talking a few months from now, a year or so, or never?

I'm afraid the minimal set of changes for in-chain-validated colored coins that Mark already explained here:


...wouldn't get you nowhere near 80% of the functionality, and it's not a "quick fix" neither.
Without sub-transactions you don't even have open signed orders so traders would need to connect directly during the execution of the trade, which is a big disadvantage.
Furthermore, we don't know the full potential functionality that sub-transactions add yet. When Satoshi designed bitcoin's scripting language he didn't knew what the current list of potential contracts would be like.
In the same way, nobody knows all the uses the freimarkets extension could have.
What we know is that from the about 10 use cases described in the example cases of our doc, 10 of them make use of sub-transactions, for example, and many of them also use validation scripts (another extension, a script that needs to be valid for the whole transaction apart from each input's script).
Indivisible tokens may seem optional at a first glance too, but they're the base for implementing inflatable assets and authorized assets, needed among other things to implement KYC compliance for those issuers who need it (that can include local currencies which operate with special custom rules defined by their respective communities).

Finally off-chain assets and their inter-operability is the best answer we have for ultimate scalability of the whole network of assets.
Hopefully Mark's work on a committed UTXO soft fork will greatly help improve the scalability of bitcoin itself and reduce the trust for SPV nodes.
But as Peter Todd and others have explained many times, there's a trade-off between in-chain scalability and centralization and there will always be a limit to the scale of in-chain validation if you don't want to compromise the p2p nature of the system.
So even if we separate private chains in a second phase, we really believe its development is of critical importance.

The reason I ask for a 'minimal' spec is that the Freimarkets spec was proposed a couple of months ago and I am not sure if you made any progress implementing the spec (or receiving the required funding).  So I am wondering if there is a quick and dirty fix that does not require the development resources you requested, but is able to get say 80% of the final functionality.

Meaning,  if the scope of Freimarkets is reduced (say remove subtransactions),  what would be the smallest scope that can be defined such that it is easy to implement?

Well..

You have to put asset definitions on the chain, so that there is a deterministic process for figuring out which coins definitions exist. Then you construct a normative UTXO index with color annotations.

But if you're going to be making changes to bitcoin's validation rules, then you might as well solve other problems that colored coins have, such as a lack of binding signatures on orders. So you start making changes to support these as well since why not? you're forking anyway. Then you notice that there isn't good reason for many of these coins to be traded on the public chain. If you have a centralized issuer, it is nearly the same level of trust to have a centralized clearing house for transactions, and there are many positive benefits to that architecture. But then you need to add primitives for these off-chain private servers to coordinate via the public chain, 3rd party time stampers, or mutual observation. Etc.

If you've read the Freimarkets spec, this should be sounding familiar. It's exactly the thought experiment which led to where we are.

Freimarkets is a locally minimal set of changes to the bitcoin protocol to directly and efficiently support colored coins and their myriad applications. There are probably other possible designs that are equally minimal or perhaps even a little more so, but I think we're pretty close.

TL;DR The answer to your question is "Freimarkets".

What I meant was, what would be needed to incorporated into the bitcoin rules to ensure that colored coins are valid without the need to traverse back to the root transaction.

Colored coins doesn't require any changes to bitcoin. That us rather the point.

Curious,  what would be the minimum amount of changes to Bitcion (based on the Freimarkets proposal) to implement the functionality of Colored Coins?

For SPV proofs you need the history tracing colored coins back to their definition transaction. You do not need to store the history of every input back to genesis, so there isn't a combinatorial explosion of history, but it's still a large concern. When you first create a colored coin definition, proofs are problably about a kilobyte or two in size. Each time they change hands the proof grows by the size of the transaciton plus a few hundred bytes of metadata. So coins can change hands a few dozen or maybe up to a hundred times before the proofs start to get unweildy, although one's tolerance probably depends on many factors.

You could have an online issuer that every so often creates new colored coin definitions pegged 1:1 with the old coins, and operates an exchange to trade in old heavyweight coins for fresh new ones.  This works, barely. It's very much non-ideal, but it's also as far as I know the very best you can do without forking bitcoin. Tons of credit goes to Alex Mizrahi, Meni Rosenfeld, and others who have pushed this concept as far as it can go.


Alas, there is no real magic in this world. SNARK systems hold out the promise for a compact proof that doesn't require the entire block chain to validate. But the downside is that creating SNARK proofs is a very, very computaitonally intensive process. I think it's an open question in what way SNARKs will end up being useful in the bitcoin ecosystem, for this reason.


No, by incorporating coloring into the validation rules, Freimarkets is the same as bitcoin in this regard. If a transaction has made it into the chain, you can assume it and its attached coloring is valid with SPV level of security. If/when commitments of the utxo state are added to the coinbase, you could use those indices to check validity and coloring as well. So proof size remains small and relatively constant over time.