[ANN][XCP] Counterparty - Pioneering Peer-to-Peer Finance - Official Thread - page 383.

BitThink

legendary

Activity: 882

Merit: 1000

Quote from: dexX7 on March 19, 2014, 07:52:17 PM

Quote from: BitThink on March 19, 2014, 07:31:10 PM

As far as I know, 80 to 40 does not change much. Previously we divide data into segments of 80 bytes, now 40 bytes. Just the number of outputs doubled. What's the big deal? Only one OP-RETURN is allowed in one transaction?

Only one allowed.

Oops. Then we have to keep using multisig for long data.
do we know how many percent of current transactions can be squeezed into 40 bytes now?
Could we use asset id instead of using the name everywhere?

reader31

full member

Activity: 221

Merit: 100

Quote from: xnova on March 19, 2014, 07:53:37 PM

Quote from: Matt Y on March 19, 2014, 07:30:35 PM

Counterwallet live on testnet

Link: https://testnet.counterwallet.co/

Report bugs here: https://forums.counterparty.co/index.php?topic=188.0
Get testnet XCP here: https://forums.counterparty.co/index.php/topic,184
Get testnet BTC here: http://tpfaucet.appspot.com/
Blog post outlining functionality: https://www.counterparty.co/counterwallet-live-testnet/

xnova has done a wonderful job developing Counterwallet and he deserves a big thanks from all of us. Let's all start testing and get Counterwallet on mainnet as soon as possible!

Thanks xnova. A truly remarkable job you've done on this.

Thanks guys. There are no doubt still a number of bugs to work through to get the wallet to a mainnet release, but this is a big accomplishment for the team and we're looking forward to bringing the capabilities of Counterparty to a larger user population. We're targeting a mainnet release within the first week of April. Of course this can change depending on how the testnet testing goes, but I'm thinking we should be able to keep that deadline.

Awesome Job guys...will give it a whirl...

jgarzik

legendary

Activity: 1596

Merit: 1100

To date, I've not seen a blockchain data dumping scheme that could not be securely replaced with a simple hash.

You don't need to store data in the blockchain. That is purely intellectual laziness. Timestamping hash(data) is just as secure, while more efficient.

Furthermore, a secondary chain can be provably pegged to bitcoin: http://sourceforge.net/p/bitcoin/mailman/message/32108143/

porqupine

full member

Activity: 214

Merit: 101

Quote from: led_lcd on March 19, 2014, 07:51:15 PM

To clarify, in my proposal the long name wouldn't be changeable.

In the leave things mostly as they are scenerio, which is what I was describing above then the 'name space' would be able to change since it is just part of the asset description. The asset name is still the primary key and has to stay the same.

I can see the disadvantage of this where people can issue spam assets to 'typosquat' or even create duplicates of the same namespace. If it's not enforced by the protocol as you said, it's going to cause confusion.

The idea is that it is possible to implement name and category functionality without altering the protocol by making some filtration standards for indexing Assets, so an asset may be DKGCEWU (some unique 8 letter base_58 asset code or 'short name') ~ Gold Troy Ounce ~ ~ (issuer) ~ ~ (comment /pgp link) ~
of course one has to trust the issuer and check the link for the valid pgp signature, but once one has done this all the problems are gui / client problems, you could exclude the ability for someone else to create ~ Gold Troy Ounce ~ on the protocol level but then you are back to the original asset name problem. Instead you link the ~ Gold Troy Ounce ~ to the DKGCEWU that identifies the asset. Sure someone else can make a ~ Gold Troy Ounce ~ but it can easily be distinguished by the 'short name' as different.

xnova

sr. member

Activity: 390

Merit: 254

Counterparty Developer

Quote from: Matt Y on March 19, 2014, 07:30:35 PM

Counterwallet live on testnet

Link: https://testnet.counterwallet.co/

Report bugs here: https://forums.counterparty.co/index.php?topic=188.0
Get testnet XCP here: https://forums.counterparty.co/index.php/topic,184
Get testnet BTC here: http://tpfaucet.appspot.com/
Blog post outlining functionality: https://www.counterparty.co/counterwallet-live-testnet/

xnova has done a wonderful job developing Counterwallet and he deserves a big thanks from all of us. Let's all start testing and get Counterwallet on mainnet as soon as possible!

Thanks xnova. A truly remarkable job you've done on this.

Thanks guys. There are no doubt still a number of bugs to work through to get the wallet to a mainnet release, but this is a big accomplishment for the team and we're looking forward to bringing the capabilities of Counterparty to a larger user population. We're targeting a mainnet release within the first week of April. Of course this can change depending on how the testnet testing goes, but I'm thinking we should be able to keep that deadline.

dexX7

legendary

Activity: 1106

Merit: 1026

Quote from: BitThink on March 19, 2014, 07:31:10 PM

As far as I know, 80 to 40 does not change much. Previously we divide data into segments of 80 bytes, now 40 bytes. Just the number of outputs doubled. What's the big deal? Only one OP-RETURN is allowed in one transaction?

Only one allowed.

led_lcd

sr. member

Activity: 262

Merit: 250

Quote from: GLaDOS on March 19, 2014, 07:35:46 PM

Quote from: led_lcd on March 19, 2014, 07:02:54 PM

Quote from: GLaDOS on March 19, 2014, 06:58:13 PM

Quote from: led_lcd on March 19, 2014, 05:08:17 PM

Quote from: PhantomPhreak on March 19, 2014, 04:46:21 PM

How much of this proposal, and of other similar proposals, can be handled by the asset description field? (Not much space is needed to store a URL.).I love namespaces, etc., but I want to keep the protocol-level as simple and as stupid as possible. We could do this sort of thing is Counterwallet and BootleXCP themselves, for example.

Extremely good point and porqupine has also communicated the same to me.

We would only need a naming convention on how to format the asset description field. The rest can be up to the client side to parse it.

As long as there was some defacto which was perhaps outlined by the Counterparty team but not necessarily enforced, then everyone would be a happy camper.

Eg Lets just use the tilde as a delimiting character in the description. Field 1 = namespace, field 2 = description, field 3 = url

It needs to be a new long asset name field that is enforced by the protocol for long names. A description will not work because users can put anything in there.

My understanding (a dev can correct me) is that the description field can be modified for an asset.

So lets just say we all agree on a defacto standard for the description field. There are 3 fields delimited by a tilde.

Each of the clients can just invalidate and fail to show any asset in their list where 3 tildes aren't found. The asset owner would have motivation to get it right.

Long asset names should not be changeable just like short names. The long names are enforced at the time of issuance to be valid only if the issuer also owns the top level asset.

Afterward everything should go through the protocol using only the short names.

To clarify, in my proposal the long name wouldn't be changeable.

In the leave things mostly as they are scenerio, which is what I was describing above then the 'name space' would be able to change since it is just part of the asset description. The asset name is still the primary key and has to stay the same.

I can see the disadvantage of this where people can issue spam assets to 'typosquat' or even create duplicates of the same namespace. If it's not enforced by the protocol as you said, it's going to cause confusion.

GLaDOS

newbie

Activity: 24

Merit: 0

Quote from: led_lcd on March 19, 2014, 07:02:54 PM

Quote from: GLaDOS on March 19, 2014, 06:58:13 PM

Quote from: led_lcd on March 19, 2014, 05:08:17 PM

Quote from: PhantomPhreak on March 19, 2014, 04:46:21 PM

How much of this proposal, and of other similar proposals, can be handled by the asset description field? (Not much space is needed to store a URL.).I love namespaces, etc., but I want to keep the protocol-level as simple and as stupid as possible. We could do this sort of thing is Counterwallet and BootleXCP themselves, for example.

Extremely good point and porqupine has also communicated the same to me.

We would only need a naming convention on how to format the asset description field. The rest can be up to the client side to parse it.

As long as there was some defacto which was perhaps outlined by the Counterparty team but not necessarily enforced, then everyone would be a happy camper.

Eg Lets just use the tilde as a delimiting character in the description. Field 1 = namespace, field 2 = description, field 3 = url

It needs to be a new long asset name field that is enforced by the protocol for long names. A description will not work because users can put anything in there.

My understanding (a dev can correct me) is that the description field can be modified for an asset.

So lets just say we all agree on a defacto standard for the description field. There are 3 fields delimited by a tilde.

Each of the clients can just invalidate and fail to show any asset in their list where 3 tildes aren't found. The asset owner would have motivation to get it right.

Long asset names should not be changeable just like short names. The long names are enforced at the time of issuance to be valid only if the issuer also owns the top level asset.

Afterward everything should go through the protocol using only the short names.

dexX7

legendary

Activity: 1106

Merit: 1026

Quote from: jl777 on March 19, 2014, 07:23:44 PM

If this data is representative, 2:1 shouldnt be a problem. "00000000000000000000000" alone saves a lot of space
"20434e545250525459" appears three times, so just using those two as special words in a simple huffman tree you can squeeze this down a lot.

I have no idea what data is in the OP_RETURN, but I am assuming there are things that are globally common, like "0000..", maybe there are other globally common things. Put all those into a dictionary and based on the relative frequency assign a variable length bit pattern to the dictionary words.

I see. 0x20434e545250525459 ("CNTRPRTY") is actually used to flag outputs as XCP data. There is no room for simple substitution, I think - "0000.." won't alway be "0000..", but the idea about compression is very interesting nevertheless.

BitThink

legendary

Activity: 882

Merit: 1000

As far as I know, 80 to 40 does not change much. Previously we divide data into segments of 80 bytes, now 40 bytes. Just the number of outputs doubled. What's the big deal? Only one OP-RETURN is allowed in one transaction?

Matt Y

hero member

Activity: 647

Merit: 510

Counterpartying

Counterwallet live on testnet

Link: https://testnet.counterwallet.co/

Report bugs here: https://forums.counterparty.co/index.php?topic=188.0
Get testnet XCP here: https://forums.counterparty.co/index.php/topic,184
Get testnet BTC here: http://tpfaucet.appspot.com/
Blog post outlining functionality: https://www.counterparty.co/counterwallet-live-testnet/

xnova has done a wonderful job developing Counterwallet and he deserves a big thanks from all of us. Let's all start testing and get Counterwallet on mainnet as soon as possible!

Thanks xnova. A truly remarkable job you've done on this.

jl777

legendary

Activity: 1176

Merit: 1134

Quote from: dexX7 on March 19, 2014, 07:08:32 PM

Quote from: jl777 on March 19, 2014, 06:48:26 PM

Could the info needed be compressed to fit into 40 bytes?
What sort of entropy does the typical XCP OP_RETURN data have?

Depending on the data, it might be possible to create a small customized compress/decompress function

If someone can post an example it would help me analyze this a bit more

Edit: due to the importance of this I think it would even make sense to create a custom "dictionary" of common bit patterns (hopefully there are some) in the desired OP_RETURN data. This would have to be put into the counterpartyd, but doing stuff like this might be what is needed to get 2:1 compression, then again maybe standard compression will just work, but I think some sort of custom code would have a good chance. It all depends on the type of data.

James

What do you mean by "dictionaries for common bit patterns"? To my knowledge XCP transactions use some bits to flag a data package as XCP package and some to define the transaction type, but the rest is acutal transaction field data without OP codes or whatsoever.

Code:

1c434e54525052545900000000000000000000000100000045d964b80000000000 

20434e5452505254590000001e5329a91040952f00000000000000000033474f4c 
20442f55534420505249434520736f757263653a207777772e7175616e646c2e63 
106f6d204c6f6e646f6e20466978696e6700000000000000000000000000000000 

20434e5452505254590000002800035325c3780000000002faf0800000000002fa
12f08040ae140000000000000013b0000000640000000000000000000000000000

20434e5452505254590000000a00000000000000000000000005f5e10000000000 
160000000100000002540be400012c000000000000000000000000000000000000

If this data is representative, 2:1 shouldnt be a problem. "00000000000000000000000" alone saves a lot of space
"20434e545250525459" appears three times, so just using those two as special words in a simple huffman tree you can squeeze this down a lot.

I have no idea what data is in the OP_RETURN, but I am assuming there are things that are globally common, like "0000..", maybe there are other globally common things. Put all those into a dictionary and based on the relative frequency assign a variable length bit pattern to the dictionary words.

However, instead of all this trouble, it seems the trick of storing data in unused multisig outputs will solve it with much less work

James

dexX7

legendary

Activity: 1106

Merit: 1026

Quote from: jl777 on March 19, 2014, 06:48:26 PM

Could the info needed be compressed to fit into 40 bytes?
What sort of entropy does the typical XCP OP_RETURN data have?

Depending on the data, it might be possible to create a small customized compress/decompress function

If someone can post an example it would help me analyze this a bit more

Edit: due to the importance of this I think it would even make sense to create a custom "dictionary" of common bit patterns (hopefully there are some) in the desired OP_RETURN data. This would have to be put into the counterpartyd, but doing stuff like this might be what is needed to get 2:1 compression, then again maybe standard compression will just work, but I think some sort of custom code would have a good chance. It all depends on the type of data.

James

What do you mean by "dictionaries for common bit patterns"? To my knowledge XCP transactions use some bits to flag a data package as XCP package and some to define the transaction type, but the rest is acutal transaction field data without OP codes or whatsoever.

Code:

1c434e54525052545900000000000000000000000100000045d964b80000000000 

20434e5452505254590000001e5329a91040952f00000000000000000033474f4c 
20442f55534420505249434520736f757263653a207777772e7175616e646c2e63 
106f6d204c6f6e646f6e20466978696e6700000000000000000000000000000000 

20434e5452505254590000002800035325c3780000000002faf0800000000002fa
12f08040ae140000000000000013b0000000640000000000000000000000000000

20434e5452505254590000000a00000000000000000000000005f5e10000000000 
160000000100000002540be400012c000000000000000000000000000000000000

led_lcd

sr. member

Activity: 262

Merit: 250

Quote from: GLaDOS on March 19, 2014, 06:58:13 PM

Quote from: led_lcd on March 19, 2014, 05:08:17 PM

Quote from: PhantomPhreak on March 19, 2014, 04:46:21 PM

How much of this proposal, and of other similar proposals, can be handled by the asset description field? (Not much space is needed to store a URL.).I love namespaces, etc., but I want to keep the protocol-level as simple and as stupid as possible. We could do this sort of thing is Counterwallet and BootleXCP themselves, for example.

Extremely good point and porqupine has also communicated the same to me.

We would only need a naming convention on how to format the asset description field. The rest can be up to the client side to parse it.

As long as there was some defacto which was perhaps outlined by the Counterparty team but not necessarily enforced, then everyone would be a happy camper.

Eg Lets just use the tilde as a delimiting character in the description. Field 1 = namespace, field 2 = description, field 3 = url

It needs to be a new long asset name field that is enforced by the protocol for long names. A description will not work because users can put anything in there.

My understanding (a dev can correct me) is that the description field can be modified for an asset.

So lets just say we all agree on a defacto standard for the description field. There are 3 fields delimited by a tilde.

Each of the clients can just invalidate and fail to show any asset in their list where 3 tildes aren't found. The asset owner would have motivation to get it right.

GLaDOS

newbie

Activity: 24

Merit: 0

Quote from: led_lcd on March 19, 2014, 05:08:17 PM

Quote from: PhantomPhreak on March 19, 2014, 04:46:21 PM

How much of this proposal, and of other similar proposals, can be handled by the asset description field? (Not much space is needed to store a URL.).I love namespaces, etc., but I want to keep the protocol-level as simple and as stupid as possible. We could do this sort of thing is Counterwallet and BootleXCP themselves, for example.

Extremely good point and porqupine has also communicated the same to me.

We would only need a naming convention on how to format the asset description field. The rest can be up to the client side to parse it.

As long as there was some defacto which was perhaps outlined by the Counterparty team but not necessarily enforced, then everyone would be a happy camper.

Eg Lets just use the tilde as a delimiting character in the description. Field 1 = namespace, field 2 = description, field 3 = url

It needs to be a new long asset name field that is enforced by the protocol for long names. A description will not work because users can put anything in there.

PhantomPhreak

sr. member

Activity: 476

Merit: 300

Counterparty Chief Scientist and Co-Founder

Quote from: led_lcd on March 19, 2014, 06:52:37 PM

Quote from: GLaDOS on March 19, 2014, 06:26:53 PM

As far as I can tell, this suggested implementation should work and be compatible with the existing protocol.
This is actually another great suggestion the more I think about it now.

It reminds me of the 8.3 short/long filename. It should be okay to allow lowercase characters, numbers, and symbols for the long asset name since it will be hash and encoded into base 26 anyway.

Yes, that's basically the gist of it. Take a long asset name and fold it into a short asset name.

The benefit of the distinction between a top level and lower level asset issuance is to give a means to tier the costing of issuing assets in a simple fashion.

I'd be happy to stay with the status quo plus a defacto standard on how to craft the asset description to include a namespace as long as:

a) If we think that the current pricing model is only a short term solution until a floating fee solution is found (similar to transmission fees in Bitcoin).

b) The defacto standard is something centralized such that there is agreement in the community.

I think that's right.

PhantomPhreak

sr. member

Activity: 476

Merit: 300

Counterparty Chief Scientist and Co-Founder

Quote from: jl777 on March 19, 2014, 06:48:26 PM

Quote from: dexX7 on March 19, 2014, 06:38:38 PM

80 bytes would be better than 40 bytes, but is this amount sufficient to support all XCP transaction types?

Could the info needed be compressed to fit into 40 bytes?
What sort of entropy does the typical XCP OP_RETURN data have?

Depending on the data, it might be possible to create a small customized compress/decompress function

If someone can post an example it would help me analyze this a bit more

Edit: due to the importance of this I think it would even make sense to create a custom "dictionary" of common bit patterns (hopefully there are some) in the desired OP_RETURN data. This would have to be put into the counterpartyd, but doing stuff like this might be what is needed to get 2:1 compression, then again maybe standard compression will just work, but I think some sort of custom code would have a good chance. It all depends on the type of data.

James

It would take some serious shoe-horning to fit all of the data we need into 40 bytes reliably, and in three months they could lower it to 35 bytes (again, for no reason) and then we'd be screwed.

On the plus side, we can fit a lot more than 80 bytes into 256 one-of-three multi-sig outputs, and now we have no incentive to use less than that except lower fees.

led_lcd

sr. member

Activity: 262

Merit: 250

Quote from: GLaDOS on March 19, 2014, 06:26:53 PM

As far as I can tell, this suggested implementation should work and be compatible with the existing protocol.
This is actually another great suggestion the more I think about it now.

It reminds me of the 8.3 short/long filename. It should be okay to allow lowercase characters, numbers, and symbols for the long asset name since it will be hash and encoded into base 26 anyway.

Yes, that's basically the gist of it. Take a long asset name and fold it into a short asset name.

The benefit of the distinction between a top level and lower level asset issuance is to give a means to tier the costing of issuing assets in a simple fashion.

I'd be happy to stay with the status quo plus a defacto standard on how to craft the asset description to include a namespace as long as:

a) If we think that the current pricing model is only a short term solution until a floating fee solution is found (similar to transmission fees in Bitcoin).

b) The defacto standard is something centralized such that there is agreement in the community.

jl777

legendary

Activity: 1176

Merit: 1134

Quote from: dexX7 on March 19, 2014, 06:38:38 PM

80 bytes would be better than 40 bytes, but is this amount sufficient to support all XCP transaction types?

Could the info needed be compressed to fit into 40 bytes?
What sort of entropy does the typical XCP OP_RETURN data have?

Depending on the data, it might be possible to create a small customized compress/decompress function

If someone can post an example it would help me analyze this a bit more

Edit: due to the importance of this I think it would even make sense to create a custom "dictionary" of common bit patterns (hopefully there are some) in the desired OP_RETURN data. This would have to be put into the counterpartyd, but doing stuff like this might be what is needed to get 2:1 compression, then again maybe standard compression will just work, but I think some sort of custom code would have a good chance. It all depends on the type of data.

James

dexX7

legendary

Activity: 1106

Merit: 1026

80 bytes would be better than 40 bytes, but is this amount sufficient to support all XCP transaction types?

Topic: [ANN][XCP] Counterparty - Pioneering Peer-to-Peer Finance - Official Thread - page 383. (Read 1277011 times)