color kernels strike back

[Alex Mizrahi]

The choice of optimal color kernel is still an open question, as there is no 'perfect' one at this point. I've narrowed it down to three designs, in order of preference:

1. CTDE: 'colorvalue transfer data encoding':

    Information about transfer is encoded as data in OP_RETURN output, not as satoshi values.

    Pros: 

      * departure from coupling colorvalues with  satoshi values means high divisibility and no weird side-effects.

      * compatible with 'partial transactions'

    Cons: 

      * currently not all pools support OP_RETURN, so confirmations will be delayed

      * there are people who dislike OP_RETURN and there is no guarantee that it will always be available

      * it is departure from the original idea of coloring satoshis

      * 80-byte limit means that we'll be limited to 4-5 colored outputs. It's OK for p2ptrade transactions, but is too little for mixing.

         (There is a way to address this issue by making protocol a bit more complex.)

2. EPOBC: a variant of POBC with variable padding. 

  Transaction can be marked either via nSequence or OP_RETURN output.

Pros:

  * fairly close to the original design, 1 satoshi is 1 atom of colorvalue, padding is accounted for separately

  * relatively easy to explain, people can get intuitive understanding

  * can be used right now, doesn't rely on OP_RETURN

  * if nSequence becomes relevant again it will be usable

Cons:

  * coupling satoshi values with colorvalues means that high divisibility costs a lot, some people believe it creates accounting problems

  * not compatible with 'partial transactions' which are useful for offline trading

  * is a bit complex

3. ITOG: information about transfer is encoded in nSequence, supports both padding and scaling.

(I already implemented it, by the way.)

Pros:

 * high divisibility without use of OP_RETURN, usable right now

 * compatible with 'partial transactions'

Cons:

 * it is fairly hard to explain how it works

 * in some cases one needs considerable amount of bitcoins to perform a transaction (temporarily)

 * extra complexity associated with satoshi value accounting

To summarize, ITOG is probably no longer relevant.

Choice between CTDE and EPOBC is largely a choice between old a new model.

EPOBC has an additional advantage that it would work right now.

There is a risk that OP_RETURN will be banned in future releases, but in that case we can do a protocol update, it isn't particular painful within colored coins model.

Thoughts?

[Tamas Blummer]

All suck, and this project are moving in circles since a year.

Cointrace is the way to go. Backward compatibility with legacy wallets, simple implementation similar limitations than these elaborate designs.

--
You received this message because you are subscribed to the Google Groups "Colored coins (BitcoinX)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoinX+u...@googlegroups.com.
To post to this group, send email to bitc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Alex Mizrahi]

All suck, and this project are moving in circles since a year.

We didn't have an option to encode data in OP_RETURN until now.

Cointrace is the way to go. Backward compatibility with legacy wallets, simple implementation similar limitations than these elaborate designs.

Well again, the point of having colored coins on the Bitcoin blockchain is trustless trade between peers. You're missing the point.

[Tamas Blummer]

You are missing the point. Cointrace is about tracking property over the block chain with simplest means beackward compatible to legacy wallets.

Decentralized exchange is a higher level layer that needs to be built no matter if its your complicated design or cointrace.

It is quite straightforward to define swap and trade transactions in cointrace that could be signed in the CoinJoin manner by

different parties. One needs just an additional channel to broadcast proposals.

I actually made an attempt on adding those trade types in: https://bitcointalk.org/index.php?topic=371984.msg4832209#msg4832209

[Alex Mizrahi]

You are missing the point. Cointrace is about tracking property over the block chain with simplest means beackward compatible to legacy wallets.

This backward compatibility essentially boils down to an ability to use same addresses (keypairs) as used for normal Bitcoin transfers.

I really doubt it is an important feature. I mean, how hard is it to have a separate set of addresses for asset transfers?

You need to add a lot of code to handle asset transfers anyway, so a little extra code to keep a separate set of addresses is barely important.

So the trade-off is essentially between use of a designated set of addresses and ability to do payment verification with much less data. Hmm.

We already have a thin web client which does coloring via backward scan of transaction graph (without trusting the server), the demo which does this was available for more than half of year: http://bitcoinx.github.io/webcoinx/

It neither needs to scan the whole blockchain, nor needs to trust the server.

Can you do that with cointrace?

>  its your complicated design

You keep saying that it's complicated, but it takes only about a day to implement the main part of it. How is it complicated? Why does it matter?

FYI the first wallet based on coloredcoinlib was implemented by a 12 year old guy over the weekend. (I'm not kidding.)

Absolutely none of people working on this project complained about complexity.

 

It is quite straightforward to define swap and trade transactions in cointrace that could be signed in the CoinJoin manner by

different parties. One needs just an additional channel to broadcast proposals.

Which is exactly what we already implemented in ChromaWallet. (And, previously, in ArmoryX.)

[Tamas Blummer]

I say it is complicated since after about a year of existence this project is where it started. All development done is in question as illustrated in

your opening post.

I did not expect you to recognize this, but hoped others are listening.

[Alex Mizrahi]

I say it is complicated since after about a year of existence this project is where it started. All development done is in question as illustrated in

your opening post.

Ah, it looks like you're simple not aware of context...

The new colored coin client design we have (NGCCC aka ChromaWallet) supports pluggable color kernels, basically everything related to kernel is encapsulated in one class.

For example, here's an implementation of order-based coloring:

https://github.com/bitcoinx/ngcccbase/blob/master/coloredcoinlib/colordef.py#L111

The whole thing is just 130 lines long.

And the most complex I've done is ITOG, it is about 250 lines long.

So color kernel implementation is not important, it can be done in matter of days.

I.e. if people decide that encoding colorvalues in OP_RETURN outputs is the way to go, I can make it working literally tomorrow.

We were working on the rest of the wallet, making it nice, fast and safe take some effort.

But you'll need same kind of functionality if you implement cointrace:

 * you need code which scans the blockchain for cointrace transactions

 * records address balances in some kind of database

 * handles reorganizations

 * shows history 

and so on.

We already implemented these parts in such a way that they are color kernel agnostic.

 

I did not expect you to recognize this, but hoped others are listening.

Others already tested ChromaWallet on the testnet and are waiting for an improved version.

At minimum the improved version will include another color kernel and an ability to work without a local bitcoind.

[Jorge Timón]

I don't have a strong opinion about the best coloring kernel, but as
said before somewhere, I think "partial transactions" or "open binding
orders" can be important even if they're not complete like in
freimarkets (ie even if you can offer CCs for BTCs but not the other
way around).

I'm glad that you're exploring better solutions than the initial order
based coloring and I think that the abundance of coloring kernels
actually shows that it was a good abstraction to have multiple
schemes with a shared codebase.

About using coinjoin for p2p trade...Mark Friedenbach is working on
coinjoin and he mentioned that he wanted to talk to you about using it
for colored coins p2p trade.
In the end we prefer freimarkets for scalability reasons and features
(we're now also studying how to better replace bitcoin's scripting
language[1]), but we still think colored coins is the best near term
solution (and you're not selling "examplecoins" to stupid speculators
or anything...).

By the way, I mentioned it to you before, but I'll try one last time...
Since chroma wallet is a nonprofit project clearly related to both
complementary currencies and free knowledge, you could list it to
receive freicoin donations matched by the foundation.
10 organizations/projects have received 40905.47 FRC in donations plus
matching so far.
It's not much, but I think any help is welcomed, no?
Well, write me if you need any help on this. If you don't say anything
this time I will assume you're not interested in receiving freicoin
donations and I won't ask again. The nonprofit colored coins could be
listed too I think.

So keep up the good work!

[1] https://groups.yahoo.com/neo/groups/concatenative/conversations/messages/4950

[Alex Mizrahi]

Since chroma wallet is a nonprofit project clearly related to both
complementary currencies and free knowledge, you could list it to
receive freicoin donations matched by the foundation.

Thanks, but currently we have enough money for ChromaWallet development and are not interested in donations in general.

Maybe other projects need it, I don't know.

I think we'll change the way how ChromaWallet development is funded soon, and then we'll be glad to accept Freicoin donations. 

(In simple terms, it needs more transparency, but I'm not ready to set it up right now.)

BTW personally I support Freicoin as a cryptocurrency, and people who were working on ChromaWallet in November got 1000 FRC bonus (each).

[Alex Mizrahi]

 

Others already tested ChromaWallet on the testnet and are waiting for an improved version.

At minimum the improved version will include another color kernel and an ability to work without a local bitcoind.

BTW now it looks like it is a good thing that we didn't release the version with ITOG coloring as planned in January: it could be really confusing, and thus damaging.

Now it looks like OP_RETURN becomes practically available sooner than I thought (it wasn't on the table for planned January release), and we have a chance for a clean start.

[Flavien]

I think we all agree ITOG is over-engineered and is probably not suitable.

Regarding 1, I think this is a big issue:

* 80-byte limit means that we'll be limited to 4-5 colored outputs. It's OK for p2ptrade transactions, but is too little for mixing.

         (There is a way to address this issue by making protocol a bit more complex.)

When you think about the use cases, it will not be practical to be limited to 4-5 colored outputs. People will want to be able to send several colors in a single transaction on a regular basis - potentially several dozens. The reason is the following: those would be physically separate colors (issued by different transactions, therefore non-mixable), but represent the same issuer and same contract. They will be separate colors because the issuer will have reissued coins over time, therefore creating a new color, but associating them with the same contract (essentially multiple issues of the same asset).

As an example, if you want to pay 10 X-coins to Bob, you will send him 2 X-coins from issue A + 5 X-coins from issue B + 3 X-coins from issue C. Those are different colors from a protocol point of view, but represent the same contract and have the same value, so Bob doesn't care whether it came from issue A or B or C as long as the total is 10. But that means that colors coins payments may be quite fragmented, and limiting the transaction to 4-5 colors will be highly impractical.

Regarding 2:

2. EPOBC: a variant of POBC with variable padding.

How exactly do you plan to store the variable padding, without resorting to OP_RETURN?

I think the best solution sits somewhere between those 2:

• a: Use a mandatory recognizable OP_RETURN output to mark a color transfer transaction
• b: Use the satoshi value + ordered based coloring to map inputs to outputs
• c: Use no padding as it is a clumsy solution in search of a real problem

Point a makes payment validation as easy as technically possible. Point b and c make it a simple protocol ("what you see is what you get"), addressing the concerns of complexity some have voiced (both for implementers and users).

[Alex Mizrahi]

 

* 80-byte limit means that we'll be limited to 4-5 colored outputs. It's OK for p2ptrade transactions, but is too little for mixing.

         (There is a way to address this issue by making protocol a bit more complex.)

When you think about the use cases, it will not be practical to be limited to 4-5 colored outputs. People will want to be able to send several colors in a single transaction on a regular basis - potentially several dozens.

Good point...

 

The reason is the following: those would be physically separate colors (issued by different transactions, therefore non-mixable), but represent the same issuer and same contract. They will be separate colors because the issuer will have reissued coins over time, therefore creating a new color, but associating them with the same contract (essentially multiple issues of the same asset).

Well, we can re-think how we handle "multiple issues", as having a dozen of colors in one transaction isn't a good thing.

There is a good solution under condition that:

1. additional issues are relatively infrequent

2. issuer is active

In that case we can make it work in such a way that a new color will replace old colors: i.e. you and issuer make a transaction where coins of the old color are destroyed and you get coins of a new color.

This is more efficient that multiple independent issues, and also it solves the problem with a long history checks: people will get fresh coins with minimal history.

So this also solves the problem with lightweight clients.

 

2. EPOBC: a variant of POBC with variable padding.

How exactly do you plan to store the variable padding, without resorting to OP_RETURN?

There are two options:

1. Encoded in nSequence. I know that people find it icky, but there is no problem with it.

Transaction replacement semantics is unusable, it depends on all miners being honest. This makes no sense without very fundamental changes to the Bitcoin protocol. (But if nSequence somehow becomes usable again, we can switch to OP_RETURN which acts as a backup plan.)

2. Magic output: the first output is sent to self by the creator of transaction, and number of satoshi in it is used to encode padding: it is of form 42 + 10^n for 10^n padding.

I.e. if first output is 142, then we have 100 satoshi padding, if it is 1042 it is 1000 satoshi padding, if it is 42 then no padding is used (it is a special case).

I do not like the second option because it creates some extra bloat: those magic outputs should be spent.

 

• c: Use no padding as it is a clumsy solution in search of a real problem

Ehm, the problem is that we want it to be usable right now. Without padding it isn't.

Sure, we can make a protocol which ignores anti-dust rules and hope that Bitcoin devs will remove those rules.

But the thing is, we face competition from Mastercoin, Counterparty, Cointrace etc.

I understand that padding is bad from aesthetics point of view, but in practice, cost is minimal: handling padding requires like 30-50 lines of code, but it makes colored coins usable right now, so benefits are tremendous.

It's not really a problem to write that code and be extra, extra careful during code review to make sure it works as advertised.

Sacrificing practicality for aesthetics reasons is a bad idea IMHO.

 

Point a makes payment validation as easy as technically possible. Point b and c make it a simple protocol ("what you see is what you get"), addressing the concerns of complexity some have voiced (both for implementers and users).

You need to take into account that Tamas managed to implement simple order-based coloring incorrectly twice.

So really there is no alternative to meticulous code reviews.

And we already spent more time on these debates than time required to implement and review code.

[Flavien]

There is a good solution under condition that:

 

I hope you realize that this is way too impractical for the majority of use cases.

 

Sure, we can make a protocol which ignores anti-dust rules and hope that Bitcoin devs will remove those rules.

But the thing is, we face competition from Mastercoin, Counterparty, Cointrace etc.

 

When I see that those guys are willing to literally burn millions of dollars just to get XCP or MSC, I'm not sure than having no padding will really be an issue here. At least you see immediate "return" for the capital you have to put: issuing colored coins.

 

I understand that padding is bad from aesthetics point of view, but in practice, cost is minimal: handling padding requires like 30-50 lines of code, but it makes colored coins usable right now, so benefits are tremendous.

 

Well aesthetics translates directly into user experience - I'm not talking about complexity in terms of code here. How do you explain to the user how much BTC they need to send (as padding) to send X colored coins?

 

Say I have 0.2 colored BTC and 0.01 BTC worth of padding (spread across several outputs), and I want to send 0.1 colored BTC. How do I know how much BTC (out of the 0.01) will go into the transaction as padding? It's actually completely random, it depends how much padding each of your inputs have, and how much padding you will include in your outputs. In UX, if you can't explain in a sentence what the rule is, that means it's not good for prime time.

 

Without padding, you can have purely colored transactions, and it's much clearer what happens to your balance. That's what I call "what you see is what you get". This is the same reason why XML prevails over binary formats today, even though XML is very inefficient, both in terms of space and in terms of parsing: it's human-friendly.

[Alex Mizrahi]

Let's consider a practical example: there is a company called ActiveMining (aka VirtualMining), its CEO, Ken Slaughter, announced that they consider using colored coins as a trading platform (previously it was on  BitFunder and BTCT.co, both of which were closed), and users are (were?) enthusiastic about it: https://bitcointalk.org/index.php?topic=297503.msg4602246;topicseen#msg4602246

So it could be one of first real colored coin deployments.

Currently 25 000 000 are outstanding, 15 000 000 are not sold yet.

40 000 000 shares total.

Let's consider how it works with different coloring schemes mentioned above: suppose Ken asks how to issue coins and distribute them among shareholders.

1. CTDE (encoded in OP_RETURN): requires about 10000 satoshi per shareholder.

2. EPOBC: 40 * 10^6 * 10^(-8) = 0.4 BTC are needed to represent 40 000 000 shares.

    And in addition to that about 10000 satoshi per shareholder to send them.

3. Your proposal: OBC with OP_RETURN marking:

Oh, Ken, you see, you have a couple of options:

a)  You can spend 0.4 BTC creating 40M shares 1 satoshi each, but then you need to make sure that you never transact less than 5700 shares, and change cannot be less than 5700 shares... Otherwise transaction might get stuck.

Inconvenient? No, it is not a colored coin problem, it is a Bitcoin problem! We hope they'll fix it next year when they see so many people using colored coins. Not good?

b) Another option is to have 10000 satoshi per share, then you can send as little as 1 share... Yes you need 400 BTC to issue shares.... Bye.

Obviously, this is 100% unusable, so Ken will either use something like Counterparty/Mastercoin/Cointrace, as they say they have a working decentralized exchange, which is slow, but doesn't have these ridiculous rules.

How do you explain to the user how much BTC they need to send (as padding) to send X colored coins?

Yes: padding is on same scale as minfee, so I'll just say that colored coin transactions have higher fees. Say, twice higher than normal fee.

You can forget about crazy transaction which send a dozen of colors at once, normally recipient gets only 1 txout.

Users who do many transactions (e.g. market makers) need to know more, but it's fairly easy to describe.

You're overthinking it. Very few Bitcoin users know how Bitcoin works under the hood, but nevertheless keep using it.

Say, recently I talked with a guy who was implementing a game and needed to process a lot of payments. He thought that using same address for different payment is beneficial because it reduces fees. I told him that total amount of fees is proportional to number of payments one receives and does not depend on which addresses one use.

Very few Bitcoin users know about transaction outputs, but transaction outputs definitely affect fees they pay. Somehow it doesn't make Bitcoin non-viable cryptocurrency.

Going back to colored coins, maybe in distant future people will need to send many colors in one transaction, and won't like spending a lot on padding... Well, maybe by that time anti-dust rules will be abolished... Or we'll use an entirely different encoding scheme.

What's important, paying padding is voluntary, and it's up to a sending party to decide on this.

Thus coloring scheme which permits padding is strictly superior to one which doesn't: padding can be turned off when it is unnecessary.

 

 Say I have 0.2 colored BTC and 0.01 BTC worth of padding (spread across several outputs), and I want to send 0.1 colored BTC. How do I know how much BTC (out of the 0.01) will go into the transaction as padding? It's actually completely random, i

Fee is also "completely random", does it bother you?

And normally padding exists is on same scale as fees.

 

In UX, if you can't explain in a sentence what the rule is, that means it's not good for prime time.

Apparently, Bitcoin isn't good for the prime time either...

 

Without padding, you can have purely colored transactions, and it's much clearer what happens to your balance. That's what I call "what you see is what you get".

Yes, this is nice... But it is unusable.

[Tamas Blummer]

I apologize, I was not aware of the context.

The cointrace proposal is really simple, but works only on a full node. 

Above proposals would be suitable for a node that knows all inputs of a transaction transitively until the color genesis, right? That's more then SPV but less then full node.