SPEEDEX Configuration Proposal

[Jonathan Jove]

Following on from Building SPEEDEX and Behind the Scenes with SPEEDEX, CAP-0044 is our first published proposal working toward SPEEDEX on Stellar.

This proposal is an early step, as it handles only the configuration aspects of SPEEDEX. With that said, this proposal still has some interesting design decisions in it. One of the main design points  is around deciding what assets should be eligible to participate in the SPEEDEX market mechanism. This proposal hands the decision making power to the validators.

I'm looking forward to any and all feedback, criticism, comments, and questions.

[Leigh McCulloch]

Hi Jon,

 

A couple questions:

 

1 –

 

>The set of assets must be configurable because the time to compute a solution increases with the number of assets, so we cannot compute a solution for all assets.

 

The motivation for specifying some way to produce a limited set of assets is clear, but it is less clear why CAP-44 is the most appropriate way to arrive at that set. I see that the design rationale specifies two alternatives considered and that both alternatives are considered ineligible because in a large part they depend on a nominating validator getting to make the decision on either which assets and configuration, or just which configuration, applies and how that power could result in "anarchy" or a "pathological" proposal. Are there alternatives where asset selection is derived from the transaction set and is something that all validators decide on, not just the nominating validator? For example, the related goal noted in the proposal is that the Stellar network will run at scale. Could an alternative be to select the assets based on the quantity of trades, selecting asset pairs with more trades since they theoretically require greater scale, and deferring transactions that trade assets not demanding scale to the existing non-SPEEDEX trading mechanism?

 

2 – How do you see CAP-44 affecting the equitable use of the network? I think the proposal should address this both from the perspective of users, and issuers in regions not represented by validators of the network.

 

Leigh

--
You received this message because you are subscribed to the Google Groups "Stellar Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stellar-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/stellar-dev/CAG4FKhwG9iDCT0oF4eTGe4OSc8jy_QVxBoXfvSbhXmiR10BA9g%40mail.gmail.com.

[Jonathan Jove]

Hi Leigh,

1. This is an interesting question. I did consider mechanisms where the set of assets are derived from the transaction set. This approach doesn’t necessarily make it something that all validators decide on because, in practice, nominated values rarely get combined. This means that the transaction set nominated by the validator is likely to be chosen as is, and the validator can select whatever transaction set they want.

With that said, it’s still worth considering the merits of deriving the set of assets from the transaction set. This approach led me to some interesting challenges, but they might not be insurmountable. The main issue was that it’s not obvious to me how to actually construct the set of assets. Imagine a world with a few dominant assets (in the world of fiat currency this approximately translates to the basket of IMF special drawing rights currencies which are USD, EUR, CNY, JPY, and GBP). It is reasonable to guess that all of these assets would appear in SPEEDEX orders in every ledger. But these assets would also participate in liquidity pools for many other, less dominant assets. Those less dominant assets are less liquid, so we shouldn’t expect all of those assets to appear in SPEEDEX orders in every ledger (more generally, there will probably always be many more assets on the network than there is capacity for operations in a transaction set). So how do we choose among the less dominant assets? We can’t easily determine which ones will trade a lot, because doing so means computing the clearing prices for that larger set of assets… which is exactly what we are trying to avoid by limiting the set in the first place. 

There might be reasonable heuristics for this problem. But it’s not obvious to me that a heuristic would consistently outperform a set of validators that are actually trying to steward the network.

2. I’ve given this some thought, but I don’t really have a particularly good answer at this point. I have already spoken to some people about this, and in general I’m very interested to hear what other people have to say on this topic.

My hope here is that validators would be open to adding high impact (in terms of value, utilization, or otherwise) assets regardless of region because it would support the network. If we find that there are more assets that validators want to include than the current implementation can support for performance reasons, then efforts can be made to optimize the implementation. It’s hard for me to predict whether this hope is reasonable or likely.

It’s also worth noting that other assets will still be just as usable on the network as they are today.

Thanks,

Jon

[Tomer Weller]

> Could an alternative be to select the assets based on the quantity of trades, selecting asset pairs with more trades since they theoretically require greater scale, and deferring transactions that trade assets not demanding scale to the existing non-SPEEDEX trading mechanism?

Leigh, is the quantity of trades enough to make these decisions? it sounds like you would also need to factor in their value, which makes it pretty hairy. 

Like Leigh, I would love to learn about other options to derive the asset list from the txset.

With that said, if we can't figure out anything that's not gameable, I support the CAP in its current form. My reasoning: 

1. The validators are the de-facto governing body of the Stellar protocol and deserve a say in what transactions get processed. This is especially true given that validators are not collecting transaction fees and are dedicating their compute power to execute these transactions. 

2. We talk a lot about how SCP gives reputable validators an opportunity to make informed decisions on what transactions to include. However, validators don't actually have tools to make these decisions. This is a good start (and I support more to come).

3. It's fairly common in the defi space to give decentralized governing bodies (DAO) the power to list assets in various protocols. This pattern is well understood and will shine a light on SCP's reputation based voting vs. the usual stake-weighted voting in other protocols.   

To view this discussion on the web visit https://groups.google.com/d/msgid/stellar-dev/CAG4FKhy45Enak%3DCdTMa2PLdZg20L3yeXN4H7OmGjhtVp1H%2BMHQ%40mail.gmail.com.

[Jonathan Jove]

I did some research to evaluate the importance of including assets not mentioned in the transaction set. As a particular fictional case study, I considered the following tractable scenario:

- A single offer to sell GBP for JPY
- Liquidity pools with liquidity in proportion to FOREX market share
    - (GBP,JPY) -> 2.78%
    - (GBP,EUR) -> 3.57%
    - (USD,JPY) -> 27.95%
    - (USD,EUR) -> 13.34%

If only assets mentioned in the transaction set are included, then the only liquidity pool used is (GBP,JPY). Note that it’s not possible to use (GBP,EUR) or (USD,JPY) because there is no source/sink for the traded EUR and USD.

If assets not mentioned in the transaction set are included, then all four liquidity pools can be used. In this case, slippage for a small trade is 37% lower than for the previous case which only uses (GBP,JPY).

The above supports my claim that assets not mentioned in the transaction set can significantly improve the quality of prices. Leigh and Tomer have asked about the possibility of deriving the full set of assets. I have done some more thinking and research, and at this point I’m not aware of any algorithm or even approximation algorithm that can efficiently solve the problem. I'm very hesitant to use market characteristics from previous ledgers like trading volume because they are gameable and transient. For example, if a huge market displacement occurred in the previous ledger then there is no reason to think that the same volume level will persist in the next ledger. Do either of you have any proposals for how one might derive the full set of assets (ie. including assets not mentioned in the transaction set) without depending on those kinds of market characteristics?

[Jonathan Jove]

Sorry, slight correction. I flipped the market shares of (GBP,JPY) and (GBP,EUR) when I sent the previous email. They were correct in my calculations, so the results should still be accurate.

[Nicolas Barry]

I have been thinking some more about various approaches to this problem of asset selection for SPEEDEX.

While there might be "on network" schemes that make it possible to govern that set in (possibly) a better way, I think that those solutions will come with other problems: more complexity, less flexibility and potentially worse security.

Like with any other protocol change, the first question I ask before embarking on more complicated endeavors is "can we do this later?" and follow the 80/20 rule.

In this case, I think the answer is a resounding yes: if we can start with something like what is described in the CAP, we reduce on-chain complexity while delivering value to users.

In that context, the lens we should be using for this CAP is: what is missing from it that would address the big concerns that people (me included) have.

Here are a few of the concerns I have and possible ways they could be addressed.

Q: How can we expect validators to decide which asset to include in the set?

A: The CAP should have a first version of the offchain agreement that validators will follow to govern that set.

The document will describe: how often the set of assets is decided (can start with something relatively slow like once a month for example), which "cost function" is used when assessing an asset (can start with "total fees" spent trading over the last period for example), etc.

Q: How can users of the network know which assets are going to be usable in SPEEDEX for the upcoming ledger?

A: This may point to a gap in the existing CAP.

Upgrades are scheduled by validators, but there is no guarantee on when a single upgrade will be ratified by the network, and as a consequence it's very hard to tell which combination of upgrades will be activated and in which sequence.

To solve this we could:

* change how upgrades are done on the network so that validators can decide in advance which upgrades will be applied for future ledgers (instead of applying upgrades as soon as a quorum votes for one). In this case, validators could agree to queue up even a large number of upgrades (1000 assets changing) for a future ledger and systems like Horizon can expose to clients those future upgrades (and specifically in the case of SPEEDEX, can expose the upcoming set with information on when that set will become active). I would recommend making this a separate CAP.

* improve the visibility on intent of upgrades from validators, so that Horizon can report on it. This would help with validator coordination, but would not help for clients.

Q: Should there be a way for some validators to "opt-out" of SPEEDEX governance?

A: Right now core implements what is described as "governing validators" in section 5.3 of Fast and secure global payments with Stellar (stanford.edu), meaning that for a validator to vote for an upgrade, it must be configured to vote for that exact upgrade. For some organizations this may not be desirable.

Instead some organizations may want to act in a "non governing" way, delegating actual governance to other (or any as described in the paper) organizations.

Other questions related to that CAP.

Should there be an explicit "maximum number of SPEEDEX assets" somewhere? Right now the CAP allows nodes to vote to add any number of assets in the set, and I *think* that this can result in a network to allow adding more assets than what any individual node would be configured for.

SpeedexConfigurationEntry

Need to expand on why it should be a ledger entry: until now, ledger wide settings are stored directly in the ledger header.

I think it's a combination of size and that it rarely changes, so storing it in the ledgerheader would be maybe wasteful.

If we think of this as a "ledger header extension", the ledger entry type that we're introducing should be aligned with that idea, and `SpeedexConfiguration` is just the first of its kind.

Nicolas

To view this discussion on the web visit https://groups.google.com/d/msgid/stellar-dev/CAG4FKhyTajPDpHFih1YbTujL0QmLpCb25f9AaX2zEkbXdX9huw%40mail.gmail.com.

[Jonathan Jove]

Interesting questions, as always. 

> Q: How can we expect validators to decide which asset to include in the set?

I think the answer to this question belongs in a SEP rather than a CAP, because the validators are in practice free to disregard the guidelines. With that said, let's focus on what those guidelines might actually be.

First, those guidelines probably should not be purely quantitative. There may be certain assets validators want to include as their presence may stimulate growth, stability, or development of the network. Perhaps there should be some small number of discretionary slots.

Second, what should the quantitative guidelines be. This is extremely tricky, because these may be both "gameable and transient" as noted in my previous correspondence. We can consider your proposal "total fees spent trading in the previous month" as an example. Right now, most operations execute at base fee. The highest fee bids we routinely see are about 100,000,000 stroops (10 lumens) which is 1,000,000 times the base fee. So a few trades executing at that fee bid would probably outweigh all other trading in the month. In other words, this proposal would mean that becoming a SPEEDEX asset could be bought for a few dollars or tens of dollars. This is not desirable. I'm very open to more proposals on quantitative guidelines, but I emphasize again that finding an approach that is hard to manipulate will likely be difficult.

> Q: How can users of the network know which assets are going to be usable in SPEEDEX for the upcoming ledger?

For the _next_ ledger, the SPEEDEX assets are exactly those SPEEDEX assets after upgrades were applied in the previous ledger. This is the case because upgrades are applied after transactions.

For ledgers beyond the next ledger, the SPEEDEX assets are approximately the current SPEEDEX assets. The maximum number of errors is linearly proportional to the number of ledgers in the future because the validators can only add and remove a single asset per ledger (multiple upgrades of the same type are invalid).

So Horizon can provide information about the present, and that information should be very helpful for the near future. The distant future, however, is almost impossible to predict.

The proposal you listed here about queuing up multiple upgrades is interesting. I could imagine this being useful in some scenarios, but it doesn't obviously solve any new problems with this proposal unless the validators actually engage in an externally governed, periodic update process. If validators are updated haphazardly, then I don't think this will help much. Regardless, it would be good for someone to write up a separate proposal about how this could work.

> Q: Should there be a way for some validators to "opt-out" of SPEEDEX governance?

This is an interesting question. In this context, "non-governing" basically means vote for any operations on the list of SPEEDEX assets that other validators vote for. This could be useful, although I'm not totally certain why a validator would choose to operate in this mode rather than the governing mode then just abstain. Do we have any evidence that validators on the network actually _want_ the non-governing behavior? If so, we should implement it. If not, we should hold off until it is desired.

> Should there be an explicit "maximum number of SPEEDEX assets" somewhere? Right now the CAP allows nodes to vote to add any number of assets in the set, and I *think* that this can result in a network to allow adding more assets than what any individual node would be configured for.

There could be such a limit, but I don't see much motivation for doing so. Given our lack of knowledge about future usage patterns, it is unlikely that we will set it just right. If we set the limit too low, then the network will have unused capacity until the next protocol upgrade. If we set the limit too high, then the limit will have no effect.

My general opinion is that the validators are responsible for the configuration of the network, and part of that responsibility is making sure that the network can handle the load permitted by that configuration. For example, the validators could just as well vote to raise the maximum transaction set size beyond what the network could handle but they do not. Given that history has shown the validators to be responsible, I'm not concerned that they will err here.

> SpeedexConfigurationEntry
> Need to expand on why it should be a ledger entry: until now, ledger wide settings are stored directly in the ledger header.

The two reasons I chose to use a ledger entry instead of the ledger header:
1. It's more time efficient in stellar-core: the list of assets will only be copied when we are specifically accessing it through LedgerTxn instead of in _every_ LedgerTxn; the list of assets will only have to be written to disk (buckets and SQL) when it changes instead of _every_ ledger because the ledger header must always be written.
2. It's more space efficient in the history archive: the list of assets will appear in the buckets once per checkpoint instead of in the ledger header once per ledger.

I think these reasons are relatively good, and they become more compelling the larger the list of SPEEDEX assets becomes. But if people generally prefer moving it into the ledger header then I'm not strongly opposed.

----
I will update the proposal to address some of these questions.

Thanks,

Jon

[Nicolas Barry]

cool.

see inline

On Tue, Mar 1, 2022 at 8:55 AM Jonathan Jove <j...@stellar.org> wrote:

Interesting questions, as always. 

> Q: How can we expect validators to decide which asset to include in the set?

I think the answer to this question belongs in a SEP rather than a CAP, because the validators are in practice free to disregard the guidelines. With that said, let's focus on what those guidelines might actually be.

First, those guidelines probably should not be purely quantitative. There may be certain assets validators want to include as their presence may stimulate growth, stability, or development of the network. Perhaps there should be some small number of discretionary slots.

Second, what should the quantitative guidelines be. This is extremely tricky, because these may be both "gameable and transient" as noted in my previous correspondence. We can consider your proposal "total fees spent trading in the previous month" as an example. Right now, most operations execute at base fee. The highest fee bids we routinely see are about 100,000,000 stroops (10 lumens) which is 1,000,000 times the base fee. So a few trades executing at that fee bid would probably outweigh all other trading in the month. In other words, this proposal would mean that becoming a SPEEDEX asset could be bought for a few dollars or tens of dollars. This is not desirable. I'm very open to more proposals on quantitative guidelines, but I emphasize again that finding an approach that is hard to manipulate will likely be difficult.

Yes I agree, a SEP is probably the best place for this. I guess in the context of this CAP, we should make sure that the way validators vote allows for "fine grain" tuning. I think that the CAP already does this by virtue of exposing add/remove of individual assets.

 

> Q: How can users of the network know which assets are going to be usable in SPEEDEX for the upcoming ledger?

For the _next_ ledger, the SPEEDEX assets are exactly those SPEEDEX assets after upgrades were applied in the previous ledger. This is the case because upgrades are applied after transactions.

For ledgers beyond the next ledger, the SPEEDEX assets are approximately the current SPEEDEX assets. The maximum number of errors is linearly proportional to the number of ledgers in the future because the validators can only add and remove a single asset per ledger (multiple upgrades of the same type are invalid).

So Horizon can provide information about the present, and that information should be very helpful for the near future. The distant future, however, is almost impossible to predict.

The proposal you listed here about queuing up multiple upgrades is interesting. I could imagine this being useful in some scenarios, but it doesn't obviously solve any new problems with this proposal unless the validators actually engage in an externally governed, periodic update process. If validators are updated haphazardly, then I don't think this will help much. Regardless, it would be good for someone to write up a separate proposal about how this could work.

I guess the solution is sound if we don't expect more than 1 change (where 1 change=add, remove or replace an asset) once in a while, otherwise the current implementation/protocol may not be good enough as the way network upgrades are done is by scheduling them on each validator and there is no way to:

* have conditions on upgrades (like: only vote to add asset X if Y was already added)

* schedule upgrades for the same type (ie: can't schedule 2 `LEDGER_UPGRADE_ADD_SPEEDEX_ASSET`).

There is also the problem of lack of transparency of future upgrades: the reason I mentioned a solution where we'd prequeue upgrades is that it allows to separate validators voting and reaching consensus on changes from when they actually take effect (atomically).

> Q: Should there be a way for some validators to "opt-out" of SPEEDEX governance?

This is an interesting question. In this context, "non-governing" basically means vote for any operations on the list of SPEEDEX assets that other validators vote for. This could be useful, although I'm not totally certain why a validator would choose to operate in this mode rather than the governing mode then just abstain. Do we have any evidence that validators on the network actually _want_ the non-governing behavior? If so, we should implement it. If not, we should hold off until it is desired.

I think the issue by only having "abstain" as a mechanism, is that a quorum of "governing nodes" is required to make those changes. This might be a good question for the validator community.

 

> Should there be an explicit "maximum number of SPEEDEX assets" somewhere? Right now the CAP allows nodes to vote to add any number of assets in the set, and I *think* that this can result in a network to allow adding more assets than what any individual node would be configured for.

There could be such a limit, but I don't see much motivation for doing so. Given our lack of knowledge about future usage patterns, it is unlikely that we will set it just right. If we set the limit too low, then the network will have unused capacity until the next protocol upgrade. If we set the limit too high, then the limit will have no effect.

My general opinion is that the validators are responsible for the configuration of the network, and part of that responsibility is making sure that the network can handle the load permitted by that configuration. For example, the validators could just as well vote to raise the maximum transaction set size beyond what the network could handle but they do not. Given that history has shown the validators to be responsible, I'm not concerned that they will err here.

What I am asking about here is to add a network wide parameter that is the maximum number of assets (that can be updated via consensus) that the network can handle, otherwise the effective set of active assets can be larger than what individual validators vote for.

Example: 2 out of 3, validators want respectively {A, B}, {B, C}, {A,C} (so validator 1 will vote to add "A" and "B", etc), the network will reach consensus to add {A, B, C}.

 

[Leigh McCulloch]

What's the consequence of an account holder submitting a transaction that includes an operation that will intend to trade via SPEEDEX, but the asset is not enabled for SPEEDEX? Will the trade fallback to the existing DEX? Will users explicitly choose to trade via SPEEDEX or will that be abstracted and hidden? I realize answering this question is out of scope of this CAP itself, but answering it may be helpful to providing more context.

[Jonathan Jove]

A transaction set is invalid if any "create SPEEDEX offer" operation uses a non-SPEEDEX asset. Users must explicitly choose to trade via SPEEDEX, and there is no fallback to the existing DEX.