Limiting transaction complexity

[indolering]

I'm trying to write a postmortem for the recent aggregation issues.  However, there is some disagreement as to whether we should implement temporary blacklists and limits on transaction sizes.

I believe the problem boils down to Ryan and Jeremy thinking we should implement those two items while Domob and Phelix disagree.

This puts me in the weird position of saying "Some people think we need this but they aren't up for coding it so I can't give an ETA."

Domob, are you up for implementing Ryan's proposed changes? Or, alternatively, is anyone else up for implementing them?

[Daniel Kraft]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi!

On 2014-09-06 20:35, indolering wrote:
> I believe the problem boils down to Ryan and Jeremy thinking we
> should implement those two items while Domob and Phelix disagree.
>
> This puts me in the weird position of saying "Some people think we
> need this but they aren't up for coding it so I can't give an
> ETA."
>
> Domob, are you up for implementing Ryan's proposed changes? Or,
> alternatively, is anyone else up for implementing them?

I'm not entirely against implementing some more limits on tx size --
and I think also phelix was actually in favour of limiting the number
of inputs for a transaction (while biolizard at first wasn't sure
about it in the BM discussion).

But I believe that the point should be to find agreement among the
community first, and not implementing something while others disagree.
No? If there is some reasonable limit proposed that has no practical
implications that could be interpreted as censorship, I'm definitely
ready to implement them.

Yours,
Daniel

- --
http://www.domob.eu/
OpenPGP: 901C 5216 0537 1D2A F071 5A0E 4D94 6EED 04F7 CF52
Namecoin: id/domob -> https://nameid.org/?name=domob
- --
Done: Arc-Bar-Cav-Hea-Kni-Ran-Rog-Sam-Tou-Val-Wiz
To go: Mon-Pri
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Icedove - http://www.enigmail.net/

iQJ8BAEBCgBmBQJUDCRXXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ5MDFDNTIxNjA1MzcxRDJBRjA3MTVBMEU0
RDk0NkVFRDA0RjdDRjUyAAoJEE2Ubu0E989SifwQALe+euvfZ+C9J4rkMJiVVlh5
FFePOYkOUIpMRaqYSavaJpRoV6Z656j+1UV479l+4uNzfUvfhBehibhvj+UJwrcx
n4jMCsZTE15xJHA4A5THD6gVvscPARpBB4NvxXJJGWWeZ+kGj9CCJ6TarR4bXfz8
76umtPQU1mCYxBEoOUPBqHI0twaLNiHHSyUyeyD1Me4BOKqgrumrL9ioPFPwGK5t
K57xKdQDs0DTVDtUD2myf6+f07q3U+6zygadVaZhi2pxUpKrxCXErRvyX+c+DzPb
g7FuKABp0dwmHyQZGnAzB6/Yc8dwl8YnJ21VyVlror7EOzud45Uclnmg73LPAaVI
7d72c0eQOSSo5DFkAZy4hEoVgz6pX9/ZTuovEOduoQiUmozGjvrDkEAOt7KzvdzZ
WJe/sXLIFfzSywLazgDjTQOxCEBS6CwtQcD5FLED7cf0A8Py+VMfp6fBOnm7hmor
jf0lYeL43SNCevnC+FqCfmNfEKnAvYwJW+9vX0q6SfupDZKqnbzCJTJbz9Y1GhKG
K8CIJ3WzBVSPiDVnljEK1/4bbpB5M7nGDQeBW2ve08kHTHUv+zHCqbj3VquNk+yF
T/j8uAvEfu6YFuYxBtGBHIkVghY9GJy1Wx9rVMVt+cznzipcvCiR9qiZ01QdzZTh
WqsKaHLnOVKlLoNBtbVE
=YGA8
-----END PGP SIGNATURE-----

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/07/2014 04:24 AM, Daniel Kraft wrote:
> Hi!

>
>
> I'm not entirely against implementing some more limits on tx size
> -- and I think also phelix was actually in favour of limiting the
> number of inputs for a transaction (while biolizard at first wasn't
> sure about it in the BM discussion).
>
> But I believe that the point should be to find agreement among the
> community first, and not implementing something while others
> disagree. No? If there is some reasonable limit proposed that has
> no practical implications that could be interpreted as censorship,
> I'm definitely ready to implement them.
>
> Yours, Daniel
>
>

My initial take on this (and keep in mind that I don't work on the
core client code) is this:

Any solution which we adopt should scale properly to both high and low
network load. This means that if the network load is low enough that
it's not harmful, large transactions should be allowed. If the
network load is extremely high, then costly transactions (including
large transactions) should be disallowed as needed to keep the network
healthy (and no more than needed).

Imposing an arbitrary limit on input count or transaction size would
run counter to this principle -- it will unnecessarily censor innocent
transactions, and it won't prevent a large-scale attack.

Some people have asked what legitimate use large transactions have in
Namecoin, given that we're not primarily a currency. This ignores the
fact that the naming system only works if the currency works. Put
differently, if people have efficiency issues transferring NMC between
each other, or exchanging NMC for other currencies, then the naming
system is made less useful. And those use cases require Namecoin to
be usable as a currency, even though that's a means to an end (the end
being naming) rather than an end itself.

Quite simply, large transactions are not inherently harmful. And they
are probably beneficial to some users. We should only be imposing
restrictions on transactions to the extent necessary to prevent harm
to the network.

So -- here's what I think would be a good idea. This is loosely based
on some of Ryan's proposals. If some of this is stupid, please tell me.

- -----

1. Modify the tx priority calculation as follows (numbers are made up,
feel free to debate those):

A. Transactions which have no name_new, name_firstupdate, or
name_update outputs, or which have at least 1 non-change currency
output, will be penalized 80% of their priority. (So final priority
is 0.2 * initial priority.)

B. Transactions which have at least 1 name_update output and have
exactly 0 currency outputs will be boosted in priority by between 400%
and 900% (so final priority is between 5 * initial priority and 10 *
initial priority.) The exact priority boost is chosen by a linear map
between time to expiration (36000 to 0; chosen as the minimum if there
are multiple name_update outputs) and the priority boost (400% to 900%).

The above changes will prioritize usage of the naming system over
currency applications, will prioritize existing names over new names,
and will prioritize infrequent updates over frequent updates. I think
these priorities are uncontroversial, although the numbers may need
tweaking.

2. getauxblock and getmemorypool RPC's should have a configurable
timeout. If this timeout is triggered, then the 10% of transactions
in the memory pool with the lowest priority are automatically dropped,
and their txid's blacklisted from entering the memory pool for 6 blocks.

This will only drop transactions from mining clients, since relaying
clients don't use getauxblock and getmemorypool. Miners with more
resources will drop transactions less often. 6 blocks isn't long to
wait for end users who are affected, and if that delay is
unacceptable, an end user can simply increase the tx fee and rebroadcast.

3. Bitcoin Core is planning to have a new tx fee calculation system,
which correlates priorities of real-world transactions with how long
they take to be mined. This is much more decentralized than
developers choosing the fees. We should absolutely switch to this
method once Bitcoin Core has done so. (Rebasing is probably a
prerequisite to this.) By doing this, we won't have to worry about
the implications of changing tx fees, since it will automatically
adjust if miners change them.

- -----

I am against arbitrary limits on tx size and input count, unless a
compelling case can be made on why they're necessary to prevent
attacks that the above proposals cannot prevent.

- -Jeremy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJUDO12AAoJEFgMI9bDV/9qMwkH/1YxxCuG+NXlvdImfHO0vslH
tHBBnihgek5KobTOpxyLfwxHyS7nJBKePl2vvA6485kWpxD9aiHyB8EQ198C8k5C
8Fe25o+QXinZ+aLRY8TFEbz3PDOr/rdelWw8c8U+k6jsvvTe818DwMkdALUVoiMw
tN92vX9c6yJG1PkDmBGiATVdNkdKtUAdAeomUcvR+SvGWX+6fv2ybj2eT8YP3Z9o
y5hEstQh/xahHxst9JK8MNISCkLm8lQe5Fyue6KleuEx35saMvsCegyjRLdRqLtB
tzXpgPFwzSxcr1/TIkz2+nnHbHYCoPwtsmBeuyAbcQwgihpE2D7i1zgfFuVR6T0=
=A7q2
-----END PGP SIGNATURE-----

[Ryan Castellucci]

I like what Jeremy proposed, but a few comments:

> A. Transactions which have no name_new, name_firstupdate, or
name_update outputs, or which have at least 1 non-change currency
output, will be penalized 80% of their priority. (So final priority
is 0.2 * initial priority.)

Change outputs are just outputs, they aren't special. Did you mean "have more than one currency output"?

> --
> You received this message because you are subscribed to the Google Groups "Namecoin Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to namecoin+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Ryan Castellucci http://ryanc.org/

[Ryan Castellucci]

domob appears to have tried to reply but got caught in spam limbo. I
tried to approve it but I'm not sure it worked.

[phelix]

Hi!

Transaction complexity is inherently limited by the new maximum tx size of 20kb.

Mempool size limiting and priorization would be nice but I don't  think it's urgent so I'd rather concentrate on other things for now. IMHO it's relatively unlikely that anybody will be able to pull something off much worse than the Aggregator.

I am aware of at least one issue that could cause trouble that I would like us to fix for 0.3.77 - before it will bubble up, saving us time and nerves.

phelix