performance overhead of transactions

[James Taylor]

We've done an initial pass to determine the overhead of transactions for write-once/append-only data through Tephra in Phoenix - see https://issues.apache.org/jira/browse/PHOENIX-1901 for details. Looks like about a 20% perf hit across the board for our concurrent query workloads. The sets of queries between the dashed lines here[1] are run in parallel. The bottom set (prefixed with "Serial") are run serially and these look to be more similar perf-wise with and without transactions. I'm wondering if perhaps we're seeing a bottleneck in the transaction manager. Two questions:

1) Do the canCommit and commit RPCs still happen if the change set is empty? This would be the case in Phoenix connection that is doing only querying.

2) What kind of tuning is available on the Transaction Manager? For example, how big is the thread pool for obtaining transaction ids and can we make it bigger through a config param?

Thanks,

James

[1] http://phoenix-bin.github.io/client/publish/pherf-txn.html

[Poorna Chandra]

(Moving the discussion to tephra-user since tephra-dev is currently moderated)

Hi James,

Thanks for sharing the performance numbers.

1) Do the canCommit and commit RPCs still happen if the change set is empty? This would be the case in Phoenix connection that is doing only querying.

Yes, today canCommit and commit RPCs still happen if the change set is empty. We should be able to reduce the transaction RPC overhead for transactions with empty change sets. Can you please file a JIRA for this?

 

2) What kind of tuning is available on the Transaction Manager? For example, how big is the thread pool for obtaining transaction ids and can we make it bigger through a config param?

There are two config parameters to control thread pool size in Transaction Manager -

1. data.tx.server.threads - specifies the number of threads of transaction manager. Default value is 20.
   
2. data.tx.server.io.threads - specifies the number of IO threads for the thrift server. Default value is 2.
   

How many concurrent transaction RPCs happen in the test? Depending on that we need to increase thread pool size.

The other setting that might need increasing is the heap memory size allocated to Transaction Manager. The default heap size might be too low.

Thanks,

Poorna.

--
You received this message because you are subscribed to the Google Groups "Tephra Developer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tephra-dev+...@googlegroups.com.
To post to this group, send email to tephr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tephra-dev/CAAF1Jdi6G55UHP2rKZUefFtTpt7FV%3DSrKxRM9R0K3a9MgEi2sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.