2 flaws in the AW5 kernel design

[William la Forge]

I've completed much of the code now for wiring the components together, and have just uncovered two design flaws--a minor setback.

#1 Managing the address maps.

Each database (except the first) currently has an address map which fell partly in the kernel and partly outside of it, as it used handles to reference its inodes and to reference the non-kernel blocks but its changes were tracked as part of the transaction context. This doesn't work well, as once one of its dirty inodes is written to disk (which happens when the number of dirty blocks becomes excessive), an entire address map effectively becomes write locked for a single transaction, and this creates a real bottleneck.

The solution is not very difficult. We move the address maps entirely within the kernel and track changes (additions only) within the transaction context.

#2 A problem with the transaction context

The transaction context started out simple enough. There was one transaction context per transaction and it was responsible for tracking all database changes to non-kernel blocks, including disk space allocations and dirty blocks, as it needs to be able to unwind failed transactions and reprocess them when a collision had occurred. The transaction context was implemented by the kernel context element and the first database of the kernel was dedicated to this. But the problem is that the tracking really needs to be done on a per database basis.

The solution requires an architectural change. We need a single transaction context, which is accessible via a thread variable, which then references a kernel context element for each database, the latter being created as needed. We no longer need to have a database dedicated to transaction context and we also eliminate it as a potential bottleneck--we were only able to create/rewind/abort/commit one transaction at a time.

In conclusion then, the design changes are relatively minor though much of the code base will need to be reviewed. But the new design will, in addition to eliminating the flaws, also eliminate some bottlenecks.

Bill

[William la Forge]

The second design flaw is about fixed. Unit tests run and the stats are OK. Only a bit of additional wiring is needed which can be addressed with the fix for the first design flaw, which requires a minor change.

I've introduced a new class, TransactionContexts, which is now passed as the argument to transaction methods. This new class tracks the KernelContextElement's created in each database while the transaction is processed and manages them. Meanwhile KernelContextElement still extends the TransactionContext trait. Doing it this way minimized the number of changes and the result is pretty clean.

On the plus side, there is no longer any bottleneck for creating transactions. Indeed, a transaction which doesn't touch any of the databases uses no locks at all. On the minus side, we will now create a TransactionContextElement in every database touched by a transaction. No choice really.

Bill