Invoice numbering and eventual consistency

[Thomas Vantroyen]

Hi,

In Belgium, invoices need to be sequentially numbered without gaps or duplicates per year: eg. 2015/1, 2015/2, 2015/3, 2015/4, ...

I've read a lot on numbering like this using CQRS/ES, but often the solution points in the direction of "consistency is overrated", but, in this case it isn't.

If I implement this, I end up with a domain service that will be responsible for assigning the next, unique number to an Invoice.  How will this domain service provide a consistent numbering?

It could do Invoice.SetNumber(x) which creates the InvoiceNumbered event on the Invoice AR which is picked up by an InvoiceNumberedHandler updating a LatestInvoiceNumberReadModel, which, in turn, can be used as input for the NumberingService in the first place.

But, as EventHandlers run separately, the latest invoice number on the ReadModel could be "not yet consistent" when another invoice needs to be numbered.  Or: storing the InvoiceNumbered event and updating the readmodel does not happen in one transaction, it's eventually consistent.  This can cause double use of the same invoice number.

How would you solve this? (please be free to do it entirely different :) )

thank you

Thomas

[Freek Paans]

Can't you have the service maintain the latest invoice number in a db and then enlist it into the transaction?

-Freek

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Vantroyen]

Yes, I considered that too.  But I was wondering if there's a more in-the-box solution.  But maybe having a sort of "sequence" provider injected is not too bad.

regards

[Alexander Langer]

> In Belgium, invoices need to be sequentially numbered without gaps
> or duplicates per year: eg. 2015/1, 2015/2, 2015/3, 2015/4, ...

> How would you solve this? (please be free to do it entirely
> different :) )

Like this:

One InvoiceNumbers AR per year
Its sole purpose is to maintain a gap-free sequence
of invoice numbers for each year.
Its only state is its year and the current invoice number.

A "create invoice" command sent to InvoiceNumbers will yield
a InvoiceNumberAssigned(InvoiceNo, UUID) event,
where InvoiceNo is the next number in the sequence
and UUID is a randomly generated UUID used for the to
be created invoice.
Transaction #1 finishes here, one AR modified.

Saga listens to InvoiceNumberAssigned events. For each event,
the saga sends a CreateInvoice(UUID, InvoiceNo) command.
Saga state change and submission of the command to the
command bus is handled in one transaction.

The handler for CreateInvoice creates the actual invoice in a separate
transaction, again one AR modified.

The invoice created is still empty and needs to be filled with items by
the user, but it already has an a UUID by which it can be retrieved from
the DB and the unique ID.

If for some reason the saga or the command handler crashes or the
transactions do not commit successfully, the cause can be fixed and the
invoice will be created as planned with the correct sequence number when
the system comes back (or the saga is restarted/retried).

As with usual accounting software, invoices can never be deleted, only
marked "invalid" (or similar), i.e., via replaying InvoiceNumberAssigned
events to a suitable projection, you can create a 1:1 mapping between
"public" invoice numbers and internal UUIDs.

Best,
Alexander

[Greg Young]

ReadModel->GetNextInvoiceNumber

Command CreateInvoice (InvoiceNumber)

Cheers,

Greg

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Studying for the Turing test

[Greg Young]

too complicated.

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Greg Young]

a question on a business level btw. What happens when on Dec 28th I
issue 5 invoices 133-137

Dec 29th invoice 135 gets cancelled or reissued as 138.

Cheers,

Greg

[Alberto Brandolini]

Hi all, 

one more thing that might help (or confuse :-) )

When talking about invoice numbering, looks like the problem is always the same, while it isn't necessarily always exactly the same.

If you have automatic invoice generation you might ask the sequence number right on invoice creation. Easy stuff. And many developers tend to get the requirement too strictly, invoices in the same day could be in 'sparse order' even if ons is coming from the morning and one in the afternoon.

If the invoice stays in draft state for a while - with people arguing whether to include given hours in the invoice (like in consulting) - then it might be a good idea to separate invoice generation & discussion (without numbering), from getting the number. One gets the number when the invoice is ready, at the last responsible moment. Acquiring the number might go with a similar strategy, but just ...delayed. You might spot the echoes of 'Draft Model' here.

The third scenario might be when entering/archiving invoices from a past year, the sequence constraint might be temporarily broken if entering invoices in wrong order. Which might be totally reasonable if you're a small company manually collecting invoices (and you can't find them in your messy office). In this case the sequence invariant might serve as a warning "Invoice 8 is still missing" but not as a blocker. An incomplete sequence might be reality, whether you like it or not. (It can be even messier, allowing invoice 1, 2, 3, 3/bis, 4 ...)

HTH (or the opposite)

Alberto

> email to dddcqrs+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

[Thomas Vantroyen]

Hi,

@Greg

Invoices are not allowed to be cancelled.

When you make an invoicing mistake and want to "remove" the invoice, a second new Invoice needs to be created with a negative balance to right your wrong.  Literally translated we call this special type of invoice a "credit note", but I don't know if that's the right word in English.

Because of this inflexibility, most invoicing software use a draft-state for invoices and only apply a number in a later state of the scenario.  But that's off-topic :)

bye

[Greg Young]

Then the mechanism I gave should work just fine.

On Wed, Jan 14, 2015 at 4:25 PM, Thomas Vantroyen

[João Bragança]

We call it a "credit memo."

Maybe you need multiple classes here. Quote -> Sales Order -> Invoice, where an Invoice is basically immutable.

Sent from my regular computer

http://twitter.com/thefringeninja

http://www.thefringeninja.com/

[Thomas Vantroyen]

Hi,

@Alberto:

Paragraph 1: agreed

Paragraph 2: i just wrote the draft-state thing to Greg, didn't read your post yet. Sorry :)

Paragraph 3: you would only use the sequencer for new invoices.  Legacy invoices already have their number, and the domain should respect the existing number, never use the NextNumber service.  Just by using a separate Command for old invoices.

> email to dddcqrs+u...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.

[Alberto Brandolini]

Yep, 

I wasn't inquiring, just clarifying that we might have three different problems, with a similar rough description. And, you know, DDD it's all about being pedantically precise. :o)  In case 3 we ended up having a different aggregate, called InvoiceStrip, representing the state of the past sequence as far as we knew it.

Cheers

[Thomas Vantroyen]

Hey Greg,

Actually my question is a problem with the approach you suggest:  

ReadModel.GetNextInvoiceNumber()

NewInvoiceCommand(number)

.. can go wrong as the ReadModel isn't always consistent in race conditions.  The EventHandler updating the ReadModel is not executing in the same transaction as the NewInvoiceCommandHandler.  So, the number in the ReadModel could be behind reality a little, causing double numbers.

Any ideas except using a consistent SequenceService?

tnx

[Greg Young]

Yes use it in the key eg invoice-0005 if one already exists you get a concurrency exception... retry with a get of invoice number. This should happen less than 1% of the time

[Tom Janssens]

I'm probably alone on this, but I'd opt for simplicity, use a guid and and assign the sequence nr on the viewmodel...

Op woensdag 14 januari 2015 11:26:27 UTC+1 schreef Thomas Vantroyen:

[Freek Paans]

Good one. When are we using which solution?

-Freek

[Thomas Vantroyen]

@Tom

I also thought about this, I don't immediately see a good case scenario where it would fail.  But... this could get a lot more messy when you're having bugs than it would when the number is explicitly assigned to the AR.  The number is a sequence, but once assigned, it should become core data of the invoice.  imho :)

[Greg Young]

What happens on a projection replay?

--

You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Greg Young]

What happens on adding a new projection say a graphdb where does it get the invoice number from?

[Peter Hageus]

What about the opposite case, the number is generated but the NewInvoiceCommand fails? (gapless is a hard requirement in this context)

 

/Peter

[Greg Young]

In my example that's no problem the next invoice will use that number. The only place it would be an issue is the last invoice of the year. That can be mitigated by not letting it fail

[Greg Young]

Btw by not failing I mean for business reasons .. It can obviously fail for technical reasons (out of disk space)

[Alexander Langer]

I'm having slight problems to understand how to recover from that situation:

Thread A: ReadModel.getNextInvoiceNumber() -> 1234
Thread B: ReadModel.getNextInvoiceNumber() -> 1235
Thread A: transactin fail
Thread B: transaction commit.

-> gap between 1233 and 1235

[Greg Young]

See rest of reply.

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Philippe Rutten]

One could indeed create the invoice with a uuid and assign the sequence number afterwards, however I don't think you would do it in a 'mere' viewmodel since your domain is probably interested--not necessarily what the number is, but at the very least the fact that it was assigned.

Instead, one can run a saga that always knows what the last-assigned number is, that knows what the sequencing rules are for generating a new number, and then assigns it via a command back to the domain.

Now one can argue whether this command should go to the InvoiceAR or to a separate SequenceNrAR or something like that, after all the fact that a sequence number is attributed to an invoice doesn't necessarily make it a property of that invoice--especially when we have dependencies on external factors such as what year we are in and how many invoices have been numbered previously. The latter of course further supports the case for a saga.

[Philippe Rutten]

...this mechanism should also cover Greg's point on projection replay.

[Tom Janssens]

I'll post on my own comment to merge the questions; please correct me if I'm missing a few concepts here.

In the end a sequential number is just a "label"; while the constraint of having unique numbers is a domain constraint, it's only the ordering that's important. The numbers can be deferred from the ordering and some initial value.

You create a persisted stream per invoiceperiod that subscribes to InvoicePeriodStarted(InvoicePerdiodId,FirstInvoiceNumber) and InvoiceFinalized(InvoicePeriodId, InvoiceId). As a stream is persisted the order is guaranteed to be the same.

a) What happens on projection replay?

Create a projection per invoiceperiod that contains an ordered list of InvoiceId's for that period.

If the list doesn't contain the InvoiceId, append it.

InvoiceNr = List.IndexOf(invoiceList, InvoiceId, default: invoiceList.length) + firstInvoiceNumber

This implies that invoicenumbers are either TBD or known. As long as they are TBD, you shouldn't send/print them.

b) What happens on adding a new projection say a graphdb where does it get the invoice number from?

From the projection mentioned in point a; after all, the number is only a graphical representation, what's important is that the order is guaranteed.

Op woensdag 14 januari 2015 22:18:55 UTC+1 schreef Tom Janssens:

[Greg Young]

Why not just the sequence number of the stream?

--

[Tom Janssens]

Op donderdag 15 januari 2015 14:19:32 UTC+1 schreef Greg Young:

Why not just the sequence number of the stream?

This might work on 2 conditions:

- The persistent stream is idempotent (I'd assume it is)

- You can easily perform a search for the invoice number based on the invoiceId (I'd assume some kind of an index table would be necessary here)

[Alexander Langer]

Thanks.

Given the official rules for invoicing in many European countries, I
have to admit I actually took the gap-less requirement for invoice
numbers in the accounting domain literally.

I'm still not sure if it is allowed to have gaps sitting there for a
random time span (hours, days, ...), i.e., until the next invoice is
issued to fill the gap. But I guess at the end of the day the
probability that there are two concurrent threads creating invoices
negligible in a small business with one invoice per day or less...

On 15.01.15 09:37, Greg Young wrote:
> See rest of reply.
>
> On Thursday, January 15, 2015, Alexander Langer <al...@big.endian.de

> <mailto:al...@big.endian.de>> wrote:
>
> I'm having slight problems to understand how to recover from that
> situation:
>

> Thread A: ReadModel.__getNextInvoiceNumber() -> 1234 Thread B:
> ReadModel.__getNextInvoiceNumber() -> 1235 Thread A: transactin fail

[Ben Kloosterman]

Dont use transactions , with transaction and multiple consumers or the cloud you always face the possibility of a hole due to roll back even in a DB. 

Perfectly valid to have an Invoice number but cancel it due to technical issue. Just make sure it i exists in the DB ! . 

I would use a synchronous service for this which manages a list of all  invoices - and creates them.  Since its a single source of truth and synchronous it should be pretty straight forward.

Ben

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

[Thomas Vantroyen]

That's my conclusion too.  Trying to do something "immediately consistent" in an "eventually consistent" way results in a complicated solution. 

 

There's nothing wrong with making an exception for the rare "immediately consistent" features and doing it... "immediately consistent" :)  Simple code, no compensational constructs.  Just number the damn thing and be done with it :)

 

But this discussion was very insightful nonetheless.

 

grtz

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.

[@yreynhout]

I would just get a homeless guy/gal to help out in the office. If you want the next invoice number you just queue in front of his desk and ask for it. This could also work remotely, using some old phone that doesn't allow multiple callers. Pretty sure they guy/gal would be nicked "Sindy" (sequential invoice number dispatcher). YMMV

[Tom Janssens]

Note to self: whenver I might need to hire someone, ask Yves first!

Op vrijdag 16 januari 2015 14:52:35 UTC+1 schreef @yreynhout:

[Kijana Woodard]

You'd be surprised how many projects I've seen [recently!] that does have a "Sindy" to assign invoice / account numbers.

--