Re: [Invoicing Issue] Delayed Invoice generation

[karan bansal]

Hi Rodrigo,

How often are you facing the delay and what is the typical delay caused? For example, does it happen on any particular day like a month end or bulk processing date? Does it cause a delay of a few seconds, minutes, hours or a day/two etc. 

It does depend on many different factors like notification processing configuration, hardware like number of nodes, database etc., load on them, the time it takes to process each of these events which itself also depends on things like hardware and whether you are using plugins or not and finally the settings for the queue (bus & notification queue which have similar properties) to state a few. 

The default notification entries are shown here. The main properties you want to look at are -  for the main notification queue:

# Default values
org.killbill.notificationq.main.sleep=3s
org.killbill.notificationq.main.notification.nbThreads=10

org.killbill.notificationq.main.claimed=10

Could you please confirm if you have setup any values different than the default values. 

You could also look for exceptions in the logs and add some monitoring to your server/DB so that you can have some reference values for the times when you see the processing delay. Also, you can look into if you are using any kind of plugins like taxation, invoicing etc. that could cause some latency. 

Regards

Karan

On Saturday, April 15, 2023 at 2:22:42 AM UTC+5:30 Rodrigo Zanel wrote:

Hi All,

I have an issue and a question.

The case:
Our plans are all prepaid plans, and to ensure that when we create a subscription we charge the account, we make sure the entitlementDate and billingDate are equal NOW.

It turns out that for any reason, sometimes Killbill is delaying the invoice generation, and consequently does not charge the account when the subscription is created. I believe this could be related to some internal processing queue, because most of the time the invoice and charge are generated as soon as the subscription is created.

Given:
- AUTO_INVOICING_DRAFT is FALSE for all accounts (invoice is committed automatically)
- When creating the subscription we send callCompletion as TRUE and callTimeoutSec as 60 (seconds)

- We're using recurring payment for the subscription, not one-time payments

My questions are:
- Do you have any suggestions on how to to check the reason why Killbill often doesn't generate the invoice as soon as the the subscription is added?
- Is there any way to only allow a subscription to be added if the invoice is generated and the charge is made?

I already tried the following solutions:

- After adding the subscription, I check if there is any payment pending for the generated invoice (since the invoice is generated). If there is a pendimgn payment, I trigger the payment for the invoice through the API POST /1.0/kb/invoices/{invoiceId}/payments
- If a payment fails for any reason, I cancel the subscription. The problem is that sometimes the invoice is not generated (as explained above), so there is no pending payment to be triggered in that case.
- To ensure that the invoice will be generated (when the invoice is NOT automatically generated after the subscription is added), I call the POST API
/1.0/kb/invoices to trigger the invoice generation. The problem is that this API takes a few seconds longer to create the invoice and process the payment, and this causes a bad experience for the user once the request takes longer than necessary

Any tips?

Thank you in advance,
Rodrigo

[Shaun Forgie]

Hi Rodrigo,

Remember that billing cycles are driven off the account. When you create the first subscription for an account it sets the billing cycle for the account. Upon creating additional subscriptions for the account those subscriptions will not trigger invoices immediately...they will be added as line items to the underlying account and invoiced at the next billing cycle.

It is possible to set the billing cycle for a subscription to be independent of the account billing cycle such that the customer will receive different invoices for different subscriptions but this is not the default behaviour.

Rgs

Shaun

On Sat, 15 Apr 2023 at 08:52, Rodrigo Zanel <rodrigo...@gmail.com> wrote:

Hi All,

I have an issue and a question.

The case:
Our plans are all prepaid plans, and to ensure that when we create a subscription we charge the account, we make sure the entitlementDate and billingDate are equal NOW.

It turns out that for any reason, sometimes Killbill is delaying the invoice generation, and consequently does not charge the account when the subscription is created. I believe this could be related to some internal processing queue, because most of the time the invoice and charge are generated as soon as the subscription is created.

Given:
- AUTO_INVOICING_DRAFT is FALSE for all accounts (invoice is committed automatically)
- When creating the subscription we send callCompletion as TRUE and callTimeoutSec as 60 (seconds)

- We're using recurring payment for the subscription, not one-time payments

My questions are:
- Do you have any suggestions on how to to check the reason why Killbill often doesn't generate the invoice as soon as the the subscription is added?
- Is there any way to only allow a subscription to be added if the invoice is generated and the charge is made?

I already tried the following solutions:

- After adding the subscription, I check if there is any payment pending for the generated invoice (since the invoice is generated). If there is a pendimgn payment, I trigger the payment for the invoice through the API POST /1.0/kb/invoices/{invoiceId}/payments
- If a payment fails for any reason, I cancel the subscription. The problem is that sometimes the invoice is not generated (as explained above), so there is no pending payment to be triggered in that case.
- To ensure that the invoice will be generated (when the invoice is NOT automatically generated after the subscription is added), I call the POST API
/1.0/kb/invoices to trigger the invoice generation. The problem is that this API takes a few seconds longer to create the invoice and process the payment, and this causes a bad experience for the user once the request takes longer than necessary

Any tips?

Thank you in advance,
Rodrigo

--
You received this message because you are subscribed to the Google Groups "Kill Bill users mailing-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to killbilling-us...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/killbilling-users/664470ef-a31d-41c0-82bd-0ed8220a23d2n%40googlegroups.com.

--

Shaun Forgie [Principal] - Method Maker Ltd
57a McIntyre Rd, Manukau 2022, Auckland, New Zealand
Mobile +64 21 666 910

[Shaun Forgie]

Hi Rodrigo,

In Kill Bill payments follow subscriptions...they do not precede them. 

You could set up a raw account without a subscription from the catalog and process a manual invoice in the first instance. If the payment was successful you could then create the subscription in arrears.

In summary if you want to process a payment before setting up the subscription you will have to follow the procedure above.

Rgs

Shaun

On Tue, 18 Apr 2023 at 10:29, Rodrigo Zanel <rodrigo...@gmail.com> wrote:

Hi Karan and Shaun,

Karan, I didn't notice a specific moment or reason where the problem happens. I've seen it happen even on days of low access to our applications. Seems a bit random to me. Our customers can create subscriptions any day of the month, so we don't have a bulk subscription processing day for example. When it comes to the properties, we use the default settings, we don't overwrite any properties. Related to plugins, we only use the Stripe plugin to charge the subscriptions. We don't have any other plugins and the Stripe plugin doesn't intercept invoice generation (as far as I know).
Looking at logs I don't see any errors or warnings. The only thing that I noticed, is that the killbill.notifications table contains a lot of records with classname 'org.killbill.billing.invoice.notification.NextBillingDateNotificationKey' in status 'AVAILABLE' and queue as 'invoice-service:next-billing -date-queue'. Do you think this could indicate a problem? Is there any documentation that I can refer to on how each notification should be processed?

Shaun, all of our plans are set up with the recurringBillingMode as IN_ADVANCE, and whenever a plan is created, we make sure that the entitlementDate and billingDate values are passed with the current date/time. This way, except in the case of the failure that I described, the invoice is always generated just after the plan is created, and consequently the charge is made. Around 500 subscriptions are created per month, and in most of the cases, the invoice is generated immediately after creating the subscription.

Sirs, as far as I could understand, in the current Killbill architecture there is no guarantee that a subscription will only be added if the payment is approved. Am I correct in thinking this?

I think an alternative would be to process a one-time payment. If approved, generate a credit to the account and then add the subscription. That way the first charge would use the account credit instead of charging the payment method and I would ensure the subscription was paid before being added.

In your experience, does this solution make sense? Do you know other use cases of platforms that also need to guarantee the payment of the first invoice before adding the subscription? How have other customers solved this use case? Let's take Spotify as an example. We only join the plan if the payment is confirmed. Both cases make sense to me, in different contexts, of course.

Thank you in advance for your considerations,
Rodrigo

--

Rodrigo Zanel