Usage billing period offset from invoice date

[ma...@solarnetwork.net]

Hello,

I am wondering if it is possible to configure a usage-based plan where the invoice is generated a few days after the billing cycle date. For example, I'd like an invoice to be generated on the 2nd of Sept, and to include usage for the previous calendar month (1st Aug - 31st Aug).

The reason for the delay is to allow for the jobs posting daily usage data for the last day of the month to have a chance to run before the invoice is generated.

Is such a configuration possible?

Thanks for any advice,
Matt

[stephane brossier]

Hi Matt,

This is currently not supported. If you guys are ready to make some code change, we could help you navigate the code and review a PR though as the feature seems interesting.

Stéphane

--
You received this message because you are subscribed to the Google Groups "Kill Bill users mailing-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to killbilling-users+unsubscribe@googlegroups.com.
To post to this group, send email to killbilling-users@googlegroups.com.
Visit this group at https://groups.google.com/group/killbilling-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/killbilling-users/0ca764fd-bd42-42bf-a452-b43f76256e96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ma...@solarnetwork.net]

Thanks, Stéphane.

I wouldn’t mind taking a stab at this. Could you point me at a good place to start, or give me any high-level pointers on how such a delay might be implemented?

— m@

[stephane brossier]

Matt,

I went through the code, and i am wondering if we could reverse the model -- implementation would be simpler, main work would be to write good tests to verify behavior.

1. You could configure your customer BCD to be on the 2nd -- not the 1st.

2. We could modify the code -- invoice config -- to fetch usage with an offset, let's say to 2 days in a such way that on sept 2nd, it pulls usage from 1st Aug - 31st Aug.

Would that work for you?

Something to keep in mind is that such code modification would only be available in our development branch `work-for-release-0.19.x` and not against current versions 0.18.x.

Stéphane

On Thu, Aug 31, 2017 at 1:11 AM, <ma...@solarnetwork.net> wrote:

[ma...@solarnetwork.net]

Yes, I think that would be great, thanks. Just so I'm clear, would this new "usage offset" property be used to offset the end date of the usage billing period by X days, and the start date of the usage billing period would be derived from that based on the billingPeriod defined on the usage?

I'm using a MONTHLY billingPeriod for this usage, but I suppose for other billingPeriod values like WEEKLY this new "usage offset" would still use the expected range of 1 week, ending X days before the BCD?

I'd be happy to test it out in the 0.19.x branch as well.

-- m@

[stephane brossier]

Matt,

On Fri, Sep 1, 2017 at 11:55 AM, <ma...@solarnetwork.net> wrote:

Yes, I think that would be great, thanks. Just so I'm clear, would this new "usage offset" property be used to offset the end date of the usage billing period by X days, and the start date of the usage billing period would be derived from that based on the billingPeriod defined on the usage?

This is how our code works: Let's say you start a subscription with a monthly in arrear usage type  on '2017-08-01T20:02:05.000':

1. On 2017-09-01T20:02:05.000, invoicing code is invoked and takes into account all items seen from 2017-08-01T20:02:05.000 ->  2017-09-01T20:02:05.000 (not included).
   
2. Next month,  on 2017-10-01T20:02:05.000 , invoicing code is invoked again takes into account all items seen from 2017-08-01T20:02:05.000 ->  2017-10-01T20:02:05.000 (not included). However, assuming you are not inserting items in the past, the period from from 2017-08-01T20:02:05.000 ->  2017-09-01T20:02:05.000 did not change and so the resulting item is for the new usage from 2017-09-01T20:02:05.000 ->  2017-10-01T20:02:05.000.

If for some reason, we were to record addition usage items for the period 2017-08-01T20:02:05.000 ->  2017-09-01T20:02:05.000 after  2017-09-01T20:02:05.000, you would see a new invoice usage item generated to account for this change as well.

So the point i am trying to illustrate here is that there is not really a startDate per say. Instead there is:

• a targetDate (or endDate, up to where to invoice)
• some boundaries (in this case one per month) to ensure we have different invoice usage items for the different month. The system always recomputes everything-- although we have some optimization code to not really do that, but that's not important for this discussion-- and generates the delta with what is missing.

In order to implement what i suggested you would need to shift the boundaries (probably in this code) by a day -- assuming your BCD would be 2nd. You could start by hardcoding it, and write some tests here and then integration tests here. If this works, we would have to discuss how to make this configurable.

Stéphane

I'm using a MONTHLY billingPeriod for this usage, but I suppose for other billingPeriod values like WEEKLY this new "usage offset" would still use the expected range of 1 week, ending X days before the BCD?

I'd be happy to test it out in the 0.19.x branch as well.

-- m@

> I went through the code, and i am wondering if we could reverse the model -- implementation would be simpler, main work would be to write good tests to verify behavior.
>
>
> 1. You could configure your customer BCD to be on the 2nd -- not the 1st.
> 2. We could modify the code -- invoice config -- to fetch usage with an offset, let's say to 2 days in a such way that on sept 2nd, it pulls usage from 1st Aug - 31st Aug.
>
>
> Would that work for you?
>
>
> Something to keep in mind is that such code modification would only be available in our development branch `work-for-release-0.19.x` and not against current versions 0.18.x.

--
You received this message because you are subscribed to the Google Groups "Kill Bill users mailing-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to killbilling-users+unsubscribe@googlegroups.com.
To post to this group, send email to killbilling-users@googlegroups.com.
Visit this group at https://groups.google.com/group/killbilling-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/killbilling-users/9ec2fcbc-2126-46b9-9da5-2b4aaa90646e%40googlegroups.com.

[ma...@solarnetwork.net]

Ah yes, of course I wasn't considering how invoices in Killbill pull in all usage *up to* the target date. Thanks for the clear explanation. And thank you for the pointers to the code and tests; I will have a crack at hard coding an offset value as you suggest.

-- m@

On Saturday, September 2, 2017 at 8:23:02 AM UTC+12, stephane brossier wrote:

> So the point i am trying to illustrate here is that there is not really a startDate per say. Instead there is:

> a targetDate (or endDate, up to where to invoice)some boundaries (in this case one per month) to ensure we have different invoice usage items for the different month. The system always recomputes everything-- although we have some optimization code to not really do that, but that's not important for this discussion-- and generates the delta with what is missing.

[Matt Magoffin]

Hi Séphane,

I started work on this as you suggested. You can see the changes here:

https://github.com/SolarNetwork/killbill/compare/eclipse-support-0.18.x...usage-offset

I copied two unit test methods from TestContiguousIntervalConsumableInArrear that failed after hard-coding an offset in all tests and then ensured the old tests plus the new ones pass.

Could you take a look and see what you think?

— m@

[stephane brossier]

Hey Matt,

I took a look at your diff, and i have some comments/questions. Let me first draw something to make sure we understand each other:

Let 's assume following subscription with BCD=1, startDate=15, endDate=14:

ux: are usage recorded on those dates:

              s               bcd                         bcd       e

  ----14-15------------1-2----------------------1--------14

         |   |     |              |              |                         |       

       u1 u2   u3          u4            u5                      u6

Prior your change:

• On the first: Period 15 -> 1: {u2, u3}
• On the first (next month): Period 1 -> 1: {u4, u5}
• On the 14 (endDate): Period 1 -> 14: {u6}

With your changes, BCD =2 and usageBcdOffsetDays=1, this means instead of being called every 1st, system would compute invoices on the 2nd, but the intent to generate the invoices as if we were called on the first, correct?

• What would the service period of each invoice be (e.g for first one, 15 -> 1 or 15 -> 2) ?
• Which usage item would be contained in each such invoice?

One question for instance  here: We would create a first transition on the 14 (15 -1), and include a hypothetical point, u1. In theory it does not make sense for such point to exist since subscription was not created yet, but then why does the code need to subtract usageBcdOffsetDays to startDate?

I suggest creating a few scenario written/drawn somewhere that we could use the following way-- ascii version is probably not optimal:

1. Make sure we understand what is the objective -- with some amount of details
2. Point to such scenario during CR
3. Write the test case -- ensure good coverage-- and also helps a lot to follow what the test is supposed to  do

[Matt Magoffin]

Hi Stéphane,

Thank you for the feedback. My understanding in the actual technical implementation details are limited as I am too inexperienced with Kill Bill. From reading the code and unit tests, I came to understand the dates added to transitionTimes as essentially BCD dates, but sometimes they are not because of that line you pointed to. Then I proposed any date added to transitionTimes should have the offset applied.

Still, the semantics of the billing dates and these transition dates are not completely clear in my head, or I suppose the significance of the rawUsageStartDate, which is the effective date of the first billing event, or which I thought was a BCD.

Looking at this line of an existing test:

https://github.com/killbill/killbill/blob/2e8156f27ee9368b1d7b815a440af5829f9000aa/invoice/src/test/java/org/killbill/billing/invoice/usage/TestContiguousIntervalConsumableInArrear.java#L238

I don’t understand why raw1 shouldn’t be included in the resulting rolled up usage, if it hasn’t been included in any invoice item yet?

I’ve tweaked the code to not offset the start date now, along with the unit tests I added so they pass again. I could add more tests, but quite frankly I’m not confident I understand what type of inputs in terms of billing events and their dates are passed into this class in the first place. Maybe it would help  me if you could explain the intention of the ContiguousIntervalUsageInArrear constructor arguments in more detail?

I know what my objective is, which for the simple case of an invoice generated on a BCD > 1 to include usage up to (but excluding) the 1st of that month… resulting in invoices generated say on the 2nd of October to include usage to to the 30th of September (inclusive). Essentially I would always want this offset to be (BCD - 1).

— m@

[stephane brossier]

Matt,

On Sat, Sep 23, 2017 at 9:27 PM, Matt Magoffin <ma...@solarnetwork.net> wrote:

Hi Stéphane,

Thank you for the feedback. My understanding in the actual technical implementation details are limited as I am too inexperienced with Kill Bill. From reading the code and unit tests, I came to understand the dates added to transitionTimes as essentially BCD dates, but sometimes they are not because of that line you pointed to. Then I proposed any date added to transitionTimes should have the offset applied.

Still, the semantics of the billing dates and these transition dates are not completely clear in my head, or I suppose the significance of the rawUsageStartDate, which is the effective date of the first billing event, or which I thought was a BCD.

Looking at this line of an existing test:

https://github.com/killbill/killbill/blob/2e8156f27ee9368b1d7b815a440af5829f9000aa/invoice/src/test/java/org/killbill/billing/invoice/usage/TestContiguousIntervalConsumableInArrear.java#L238

I don’t understand why raw1 shouldn’t be included in the resulting rolled up usage, if it hasn’t been included in any invoice item yet?

The reason why raw1 is not included is because raw1 has been inserted before the subscription starts -- there is no api to even be able to register such a data point, as the api takes the subscriptionID; however because this is a unit test, we still want to verify that such condition would be ignored.

 

I’ve tweaked the code to not offset the start date now, along with the unit tests I added so they pass again. I could add more tests, but quite frankly I’m not confident I understand what type of inputs in terms of billing events and their dates are passed into this class in the first place. Maybe it would help  me if you could explain the intention of the ContiguousIntervalUsageInArrear constructor arguments in more detail?

• final Usage usage,  // Usage section as described in the catalog
  
• final UUID accountId, // obvious
  
• final UUID invoiceId, // obvious
  
• final List<RawUsage> rawSubscriptionUsage, // All the usage data associated with this subscription
• final LocalDate targetDate, // The date at which we want to bill for; in normal use case, should align to BCD
• final LocalDate rawUsageStartDate, // This is an setting to limit how far we look back for usage data, this configurable is and by default set to 2 billing periods.
• For one thing we want to avoid modifying old invoices if we realize that usage data from 2 years ago suddenly got modified
• Another aspect is that since algorithm always recomputes the whole state, it becomes expensive and so limiting that to an acceptable period is recommended
• final InternalTenantContext internalTenantContext
  
  

By just looking at your tests, i am still confused. for instance here: The BCD is set to 15 in the test, so your offset is BCD - 1 = 14. Why?

 

I know what my objective is, which for the simple case of an invoice generated on a BCD > 1 to include usage up to (but excluding) the 1st of that month… resulting in invoices generated say on the 2nd of October to include usage to to the 30th of September (inclusive). Essentially I would always want this offset to be (BCD - 1).

The work is challenging on both sides -- and to be fair there is so much time i have to spend on this. Before any code change, i really need to ensure we first understand precisely on both sides what it is we are trying to achieve. I really think having some picture describing use cases  -- showing startDate, BCD, offset, endDate and examples of usage points would be the best to communicate. Then tests could follow, implementing exactly those pictures -- at this point following test case becomes practical. Once we have tests in place, i could do a code review to spot any obvious issues and if both regression tests and your new tests matching the described scenario pass, i am happy to merge.

Stéphane

[Matt Magoffin]

Hi Stéphane,

Thank you for the additional info on those constructor arguments. In answer to your question, I set the offset in the test to (BCD - 1) because if I choose to generate invoices on the 15th of the month I want the usage window to end at the start of the month.

Or to put another way, whatever day this monthly invoice is generated on, I want to only include usage for the previous calendar month, because in our system we have other reports generated outside of Kill Bill that show the same usage data but are based on calendar months. I would like the aggregate usage numbers on the invoices to match the numbers reported on those reports. I understand that in Kill Bill invoices might get generated on days other than the BCD, but for the typical case when they are, an agreement between Kill Bill and the other system is what I’m trying to achieve.

And to reiterate why I’d like to have the BCD not on the 1st of the month, it is because it might take some time for the usage data to get fully populated in Kill Bill, so I’d like to delay the generation of the invoice a day (or few) just to make it more likely Kill Bill has all the usage data available when it generates the invoices.

— m@

[stephane brossier]

Ok, i understand the overall desire -- and why you set usageBcdOffsetDays = BCD - 1. 

In order to move forward, i need some solid test cases -- and again, a small picture, or at the very least test case must have detailed explanations of what they do. For instance -- but not limited to -- assuming BCD = 3, offest = 2:

• startDate = end of month, 1st, 2nd, 3rd + few regular periods
• early cancelation: 
• endDate =  end of month, 1st, 2nd, 3rd 
• case where endDate < 1 full period

Each test should should have unit usage in  between boundaries and on each specifc boundary

At this point i would open a PR, so i can comment on it and we can iterate.

Stéphane

[Matt Magoffin]

OK, I’d like to understand better the variations of input data you’re after for the tests. For a first step, would it be OK to focus solely on the outcome of the 

org.killbill.billing.invoice.usage.ContiguousIntervalUsageInArrear.getTransitionTimes()

method? From looking at the inputs that method relies on, I started trying to document a few test case scenarios:

Test Case	BillingEvent count	BillingEvent[0] effectiveDate	BillingEvent[n] effectiveDate	targetDate	rawUsageStartDate
First invoice event on BCD	2	2017-01-10	2017-02-03	2017-02-03	2017-01-10
Second invoice event on BCD	3	2017-01-10	2017-03-03	2017-03-03	2017-01-10
Subscription ends not on BCD	4	2017-01-10	2017-03-20	2017-03-20	2017-01-10

Does that look sensible, just focusing on a subscription that runs for a few months? Does the List<BillingEvents> always contain all events for the life of the subscription like I’m showing here with the first effectiveDate basically always the subscription start date?

If these assumptions are valid, then for those inputs, I was imagining resulting transition times like:

1. 2017-01-10, 2017-02-01
2. 2017-01-10, 2017-02-01, 2017-03-01
3. 2017-01-10, 2017-02-01, 2017-03-01, 2017-03-20

Am I getting this right?

— m@

[stephane brossier]

Matt,

This is a good start, but i would need more detail-- input:

• Subscription:
• startDate
• endDate (or null)
• BillingPeriod (Monthly, ...)
• BCD = ?
• Offset = ?

Stéphane

[Matt Magoffin]

Hi, I meant to build off your example, so

• Subscription:
• startDate: 2017-01-10 (I thought this was always BillingEvent[0].effectiveDate?)
• endDate: null until #3, then 2017-03-20
• BillingPeriod: Monthly
• BCD: 3
• Offset: 2

— m@

[stephane brossier]

On Thu, Sep 28, 2017 at 5:36 PM, Matt Magoffin <ma...@solarnetwork.net> wrote:

Hi, I meant to build off your example, so

• Subscription:
• startDate: 2017-01-10 (I thought this was always BillingEvent[0].effectiveDate?)
• endDate: null until #3, then 2017-03-20
• BillingPeriod: Monthly
• BCD: 3
• Offset: 2

In that case, you would see following (because BCD =3):

• BillingEvent[0].effectiveDate = 2017-01-10 
  
• BillingEvent[1].effectiveDate = 2017-02-03  -- leading pro-ration from  2017-01-10  -> 2017-01-10
  
• BillingEvent[2].effectiveDate = 2017-03-03 
  
• Case #3: BillingEvent[3].effectiveDate = 2017-03-20 --  trailing pro-ration from 2017-03-03  ->  2017-03-20

Now, with that specific scenario, add usage points on each boundary and in between, and define what you expect to see when your offset is taken into account.

Stéphane

[Matt Magoffin]

OK, your dates match my BillingEvent[n].effectiveDates in the table, so I think we’re on the same page. What I was asking was about the expected output from the getTransitionTimes() method, because that is what I’m effectively changing. Going back to these inputs then, is it correct to say the output of calling getTransitionTimes() on each of the invocation corresponding to rows in the table would be:

1. 2017-01-10, 2017-02-01
2. 2017-01-10, 2017-02-01, 2017-03-01
3. 2017-01-10, 2017-02-01, 2017-03-01, 2017-03-20

So essentially to me it is like the first transition date is the subscription start date (the offset is not applied), if the subscription has ended the final transition date is the subscription end date (the offset is not applied), and all other dates are BCD dates with the offset applied.

Is this reasoning correct?

— m@

[stephane brossier]

On Thu, Sep 28, 2017 at 5:55 PM, Matt Magoffin <ma...@solarnetwork.net> wrote:

OK, your dates match my BillingEvent[n].effectiveDates in the table, so I think we’re on the same page. What I was asking was about the expected output from the getTransitionTimes() method, because that is what I’m effectively changing. Going back to these inputs then, is it correct to say the output of calling getTransitionTimes() on each of the invocation corresponding to rows in the table would be:

1. 2017-01-10, 2017-02-01
2. 2017-01-10, 2017-02-01, 2017-03-01
3. 2017-01-10, 2017-02-01, 2017-03-01, 2017-03-20

Actually forgive me for the confusion, what i output'ed in the previous email, was not the billing events, but the transition times -- for case #3 which includes previous case:

• targetDate = 2017-03-20
  
• Billing events = {2017-01-10, 2017-03-20}, where startDate=2017-01-10, endDate=2017-03-20 (there are not other billing events, nothing else happen on this subscription
  
• TransitonTimes = {2017-01-10, 2017-02-01, 2017-03-01, 2017-03-20}
  

 

So essentially to me it is like the first transition date is the subscription start date (the offset is not applied),

I think so, but i 'd like you to think of use case where 1 <= startDate <= BCD = 3; how would that work with your scenario? For instance on the 3rd, invoice gets generated and using your offset '2', you would compute usage for the 1st -- by then your usage data might be up to date, so you might bill for that specific day. 

I think those are the edge case, i would like to figure out so 1/ they work for you and 2/ we have test cases and understand what code should be doing.

 

if the subscription has ended the final transition date is the subscription end date (the offset is not applied),

Same thing here: In the use case above, invoice code tries to generate an invoice on the 2017-03-20, but if you have a lag in getting your usage data, then will you really end up billing for the right thing -- assuming lag is 2 days, won't you just bill up to 2017-03-18?

and all other dates are BCD dates with the offset applied.

For those, this probably make sense.

 

Is this reasoning correct?

We are getting there ;-) 

Stéphane