Bug Report — Error 1019 (SUB_CREATE_AO_NOT_AVAILABLE) when adding addon to existing subscription after catalog update

63 views
Skip to first unread message

TAMIL THENDRAL SENTHAMIZH

unread,
Apr 17, 2026, 11:20:36 AMApr 17
to Kill Bill users mailing-list

Hi Kill Bill Team,

I'd like to report a bug related to addon subscription creation for existing subscriptions when a new catalog version introduces addon availability that was not present in the original catalog version.

## Kill Bill Version

This bug exists in ALL Kill Bill versions including the latest (0.24.16). The affected code (AddonUtils.java, CatalogResource.java, SubscriptionCatalog.java) is identical across 0.22.33 and the latest master branch. We verified this by comparing the source code line by line.

## Summary

When a base subscription is created under catalog version V1 (where no addons are marked as <available>), and a new catalog version V2 is uploaded that adds an addon as <available> for the base product, existing subscriptions:

1. Cannot subscribe to the newly available addon — fails with error code 1019 (SUB_CREATE_AO_NOT_AVAILABLE)
2. The GET /catalog/product?subscriptionId= API returns an empty available list instead of showing the newly added addon

The addon can only be subscribed if the base subscription is changed via PUT /subscriptions/{id} (change plan to the same plan), which is not acceptable as it triggers re-billing (fixed + recurring charges).

## Steps to Reproduce

### Step 1: Upload Catalog V1 (Base product WITHOUT addon availability)
POST /1.0/kb/catalog/xml
Content-Type: application/xml
xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<catalog>
    <effectiveDate>2026-04-18T12:40:12Z</effectiveDate>
    <catalogName>DEFAULT</catalogName>
    <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
    <currencies>
        <currency>USD</currency>
    </currencies>
    <units>
        <unit name="cell-phone-message-one"/>
        <unit name="cell-phone-message-two"/>
        <unit name="cell-phone-message-three"/>
        <unit name="cell-phone-message-four"/>
        <unit name="cell-phone-message-five"/>
    </units>
    <products>
        <product name="Standard">
            <category>BASE</category>
            <!-- NOTE: No <available> section — addon is NOT mapped -->
        </product>
        <product name="RemoteControl">
            <category>ADD_ON</category>
        </product>
        <product name="OilSlick">
            <category>ADD_ON</category>
        </product>
    </products>
    <rules>
        <changePolicy>
            <changePolicyCase>
                <policy>IMMEDIATE</policy>
            </changePolicyCase>
        </changePolicy>
        <changeAlignment>
            <changeAlignmentCase>
                <alignment>START_OF_BUNDLE</alignment>
            </changeAlignmentCase>
        </changeAlignment>
        <cancelPolicy>
            <cancelPolicyCase>
                <productCategory>BASE</productCategory>
                <policy>IMMEDIATE</policy>
            </cancelPolicyCase>
            <cancelPolicyCase>
                <policy>IMMEDIATE</policy>
            </cancelPolicyCase>
        </cancelPolicy>
        <createAlignment>
            <createAlignmentCase>
                <alignment>START_OF_BUNDLE</alignment>
            </createAlignmentCase>
        </createAlignment>
        <billingAlignment>
            <billingAlignmentCase>
                <alignment>ACCOUNT</alignment>
            </billingAlignmentCase>
        </billingAlignment>
        <priceList>
            <priceListCase>
                <toPriceList>DEFAULT</toPriceList>
            </priceListCase>
        </priceList>
    </rules>
    <plans>
        <plan name="standard-daily" prettyName="standard-daily-prepaid-prettyname">
            <product>Standard</product>
            <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
            <initialPhases/>
            <finalPhase type="EVERGREEN">
                <duration>
                    <unit>UNLIMITED</unit>
                </duration>
                <fixed type="ONE_TIME">
                    <fixedPrice>
                        <price>
                            <currency>USD</currency>
                            <value>723</value>
                        </price>
                    </fixedPrice>
                </fixed>
                <recurring>
                    <billingPeriod>DAILY</billingPeriod>
                    <recurringPrice>
                        <price>
                            <currency>USD</currency>
                            <value>234</value>
                        </price>
                    </recurringPrice>
                </recurring>
            </finalPhase>
        </plan>
        <plan name="remotecontrol-daily" prettyName="remotecontrol-daily-prettyname">
            <product>RemoteControl</product>
            <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
            <initialPhases/>
            <finalPhase type="EVERGREEN">
                <duration>
                    <unit>UNLIMITED</unit>
                </duration>
                <recurring>
                    <billingPeriod>DAILY</billingPeriod>
                    <recurringPrice>
                        <price>
                            <currency>USD</currency>
                            <value>0</value>
                        </price>
                    </recurringPrice>
                </recurring>
                <usages/>
            </finalPhase>
        </plan>
        <plan name="oilslick-daily" prettyName="oilslick-daily-prettyname">
            <product>OilSlick</product>
            <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
            <initialPhases/>
            <finalPhase type="EVERGREEN">
                <duration>
                    <unit>UNLIMITED</unit>
                </duration>
                <recurring>
                    <billingPeriod>DAILY</billingPeriod>
                    <recurringPrice>
                        <price>
                            <currency>USD</currency>
                            <value>0</value>
                        </price>
                    </recurringPrice>
                </recurring>
                <usages/>
            </finalPhase>
        </plan>
    </plans>
    <priceLists>
        <defaultPriceList name="DEFAULT">
            <plans>
                <plan>standard-daily</plan>
                <plan>remotecontrol-daily</plan>
                <plan>oilslick-daily</plan>
            </plans>
        </defaultPriceList>
    </priceLists>
</catalog>

### Step 2: Create a base subscription on the "standard-daily" plan
POST /1.0/kb/subscriptions
Content-Type: application/json
json
{
    "accountId": "4dfde893-a220-4820-ace9-44b47985cbef",
    "planName": "standard-daily"
}

Response: Subscription created successfully (subscriptionId: bb3bf566-7b4d-4b90-8ba4-2302596b52ab)

### Step 3: Verify product info — available list is empty (expected)
GET /1.0/kb/catalog/product?subscriptionId=bb3bf566-7b4d-4b90-8ba4-2302596b52ab

Response (correct — no addons were configured as available in V1):
json
{
    "type": "BASE",
    "name": "Standard",
    "prettyName": "Standard",
    "plans": [],
    "included": [],
    "available": []
}

### Step 4: Upload Catalog V2 (Base product WITH addon availability)

Note: The only change from V1 is adding <available><addonProduct>RemoteControl</addonProduct></available> to the Standard product. The effective date is 1 second later.
POST /1.0/kb/catalog/xml
Content-Type: application/xml
xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<catalog>
    <effectiveDate>2026-04-18T12:40:13Z</effectiveDate>
    <catalogName>DEFAULT</catalogName>
    <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
    <currencies>
        <currency>USD</currency>
    </currencies>
    <units>
        <unit name="cell-phone-message-one"/>
        <unit name="cell-phone-message-two"/>
        <unit name="cell-phone-message-three"/>
        <unit name="cell-phone-message-four"/>
        <unit name="cell-phone-message-five"/>
    </units>
    <products>
        <product name="Standard">
            <category>BASE</category>
            <available>
                <addonProduct>RemoteControl</addonProduct>
            </available>
        </product>
        <product name="RemoteControl">
            <category>ADD_ON</category>
        </product>
        <product name="OilSlick">
            <category>ADD_ON</category>
        </product>
    </products>
    <rules>
        <changePolicy>
            <changePolicyCase>
                <policy>IMMEDIATE</policy>
            </changePolicyCase>
        </changePolicy>
        <changeAlignment>
            <changeAlignmentCase>
                <alignment>START_OF_BUNDLE</alignment>
            </changeAlignmentCase>
        </changeAlignment>
        <cancelPolicy>
            <cancelPolicyCase>
                <productCategory>BASE</productCategory>
                <policy>IMMEDIATE</policy>
            </cancelPolicyCase>
            <cancelPolicyCase>
                <policy>IMMEDIATE</policy>
            </cancelPolicyCase>
        </cancelPolicy>
        <createAlignment>
            <createAlignmentCase>
                <alignment>START_OF_BUNDLE</alignment>
            </createAlignmentCase>
        </createAlignment>
        <billingAlignment>
            <billingAlignmentCase>
                <alignment>ACCOUNT</alignment>
            </billingAlignmentCase>
        </billingAlignment>
        <priceList>
            <priceListCase>
                <toPriceList>DEFAULT</toPriceList>
            </priceListCase>
        </priceList>
    </rules>
    <plans>
        <plan name="standard-daily" prettyName="standard-daily-prepaid-prettyname">
            <product>Standard</product>
            <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
            <initialPhases/>
            <finalPhase type="EVERGREEN">
                <duration>
                    <unit>UNLIMITED</unit>
                </duration>
                <fixed type="ONE_TIME">
                    <fixedPrice>
                        <price>
                            <currency>USD</currency>
                            <value>723</value>
                        </price>
                    </fixedPrice>
                </fixed>
                <recurring>
                    <billingPeriod>DAILY</billingPeriod>
                    <recurringPrice>
                        <price>
                            <currency>USD</currency>
                            <value>234</value>
                        </price>
                    </recurringPrice>
                </recurring>
            </finalPhase>
        </plan>
        <plan name="remotecontrol-daily" prettyName="remotecontrol-daily-prettyname">
            <product>RemoteControl</product>
            <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
            <initialPhases/>
            <finalPhase type="EVERGREEN">
                <duration>
                    <unit>UNLIMITED</unit>
                </duration>
                <recurring>
                    <billingPeriod>DAILY</billingPeriod>
                    <recurringPrice>
                        <price>
                            <currency>USD</currency>
                            <value>0</value>
                        </price>
                    </recurringPrice>
                </recurring>
                <usages/>
            </finalPhase>
        </plan>
        <plan name="oilslick-daily" prettyName="oilslick-daily-prettyname">
            <product>OilSlick</product>
            <recurringBillingMode>IN_ADVANCE</recurringBillingMode>
            <initialPhases/>
            <finalPhase type="EVERGREEN">
                <duration>
                    <unit>UNLIMITED</unit>
                </duration>
                <recurring>
                    <billingPeriod>DAILY</billingPeriod>
                    <recurringPrice>
                        <price>
                            <currency>USD</currency>
                            <value>0</value>
                        </price>
                    </recurringPrice>
                </recurring>
                <usages/>
            </finalPhase>
        </plan>
    </plans>
    <priceLists>
        <defaultPriceList name="DEFAULT">
            <plans>
                <plan>standard-daily</plan>
                <plan>remotecontrol-daily</plan>
                <plan>oilslick-daily</plan>
            </plans>
        </defaultPriceList>
    </priceLists>
</catalog>

### Step 5: Verify product info — available list is STILL empty (BUG)
GET /1.0/kb/catalog/product?subscriptionId=bb3bf566-7b4d-4b90-8ba4-2302596b52ab

Actual response (WRONG — should show RemoteControl):
json
{
    "type": "BASE",
    "name": "Standard",
    "prettyName": "Standard",
    "plans": [],
    "included": [],
    "available": []
}

Expected response:
json
{
    "type": "BASE",
    "name": "Standard",
    "prettyName": "Standard",
    "plans": [],
    "included": [],
    "available": ["RemoteControl"]
}

### Step 6: Attempt to subscribe the addon — FAILS with error 1019
POST /1.0/kb/subscriptions/createSubscriptionWithAddOns
Content-Type: application/json
json
[
    {
        "accountId": "4dfde893-a220-4820-ace9-44b47985cbef",
        "bundleId": "6332f4e4-e70f-48aa-8199-0510e290ad7c",
        "planName": "remotecontrol-daily"
    }
]

Error response:
json
{
    "className": "org.killbill.billing.subscription.api.user.SubscriptionBaseApiException",
    "code": 1019,
    "message": "Can't create AddOn remotecontrol-daily for BasePlan Standard (Not available)",
    "causeClassName": null,
    "causeMessage": null,
    "stackTrace": []
}

## Root Cause Analysis

The issue is in the catalog version resolution logic. Two files are affected:

### 1. AddonUtils.checkAddonCreationRights() (subscription module)

File: subscription/src/main/java/org/killbill/billing/subscription/engine/addon/AddonUtils.java

When validating addon creation, this method does:
java
final Plan currentOrPendingPlan = baseSubscription.getCurrentOrPendingPlan();
final Product baseProduct = currentOrPendingPlan.getProduct();

The currentOrPendingPlan is resolved from the subscription's transitions, which were built during DefaultSubscriptionBase.rebuildTransitions() using:
java
catalog.findPlan(nextPlanName, cur.getEffectiveDate(), lastPlanChangeTime)

Where lastPlanChangeTime = the subscription's CREATE event time. This calls SubscriptionCatalog.findCatalogPlanEntry(), which walks catalog versions backwards. Since the subscription was created BEFORE catalog V2's effective date, and the plan in V2 has no effectiveDateForExistingSubscriptions set, the code resolves to catalog V1 — the version where <available> is empty.

Meanwhile, the addon plan (remotecontrol-daily) is resolved from the LATEST catalog version (V2) via catalog.versionForDate(effectiveDate) in DefaultSubscriptionBaseCreateApi.createPlansIfNeededAndReorderBPOrStandaloneSpecFirstWithSanity().

So there's a mismatch: the addon plan comes from V2, but the base product comes from V1. The isAddonAvailable() check fails because V1's Standard product has no addons in <available>.

### 2. CatalogResource.getProductForSubscriptionAndDate() (jaxrs module)

File: jaxrs/src/main/java/org/killbill/billing/jaxrs/resources/CatalogResource.java

The /catalog/product?subscriptionId= endpoint returns the product from the subscription's last event transition, which was also resolved from catalog V1. It never looks at the latest catalog version.

### Why does PUT /subscriptions/{id} (change plan) fix it?

Because a plan change creates a new CHANGE event with lastPlanChangeTime = now. Since "now" is after V2's effective date, findCatalogPlanEntry() resolves to V2, and the base product now has RemoteControl in <available>. However, this triggers re-billing (fixed + recurring charges), which is not acceptable.

### Why effectiveDateForExistingSubscriptions doesn't help

This catalog feature was designed for price changes, not for addon availability changes. Even if set, it only affects plan/price resolution — the AddonUtils.checkAddonCreationRights() code still gets the base product from the subscription's current transition, which may or may not have been updated depending on timing.

### Interesting contrast: Dry-Run API does it correctly

The getDryRunChangePlanStatus() method in DefaultSubscriptionInternalApi already uses the correct approach:
java
final StaticCatalog catalogVersion = catalog.versionForDate(requestedDate);
final Product baseProduct = catalogVersion.findProduct(baseProductName);

It looks up the base product from the latest catalog version. But this logic is not applied to the actual addon creation flow.

## Proposed Fix

### Fix 1: AddonUtils.java — Allow addon creation using latest catalog version

When isAddonAvailable() fails using the base product from the subscription's old catalog version, add a fallback check using the base product from the addon plan's catalog version (which is always the latest):
java
if (!isAddonAvailable(baseProduct, targetAddOnPlan)) {
    // Fallback: check the base product from the addon plan's catalog version (latest catalog)
    try {
        Product baseProductFromAddonCatalog = targetAddOnPlan.getCatalog().findProduct(baseProduct.getName());
        if (baseProductFromAddonCatalog == null || !isAddonAvailable(baseProductFromAddonCatalog, targetAddOnPlan)) {
            throw new SubscriptionBaseApiException(ErrorCode.SUB_CREATE_AO_NOT_AVAILABLE, ...);
        }
    } catch (CatalogApiException e) {
        throw new SubscriptionBaseApiException(ErrorCode.SUB_CREATE_AO_NOT_AVAILABLE, ...);
    }
}

### Fix 2: CatalogResource.java — Return product from latest catalog version

In getProductForSubscriptionAndDate(), look up the product from the current catalog version instead of using the stale product from the subscription's transition:
java
Product latestProduct = product;
try {
    TenantContext tenantContext = context.createTenantContextNoAccountId(request);
    StaticCatalog latestCatalogVersion = catalogUserApi.getCurrentCatalog("unused", tenantContext);
    latestProduct = latestCatalogVersion.findProduct(product.getName());
} catch (CatalogApiException e) {
    // Fall back to original product
}
final ProductJson productJson = new ProductJson(latestProduct);

Both fixes are backward-compatible — if the catalog lookup fails, they fall back to the original behavior.

## Affected Versions

ALL versions of Kill Bill are affected, including the latest release (0.24.16) and the current master branch. We compared the source code of AddonUtils.java, SubscriptionCatalog.java, and CatalogResource.java between version 0.22.33 and the latest master — the code is identical. No fix for this issue was found in any release notes from 0.22.33 through 0.24.16.

## Impact

Any Kill Bill deployment that uses versioned catalogs and needs to add addon availability for existing base products after initial catalog creation is affected. The only current workaround is to change the base subscription's plan via PUT (which triggers unwanted re-billing).

Thank you for looking into this.

karan bansal

unread,
Apr 20, 2026, 4:29:47 AMApr 20
to Kill Bill users mailing-list
Hi there,

Thank you for providing the detailed info about your requirement. The behavior that you have described is actually the expected behavior by design. The new catalog is not available for the existing subscriptions and changing the plan is the appropriate method to subscribe to the new variant. 

I understand that this triggers duplicate charges in your case based on how the catalog is setup. Could you please try your scenario using this as your original catalog and this as your upgrade catalog and let me know if you see the duplicate charge.

Regards
Karan

TAMIL THENDRAL SENTHAMIZH

unread,
May 13, 2026, 3:47:18 PMMay 13
to Kill Bill users mailing-list

Hi Karan,


Thank you for the response and for sharing the catalog examples.


We tried the approach you suggested — using the catalog structure from the links you shared — but unfortunately the issue persists. The addon subscription still fails with error code 1019 for existing subscriptions created under the earlier catalog version. The plan change (PUT) does make it work, but as mentioned, it triggers duplicate fixed + recurring charges which is not acceptable for our production environment with existing tenants.


To summarize what we've verified:


Your suggested catalog structure — tried, same error 1019

Plan change to same plan (PUT /subscriptions/{id}) — works but triggers re-billing

effectiveDateForExistingSubscriptions — doesn't help for addon availability changes

We respectfully disagree that this is purely "expected behavior by design" for the following reason:


Kill Bill's own internal code already handles this correctly in one place — getDryRunChangePlanStatus() in DefaultSubscriptionInternalApi resolves the base product from the latest catalog version using catalog.versionForDate(requestedDate). This proves the system already supports looking up product metadata from newer catalog versions. The gap is only in AddonUtils.checkAddonCreationRights(), which doesn't apply the same logic.


We believe this is a valid enhancement (if not a bug fix) and we're happy to contribute a backward-compatible PR that adds a fallback check in AddonUtils — only activating when the original check fails, preserving existing behavior for all other cases.


Would the team be open to reviewing such a PR? Or is there an alternative approach you'd recommend that avoids re-billing existing subscribers?


Thank you for your time.


Best regards, Tamil Thendral M

karan bansal

unread,
May 14, 2026, 4:01:14 AMMay 14
to Kill Bill users mailing-list
Hi Tamil,

As I mentioned in my last update, the subscriptions being pinned to the catalog version at the time of creating the subscription is the intended behavior. It is mentioned here in the docs. The newer versions of the catalog do not impact existing subscriptions. The only way is to change the plan to the new variant.

What I was trying in the newer version of the catalog, was to move the fixed charge out of evergreen phase. So the existing subscription would move from evergreen phase to evergreen phase and will not re-trigger the fixed charge. I have tried the same again using these bash scripts https://github.com/KBbitsP/FileUploads/blob/main/reproduce_1019_original.sh and https://github.com/KBbitsP/FileUploads/blob/main/reproduce_1019_newCatalog.sh and it does not trigger the duplicate charge for me. Could you give it a try using these scripts.

Regards
Karan

TAMIL THENDRAL SENTHAMIZH

unread,
May 14, 2026, 11:37:02 AMMay 14
to Kill Bill users mailing-list

Hi Karan,

Thank you for the detailed scripts and the DISCOUNT phase suggestion. 
We tested it and it does prevent the duplicate FIXED charge — we truly appreciate 
the time you spent on this.

However, we noticed that both your test scripts (reproduce_1019_original.sh and 
reproduce_1019_newCatalog.sh) use DAILY billing period. Our production environment 
uses MONTHLY billing, and the behavior is quite different when the plan change 
happens mid-cycle.


THE ISSUE WITH MONTHLY BILLING:
-------------------------------

With DAILY billing, the remaining period after a plan change is less than a day, 
so any duplicate recurring charge is negligible or zero. But with MONTHLY billing, 
when the plan change happens days or weeks after subscription creation, we see a 
duplicate RECURRING charge for the remaining days — without a corresponding 
REPAIR_ADJ to offset it.

Example:

  Invoice 1 (Jan 5 - Subscription Created):
    RECURRING   $100    Jan 5 -> Feb 5    (already paid)

  Invoice 2 (Jan 20 - Plan Change to same plan):
    RECURRING    $50    Jan 20 -> Feb 5   (charged AGAIN for remaining days)

The customer has already paid for Jan 20 to Feb 5 as part of the original $100 
invoice. After the plan change, they get charged $50 again for the same period.

Total paid: $150 for a single month. This is a real duplicate charge.


REQUEST:
--------

Could you kindly try the same scenario with MONTHLY billing period, with the 
plan change happening 15-20 days after subscription creation? We believe you 
will observe the same duplicate recurring charge that we are seeing.


WHY PLAN CHANGE IS NOT ACCEPTABLE FOR THIS USE CASE:
----------------------------------------------------

We fully understand that subscriptions are pinned to their catalog version by 
design. However, the requirement here is very simple — we just want to make a 
newly added addon available to existing subscribers. We are not changing the 
price, product, or billing terms. Only the addon availability metadata changed 
in the new catalog version.

Forcing a plan change for this purpose causes real financial impact to customers 
in production, which is not acceptable.


FEATURE SUGGESTION FOR FUTURE VERSIONS:
----------------------------------------

We would like to respectfully suggest that Kill Bill consider supporting addon 
availability resolution from the latest catalog version for existing subscriptions 
in future releases.

This is a very common and critical requirement for SaaS platforms:

  - Products evolve over time
  - New addons are developed and need to be offered to ALL existing subscribers
  - Currently the only path is a plan change which triggers duplicate billing
  - A catalog metadata change (addon availability) should not require re-billing

The architecture already supports this in one place — getDryRunChangePlanStatus() 
in DefaultSubscriptionInternalApi correctly resolves the base product from the 
latest catalog version using catalog.versionForDate(requestedDate).

Extending the same fallback logic to AddonUtils.checkAddonCreationRights() would:

  - Be fully backward-compatible (only activates when original check fails)
  - Have zero billing impact (no plan change needed)
  - Allow platforms to iterate on their catalog without disrupting existing customers

We believe this flexibility is essential for real-world SaaS billing and would 
benefit the entire Kill Bill community. We are happy to contribute a 
backward-compatible PR if the team is open to reviewing it.

Thank you for your continued support and patience.

Best regards,
Tamil Thendral

karan bansal

unread,
May 18, 2026, 3:51:14 AMMay 18
to Kill Bill users mailing-list
Hi Tamil,

The initial catalog that you provided in the very first email was based on daily billing period. I will give the monthly billing period also a try. I am also discussing the requirement with the Devs team, if we can consider this feature. 

Regarding the client being charged again for the already paid period in the new plan : could you please confirm what endpoint/steps are you using. For example, cancelling the existing subscription and creating a new one for the new variant, OR changing the plan to the new variant. In both these cases, the system should generate a pro rated credit for the remaining days of the original subscription.

Regards
Karan
Reply all
Reply to author
Forward
0 new messages