Is changing the sort order of results, when no sort order was documented, a breaking change?

[Cooper Marcus]

Our API can provide a list of results - typically the list is large and the results are paginated. 

For the past few years, the sort order of the list has been alphabetical.

We've never documented any particular sort order - it was just "accidental" that we were returning the items in alphabetical order. 

We've found that for performance reasons, we need to change the order in which the items are returned - the items will soon sorted roughly by their creation date. 

Is this a breaking change? 

I'm pretty sure it isn't, as we never documented any particular sort order. But... we were accidentally consistent, for many years, and now we will break that consistency - have we inadvertently created a contract with our users regarding the sort order?

Assuming you agree that this isn't strictly a breaking change, would you nevertheless consider proactively notifying your users? Or is this something you'd just change, and let your Support colleagues handle any customer issues? (there are certainly pros and cons to both approaches ; )

Thanks, Cooper

[Eric Stein]

Does your current (old) iteration provide a query parameter to clients to let them specify the sort order? If clients had no sorting options before, then changing the only ordering available to them is definitely a breaking change. Any clients who want a consistent experience after your change will presumably need to add a query parameter - you are adding one for them, right?

I would argue that you had an implicit contract (default ordering = alphabetical) even if there was a query parameter available, but it's definitely less damaging because clients who felt strongly should have used it. If you did have such a query parameter, you could have made it mandatory and saved some pain, but that's water under the bridge.

You should unquestionably notify any API users well in advance of making the change, to give them a chance to update their codebases. I can't see any benefit to the business by surprising your clients in this fashion, and I can't see any good reason to make your support team's life harder.

Eric

[Cooper Marcus]

Good points Eric, thanks!

See below...

On Thu, Jul 7, 2016 at 8:43 AM, Eric Stein <er...@fulminatus.com> wrote:

Does your current (old) iteration provide a query parameter to clients to let them specify the sort order? If clients had no sorting options before, then changing the only ordering available to them is definitely a breaking change. Any clients who want a consistent experience after your change will presumably need to add a query parameter - you are adding one for them, right?

No query parameter to specify a sort order in the past, nor will one be added.

The sort order was not documented or specified in the past, nor would it be if we make this change - it'll just change.

To get all the results, a user would and will have to page through the results. To get only a subset of results, users could specify a filter on the name of the items - but the results would still have no specified order, and it might still require paging to get all the matching results.

Does this clarification adjust your thoughts on this matter?

 

I would argue that you had an implicit contract (default ordering = alphabetical) even if there was a query parameter available, but it's definitely less damaging because clients who felt strongly should have used it. If you did have such a query parameter, you could have made it mandatory and saved some pain, but that's water under the bridge.

Note that the default alphabetical ordering could only be ascertained by _using_ the API - nowhere in our documentation did we specify a sort order, and only after retrieving a variety of results could a user start to guess that the results will typically be returned in alphabetical order.

 

You should unquestionably notify any API users well in advance of making the change, to give them a chance to update their codebases. I can't see any benefit to the business by surprising your clients in this fashion, and I can't see any good reason to make your support team's life harder.

Eric

On Thursday, July 7, 2016 at 12:57:24 AM UTC, Cooper Marcus wrote:

Our API can provide a list of results - typically the list is large and the results are paginated. 

For the past few years, the sort order of the list has been alphabetical.

We've never documented any particular sort order - it was just "accidental" that we were returning the items in alphabetical order. 

We've found that for performance reasons, we need to change the order in which the items are returned - the items will soon sorted roughly by their creation date. 

Is this a breaking change? 

I'm pretty sure it isn't, as we never documented any particular sort order. But... we were accidentally consistent, for many years, and now we will break that consistency - have we inadvertently created a contract with our users regarding the sort order?

Assuming you agree that this isn't strictly a breaking change, would you nevertheless consider proactively notifying your users? Or is this something you'd just change, and let your Support colleagues handle any customer issues? (there are certainly pros and cons to both approaches ; )

Thanks, Cooper

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at https://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Ashwin Rao]

In my opinion, you are not breaking a contract, but it is up to you (and not your support team) to make sure that the clients' expectations are managed. I don't know your API, so I don't know if the order is important for the clients to function... if it not, then it should be a "nice-to-know" or an non-event. if your clients have business logic that depend on the sort-order, then not only do you need to manage the expectation, you need to also provide your customers some time to re-tool their consumer applications to use the new sort-order.

[Eric Stein]

On Thursday, July 7, 2016 at 10:04:31 PM UTC-4, Ashwin Rao wrote:

In my opinion, you are not breaking a contract, [..] if your clients have business logic that depend on the sort-order, [..] you need to also provide your customers some time to re-tool their consumer applications to use the new sort-order.

Maybe we're having a disagreement about terminology, but to me you just defined breaking a contract. If clients need to rewrite code to keep getting what they used to get, that's you breaking your contract with them. If your clients *can't* get what they used to, you've definitely broken the contract. That you never specified the order is not germane .. you gave them something consistent that they could rely on, gave them no control over the shape of the data they got, and didn't specifically warn them it might change.

Eric

[Jørn Wildt]

What do you know about your clients use cases? Is it important to get the list sortede in alphabetical order? Are clients always requesting pages large enough to see everything? Or do they rarely venture further than the first page?

If ordering *is* important for your clients, then, yes, I would say you are introducing a breaking change as they rely on users being able to page through the output in a sensible order.

If the typical use case is independent of the sort order then, no, I would not say you brake anything (for all practical purposes ...).

But what is the cost of making the change? For your company (including support)? For your clients? What would your company value the most? Are there any legal considerations?

Breaking or not is somewhat an irrelevant question. What matters are the consequences of which you have said nothing :-)

/Jørn

--

[Ashwin Rao]

Eric, I define contract is something that is explicitly stated... such as via documentation, API parameters, etc. So based on the definition, you are not necessarily breaking it if it was not specified explicitly.  But, I do agree with one of the comments after this when it was noted that if users have begun to use the behavior before a change, then you have to manage it. 

Ashwin Rao
9467 Lantern Way
Centerville, OH 45458
Tel: 937-885-0974
Cell: 937-985-2726
Fax: 707-832-0974
E-Mail: ashwin...@gmail.com
Linkedin: ashwingrao
facebook: ashwingrao
twitter: ashwingrao

--
You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/vsrbphwiHF0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

[Manu Whig]

Just out of curiosity, how can pagination be provided without sorting? That could result in inconsistent pagination results for repeated invocations of the same API. 

[Chris Mullins]

Yes. This would be a breaking change, as it would cause clients to improperly behave and likely break. 

The exception would be if your current implementation randomized the items on every call, just to make sure clients didn't rely on the sort order. 

You should always define the sort ordering, as other properties like pagination, skip, take, and so forth rely on it. 

Cheers,

Chris

[Cooper Marcus]

Follow-up:

I decided that the change in sort order was NOT breaking (despite some of the very much appreciated advice herein!). 

I nevertheless contacted the owners of all the accounts that had used the affected API endpoints and gave them a couple days notice about the change - I encourage them to share the news with any relevant colleagues, and I welcomed them to email me directly if they had any concerns about the change. 

I also posted the notification to our discussion forum https://discuss.newrelic.com/t/upcoming-changes-to-all-metric-names-api-endpoints/38993 

I got one email of concern (out of, very roughly, 2,000 accounts notified), and we quickly ascertained that the change would not, in fact, cause their integration to break. 

The change to our endpoints was made, and we've received zero complaints.

Cheers, Cooper

--

You received this message because you are subscribed to the Google Groups "API Craft" group.

To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.

[Ashwin Rao]

Congrats! Glad you could change the sort order without negative consequences!

Get Outlook for iOS

You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/vsrbphwiHF0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.