Designing API's For Enterprise Data Integration

[API Dev]

I have questions around API Design. We want to expose tabular data (in other words RDBMS table records) through API's for data integration needs. The data in these tables could range from few thousands to millions of records. The api's should allow reading and adjusting records. I have a couple of questions related to this

1. 1. Ease Enterprise Data Integrations - What are the standards for designing such API's to ease enterprise application integrations? For e.g. connecting to ERP, BI, Finance, HCM type of systems. Would implementing a standard like OData be helpful?
   

2. 
   

3. 2. Real-time Data Integration through API's - How do I enable real time data integrations from my enterprise application? i.e. How should the API's be designed in order to enable real time data integrations with other applications?

Thanks !

[MattM]

In our experience with the API Academy, the most important thing you can do to guide your design is first think through what client interactions you will need to support with your APIs. If you just start with the data and offer come CRUD access, you are probably going to provide too much data and hurt usability for the developers who will be consuming your APIs. This is one of the downsides of OData, since it just creates a regurgitation of the data model that can lead to brittleness of the interface over time.

So I would start by answering these questions:

- Who is going to use the data?

- What tasks are they trying to complete in using this data?

- What data is needed to support each task?

This may sound very basic, but it will help you design useful and usable APIs, rather than creating a distributed mess.

Thanks, m@

[Peter Williams]

I second MattM's thoughts and approach. Exposing your relational data model (or even your application data model) via an HTTP API is a recipe for future pain and suffering on all kinds of fronts. Understanding what clients actually need to accomplish, and then creating a good solution to those problems is a much better way to design APIs.

Peter

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at https://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Erik Wilde]

hello.

On 2019-01-24 10:05, MattM wrote:
> In our experience with the API Academy <https://www.apiacademy.co/>, the

> most important thing you can do to guide your design is first think
> through what client interactions you will need to support with your
> APIs. If you just start with the data and offer come CRUD access, you
> are probably going to provide too much data and hurt usability for the
> developers who will be consuming your APIs. This is one of the downsides
> of OData, since it just creates a regurgitation of the data model that
> can lead to brittleness of the interface over time.

just to drive matt's very valid point home: if you directly expose a
database, you expose a very specific implementation of a data model, and
probably one that has all kinds of funky details in it that are a
side-effects of the implementation platform, how it was designed, and
how it evolved over time. trying to avoid this implementation-centric
view can be extremely valuable when it comes to the developer experience
(DX) of your API. so definitely do try to start from users, use cases,
any kind of information that is not about the implementation of the data
model, and then use this as the foundation for your API design.

cheers,

dret.

--
erik wilde | mailto:erik....@dret.net |
| http://dret.net/netdret |
| http://twitter.com/dret |

[API Dev]

I totally agree to these points and there is no second thought to it. The client requirements right now are to allow data integration with other enterprise software solutions and as an MVP this means we mainly provide an ability to pull and push data in the tables exposed by our application. We do have a "tables" concept in our software which is a layer over the RDBMS table. 

With this ability to pull and push data, clients want to synchronize the data set across various enterprise applications that they use for their needs. Since we aren't aware of what other enterprise applications the clients would have, we are thinking on how we can create standard api's that ease the integration (data pull and push) and also provide real time updates to the tables. 

Kindly share your thoughts. Thanks !

[API Dev]

@MattM, @Peter Williams and @dret - any further thoughts on this? Given the situation we are in, do you think we could do something to ease the integration effort without knowing the exact use cases that clients have?

Thanks.

[Erik Wilde]

hello.

On 2019-02-04 06:49, API Dev wrote:
> @MattM, @Peter Williams and @dret - any further thoughts on this? Given
> the situation we are in, do you think we could do something to ease the
> integration effort without knowing the exact use cases that clients have?

nothing further, just repeating: if you have zero ideas about users or
use cases, delay doing anything until you have first teams or at least
ideas what users and use cases look like.

from my experience, it is rather unlikely that anybody will have an
interest in gaining access to the implementation details of a data
model. if anybody does, i would still recommend to stay clear of this.

loose coupling is a vague term, but almost everybody agrees that it is a
good thing. being able to rip out and replace an implementation without
consumers having to know is one good indicator of loose coupling. i am
not quite sure what your "higher level table model" looks like, but
maybe you can test it against this constraint?

another useful thing to do might be to practice "API Archaeology" as we
describe it in our CAM book (https://twitter.com/thecambook). while you
might not have APIs yet, you very likely have existing integrations.
looking at why they exist and what they try to do might give you a
better understanding of actual integration needs, instead of exposing
too many details.

[Victor Olex]

CRUD API to a database when done right is a huge time saver and not one bit of hindrance. I would like to invite you to take a look at our product SlashDB, which does exactly that. You will have a read-write API in a matter of minutes. https://slashdb.com

[MattM]

Product pitches are generally not useful when it comes to design guidance (the purpose of this group). And I think we've elaborated in the posts above on what those "bits of hindrance" might be ;-)

m@