OpenAPI Explorer in TW

[ludwa6]

Catching up on TW news (busy times down here on the farm!), i was rewarded by yet another serendipitous find in this little gem from Mario, with an embedded link to the TiddlyWeb API Explorer -what looks on the face of it like just the solution to the biggest problem i've got sitting on my desk right now. 

The problem is: the farmOS system i use for operational record-keeping is undergoing a major upgrade to version 2.0; this presents a data migration challenge for me & the agronomist i'm working with that would be greatly facilitated by an API Explorer.  Good news is: farmOS has an OpenAPI that can be easily explored using this open-source farmOS.py library -by anyone with adequate python chops, that is.  If only that were me!  But really: wouldn't it be so much nicer to explore the API using something like that TiddlyWeb API Explorer -a python project, as it happens.

So following Mario's pointer, i reached out to developer Chris Dent, who confirmed: not only is interface written in TW Classic, it is moreover >7 years old now, and he has had nothing to do w/ it (or TW, for that matter) in the years since (typical story of funding having run out).  Finally: code being targeted to the TiddlyWeb API itself -which uses the same OpenAPI standard, but is a very different data model- it's a project that the developer has no capacity to take on right now. Still: source code is online and available for anyone to run with. 

All that being said: i'm just captivated by the notion of using TW5 as the ultimate OpenAPI Explorer.  In such a front end, one might not only browse, but also mark-up and annotate such a typically complex document, writing & sharing comments & scripts in a most digestible & reusable form. 

So while i believe the developer that this would be no trivial undertaking, i am sufficiently intrigued by the possibility that i have to tap this collected wisdom here just to ask: Do you think like me that TW5 would make a very good OpenAPI Explorer? Whether using any of this code or some other language, i'd just like to know if this is a likely prospect, or just my wishful thinking, over-estimating the synergy between these softwares.  ?

/walt

[Joshua Fontany]

Hi walt,

A french ebook company worked with Jeremy to develop plugins that allow you to dynamically render only the text that is "onscreen", and also allows you to highlight a section of text & leave annotations attached to that highlight.

See: https://groups.google.com/g/tiddlywiki/c/_VLufc4Svp8/m/jALzYZ09BAAJ

Have fun.

Best,
Joshua F

[ludwa6]

Yes @Joshua : that one did catch my eye -and, now that you mention it, i can see how much overlap there is between this particular problem space, and the one that TW eBooks app solves.  In both cases, it's about using TW to load and browse these long docs (whether eBook or API endpoint), and then leverage TW's magic powers of annotating, transcluding w/ backlinks, etc. 

In fact: seeing as how OpenAPI standard is delivering content in JSON format, which is native to TW core, this one should be a breeze for TW5 to handle, right?   After all: we're not talking here about full CRUD interactions w/ a db (i mean: Post, Get, Put, Delete) via API;  i just want to use it to browse my app's API, so i can understand & annotate it.  Perhaps my naive (i.e. non-dev) user perspective is overly simplistic, but still, the question remains: is this an ugly problem or a juicy opportunity?

Bottom line: If anyone here sees this as either an opportunity they can see themselves solving, or else a problem that they see as too hard for some technical reason, i would love to hear more about it!

/walt

[PMario]

On Thursday, July 15, 2021 at 6:10:22 PM UTC+2 ludwa6 wrote:

The problem is: the farmOS system i use for operational record-keeping is undergoing a major upgrade to version 2.0; this presents a data migration challenge for me & the agronomist i'm working with that would be greatly facilitated by an API Explorer.

I don't understand this. ... As far as I know, Drupal which is used as a backend for farmOS uses a database to store all the values. ... The "data migration" which is needed between different application versions should be done on the database level and not on the "user level".

There may be new or different API options, but they need to be documented, otherwise an API explorer can't be created, because nobody would know how the different URLs need to look like. ..

 

  Good news is: farmOS has an OpenAPI that can be easily explored using this open-source farmOS.py library -by anyone with adequate python chops, that is.  If only that were me!  But really: wouldn't it be so much nicer to explore the API using something like that TiddlyWeb API Explorer -a python project, as it happens.

There seems to be an farmOS.js, which would probably be easier to use with TW.

 

-mario

[PMario]

Hi Walt,

After I did send my last post, I did find the new docs for version 2 of farmOS. I can see your dilemma now.

The question is, do you want to explore the V2 API or the V1.x API. It seems they are fundamentally different.

-mario

[ludwa6]

@Mario: It's the v2 API of farmOS that i'm trying to understand right now, so that is the focus.  

As compared w/ the 1.x version, v2 is said to be more powerful -and, i am finding, more exacting in its requirements.  In trying to .GET data from both (all that i've done so far[1]), using the farmOS.py library, there are some interesting methods available for client 2.x -iterate, for example- but i've been having real trouble getting some of them to run (in fact, when i passed the traceback on my .iterate fail to developer, he admitted i may have found a bug).

[1] As a volunteer tester for the data migration program, all i've done so far is download .csv exports from those tables in the database most of interest to me, and try to spot any diffs in the downloads from v1.x and v2 instances, so i've been avoiding the PUT, POST and DELETE methods for now.  But i've got a bigger challenge, once this phase of testing is complete, to load a bunch of data from spreadsheets where we do our crop planning, which process will be greatly facilitated by API scripts... So i'm feeling the need to understand this API, what we can and can't do with it, just as quick as i can get up the learning curve!

SO @Mario: do you think as i do that an API Explorer in TW could be a useful tool in this pursuit? and, if so, then is it something that might be built w/ relative ease?

/walt

[PMario]

Hi,

>SO @Mario: do you think as i do that an API Explorer in TW could be a useful tool in this pursuit? and, if so, then is it something that might be built w/ relative ease?

I think, it would be a useful tool. .. The problem creating it from scratch with TW is, that it would be like "reverse engineering".  The programmers that create the API should be able to create an API explorer with ease. .. It may be a possible "side-product" of the API unit tests ... (just a guess!)

From my point of view the farmOS.py library is a user-level library already. It abstracts the API - URL building process away, so imo it can't be used to create an API explorer.

The info needed for an API explorer would come from https://github.com/farmOS/farmOS/blob/2.x/docs/development/api/changes.md#resource-endpoints ...

An API explorer would know the different "entity-type" and "bundle" names and replace them with some dropdowns to build a valid URL eg: /api/[entity-type]/[bundle]  would result in /api/log/activity and so on.

The different possible schemas may be seen at: https://github.com/farmOS/farmOS/blob/2.x/docs/development/api/index.md ... Since there is no link to a demo API, I don't know how this looks like.

-mario

[ludwa6]

Sorry @Mario, i forgot: to browse an instance of the v2 farmOS API, here's mine: https://vdl.farmos.dev/api/ 

Viewed in Firefox (tho not in Chrome or Safari, which fail to render this in human-readable form), one can pull out links to the available endponts, and then use those to browse details, tho not easily. 

Is this all the info that would be needed to build a UI with filters and dropdowns, etc. -like that TiddlyWeb API Explorer you shared?

/walt