Admin UI Spike and discussions around it

[Dipanjan Laha]

Hello,

I have been spiking an Admin UI for Wiremock using the "__admin" APIs as we would need a visual representation of the mocks and also a quick way to add/edit them. It's a simple Angular app with bootstrap.

 

I found that there were some admin ui code present in the repo which were abandoned quite sometime back. If you think this will be useful, we can discuss the best place to put this in the project. I can raise a PR once I am more or less done and we can take it from there.

This brings me to the next question. I observed that we can only add mappings dynamically and not edit/delete them (correct me if I am wrong). I was thinking of adding admin APIs to modify and delete the mappings which I could then use in the UI. However, for that I would need a way to uniquely identify the mapping with something like an ID or an UUID. Is there any concern around adding an UUID in the StubMapping and admin APIs to edit and delete them?

Cheers

Dip

[Tom Akehurst]

I'm not particularly fussed about the UI I started. An Angular app sounds like a better idea.

I'm fine in principle with GUIDs for mappings. Please make sure you raise the PR against the 2.x branch though.

[Dipanjan Laha]

Great. I will need a few admin APIs for this.

1. APIs to Edit and delete a mapping

2. A way to get the response body file content if the body file name is specified instead of the body. This would help me to display the response in the UI instead of just the body file name. For this I was thinking of adding an API which would serve me the file from the "__files". What do you think?

I will try and raise separate PRs for these.

[Dipanjan Laha]

Ignore my 2nd point, I did not notice that it's already there...

[Tom Akehurst]

One thing to consider when you're implementing the edit function - the ordering of the stub collection matters. Mostly recently inserted mappings take precedence, except where one has a lower priority number than the other. Editing a mapping could affect this ordering.

[Dipanjan Laha]

Good point, what do you think the correct behaviour should be? imho, I think edited mappings should maintain their precedence... in that case i'll just maintain their insertionIndex.

[Tom Akehurst]

It might be easiest not to allow changing of the priority at all. I can't think of a way to do it that won't cause surprise and annoyance - either you re-insert the stub, in which case it changes position relative to others of the same priority, or you leave it in place, which would just be incorrect.

[Dipanjan Laha]

Here's what I have done  (raised a PR a while back : https://github.com/tomakehurst/wiremock/pull/306). I copy the insertion index from the older mapping and add the edited stub through a newly added replace method in SortedConcurrentMappingSet which doesn't touch the insertion index at all. So if the priority is not changed, it stays where ever it was. But becomes slightly tricky when the priority is changed, but it's not that bad as the insertionIndex remains the same. 

I am working on the UI, so should be able to have something simple pretty soon.

Let me know what you think of the PR as well :)

[Dipanjan Laha]

Hey Tom, I have build a basic UI, needs quite a bit of more work, but works for the basic use cases. I can't raise a PR for this as this needs the edit API of my earlier PR. However, if you want to take a look, check out the "admin-ui" branch on my fork https://github.com/dlaha21/wiremock. I have mapped it to /__admin_ui . Let me know what you thing. 

[Sam Edwards]

Hey Dip,

I checked out your fork and your UI is pretty awesome for adding/viewing and editing mocks.  It looks like it hasn't been touched in a while.  Do you have any plans to continue work on it?  I've been interested in a way to view mocks in a user-friendly way, and this was a great start.

Cheers,

Sam

[Dipanjan Laha]

Hi Sam,

Thanks for checking it out :)

Yes, I do have plans to take it forward. I have fixed a few bugs but haven't pushed it, so it's a bit out of date. As for the plan, here it is

1. Add an unique identifier and a way to edit the mocks. This has been recently merged by Tom.

2. Add the ability to persist the edited mapping onto the saved mapping json files. Without this the mappings edited via the ui would be transient. I plan to start work on this, Tom had some ideas around the best way to do this. 

3. The UI is pretty much there functionally. We can add it in with an necessary modifications when step 2 is done. The UI can be there without step 2 as well, just that the edited mappings won't be persisted permanently. 

Tom, would love to hear your thoughts as well.

~Dip

[Tom Akehurst]

The edit stuff is in there now, so we should be good on that front.

One option that would enable recordings to be edited would be to use the mapping's UUID (or part of it) in the filenames (in place of the short random string currently inserted to avoid clashes) so that they can easily be located and updated.

[Sam Edwards]

UUID: I like the idea of naming the mock files after the UUID in record mode and then for easy updating.  It is kinda nice to have part of the URI in the file name to distinguish between a lot of requests, but if the UUID was a prefix, it doesn't sound like that would be outrageous.

UI Component: As far as this whole admin_ui editor thing goes, there are two use cases I can think of.  There is definitely the pre-bundled use case where it's embedded within wire_mock and I think that's very powerful, but I could see that same webapp (html/js/css) being packaged as a standalone app.  It could be pointed to any WireMock server admin endpoint and using it that way.  This could be distributed as a Chrome App or something just as Dip has it now, but with a base url input/config.

(Idea) Cloud/remote hosted WireMock mappings:  This would replace the local __files and mappings directories (which I can't figure out how to deal with on Android) with a persistent cloud/remote backend.  You could push a set of recordings to the cloud, pull in a set of recordings and edit recordings (by adding scenarios, etc).  This way you could just tell WireMock to load up "Recorded Mocks 1" or "Recorded Mocks 2" and serve away, but we'd need some sort of protocol for loading in remote mappings.  You could also easily edit these mappings using Dip's UI and Edit PR.  This seems possible through the use of FilesSource in ResponseTransformer (haven't fully investigated).  I don't know how well this would work with the use of responding with recorded assets though.  It would either need to define virtual assets or just keep the entire response payload in the mapping.

[Dipanjan Laha]

Save Edits: I like the idea of UUIDs being a part of the filename, but that would make this feature backward incompatible. This won't work for anyone who has an existing set of saved mappings. It would be better if we can maintain a map of UUIDs and the filename. I'll give this a shot and see how it goes.

UI: We should definitely bundle this with wiremocks. This can be distributed as a standalone web-app (while developing this I was using a nodejs express server for a faster dev loop), but I don't quite see a point as every wiremock standalone would have it's own UI once we start bundling this with it. But I am easy with this being a standalone app as long as it's also bundled with wire mocks.

[Dipanjan Laha]

I have raised a PR for the Save Edits feature, https://github.com/tomakehurst/wiremock/pull/356

Let me know what you think..

[Tom Akehurst]

Thanks for this Dipanjan. I'll try and review this over the weekend.

[Tom Akehurst]

I've been mulling over some ideas around being able to store/manage stubs in the cloud for a while as it happens.

Serious question - would you considering paying for a SaaS service for managing and distributing (to mobile devices and servers) API mocks?

[Sam Edwards]

Dip, I have thought about it more and have to agree that the packaged into WireMock UI for editing mocks is more beneficial than a Chrome App.  I'm thinking about it this way right now probably because there is not way to currently edit mocks, and the ability to do it remotely on an existing WireMock instance from your desktop sounded appealing.  This is also the case on Android where we don't have great control over the filesystem and accessing/modifying files are a major pain.  The issue with that is that the edit APIs you added are only available in the latest version and no one would have that capability anyways, so the bundled UI sounds the best.

Also, I think for the majority of use cases, the ability to edit the files and mappings on the system and have it backwards compatible with current recordings is probably the best way to go for most users.  The UUID thing sounds nice, but it's a little more user friendly the way it is now, and if you're editing those mappings, you probably aren't getting dirty with wiring up of the API.

Finally, while this bundled admin UI is going to be great for a lot of use cases, I still see there being some way of remotely managing mocks as well.

[Dipanjan Laha]

Tom, What exactly do you want to mean by distributing mocks? Are you talking about local Wiremocks pulling down mocks from the cloud? Or do you mean mocks being served from the cloud.

Either way, I can imagine use cases where a particular service is mocked out in cloud and all the integration tests & multiple dev setups use the mocks from the cloud. This use case (or variations) may be specially handy for mocking out third party service such as say payment gateways. It may also become a critical piece in mobile dev loops. So companies may choose to pay for it, but don't quote me on this ;)

Sam, I think I understand the use case now. What if we also bundle a very simple nodejs app (along with the UI) which can be used to point the UI to any other wiremock instance. So we would have two ways of serving the same UI. 

1. Through the Wiremock Standalone itself, served alongside the mocks.
2. Through the nodejs server where we provide the path of the wiremock instance we want to point to. Something like  "node admin-ui.js http://somewhere.com/wiremock".  I mentioned nodejs because it's very easy and simple to create such an app with it.

Let me know what you guys think.

[Sam Edwards]

I'd just go with it bundled and forget about the node.js thing.  You could still do the Chrome app if you wanted to have one place to manage multiple instances, but I think the lighter the better in the core tool and a minimal UI to edit seems like what most people would need.  Any fancy features would be better distributed as a Chrome app so that you keep the core library lightweight.  It might even be something where two build artifacts are created, one with and one without the UI. But again that adds complexity.

The distributed mocks I'm thinking of would be remotely hosted and managed.  You would pull down a copy locally and serve.  There are a lot of ways to do this... First I would like to see an API endpoint under WireMock __admin to load remote mappings.  You request could be of two types.  One where you feed a full payload to your WireMock instance.  The other where you provide a URL containing the remote mappings for WireMock to pull down. I've proposed some formats for these mappings below.  The only issue is the referencing of the remote assets, but both of these examples below would handle that.

1) Inlined File Contents
{
  mappings: [{TYPICAL MAPPING HERE}],
  files: {"response1.json": "JSON HERE", "response2.png": "ENCODED_PNG"}
}

2) Remote File References
{
  mappings: [{TYPICAL MAPPING HERE}],
  filesBaseUrl: "http://myfilesgohere.com/files/

[Sam Edwards]

In conjunction with that last idea... You might want another __admin endpoint that will send POST request containing all the mappings to an arbitrary URL in format #1.  That way you could use WireMock in the typical manner (stubbing or recording), but could then save the state of that WireMock instance remotely to store and later edit the mappings.  Then you could pull them back down at a later point in time.  This is especially useful when dev servers are not available in a public URL and you've recorded traffic.

[Tom Akehurst]

Definitely the first of those two, but probably both since doing the second would be pretty straightforward.

[Sam Edwards]

Yeah, probably just sticking to format #1 for in and out would make things more straightforward.  Probably better not to complicate it with option #2, unless it becomes apparently necessary.  Why make it more complex if it doesn't need to be?

[Dipanjan Laha]

Hi Tom, did you find some time to take a look at the Save edit mapping PR? (https://github.com/tomakehurst/wiremock/pull/356)

[Tom Akehurst]

Apologies, I haven't. I'll try to this weekend.

[Dipanjan Laha]

Apologies for the delay in creating the Admin UI PR, since all the prerequisites have been merged. Let me know your thoughts

https://github.com/tomakehurst/wiremock/pull/428