Developing with Blue vs individual services

53 views
Skip to first unread message

Paweł Nawrocki

unread,
Sep 23, 2016, 12:38:02 PM9/23/16
to OpenLMIS Dev
Hello everybody!

We are currently starting to use openlmis-blue as our development environment for usage of multiple services. Each service no longer links another instances to be booted along with it. Instead, we run openlmis-blue to have them all functional and have a look at how they communicate with each other.

The above sentences seem fine, however, there's a problem that now might not be too troublesome, but in future, when we'll have more and more services, it might become really painful for developers.

Right now, in our services, we have urls to our services hardcoded like "http://auth:8080". Of course, our goal would be to have them host-based. This means, that our in-app links for developers would look like, for example, "http://localhost/auth" (or even "http://localhost", but that's not relevant for this issue). This would perfectly work with openlmis-blue settings, as we have our proxy server configured for such routing. However, if any developer would want to run, for example, just referencedata and auth individually (from services' own docker-compose files), without bringing the whole Blue up, the services would no longer communicate correctly, as referencedata would be "http://localhost:8082", auth would be "http://localhost:8081", and referencedata would still call "http://localhost/auth" for communication, and that would of course fail constantly.

This means that a developer would never be able to run just a few services instead of whole openlmis-blue and have them working properly. Also, he would be able to debug only if running singular services, which would be actually useless, because nearly everything needs to contact auth (besides other inter-service communication), and that wouldn't work. So, I was thinking about a solution to this problem, and there seem to be 2 ways of solving it.

1. Make this similar to blue. We would have something like a basic docker-compose file which runs all the stuff that is essential to our system, like nginx-proxy, consul, db, etc. Then, after running it, developer would just run docker-compose files for individual services (by the way, they would no longer need to have db and log included), and those services would be also automatically registered and being routed to, as they normally would on openlmis-blue. So overall, this would be similar to just running openlmis-blue, but developers would have choice of what they want to run, and more importantly, they could run each service on debug mode.

2. Have a separate configuration for this case. We would have another set of configuration files, stored for example on openlims-config. It would probably consist of alternative docker-compose files, that would run each service under specified port, and sort of envronment file for each, that would contain this service-url mappings that would be used by other services to communicate with it. This would, hovewer, involve additional work for keeping all the references up to date and running them separately.

Let me know which direction you think we should follow.

Regards,
Paweł

Nick Reid

unread,
Sep 23, 2016, 3:17:44 PM9/23/16
to Paweł Nawrocki, OpenLMIS Dev

I like option (1) -- that best reflects the model I assumed we were following


Keeping up a configuration file with URL paths seems complicated...


-- nick --


Nick Reid | nick...@villagereach.org
Friendly Neighborhood Spiderman, Information Systems Group


VillageReach Starting at the Last Mile
2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA
CELL: +1.510.410.0020
SKYPE: nickdotreid
www.villagereach.org


From: openlm...@googlegroups.com <openlm...@googlegroups.com> on behalf of Paweł Nawrocki <pnaw...@soldevelo.com>
Sent: Friday, September 23, 2016 9:38:02 AM
To: OpenLMIS Dev
Subject: [openlmis-dev] Developing with Blue vs individual services
 
--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.
To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/0ed91c23-cfad-4ca1-80f3-57921194cb56%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Darius Jazayeri

unread,
Sep 23, 2016, 3:26:48 PM9/23/16
to Nick Reid, Paweł Nawrocki, OpenLMIS Dev
I also think #1 is the way to go. There should be a way to fire up the "minimal platform", to which a dev (or implementation!) could add whichever other services they want.

-Darius

On Fri, Sep 23, 2016 at 12:17 PM, Nick Reid <nick...@villagereach.org> wrote:

I like option (1) -- that best reflects the model I assumed we were following


Keeping up a configuration file with URL paths seems complicated...


-- nick --


Friendly Neighborhood Spiderman, Information Systems Group


VillageReach Starting at the Last Mile
2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA
CELL: +1.510.410.0020
SKYPE: nickdotreid
www.villagereach.org


From: openlm...@googlegroups.com <openlm...@googlegroups.com> on behalf of Paweł Nawrocki <pnaw...@soldevelo.com>
Sent: Friday, September 23, 2016 9:38:02 AM
To: OpenLMIS Dev
Subject: [openlmis-dev] Developing with Blue vs individual services
 
Hello everybody!

We are currently starting to use openlmis-blue as our development environment for usage of multiple services. Each service no longer links another instances to be booted along with it. Instead, we run openlmis-blue to have them all functional and have a look at how they communicate with each other.

The above sentences seem fine, however, there's a problem that now might not be too troublesome, but in future, when we'll have more and more services, it might become really painful for developers.

Right now, in our services, we have urls to our services hardcoded like "http://auth:8080". Of course, our goal would be to have them host-based. This means, that our in-app links for developers would look like, for example, "http://localhost/auth" (or even "http://localhost", but that's not relevant for this issue). This would perfectly work with openlmis-blue settings, as we have our proxy server configured for such routing. However, if any developer would want to run, for example, just referencedata and auth individually (from services' own docker-compose files), without bringing the whole Blue up, the services would no longer communicate correctly, as referencedata would be "http://localhost:8082", auth would be "http://localhost:8081", and referencedata would still call "http://localhost/auth" for communication, and that would of course fail constantly.

This means that a developer would never be able to run just a few services instead of whole openlmis-blue and have them working properly. Also, he would be able to debug only if running singular services, which would be actually useless, because nearly everything needs to contact auth (besides other inter-service communication), and that wouldn't work. So, I was thinking about a solution to this problem, and there seem to be 2 ways of solving it.

1. Make this similar to blue. We would have something like a basic docker-compose file which runs all the stuff that is essential to our system, like nginx-proxy, consul, db, etc. Then, after running it, developer would just run docker-compose files for individual services (by the way, they would no longer need to have db and log included), and those services would be also automatically registered and being routed to, as they normally would on openlmis-blue. So overall, this would be similar to just running openlmis-blue, but developers would have choice of what they want to run, and more importantly, they could run each service on debug mode.

2. Have a separate configuration for this case. We would have another set of configuration files, stored for example on openlims-config. It would probably consist of alternative docker-compose files, that would run each service under specified port, and sort of envronment file for each, that would contain this service-url mappings that would be used by other services to communicate with it. This would, hovewer, involve additional work for keeping all the references up to date and running them separately.

Let me know which direction you think we should follow.

Regards,
Paweł

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/BY2PR02MB330C7D2348E06BFF37FC52E94C80%40BY2PR02MB330.namprd02.prod.outlook.com.

For more options, visit https://groups.google.com/d/optout.



--

Darius JazayeriPrincipal Architect - Global Health
ThoughtWorks

Paweł Nawrocki

unread,
Sep 26, 2016, 1:27:01 PM9/26/16
to OpenLMIS Dev, nick...@villagereach.org, pnaw...@soldevelo.com
Where do you think we should put such startup file? Blue, deployment or maybe another repository?


On Friday, 23 September 2016 21:26:48 UTC+2, djazayer wrote:
I also think #1 is the way to go. There should be a way to fire up the "minimal platform", to which a dev (or implementation!) could add whichever other services they want.

-Darius
On Fri, Sep 23, 2016 at 12:17 PM, Nick Reid <nick...@villagereach.org> wrote:

I like option (1) -- that best reflects the model I assumed we were following


Keeping up a configuration file with URL paths seems complicated...


-- nick --


Nick Reid | nick...@villagereach.org

Friendly Neighborhood Spiderman, Information Systems Group


VillageReach Starting at the Last Mile
2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA
CELL: +1.510.410.0020
SKYPE: nickdotreid
www.villagereach.org


From: openlm...@googlegroups.com <openlm...@googlegroups.com> on behalf of Paweł Nawrocki <pnaw...@soldevelo.com>
Sent: Friday, September 23, 2016 9:38:02 AM
To: OpenLMIS Dev
Subject: [openlmis-dev] Developing with Blue vs individual services
 
Hello everybody!

We are currently starting to use openlmis-blue as our development environment for usage of multiple services. Each service no longer links another instances to be booted along with it. Instead, we run openlmis-blue to have them all functional and have a look at how they communicate with each other.

The above sentences seem fine, however, there's a problem that now might not be too troublesome, but in future, when we'll have more and more services, it might become really painful for developers.

Right now, in our services, we have urls to our services hardcoded like "http://auth:8080". Of course, our goal would be to have them host-based. This means, that our in-app links for developers would look like, for example, "http://localhost/auth" (or even "http://localhost", but that's not relevant for this issue). This would perfectly work with openlmis-blue settings, as we have our proxy server configured for such routing. However, if any developer would want to run, for example, just referencedata and auth individually (from services' own docker-compose files), without bringing the whole Blue up, the services would no longer communicate correctly, as referencedata would be "http://localhost:8082", auth would be "http://localhost:8081", and referencedata would still call "http://localhost/auth" for communication, and that would of course fail constantly.

This means that a developer would never be able to run just a few services instead of whole openlmis-blue and have them working properly. Also, he would be able to debug only if running singular services, which would be actually useless, because nearly everything needs to contact auth (besides other inter-service communication), and that wouldn't work. So, I was thinking about a solution to this problem, and there seem to be 2 ways of solving it.

1. Make this similar to blue. We would have something like a basic docker-compose file which runs all the stuff that is essential to our system, like nginx-proxy, consul, db, etc. Then, after running it, developer would just run docker-compose files for individual services (by the way, they would no longer need to have db and log included), and those services would be also automatically registered and being routed to, as they normally would on openlmis-blue. So overall, this would be similar to just running openlmis-blue, but developers would have choice of what they want to run, and more importantly, they could run each service on debug mode.

2. Have a separate configuration for this case. We would have another set of configuration files, stored for example on openlims-config. It would probably consist of alternative docker-compose files, that would run each service under specified port, and sort of envronment file for each, that would contain this service-url mappings that would be used by other services to communicate with it. This would, hovewer, involve additional work for keeping all the references up to date and running them separately.

Let me know which direction you think we should follow.

Regards,
Paweł

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.
To post to this group, send email to openl...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.

Nick Reid

unread,
Sep 26, 2016, 4:17:11 PM9/26/16
to Paweł Nawrocki, OpenLMIS Dev, Josh Zamor

What do you mean by 'deployment'


I feel like there is a thin line between a OpenLMIS that is implemented/configured and OpenLMIS blue


Josh - do you have opinions on this thread?


Nick Reid | nick...@villagereach.org
Friendly Neighborhood Spiderman, Information Systems Group


VillageReach Starting at the Last Mile
2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA
CELL: +1.510.410.0020
SKYPE: nickdotreid
www.villagereach.org


Sent: Monday, September 26, 2016 10:27:01 AM
To: OpenLMIS Dev
Cc: Nick Reid; pnaw...@soldevelo.com
Subject: Re: [openlmis-dev] Developing with Blue vs individual services
 

Josh Zamor

unread,
Sep 29, 2016, 1:07:41 AM9/29/16
to Nick Reid, Paweł Nawrocki, OpenLMIS Dev
Hi Pawel (and Nick please see below),

I prefer Option 1 over option 2 as well.  However before we proceed with this I’d like to first review an example docker-compose that shows how we’d do this.  However before we do that I question weather this is needed for Beta scope.  At present, and for Beta, we’ll have Auth, Requisition, Reference Data, and Notification* Services.  As you pointed out, with so few services this doesn’t seem to warrant a solution yet for this deadline.

Nick, I’m thinking that if we prioritized anything like this for Beta, we might prioritize what we were discussing for your UI development, and leave co-development/debug of Java services till post-Beta.  Thoughts?

Best,
Josh

Sebastian Brudziński

unread,
Sep 29, 2016, 8:24:26 AM9/29/16
to openlm...@googlegroups.com, Nick Reid
Hello,

I just wanted to mention that I consider being able to debug the application with several services started and communicating with each other essential for the further development work and therefore would not leave that post-beta. Since the extraction of the reference data service we now have at least several places in the code that do such communication, moreover nearly everything requires auth. Finally we do also have a work on the UI in progress and I assume we may quickly run into situations like "why isn't this giving me expected response" and wanting to check what is actually happening in the code. Of course you can start single services and debug them one by one or add extensive logging but this is really inefficient and time-consuming and I'm afraid it may really slow down the actual development.

Having said that, I'm OK with having a complete and proper solution for post-beta if there's no other way, but we should at least have an immediate workaround or even a hack that I can do locally to start blue/several chosen services and debug them properly (with communication between services and UI).

Kind regards,
Sebastian Brudziński.

Josh Zamor

unread,
Sep 29, 2016, 11:57:41 AM9/29/16
to Sebastian Brudziński, OpenLMIS Dev, Nick Reid
Hi Sebastian,

I see that Pawel started a conversation in Slack with a proposal on this, we can continue the proposal discussion there, though I would suggest we tack that here to keep it consistent.

I hear you on the desire to be able to work on multiple services at once for convenience.  However I have some concerns:  

  • Achieving this in time for Beta - the proposed solution in Slack doesn’t:
    • allow you to work/debug in Reference Data and Requisition at the same time 
    • route requests through Nginx-proxy
  • I’d rather have a focus on writing tests following our testing strategy.
  • In our Service architecture we wanted to get to a place where the teams are developing their Services in parallel with development by other team’s working on their Service(s).  If we achieved this, then we wouldn’t expect a developer of team A to be opening Team B’s Service in debug mode as a normal thing.

I would not invest more time in debugging two services at this time.  I think it’s reasonable to proceed with using this approach to debug the Requisition UI that’s being ported over, with the Requisition Service endpoints.  Do we agree on that?
  
In the future I’d expect us to refocus in favor of design on endpoints using RAML, and ensuring we meet that contract with testing.

Best,
Josh


Paweł Gesek

unread,
Sep 30, 2016, 8:21:26 AM9/30/16
to openlm...@googlegroups.com

So is everyone OK with closing this ticket? Or are there any concerns?

Regards,
Paweł

Darius Jazayeri

unread,
Sep 30, 2016, 10:59:09 PM9/30/16
to Paweł Gesek, OpenLMIS Dev

I agree with Josh that (1) testing and (2) independent development of services are important.

So, debugging *two* services at a time means we are doing it wrong, and we should not assume we are going to need to do this.

-Darius (by phone)


Friendly Neighborhood Spiderman, Information Systems Group

VillageReach Starting at the Last Mile
2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA
CELL: +1.510.410.0020
SKYPE: nickdotreid
www.villagereach.org

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.
To post to this group, send email to openlmis-dev@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/57ED07F5.5020308%40soldevelo.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/779DE6F9-535D-4CC9-BF2D-CAA68274EA77%40villagereach.org.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/02e791d8-8142-62ba-2738-6f340af60257%40soldevelo.com.

Paweł Gesek

unread,
Oct 5, 2016, 9:36:53 AM10/5/16
to OpenLMIS Dev

I've closed the ticket.

I understand the idea of not needing this from our perspective, since it's not the perfect way of developing services. Something that will help stimulate this will be smaller tickets, more along the lines of services, rather than lines of end user features. But I am aware it may take time to get there and will not always be possible.

On the other side, from the perspective of someone who picks up OpenLmis and tries implementing something using it, debugging anything you want can be helpful - I understand that documentation and all that should remove this need in a perfect world, but the world is not always perfect. I am guilty  of debugging things in the past in order figure out what happens rather than going through docs, since it was faster. As far I understand this will be a real chore with OpenLmis, so I will probably be forced to read the docs instead.

So I would at least take some steps to make this more implementer friendly. But this is not for beta.

From my understanding Paweł came very close to allowing this multi debug - it's on branch https://github.com/OpenLMIS/openlmis-blue/compare/OLMIS-DEBUG for your reference.


Regards,
Paweł
Reply all
Reply to author
Forward
0 new messages