DRY - Java code duplication

[Łukasz Lewczyński]

Hi,

I looked closer into our java code and it seems like we have a lot of code duplication between several services. I created a wiki page about this: Java code duplication

Could you take a look and describe your feelings about those changes?

Regards,

Lukasz

Łukasz Lewczyński
Software Developer
llewc...@soldevelo.com

SolDevelo Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

[Paweł Albecki]

Hi Łukasz,

I think that it looks promising, especially create communication libraries and openlmis-csv. One question: does moving spring dependent classes to shared libraries mean that all services will need to use the same version of Spring Boot?

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.
To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CAAdp53x%3DyNPt3Xy6yAsfP9-J-R7%2BY2cfJfedJYDtcLqtptZOmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Paweł Albecki
Software Developer
palb...@soldevelo.com

[Łukasz Lewczyński]

Hi Pawel,

It depends. If we decide to create openlmis-base library and:
* use it in all services in the same time then yes all services will use the same version of Spring Boot but only at the beginnnig.

* update services where there will be some tickets related with the given service then no because only some services will be use the library.

Independents of that after some time there will be probably few versions of the library. I can imagine that we will create a new version of library with new version of Spring Boot but not all services will use it.

Correct me if I am wrong but is it a problem that all core services will use the same version of Spring Boot? There still be possible to create a service without the library. This is only to avoid code duplication and bugs fixing in several places.

Regards,

Lukasz

Łukasz Lewczyński
Software Developer
llewc...@soldevelo.com

[Paweł Albecki]

Releasing new version for shared library when new Spring Boot version is released sounds good to me. Additionally I think using library will encourage component to always use the latest Spring Boot patch version with security updates in every service. 

It would be good to use same version by all services but as update to latest patch version should be easy and should always be done soon after release, update to minor or major version is not always trivial and we had some issues before e.g. in Auth service. But I believe it won't be big issue when shared library is properly versioned, to stay with old Spring Boot version.

[Josh Zamor]

Thanks Lukasz for your thoughts and analysis.

I don't think it's any secret that I'm very skeptical of shared libraries that OpenLMIS creates.  I think I can fairly break the shared libraries you propose down into two groups:

1. Those for RESTful representations between Services (commonly we refer to these as DTOs)
2. Utility classes (which have Spring dependencies)
   

For #1 I'd encourage you to get a copy if you don't have it, and read Sam Newman's Building Microservices.  I was able to find a PDF of a pre-release, and I'd direct you to the section on DRY and the Perils of Code Reuse in a Microservice World (page 33 in pdf).  I very much agree with the sentiments laid out in that section - and it's why I'd continue to encourage us to keep our service and UI templates up-to-date, and to discourage shared libraries.

For #2, we have a shared library which hasn't been updated in sometime.  My concern is mostly the same as I have laid out in #1 above.  What I'll add here is that I think it's a strength of our Service architecture to allow Services to be on different versions of Spring Boot.  Of the many reasons I have for this, perhaps the most tangible is Service size measured in Lines of Code (LOC).  When we were a monolith it was even more difficult than it is today to upgrade our version of Spring, as you had to inevitably address more corner cases that popup from a 100k+ LOC program being updated raher than one which only has 10k LOC.

In short I'm not very concerned about our level of duplication between Services.  We tend to duplicate utility libraries which have dependencies to Spring or RESTful representations.

What I'd recommend:

• We disable Sonar from reporting on DRY issues cross-service.  It'll report on DRY issues within a Service still.  This fits better with our architecture.
• Please obtain a copy (if you haven't already) of Sam Newman's Building Microservices and study it. 
  

Again thanks for pushing this forward.  I love that you're trying to find ways we can improve our quality.

Best,

Josh

[Łukasz Lewczyński]

Yes I know that the idea of creating additional libraries will be hard to present and implement because theoretically each of our core service is treated as independent service. I really hate when I have to do the same thing in several services - like modifying the DTO presentation in the reference data service and then looking into other services to update their DTO presentation - that's why I love to have a small libraries that contain classes that are used by several services. If we really don't want to have additional libraries we should disable DRY issues between services in the sonar.

Łukasz Lewczyński
Software Developer
llewc...@soldevelo.com

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/0178b8b3-ff71-406a-b9e9-20dd59d0b865%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Darius Jazayeri]

A microservice friendly pattern to avoid duplicating DTOs would be to let each service publish its own library of DTOs, on the release schedule of the service. Philosophically, this is not a shared library between multiple services, but rather it is maintained by a single service and consumed by others.

And of course the service is responsible to do whatever it deems appropriate for backwards compatibility if it changes its DTOs in a breaking way.

-Darius (by phone)

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CAAdp53yH1gR0g8pJ%2BKaNOsVuCak87FjSZ4-ptFgi93TqCDTBLg%40mail.gmail.com.

[Łukasz Lewczyński]

I read Sam Newman's Building Microservices book I now I see why we don't want to introduce shared/clients libraries. Briefly:

1. Shared libraries add new connection between services so if there will be any change or a bug in the shared library, we will have to update all related services. In the first look the issue looks like something small and easy to maintain but if there will be several versions of the given shared library like 1.0.0, 1.1.0, 1.2.4, 2.3.8 and we found out that there is a bug that exists from version 1.0.0 we will have to update all related services to version 2.3.9 because the bug fix will be presented only there. This provides additional problems because from version 1.0.0 to version 2.3.9 was a lot of changes that could break a microservice and we don't want this. Without shared libraries when we find a bug we will simple update single class in each services (yes I know that this sounds like We Enjoy Typing but only if you will think about services as a one system). Firstly this is fast because we only update single class not a library, secondly with this change we will not break other services.

2. I understood that there is no problem with client libraries but only if there are not in the same repository with the core code and it is created by another team. The problem here is that if we have access to core and client library codes we could move some logic out of service to the library. I think the good example here is expanded/limited pattern that we are currently trying to implement in our endpoints. With client libraries we probably move this pattern to the library because this would be easier for us.

I think some code duplication like DTO duplication will be solved when we will use expanded/limited pattern because we will not need full representation of data. Also I think if we modify few endpoints we could remove some DTO classes. In my opinion a good example here is hasRight endpoint in the reference data service which needs user and right IDs. Because of that each time we want to check if the given user has right to do some action we have to retrieve a full representation of User and Right objects only to retrieve their IDs. If the endpoint will accept user and right names we will not need to do that.

Łukasz Lewczyński
Software Developer
llewc...@soldevelo.com

[Łukasz Lewczyński]

One more think I disabled the Cross project duplication detection in the sonar.

Łukasz Lewczyński
Software Developer
llewc...@soldevelo.com

[Paweł Albecki]

Speaking of DRY, what is the status of OpenLMIS Report service? Some time ago we were asked not to use that service but I can see Version 1.0.0 was released recently. Jasper and report templates logic/configuration/endpoints in every service is good example of DRY violation.

Regard,

Paweł

[Josh Zamor]

The status of the OpenLMIS Report Service is it should be in maintenance mode, and I don’t expect to invest too much more into it at this point:

• Reporting means many things, however we do have our Reporting team working on a data warehouse solution.  It’s not a direct replacement at this point. however as it evolves, so too I expect will our requirements.
• The Reporting service could have new Reports added.  The architecture of the Reporting service never met the design, and at this point I don’t think we should fix that.  It is still a useful service to keep and maintain, and if we need a new Jasper report, lets add it there.  As I said we’ll evaluate its future as our data warehousing solution is developed.

Best,

Josh

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/3d6452e8-9173-47c6-94da-c87901b0c1b1%40googlegroups.com.