Django example

[Samuel Rose]

Hi all,

I am exploring integrating Cloudi with both Django Project, and https://zato.io/

I am trying to conceptualize with frameworks like these, where the best place will be to insert an integration.

I have been exploring middleware for Django, and still thinking about it for Zato

Any thoughts on integrations like this? 

I've also been exploring symphony php, Rails, and Phoenix Framework (elixir)

[Michael Truog]

Zato is written in Python and claims to provide SOA, but doesn't seem to provide microservices, due to the problems of concurrency in Python with the global interpreter lock. I really haven't looked at it much, but that is my impression from a glance.

The question you have asked is similar to the question "How do I break up my monolithic architecture source code into microservices architecture source code?". There is a book "Building Microservices" by Sam Newman (https://www.amazon.com/Building-Microservices-Designing-Fine-Grained-Systems/dp/1491950358/) that attempts to answer this question, though without focusing much on having minimal latency with microservices and instead cares more about making microservices based on the teams involved, etc. Also, current wisdom appears to believe that you put something into a container to make it a microservice, but a monolith in a container isn't a microservice.

If you took a Django python project, added an instance of the CloudI API, and ran it with CloudI, you would only be using CloudI as an application server. That means most of the CloudI features wouldn't be used, except for things like the initialization timeout, the termination timeout, and things related to how a restart occurs. You would need to have use of CloudI service requests to have scalability/fault-tolerance benefits and the ability to update without downtime.

The straight-forward approach would be to rely on cloudi_service_http_cowboy for incoming HTTP and have it sent into Python Django source code, where Python source code is not acting as the HTTP server, but only handling the incoming HTTP data. However, I haven't attempted to pursue that development, and I am not aware of other people doing that. It would also be possible to use Python Django to handle the incoming HTTP as it does outside of CloudI, and only use CloudI service requests to communicate with source code that needs to be reliable elsewhere, that may or may not be written in Python. CloudI helps you create a fault-tolerant system with any programming language and minimal latency/memory. You would only want to use CloudI if you need something better than a load balancer with a few machines for failover and it is uncommon for people to care about making reliable systems (see http://cloudi.org/faq.html#1_Messaging for more information about that). There is some
basic information about creating a CloudI service at http://cloudi.org/tutorials.html#guidelines_for_creating_a_cloudi_service without focusing on a specific programming language that should help provide more information.

Best Regards,
Michael

[Samuel Rose]

Thank you for the response. I hear all that you are saying here, and understand the purpose and intention.

My goal was more to achieve ways to integrate cloudi with existing software

But, it appears the sanest way is to create http microservices, and integrate in http service level.

So those monolithic apps can't really leverage cloudi internal API due to their monolithic design. But, they can communicate into and receive data from a cloudi network. If most http service transactions from monolithic app to cloudi were asynchronous both ways, the whole system likely would be very robust. One could use cloudi microservices for critical reliable system needs, and integrate the via API to existing monolithic app when required or desired.

Thanks for helping me think more clearly on this.

[Samuel Rose]

The part where you wrote:

"The straight-forward approach would be to rely on cloudi_service_http_cowboy for incoming HTTP and have it sent into Python Django source code, where Python source code is not acting as the HTTP server, but only handling the incoming HTTP data. "

This is what I had in mind. But I understand that it is basically outside of the scope of what CloudI is intended to do, and that is fair enough. 

I think I can see ways to leverage CloudI microservices where reliable systems are needed employing many different languages. And, using existing software integrated on an HTTP service level with the CloudI system

[Michael Truog]

On 08/13/2017 01:51 PM, Samuel Rose wrote:
> The part where you wrote:
>
> "The straight-forward approach would be to rely on cloudi_service_http_cowboy for incoming HTTP and have it sent into Python Django source code, where Python source code is not acting as the HTTP server, but only handling the incoming HTTP data. "
>
> This is what I had in mind. But I understand that it is basically outside of the scope of what CloudI is intended to do, and that is fair enough.

It isn't outside the scope of what CloudI is intended to do, but there currently isn't an example for Python/Django integration. Its possible that a WSGI feature could be added to cloudi_service_http_cowboy to make integration a bit simpler, but I haven't gotten a chance to pursue that.

>
> I think I can see ways to leverage CloudI microservices where reliable systems are needed employing many different languages. And, using existing software integrated on an HTTP service level with the CloudI system

Keep in mind you don't necessarily need to use multiple programming languages with CloudI, since there is often a desire in a software team to stick with one. However, it is beneficial to choose the language to minimize latency if the processing is latency sensitive, while considering the dependencies you need and the development time that the language requires for the task. Also, you don't necessarily need to use HTTP for CloudI service communication in CloudI service requests, since a service request can pass any binary data without caring about the contents. With the Python/Django situation you mentioned, it makes sense to handle the socket connection concurrency and quantity of HTTP data in Erlang with cloudi_service_http_cowboy because it automatically makes CloudI service requests from HTTP requests, but you shouldn't feel forced to use HTTP for all CloudI integration (some microservices literature attempts to convey that microservices require HTTP, and often people
will suggest that REST requires HTTP, but that isn't true).

Best Regards,
Michael