Best Method for Rapid Code / Test / Fix cycle with CloudI

[bruce kissinger]

I am experimenting with CloudI to potentially replace a legacy web application written with MySQL, Java, and PHP.  Since I am learning more about CloudI as I go along, I find myself in a rapid code / test / fix type of cycle.  Originally, my steps were:

1) Start CloudI
2) Remove the existing service from CloudI using the "services_remove" API command
3) Change some of the Erlang code that I'm working on and recompile it
4) Add the service to back to CloudI using the "services_add" API command
5) Test the service
6) Go to Step 2

What I found was that when I added the service back again (Step 4) and tested the service (Step 5), the old original code was still being executed.  It seems like when the service is removed (Step 2) that the Erlang module must still be loaded somewhere in memory.  The only steps that I have found to be effective are to shutdown Cloudi and restart it.  This seems to clear what ever is being cached.

Has anyone experienced this?  Does anyone have advice on the best way to perform a rapid code / test / fix cycle that does not involve restarting Cloudi?

Regards,

Bruce

[Michael Truog]

You are correct that you should not need to restart CloudI to update a service.  However, the solution is a little different depending on whether your internal (Erlang) service is using the service configuration option automatic_loading set to false.  You would need to be explicitly setting automatic_loading to false due to the default value being true (the only way it is set is within the service configuration options, i.e., the last element of the service configuration data that is a tuple list).  If you are basing your source code on the hello_world5 example, then you are likely setting automatic_loading to false, as shown at https://github.com/CloudI/CloudI/blob/develop/examples/hello_world5/apps/hello_world5/src/hello_world5_app.erl#L34-L65 .

If automatic_loading is true, a services_remove should be unloading the Erlang modules within the Erlang application.  However, all the modules the Erlang application provides must be listed in the modules entry within the Erlang application file (i.e., .app file).  If your service is a single module, not using an Erlang application file, only the service module should be unloaded.  I will check to make sure that is the case, since if that wasn't working that would be a bug.

If automatic_loading is false, you need to handle reloading the Erlang modules yourself.  So you could use "l(insert_module_name_here)." at the Erlang shell prompt on each module to load the new version you have just compiled, but there are simpler ways.  If you are using an Erlang application you could use "cloudi_x_reltool_util:application_remove(insert_application_name_here, infinity, [cloudi_core]).".  That is simulating what the automatic_loading option is doing when it is set to true, when it does a services_remove on an internal service that is an Erlang application.  The cloudi_core Erlang application is listed to prevent it from being unloaded despite it possibly being a dependency of your Erlang application, since the function is removing your Erlang applications and any of its dependencies which are no longer used once the removal occurs.

While the service is running you could use "l(insert_module_name_here)." at the Erlang shell prompt to load a new version of your internal service module (i.e., the Erlang module you have which implements the cloudi_service behavior) and that won't cause problems unless you have changed the data structures used (it is a little dirty).  To have the module updated automatically you could use the service configuration option 'reload' set to true (it default to false) and it will automatically do the equivalent of "l(insert_module_name_here)." when your internal service module changes.  The difference is that the 'reload' option will also handling updating any additional application modules if your internal service is an Erlang application (currently it does not try to reload the .app file, so adding modules would be something that currently would require restarting the service instance, and that it is safest to not have it reload the .app file in the future, so I don't see that changing).

One thing to keep in mind is that you need to make sure the compilation output has been added to the code search path of the Erlang VM (it is a common problem to forget to do that).  The CloudI Service API has a convenience function for that as 'code_path_add'.  If you always want the directory in the code search path, you just need to modify the /usr/local/etc/cloudi/vm.args file (assuming the default installation prefix of /usr/local/) below the existing entries as shown at https://github.com/CloudI/CloudI/blob/develop/src/rel/files/vm.args#L29-L33 .

Please note that if you do a services_add before a services_remove with the same service doing the same subscribe function calls, the service request traffic will naturally be load balanced between the old and new instance(s) before the services_remove completes, so that can simplify a switchover.

Regards,
Michael

Regards,

Bruce

--
You received this message because you are subscribed to the Google Groups "CloudI Questions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudi-questio...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.