Development progress

[Fraser Scott]

Hi all,

All changes so far have been merged into the following main repo / branches:

 * agent / master

 * admin / development

 * controller / master

 * core / development

As you can see, the controller and agent repos still live in master, so next on the list would be to create a development branch I guess. Also, controller and agent have no tests and obviously don't have any meaning travis config.

I'll try to go through the list of things to do and will create / update github issues in case anyone has a few spare moments to tackle some.

The main task next is to actually write a meaningful agent and get the admin tool to generate remote nodes using the agent.

If anyone wants to help out with this specific feature, let me know and I'll start a discussion here outline the approach etc.

Cheers,

Fraser

[Athanasios Kostopoulos]

Hi all,

I will try to help (even though I am swamped at the moment with $DAYJOB). Feel free to kickstart the discussion.

Best!

Athanasios

--
You received this message because you are subscribed to the Google Groups "pki.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pki_io+un...@googlegroups.com.
To post to this group, send email to pki...@googlegroups.com.
Visit this group at https://groups.google.com/group/pki_io.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pki_io/54fe43b7-f140-47f9-bda5-230e595af19b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Fraser Scott]

Right, so here is a brief summary of the code layout etc.

There are four main repos that follow an MVC like structure: admin, agent, controller and core. The admin and agent code is like a view; it handles user CLI input and renders output to the user. It is also responsible for setting up logging and handling environment variables etc. The controller code holds most of the business logic and is shared between the admin and agent code. Some stuff would only be used by the admin, some by the agent, and some of it both. Controllers also generally need to use other controllers. All the controller code exists inside a single controller package. Finally there is the core code which holds all the packages that are independent of each other and could potentially be used for other unrelated projects. Generally speaking, only the controllers should be using core packages.

There is one particularly interesting controller, namely the Environment. This is used to keep state between different actions and is available across multiple controllers. It holds access to the logging so that the controller is able to log what it's doing.

If you take a look at an admin source file such as run_node.go (can probably rename to just node.go now) you'll see it defining commands to be processed by mow.cli. The command functions then set up controller parameters based on the CLI switches. All validation is done by the controller. The command's action function creates a new application which is just a shortcut for setting up all the common stuff like logging, environment, config etc. The action function then calls the main entry point into the controller for that particular command. Finally, it handles the output. The controllers will generally pass back objects so the admin/agent code can decide how or what to display. For example, creating a new node would return the node that was created and the admin may decide to show the name and public keys.

The controllers have functions that a generally called by the agent/admin code, plus a ton of internal stuff (although still exported). The main functions do the validation then whatever business logic is required. At the comment the controllers completely lack unit tests, something that will need to be addressed asap.

The core stuff hasn't really changed, although there is now an ssh package to handle communication over SSH. It works by calling the openssh command rather than using a pure-Go implementation. This is so that the user can use their .ssh/config file to the full extent. The ssh package also needs tests, but the theory is that you establish a connection using ssh.Connect(), which actually sets up a multiplexed connection. This means you only have to authenticate once, then that connection can be used multiple times; e.g. first to upload some files, then to run some remote commands. The Close() function cleans up the multiplexing.

Now, getting to the more interesting agent stuff. The idea is that an admin can easily set up pki.io on a remote server, generating the private keys for the node on the remote host. This requires running some Go code remotely which is basically what the agent is for. Initially the agent will just have to be called directly, but eventually it will be able to run as a service that makes API calls to a central server. When the admin wants to set up a remote node, the use the --host switch when running 'node new'. This would provide the hosts hostname or IP address and the admin tool would know to try to establish a connection. Next, the admin tool would SSH in and "upload" the agent tool and an installer script. Then it runs the installer script, and finally it would run something like 'pki.io.d init' to create the remote node and generate the key pairs. Remote logging would be streamed back to the admin user of stderr, and json output would be returned to the admin code over stdout. I was hoping to refactor the code so that 'node new' would do the complete 3-way setup, but for now it makes sense to just keep doing it in stages.

So setting up a remote node would involve something like:

    # this would install the agent, run a local version of 'node init' and return the node registration json which is added to the org

    admin@localhost$ pki.io node new web01 --host 10.0.0.1 ...

    # this would process the new node registration

    admin@localhost$ pki.io org run

    # this would run 'node run' remotely and would finalise the node creation

    admin@localhost$ pki.io node run web01 --host 10.0.0.1  ...

I think next steps should be adding tests to ssh and the controllers for obvious reasons. The controller package doesn't work in travis yet, so that should also be fixed. I'll create a bunch of issues tonight for each of the different tests.

Cheers

[Fraser Scott]

Bunch of issues added to https://waffle.io/pki-io/controller and https://waffle.io/pki-io/core