thoughts for next steps in docker fun?

[jkn]

Hi all
showing my ignorance a bit here, and this is probably a 'how long is a piece of strong' question...

I have a lenovo USFF box running a few apps as docker containers - nextcloud, a kanban board, issue tracker, things like that. It all works, but managing it all via ssh-ing in and manipulation of the docker-compose.yaml files feels a bit ... fragile.

I'd like to improve this but am not very clear about the best way to go.
Do I just need something like a Docker Desktop, or Portainer, or would I benefit from eg. running Proxmox? the latter sounds like it might be an new interesting learning curve, but I don't necessarily want to go down too many rabbit holes just fro the sake of it, as it were...

Thanks for any thoughts
J^n

[Theo]

jkn <jkn...@nicorp.f9.co.uk> wrote:
> I have a lenovo USFF box running a few apps as docker containers -
> nextcloud, a kanban board, issue tracker, things like that. It all works,
> but managing it all via ssh-ing in and manipulation of the
> docker-compose.yaml files feels a bit ... fragile.
>
> I'd like to improve this but am not very clear about the best way to go.
> Do I just need something like a Docker Desktop, or Portainer, or would I
> benefit from eg. running Proxmox? the latter sounds like it might be an
> new interesting learning curve, but I don't necessarily want to go down
> too many rabbit holes just fro the sake of it, as it were...

I think Docker Desktop is more for developing with Docker on your
Windows/Mac.

I suppose it depends what you want: is this a dedicated box just for such
services? In that case Proxmox might fit, since it's a dedicated OS. It'll
run VMs as well as LXC containers. It looks like Proxmox doesn't support
Docker natively, but you can run Docker inside a VM.

Portainer is more like a web UI on Docker(-Compose) command line, which you
don't need a dedicated machine for. I installed Portainer and it adopted my
previous Compose setups automatically (although in a limited mode compared
with creating them in Portainer)

So you could end up with Portainer inaide a VM on Proxmox.

There is also atuff with Kubenetes, but even the 'micro' instances I found
too complicated for a single machine - don't really make sense unless you
have several machines to deploy to.

VMs take more memory than containers, so depends if you are limited by that.
Proxmox can run LXC containers, but Docker inside LXC has complications (I
must try sometime).

Really depends what your objectives are, and how much extra complication you
want to deal with.

If you have a spare machine, maybe experiment and see if any if them are the
right shape for what you want?

Theo

[Pancho]

On 08/12/2023 18:45, jkn wrote:
> Hi all
> showing my ignorance a bit here, and this is probably a 'how long is a piece of strong' question...
>
> I have a lenovo USFF box running a few apps as docker containers - nextcloud, a kanban board, issue tracker, things like that. It all works, but managing it all via ssh-ing in and manipulation of the docker-compose.yaml files feels a bit ... fragile.
>

I like docker compose. If you use git/SCC to maintain the YAML files, I
don't see that it is fragile, for non-industrial use.

You don't need to ssh, you can use a DOCKER-HOST environment variable to
access Docker containers on a remote machine. I typically work on a
Windows PC and run Docker containers on a Raspberry Pi. To some extent I
need to, because I run my git server in a docker container, and you
cannot access a docker macvlan IP address from the host machine. I use a
macvlan network for my git container so that I can keep the standard ssh
port.

So endless, nonsensical, bollocks to learn :-).

> I'd like to improve this but am not very clear about the best way to go.
> Do I just need something like a Docker Desktop, or Portainer, or would I benefit from eg. running Proxmox? the latter sounds like it might be an new interesting learning curve, but I don't necessarily want to go down too many rabbit holes just fro the sake of it, as it were...

Orchestration isn't really helpful at home, unless you specifically
want to learn it. Docker compose and bash scripts are generally enough.

Since we now have Docker, Linux available via WSL2, and I have Linux on
a Raspberry Pi (actually 4 rpis) I don't have any real need for virtual
machines. About 10 years ago, my interest in VMs plummeted. I suppose
WSL2 is kind of a virtual machine, but I don't think about it much.

[Theo]

[repost, as the first one didn't appear]

[Andy Burns]

Theo wrote:

> [repost, as the first one didn't appear]

Did appear, yesterday 12:21

[jkn]

On Saturday, December 9, 2023 at 12:21:41 PM UTC, Theo wrote:
> jkn <jkn...@nicorp.f9.co.uk> wrote:
> > I have a lenovo USFF box running a few apps as docker containers -
> > nextcloud, a kanban board, issue tracker, things like that. It all works,
> > but managing it all via ssh-ing in and manipulation of the
> > docker-compose.yaml files feels a bit ... fragile.
> >
> > I'd like to improve this but am not very clear about the best way to go.
> > Do I just need something like a Docker Desktop, or Portainer, or would I
> > benefit from eg. running Proxmox? the latter sounds like it might be an
> > new interesting learning curve, but I don't necessarily want to go down
> > too many rabbit holes just fro the sake of it, as it were...
> I think Docker Desktop is more for developing with Docker on your
> Windows/Mac.
>
> I suppose it depends what you want: is this a dedicated box just for such
> services? In that case Proxmox might fit, since it's a dedicated OS. It'll
> run VMs as well as LXC containers. It looks like Proxmox doesn't support
> Docker natively, but you can run Docker inside a VM.

Yes; one of my followup questions to self is whether I can get "the services
I want" as LXC containers rather than Docker ones. That would give me
another learning curve to climb ;-)

> Portainer is more like a web UI on Docker(-Compose) command line, which you
> don't need a dedicated machine for. I installed Portainer and it adopted my
> previous Compose setups automatically (although in a limited mode compared
> with creating them in Portainer)
>
> So you could end up with Portainer inaide a VM on Proxmox.

Hmm, OK. So a bit like what Pancho suggests below, I think.

>
> There is also atuff with Kubenetes, but even the 'micro' instances I found
> too complicated for a single machine - don't really make sense unless you
> have several machines to deploy to.

Yes, I think Kubernetes is not relevant for my purposes.

> VMs take more memory than containers, so depends if you are limited by that.
> Proxmox can run LXC containers, but Docker inside LXC has complications (I
> must try sometime).
>
> Really depends what your objectives are, and how much extra complication you
> want to deal with.
>

Well, quite ;-)

> If you have a spare machine, maybe experiment and see if any if them are the
> right shape for what you want?

Well, quite... it's just that I don't want to break what I have running whilst I
experiment. I probably should decide on a few 'new' test services and experiment
with those on a spare box, as you suggest. I do have another small Lenovo Tiny ...
it might be an excuse to buy another, I do quite like them...

Thanks a lot
J^n

[jkn]

On Saturday, December 9, 2023 at 12:54:32 PM UTC, Pancho wrote:
> On 08/12/2023 18:45, jkn wrote:
> > Hi all
> > showing my ignorance a bit here, and this is probably a 'how long is a piece of strong' question...
> >
> > I have a lenovo USFF box running a few apps as docker containers - nextcloud, a kanban board, issue tracker, things like that. It all works, but managing it all via ssh-ing in and manipulation of the docker-compose.yaml files feels a bit ... fragile.
> >
> I like docker compose. If you use git/SCC to maintain the YAML files, I
> don't see that it is fragile, for non-industrial use.
>
> You don't need to ssh, you can use a DOCKER-HOST environment variable to
> access Docker containers on a remote machine. I typically work on a
> Windows PC and run Docker containers on a Raspberry Pi. To some extent I
> need to, because I run my git server in a docker container, and you
> cannot access a docker macvlan IP address from the host machine. I use a
> macvlan network for my git container so that I can keep the standard ssh
> port.
>

You are right of course that I could use git to maintain all my docker-compose
files. I was steering clear of that as I didn't want many git repos ... but I
appreciate there are ways to have a .git directory separate from the files
it maintains ... I will have a think about that.

I hadn't heard of the DOCKER-HOST environment variable, thanks for that.
The first reply in my google search said something like "it is rarely a good idea
to set DOCKER-HOST" ;-). I'm not very clear if there are then standardised ways of
using the HTTP API, or you have to roll your own for each service?

> So endless, nonsensical, bollocks to learn :-).
> > I'd like to improve this but am not very clear about the best way to go.
> > Do I just need something like a Docker Desktop, or Portainer, or would I benefit from eg. running Proxmox? the latter sounds like it might be an new interesting learning curve, but I don't necessarily want to go down too many rabbit holes just fro the sake of it, as it were...
> Orchestration isn't really helpful at home, unless you specifically
> want to learn it. Docker compose and bash scripts are generally enough.

Fair enough, thanks

[Pancho]

On 18/12/2023 20:57, jkn wrote:


> I hadn't heard of the DOCKER-HOST environment variable, thanks for that.
> The first reply in my google search said something like "it is rarely a good idea
> to set DOCKER-HOST" ;-). I'm not very clear if there are then standardised ways of
> using the HTTP API, or you have to roll your own for each service?
>

My apologies, DOCKER_HOST has been superseded (circa 2020-2021?) by the
"docker context" commands. "docker context" is clearly better, it caters
for many remote contexts, and is usable from the command line via the
--context argument.

Having the DOCKER_HOST environment variable set overrides docker context
operations, silently. Which I guess is why you are seeing warnings not
to use it. I will stop using DOCKER_HOST now, and use docker context
instead.

I dunno what you mean by HTTP API?

[jkn]

Hmm - I'm a bit confused now. IIUC then setting DOCKER-HOST allows you
to use the 'HTTP API' to access the docker daemon (eg from a remote machine).
So instead of ssh'ing, you can send requests (eg. via curl, or your browser) to
monitor and control your docker images. That all makes sense, but since you
were advocating the use of this I am surprised you are puzzled by the use of
the term 'HTTP API'?

'docker context' seems to be something different?

J^n

[Pancho]

On 23/12/2023 10:03, jkn wrote:
> On Saturday, December 23, 2023 at 12:40:56 AM UTC, Pancho wrote:
>> On 18/12/2023 20:57, jkn wrote:
>>
>>
>>> I hadn't heard of the DOCKER-HOST environment variable, thanks for that.
>>> The first reply in my google search said something like "it is rarely a good idea
>>> to set DOCKER-HOST" ;-). I'm not very clear if there are then standardised ways of
>>> using the HTTP API, or you have to roll your own for each service?
>>>
>> My apologies, DOCKER_HOST has been superseded (circa 2020-2021?) by the
>> "docker context" commands. "docker context" is clearly better, it caters
>> for many remote contexts, and is usable from the command line via the
>> --context argument.
>>
>> Having the DOCKER_HOST environment variable set overrides docker context
>> operations, silently. Which I guess is why you are seeing warnings not
>> to use it. I will stop using DOCKER_HOST now, and use docker context
>> instead.
>>
>> I dunno what you mean by HTTP API?
>
> Hmm - I'm a bit confused now. IIUC then setting DOCKER-HOST allows you
> to use the 'HTTP API' to access the docker daemon (eg from a remote machine).

I use docker commands from a bash shell, docker CLI.

e.g.

$ docker run hello-world
$ docker compose up

Under the hood these CLI commands use a REST API to communicate with the
docker server/daemon. As REST is HTTP based you can use REST directly. I
never have, and until you made this point I'd never thought much about
how docker performed interprocess communication.

> So instead of ssh'ing, you can send requests (eg. via curl, or your browser) to
> monitor and control your docker images. That all makes sense, but since you
> were advocating the use of this I am surprised you are puzzled by the use of
> the term 'HTTP API'?
>

Explained above.

> 'docker context' seems to be something different?
>

DOCKER_HOST redirects the REST call, and hence command line CLI to the
specified endpoint (HOST) server/daemon. For instance

old style
'''
$ DOCKER_HOST=tcp://docker-server.home:2375 docker run -it bash
'''


new context style
'''
$ context create docker-server --docker host=tcp://docker-server.home:2375
'''
then
'''
$ docker --context docker-server run -it bash
'''

These commanda are usning HTTP under the hood but I can interact with
docker using the CLI commands without worring about the HTTP/REST api