Shared Docker machine?

[Lassi Kortela]

Apologies in advance for abusing the wg mailing list for an ad, but
would some Schemers here be interested in pooling funds for a
development server to run Docker containers? Several of us have noted
that Docker is liable to break easily and can slow down a computer, so
it's most convenient to run it on a dedicated machine.

We could get one of those $5/month VPSes and install Debian or another
Linux distro on it. I can quite easily set up such a server but it's
just as well if someone else does it. With luck, we could even get a
spare server for free from someone's university or employer.

[John Cowan]

I'd be willing to pay reasonable amounts for people to run Schemes on Amazon Fargate, which basically lets you run a Docker container for US$0.04 per CPU per hour and US$0.004 per GB of memory per hour.  I assume people won't want to leave their containers running for a long time, but will try something and then abandon it.  Does that sound like the correct use case?

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/7da037a2-cc83-a8d3-a209-9b7dec889375%40lassi.io.

[Marc Nieper-Wißkirchen]

I have (root) access to my server at my university, so, in principle,
it could be used. I would, however, probably be responsible for some
security aspects. The server is connected to the intranet so it could
be used to circumvent the university's firewall by a malicious user.

Thus, I would have to restrict access to individuals and I wouldn't be
able to open such a service to just everyone in the Scheme community.

As for paying and renting, I would order a VPS at my web hosting
company. It would cost me 5 € per month for 100 GB SSD and 8 GB of
RAM. I would just like to ask someone else to do the administration
because I would like to concentrate on other things. :)

Marc

> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CAD2gp_TkfoXqKf9DTfNetq4wxzCR7rLab9vsjiKzUHwF-JpMdA%40mail.gmail.com.

[Lassi Kortela]

> I'd be willing to pay reasonable amounts for people to run Schemes on
> Amazon Fargate, which basically lets you run a Docker container for
> US$0.04 per CPU per hour and US$0.004 per GB of memory per hour.  I
> assume people won't want to leave their containers running for a long
> time, but will try something and then abandon it.  Does that sound like
> the correct use case?

Thanks for tipping us off to Fargate. I didn't know it exists.

I suspect a generic Linux server would be easier for our use case, since
we'd be running `docker build` quite often to make test builds of
containers from a local git clone. So a traditional Linux user account
with the flexibility to clone repos, edit files, etc. would be useful.

Our requirements would be quite modest for a VPS - even a 20G disk would
probably be comfortable (10G for Linux and 10G for the containers and
miscellaneous work files). 1-2G RAM ought to be enough.

[Lassi Kortela]

> I have (root) access to my server at my university, so, in principle,
> it could be used. I would, however, probably be responsible for some
> security aspects. The server is connected to the intranet so it could
> be used to circumvent the university's firewall by a malicious user.
>
> Thus, I would have to restrict access to individuals and I wouldn't be
> able to open such a service to just everyone in the Scheme community.

Thanks for the offer. That sounds great otherwise, but I wouldn't be
comfortable trusting us with access to your intranet :) Is Scheme or
programming an official part of your research activities, and if so,
would your IT department be willing to provision a server shielded from
the intranet? Our resource consumption would be next to nothing on a
university's scale.

> As for paying and renting, I would order a VPS at my web hosting
> company. It would cost me 5 € per month for 100 GB SSD and 8 GB of
> RAM. I would just like to ask someone else to do the administration
> because I would like to concentrate on other things. :)

That sounds incredibly cheap for so much SSD and RAM. How did you find
an offer like that? If the server is shielded from any private networks,
we can install Debian on it and set up shared sudo access. Then whoever
has time can perform sysadmin duties.

Docker can be installed from Debian's standard repos (apt-get install
docker.io) so no special effort is required to admin it. Just `sudo
reboot` if/when it gets wedged :D This latter part is why it's best to
have a dedicated development server instead of running Docker on a
production server...

[Marc Nieper-Wißkirchen]

Am Mi., 20. Mai 2020 um 10:59 Uhr schrieb Lassi Kortela <la...@lassi.io>:

> > As for paying and renting, I would order a VPS at my web hosting
> > company. It would cost me 5 € per month for 100 GB SSD and 8 GB of
> > RAM. I would just like to ask someone else to do the administration
> > because I would like to concentrate on other things. :)
>
> That sounds incredibly cheap for so much SSD and RAM. How did you find
> an offer like that? If the server is shielded from any private networks,
> we can install Debian on it and set up shared sudo access. Then whoever
> has time can perform sysadmin duties.

I haven't checked the details so I may have missed a pitfall. It is
the "V-Server Linux V10" offer here:
https://www.strato.de/server/linux-vserver/. The offer is in German
(and Buzzword-Pseudo-English) but you are hopefully able to understand
it.

If you think this is a sold offer, I can order it for us. We may want
to get a nice DNS entry as well. ;) I could order
"www[dot]schemers[dot]net" along.

Marc

[Lassi Kortela]

We should establish a server pool at *.servers.scheme.org, and
eventually a mailing list to go with it where people can coordinate
access, funding and system administration.

We currently have a very modest VPS running at scheme.org with Arthur.
It can easily handle the initial scheme.org website and many subdomains.
Assorted development/build/docker servers will probably come in handy
later this year and the next.

As a BSD fan I'd like to donate some BSD servers to help with
portability testing, but can't afford to pay for a separate server for
each OS. It would be cheaper to get a slightly bigger server with
unrestricted access to the underlying virtualization hardware (there are
a few commercial hosts at any given time who offer that feature) and
install QEmu or Xen on it to do our own virtualization.

Thankfully, commercial CI services are getting better and better, faster
and with more features, so that takes a significant hosting load off us.

Still, it would be really cool if we had a build server that could run
unit tests on Scheme libraries under lots of implementations. That kind
of thing is not trivial to do with standard CI solutions. It would
probably require the Docker containers we have under schemers/, and some
kind of custom system to schedule jobs with them.

[Lassi Kortela]

>>> It would cost me 5 € per month for 100 GB SSD and 8 GB of RAM.
>>

>> That sounds incredibly cheap for so much SSD and RAM.
>

> I haven't checked the details so I may have missed a pitfall. It is
> the "V-Server Linux V10" offer here:
> https://www.strato.de/server/linux-vserver/. The offer is in German
> (and Buzzword-Pseudo-English) but you are hopefully able to understand
> it.
>
> If you think this is a sold offer, I can order it for us.

The page does indeed seem to claim that, and if you already have a VPS
there and it works fine, the company is probably legit. Nevertheless,
it's several times cheaper than the industry standard. For example,
Digital Ocean offers only 1G RAM and 25G SSD for 5/month.
<https://www.digitalocean.com/pricing/>. The VPS market is almost
saturated, so how can a >4x cheaper host in a first-world country be
commercially viable?

> We may want
> to get a nice DNS entry as well. ;) I could order
> "www[dot]schemers[dot]net" along.

We are now free to make arbitrary DNS entries under scheme.org. How
about it? :)

[Marc Nieper-Wißkirchen]

Am Mi., 20. Mai 2020 um 11:52 Uhr schrieb Lassi Kortela <la...@lassi.io>:
>
> >>> It would cost me 5 € per month for 100 GB SSD and 8 GB of RAM.
> >>
> >> That sounds incredibly cheap for so much SSD and RAM.
> >
> > I haven't checked the details so I may have missed a pitfall. It is
> > the "V-Server Linux V10" offer here:
> > https://www.strato.de/server/linux-vserver/. The offer is in German
> > (and Buzzword-Pseudo-English) but you are hopefully able to understand
> > it.
> >
> > If you think this is a sold offer, I can order it for us.
>
> The page does indeed seem to claim that, and if you already have a VPS
> there and it works fine, the company is probably legit. Nevertheless,

So far, I have never had any problems with them. They are the second
largest provider for shared hosting in Europe. Their servers are also
in Europe so hopefully outside the reach of the NSA for those who
care.

> it's several times cheaper than the industry standard. For example,
> Digital Ocean offers only 1G RAM and 25G SSD for 5/month.
> <https://www.digitalocean.com/pricing/>. The VPS market is almost
> saturated, so how can a >4x cheaper host in a first-world country be
> commercially viable?

I can't say. If it doesn't work, they have a 30-day-money-back
guarantee. So when I order it, we should be ready to do a lot of
testing.

> > We may want
> > to get a nice DNS entry as well. ;) I could order
> > "www[dot]schemers[dot]net" along.
>
> We are now free to make arbitrary DNS entries under scheme.org. How
> about it? :)

That's also a possibility.