single user pod // installer
Ricardo Wurmus
I would like to revive the discussion about a single user pod
installer for Diaspora. Prepare for brain farts.
These are things that I think of when I imagine a single user pod:
1 - can be installed from tarball + script
2 - should use a self-contained database like sqlite3
3 - should not need to have a resque server instance
4 - should run on an unprivileged port
5a - the core should run on as little as router (e.g. 32MB RAM +
400Mhz), the front-end on a desktop
5b - the whole thing runs on ARM (router / raspberry pi)
1+2 are trivial, 3 could require retries when a pod is unavailable or
not responding, 4 and 5 would require massive changes. 4 would require
a change to the webfinger stuff to talk to pods on the unprivileged
port when 80 is not available (meh). 5a seems impossible, but I feel
it's quite important to separate server stuff from front-end stuff. It
would be sweet if the inter-pod communication could be handled by a
small compiled program that runs on a router. A full-blown RoR
installation won't work in this scenario.
My questions:
Do these requirements describe a desirable outcome? If so, who would
be interested to join me in designing and coding a tiny, very
constrained daemon that listens to pod chatter, queues messages and
uses sqlite (or similar) as a database?
-- rekado
Dana Priesing
I'm not a coder so I wouldn't of any use technically, but I could lend a hand with the github wiki page construction/editing if/when the time comes.
But given how often there are updates to the alpha code right now, isn't this something that makes best sense to do once D* reaches beta stage?
Ricardo Wurmus
> this something that makes best sense to do once D* reaches beta stage?
Correct. This is the reason for the first point in the list:
> 1 - can be installed from tarball + script
Still, intra-pod communication has not changed much (at all?) in
months, so a project like this could be targeted now.
Dana Priesing
me know what I can do.
Best,
Dana Priesing
Tray Torrance
This sounds very interesting to me. Have you thought through any implementation details at this point?
-torrancew
--
You received this message because you are subscribed to the Google Groups "diaspora-dev" group.
To post to this group, send email to diaspo...@googlegroups.com.
To unsubscribe from this group, send email to diaspora-dev+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/diaspora-dev?hl=en.
Maxwell Salzberg
To unsubscribe from this group, send email to diaspora-dev...@googlegroups.com.
Tom Scott
Jason Lewis
I think it's a neat idea... I would like to point out that the constraints on hardware seem a bit draconian... Particularly in terms of RAM.
Phones nowadays have more processing power and RAM than that. Plus, just spinning up a single Ruby VM instance is ~40MB. That being said, I've run Rails apps on an old Sun Netra T1 with 400MHz SPARC processor and 512MB of RAM. You can get those refurbed for <$100. So I think coming up with a DVD install of Linux and pre-configured D* would do more for single user pods than optimizing for lowest-end hardware.
Jason
Raven24
On Feb 8, 11:19 pm, Jason Lewis <jasonlewi...@gmail.com> wrote:
> I think it's a neat idea... I would like to point out that the constraints
> on hardware seem a bit draconian... Particularly in terms of RAM.
possible target devices (which I find an excellent idea) the RAM and
CPU limits are not that unreasonable. But I guess in that case that
would be some kind of minimal daemon that only receives the pod
communication. It could make use of existing OSS libraries for ssl,
xml and sqlite - which very often already come with minimal linux
installations - then the filesize, memory and processing requirements
should no problem at all. (I'm talking about a compiled binary in,
like, C or C++)
I think there are several possibilities to implement such a system...
- "shared" database
The database storage could be directly attached to that 'minimal
device' (e.g. stored on a thumb drive, memory card or even usb hard
drive) or available via a network share. Then, the actual full-blown
Diaspora installation can run on any other machine with more
ressources. It would then be able to access the "shared" database,
with the advantage that the Diaspora server wouldn't have to run all
the time to stay reachable and up-to-date.
- communication cache
Another possible scenario is that the minimal daemon just caches the
incoming pod communication while the actual Diaspora installation is
not reachable (the Diaspora server wouldn't have to run all the time,
e.g. if it is installed on a PC that gets shut down over night). Then,
when the Server comes up again or requests the cached data, it can be
forwarded to it as if it were comming in live and everything is on par
with the current pod network status again.
> So I think coming up with a DVD
> install of Linux and pre-configured D* would do more for single user pods
> than optimizing for lowest-end hardware.
requires quite a lot of maintenance and, the way I see it, is only
possible with a working "setup" install procedure. In a similar way as
for a linux distribution, but for the Diaspora installation.
(See next section for thoughts about that...)
> > While a one-line install would be nice, I don't think we should install
> > Diaspora with a shell script. Rather, we should build off of the existing
> > Ruby systems for packaging and downloading libraries. Perhaps we could
> > install Diaspora by way of an executable Thor script…
using common Ruby/Rails technologies for deployment is certainly not a
bad technology choice.
On the other hand, a simple bash script that checks if the environment
is sane and then installs all the components of Diaspora would
probably lower the barrier for first contact.
RVM, for example, is a one-liner you can copy from the project
homepage and execute in your bash and end up with a working ruby
environment that needs no further setting up.
You wouldn't need ruby installed, because the script would detect/
install and run rvm which takes care of that. You don't need a lot of
explaining or experience with some specific deployment structure if
there are no further technologies involved other than copy&paste. You
can check the host environment: see if resque is running, if all the
packages are installed, specify sane default arguments... and if
something is missing explain the necessary steps in the error message.
All that to create a self-contained diaspora environment.
But granted, I think down the line a install script and a "gem
version" are definitely both desireable.
>
> > The install process would go as follows:
>
> > 1.) Download DIASPORA* from RubyGems: `gem install diaspora`
>
> > 2.) Install DIASPORA* from the binary: `diaspora install /path/to/diaspora
> > --branch=<name> --commit=<hash>`
>
> > 3.) Run DIASPORA* on Thin as well as all of its processes: `diaspora`
>
> > If you use Pow for local development, you can run `diaspora powify` to
> > processes, not Thin, when you run `diaspora`.
>
> > The binary would be located at script/diaspora, to conform with the Rails
> > folder structure.
>
> > - T
>
> > On Feb 8, 2012, at 1:19 PM, Maxwell Salzberg wrote:
>
> > Hey!
>
> > Cool idea. Ive always wanted to run 'rake newb' and have everything
> > magically work.
>
> > My wish list for this would be to get it working on one type
> > of environment really well first, then worrying about
> > cross compatibility stuffs. I'm sure I will get some flack for it, but I
> > do like the way rvm and homebrew make packaging things from tags as well as
> > GIT HEADs pretty easy.
>
> > Also worth noting, the federation protocol has not changed much, as we
> > want to get these new interactions working awesome, but then the federation
> > protocol v2(codename: K-Fed) should be rolling out shortly thereafter,
> > possibly as a separate service(subject to change).
>
> > -m
>
>
> >> rekado,
>
> >> This sounds very interesting to me. Have you thought through any
> >> implementation details at this point?
>
> >> -torrancew
>
Maxwell Salzberg
Ricardo Wurmus
1. make it easier to install/deploy Diaspora (for dev purposes or
running a traditional multi-user pod)
2. make it possible to deploy Diaspora to non-professional hardware
and join the network
What Max, Tom, and Jason said is aimed at (1). Though I realise that
this is important, my intent with this message was to look at (2).
I've repeatedly seen comments on D* expressing the wish to *really*
own one's data by running a simple pod as someone without access to a
VPS or dedicated server. I know that this is not something the core
team should spend time on and that new developer resources should not
be diverted away from contributing to squashing bugs in D*.
Still, I think that the code is reasonably well compartmentalised to
allow for something Raven24 and I proposed. I especially like the idea
of running a caching daemon on a router that handles webfinger
enquiries and caches posts until they can be committed to the database
on the desktop. I think this wouldn't be too difficult to implement as
we would hardly need the RoR stuff to change. A simple installer would
still be needed to get the desktop application running, but the
difficult part of proxying requests from the outside world would be
handled by the router daemon.
It's an additional program, compiled, no ruby, running on ARM under
very tough resource constraints. Anyone with a router running dd-wrt /
openwrt (e.g. Buffalo routers) should be able to install the daemon
and be done with it.
In this light, task 2 is contingent upon task 1. Let's get a simple
ruby installer ready, while I continue to think about the requirements
for the D* daemon in addition. Whoever wants to discuss requirements
for the daemon: post to #RouterDiaspora on D*.
Cool?
Now that this is out of the way, let's talk about the installer.
There really is only MacOS and Ubuntu (representative for any other
GNU/Linux distribution); I don't see the need for a VM image which
would require a lot of maintenance.
On both Mac and GNU/Linux we can use rubygems for handling most
dependencies. For those gems that require native extensions we need
distro packages as rubygems can pollute system library directories
which are difficult to clean up later.
How far should the installer magic go?
Should an installed pod come with a working minimal nginx configuration?
diaspora_noob
currently used for family and friends, while I figure out some RoR
basics. I'm no Ruby maven, but count me in for any future testing,
debugging, installs, possible doc write-ups, etc. An installer script
for Diaspora making deployment as easy as possible sounds pretty
exciting. As far as GNU/Linux flavors go, it might be good to have a
Debian installer, as it does serve as the basis for a number of widely
used distros: Ubuntu, Crunchbang, Linux Mint, etc.
arto
Ricardo Wurmus
with some code here:
http://github.com/rekado/diaspora-daemon
There's not much to see yet, but you may contribute your ideas on the
wiki or in the form of Haskell code.