Mostly for Bryan: Subversion, git, darcs; JSON, Yaml; Pointrel; distributed architectures
Paul D. Fernhout
better choice for SKDB than darcs btw based on popularity):
"The Future of Subversion"
http://tech.slashdot.org/article.pl?sid=08/05/09/1528250
"As the open source version control system Subversion nears its 1.5 release,
one of its developers asks, what is the project's future? On the one hand,
the number of public Subversion DAV servers is still growing quadratically.
On the other hand, open source developers are increasingly switching to
distributed version control systems like Git and Mercurial. Is there still a
need for centralized version control in some environments, or is Linus
Torvalds right that all who use it are 'ugly and stupid'?"
For the record, the more I think about Bryan's idea of using git (or
something similar) as a platform to do distributed work like with SKDB, the
more I like it. And I think as a technology approach, it can let me easily
make distributed Pointrel-based systems, as the first version of Pointrel
checked in to the code project merges easily as all changes are made at the
end. :-)
By the way, on yaml,
http://yaml.org/
I've liked the idea of JSON in the past, but not used it:
http://www.json.org/
"JSON (JavaScript Object Notation) is a lightweight data-interchange format.
It is easy for humans to read and write. It is easy for machines to parse
and generate. It is based on a subset of the JavaScript Programming
Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text
format that is completely language independent but uses conventions that are
familiar to programmers of the C-family of languages, including C, C++, C#,
Java, JavaScript, Perl, Python, and many others. These properties make JSON
an ideal data-interchange language."
A comparison somewhat in YAML's favor of the two:
http://en.wikipedia.org/wiki/JSON#YAML
"Both functionally and syntactically, JSON is effectively a subset of YAML."
By the way, YAML is a little like this proposal of mine (easier in some
ways, harder than others):
RFC: Lisp/Scheme with less parentheses through Python-like significant
indentation?
http://groups.google.com/group/comp.lang.lisp/msg/6d55f641ea635982
(Lisp code is essentially just a tree of symbols, numbers, other trees, etc.).
Still, anything that is encoded that way could be encoded in the Pointrel
system or RDF. :-) But not really by hand with the Pointrel system, since it
is more a library for managing triples then a way to specify them easily as
text. The intent was always to do that via a full programming language. I'm
kind of now thinking more and more of the value of simpler (safer) languages
like Yaml/JSON for specifications (the Pointrel web version used Python
which is unsafe, but it did different stuff with that). Still, there is
something to be said for a potentially sandboxed but full featured language
like Jython. Different strengths and weaknesses.
So, for a distributed OSCOMAK (or SKDB :-) this is a thought:
Content including stuff written in one or more computer languages
Pointrel
git & content interpreters
Various OSs
Each layer rests on the one below, and some of the content defines the GUI
and simulation tools, same as was done with the Pointrel web version Mike
played with, which pulled programs from the repository and evaluated them to
produce web pages. :-)
Of course, the useful core of git in this application perhaps could probably
be replaced with a few dozen pages of code of Python using Pointrel (not all
of git, just a simple merging process).
And for the content to include cross-platform programs easily, the JVM is nice.
So then, for easy portability of GUIs and simulations: :-)
Content (including Jython code and JSON/YAML/indented-Lisp/Semantic/etc.)
Pointrel
Bootstrap Jython program
Jython
JVM
In practice, this brings me back to the first stuff I checked into the code
project and Mike downloaded :-) But now thinking it could eventually benefit
from adding distributed versioning code which I am implicitly using
Subversion for now. But in my model, that is just more content. And also
that it should support "safer" content like JSON, YAML. and/or Semantic
MediaWiki-like tags.
Bryan is apparently implicitly looking at GUIs via ikiwiki generating
cross-platform HTML for a web browser. So he goes in a different direction;
ikiwiki requires Perl. Perhaps he is also really targeting GNU/Linux
implicitely, not Mac/Win/Linux as I was with the JVM? I'm not sure.
So he has:
Content, some in YAML
ikiwiki
Configuration files and dedicated SKDB application code?
Perl & git
GNU/Linux (but maybe others)
Maybe this is not correct as to what Bryan intends, but it is a first cut.
And so everyone can see the side-by-side comparison of software stacks.
But that *still* leaves the issue of what to encode. :-)
Which is the hard stuff at this point IMHO. :-)
So, right now I am still thinking mostly about content and metadata to put
in the centralized Semantic MediaWiki on the server, not infrastructure
(this note excepted :-).
This is all by way of saying I'm still paying attention to what Bryan is
saying (and not saying :-).
--Paul Fernhout
Bryan Bishop
> Just for reference on what Bryan talks about with git (which I think
> is a better choice for SKDB than darcs btw based on popularity):
> "The Future of Subversion"
> http://tech.slashdot.org/article.pl?sid=08/05/09/1528250
> "As the open source version control system Subversion nears its 1.5
> release, one of its developers asks, what is the project's future? On
> the one hand, the number of public Subversion DAV servers is still
> growing quadratically. On the other hand, open source developers are
> increasingly switching to distributed version control systems like
> Git and Mercurial. Is there still a need for centralized version
> control in some environments, or is Linus Torvalds right that all who
> use it are 'ugly and stupid'?"
>
> For the record, the more I think about Bryan's idea of using git (or
> something similar) as a platform to do distributed work like with
> SKDB, the more I like it. And I think as a technology approach, it
> can let me easily make distributed Pointrel-based systems, as the
> first version of Pointrel checked in to the code project merges
> easily as all changes are made at the end. :-)
I came back after finishing this email to comment here. You're still
taking the approach of asking when we'll get our hands down and
dirty .. but it's really quite simple. Just start a git repository,
dump content in it via wget, or via your own ~/cache/ (like mine (it's
approaching its third gigabyte of papers, plaintext, etc.)). That's
all. :-)
> Still, anything that is encoded that way could be encoded in the
> Pointrel system or RDF. :-) But not really by hand with the Pointrel
> system, since it is more a library for managing triples then a way to
> specify them easily as text. The intent was always to do that via a
> full programming language. I'm kind of now thinking more and more of
> the value of simpler (safer) languages like Yaml/JSON for
> specifications (the Pointrel web version used Python which is unsafe,
> but it did different stuff with that). Still, there is something to
> be said for a potentially sandboxed but full featured language like
> Jython. Different strengths and weaknesses.
Before I get contaminated by the rest of the email, let me jot out the
overall structure that I have had in the back of my head for a while
now:
== Physical layer ==
* hardware/peripherals
** microprocessors/controllers
** instruments (arms, rollers, "robots")
[[This is all standardized, see below].
* internet
== Digital layer, offsite wrt Avg Joe ==
* various 'distributions'
** debian, gentoo, redhat meets debfab, fabgen, fabhat ;-)
* main project websites, communities
** skdb, metarepo, oscomak
(BTW: metarepo sounds immediately obvious to me, FWIW).
Super offsite:
* amorphous network of projects on private servers
* crawlers that aggregate latest git repos into skdb
== Digital layer, onsite wrt Avg Joe ==
* Standardized distribution software
** debian, gentoo, etc. I like debian, so I'll be using it.
* agx-get -- download from the metarepo
** metadata & semantic data are via YAML
* autospec - put packages together, re: manufacturing
** GNU units
* material inventory management
* matter compiler
** this one is *tough* -- but I have some really *tough* guys helping me
out on this one. We can do it. A good challenge to have, IMHO.
* cupsd but for more than inkjet printer jobs
** job generator
*** to some extent already done by prewritten scripts from SKDB, python.
**** this is because of Py-YAML. Object serialization is good.
** job scheduler
** machinery orchestration
> So, for a distributed OSCOMAK (or SKDB :-) this is a thought:
>
> Content including stuff written in one or more computer languages
(1) Formalized content: plug and play ;-)
(2) Natural language content: have to read it and suggest some new
programs to make it do something interesting instead of just sitting
there and being dry.
> Pointrel
Package relationship management? Maybe, but this could also be done in
hardcode too, since the package maintainers would have to modify this
anyway.
> git & content interpreters
> Various OSs
OSes for the microprocessor hardware; and now 'fabhat' for the OS of the
entire manufacturing operating system [obviously still an OS; just an
extension of the drivers and so on, as unix was originally meant to be
used [like with mainframes]].
> Each layer rests on the one below, and some of the content defines
> the GUI and simulation tools, same as was done with the Pointrel web
> version Mike played with, which pulled programs from the repository
> and evaluated them to produce web pages. :-)
>
> Of course, the useful core of git in this application perhaps could
> probably be replaced with a few dozen pages of code of Python using
> Pointrel (not all of git, just a simple merging process).
Hm. That might be possible, but I'm skeptical, since merging in terms of
git pulling is not the same thing as merging different project branches
entirely, since functional specifications and requirements can change
to an extent that can't be automatically detected. Although I wouldn't
be ideologically opposed to programmers writing tools to help automate
mergers like that. I just think the reusability might be slim. Might be
wrong. Not a big deal, it seems like something that wouldn't take too
long to code up?
> And for the content to include cross-platform programs easily, the
> JVM is nice.
Yep, that's what I've been considering for the instrumentation problem.
Since you could have a mechanical arm that looks a certain way, and I
could have a mechanical arm that is something totally different, how
the hell do we make this all work as intended without forcing everybody
to ever use a non-exact instrument to rewrite all of the code that they
download? :-(
> So then, for easy portability of GUIs and simulations: :-)
> Content (including Jython code and
> JSON/YAML/indented-Lisp/Semantic/etc.) Pointrel
> Bootstrap Jython program
> Jython
> JVM
I guess. My experiences with java have never been good. Always kinda
clunky. Not the rapid experimentation that a scripting platform would
provide. I think I heard once that Sun came out with a java bytecode
platform, a physical microprocessor that ran the bytecode directly, and
even *then* there was performance issues. ;-)
Am I wrong?
> Bryan is apparently implicitly looking at GUIs via ikiwiki generating
> cross-platform HTML for a web browser. So he goes in a different
> direction; ikiwiki requires Perl. Perhaps he is also really targeting
> GNU/Linux implicitely, not Mac/Win/Linux as I was with the JVM? I'm
> not sure.
Hm. Well. First, the reason ikiwiki is interesting is because it uses
git under the hood, and the fact that it's web accessible means you get
to let all of the little fishies out there bite and see it from the
web. It's just a way to get them interested. When you download a git
repo, you are downloading what the wiki is, basically. So the
presentation hooks (ikiwiki is one of them) can easily be modified. I
think that an underlying YAML syntax is close enough to wiki syntax to
play around with, or even using ikiwiki plugins to show YAML as YAML,
and let separate wiki-formatted files be there for web presentation
purposes. Dunno. Frankly, I'm hoping graphviz will have some
visualization ideas [maybe to be integrated with pointrel and dia].
> So he has:
> Content, some in YAML
> ikiwiki
> Configuration files and dedicated SKDB application code?
Possibly - backup stuff, spidering/aggregation, that sort of thing.
> Perl & git
Not necessarily perl. Rewriting ikiwiki should be easy.
> GNU/Linux (but maybe others)
Re: my focus on GNU. I mentioned above the issue of OSes. Basically,
you're going to need an operating system for the manufacturing
installation. A distribution, like 'fabhat'. The apt-get model is there
for a reason. So Windows is largely out of the question (no way in hell
we want viruses here, but this isn't my main reason). I hear Mac OS X
has a linux kernel under the hood now. Does it have an apt-get equiv?
> But that *still* leaves the issue of what to encode. :-)
>
> Which is the hard stuff at this point IMHO. :-)
Infohoard into zip files and git repositories all of the information and
do rapid prototyping. Mike was nearly doing this, but 'solar energy'
isn't actually a package, it's more like a unit.And then you two come
along with your licensing and rights stuff ;-).
> So, right now I am still thinking mostly about content and metadata
> to put in the centralized Semantic MediaWiki on the server, not
> infrastructure (this note excepted :-).
Just put as much content as you can; for example, let's say that we were
going to make a microprocessor. Actually, this is an excellent example
thanks to OpenCores. So you'd go fetch all of their files, their
manuels, documentation, FAQs, news articles, etc. Then we'd package
them up with the metadata stuff. Put them into git repositories. They
are probably already in a versioning system. So maybe we can run some
cvs-to-git programs on them (if they aren't already using git).
Then there's the matter of programs related to the OpenCores project
themselves. This would include stuff like gEDA, and eventually the
manufacturing code that would tell the JVM (or its equivalent?) what to
do to make it [in general [subject to how discussions on the matter
compiler turn out]]. In python.
- Bryan
________________________________________
http://heybryan.org/
mike1937
> isn't actually a package, it's more like a unit.And then you two come
> along with your licensing and rights stuff ;-).
going back to it and having projects stem from it, kind of. Its more
of a tutorial for middle schoolers than anything (and a refresher
course for me at the same time; I like the ISRU articles' organization
better and will return to it). On wednesday my AP tests will be over
and I can finaly write some more.
> for a reason. So Windows is largely out of the question (no way in hell
gonna leave it to those who know what they're talking about.
>
> - Show quoted text -
Bryan Bishop
> > for a reason. So Windows is largely out of the question (no way in
> > hell
>
> Err... that doesn't sound good (I still have windows :-( ). But I'm
> gonna leave it to those who know what they're talking about.
Hey, next time *you* want to update your kernel, you just *try* asking
Microsoft nicely and getting something done. Just try. :-)
mike1937
question, do you mean for everyone using the finished product or just
the people making and maintaing the infrastructure? Or does it depend
on the definition of "finished product"?
Linux is better than windows. Linux has a spirit compatible with this
project. Linux is used by 1-8% of people.
Bryan Bishop
> just to be utterly clear... when you say windows is out of the
> question, do you mean for everyone using the finished product or just
> the people making and maintaing the infrastructure? Or does it depend
> on the definition of "finished product"?
If you want to insert data, Windows is fine. If you want to evaluate the
data and see if it clicks, Windows is fine. If you want to do some
manufacturing, you're out of luck. And diy manufacturing is a very
significant portion of the whole point.
mike1937
and accidently downloaded one of the full ISO's :-( I think. It's
still downloading. Do you happen to know a place where I can download
a liveCD easily? I've already got jigdo. Whenever I try googling it it
just comes up with places to make your own live cds, I just want to
burn one that I can boot off of to try it out.
mike1937
Paul D. Fernhout
have upgraded kernels and upgraded motherboards etc.
You might also consider going with Ubuntu (a flavor of Debian) if you are a
first timer with a desktop,
http://www.google.com/search?hl=en&q=debian+versus+ubuntu (not all agree)
Ubuntu is generally easier to get running usually on common hardware. Download:
http://www.ubuntu.com/getubuntu/download
I think they may now integrate both the live CD and the installer for Ubuntu.
--Paul Fernhout
Bryan Bishop
> Whie on the topic, I just attempted to make a liveCD to boot debian,
> and accidently downloaded one of the full ISO's :-( I think. It's
> still downloading. Do you happen to know a place where I can download
> a liveCD easily? I've already got jigdo. Whenever I try googling it
> it just comes up with places to make your own live cds, I just want
> to burn one that I can boot off of to try it out.
Get the debian netinstall CD (30 MB ISO). As for a livecd, go for ubuntu
or knoppix. Be careful if you're using a wireless card. Typically, I've
never been able to get them configured on a livecd installation, but
when I installed the actual debian system, things get better, as long
as you first do some searching over the net with your wireless card
number and check the ubuntu forums and so on.