freebsd-hackers Digest, Vol 365, Issue 4
freebsd-hac...@freebsd.org
freebsd...@freebsd.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
or, via email, send a message with subject or body 'help' to
freebsd-hac...@freebsd.org
You can reach the person managing the list at
freebsd-ha...@freebsd.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of freebsd-hackers digest..."
Today's Topics:
1. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Ivan Voras)
2. Re: Another tool for updating /etc -- lua||other script
language bikeshed (John Baldwin)
3. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Ivan Voras)
4. Re: Another tool for updating /etc (Bakul Shah)
5. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Peter Jeremy)
6. Re: Another tool for updating /etc (John Baldwin)
7. Re: Another tool for updating /etc -- lua||other script
language bikeshed (John Baldwin)
8. Review/testing request: changing the detection of
data_addr/text_addr (Nathan Whitehorn)
9. Re: kenv - output needed (Atom Smasher)
10. Review/testing request: changing the arguments of
exec_setregs() (Nathan Whitehorn)
11. Re: Review/testing request: changing the arguments of
exec_setregs() (John Baldwin)
12. Re: Review/testing request: changing the detection of
data_addr/text_addr (John Baldwin)
13. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Dag-Erling Sm?rgrav)
14. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Garrett Cooper)
15. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Ivan Voras)
16. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Ivan Voras)
17. Re: Review/testing request: changing the detection of
data_addr/text_addr (Nathan Whitehorn)
18. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Robert Watson)
19. Re: Another tool for updating /etc -- lua||other script
language bikeshed (Ivan Voras)
20. Solved: How to slow down SATA to 1.5 GBit/s ? (Thomas Schmitt)
----------------------------------------------------------------------
Message: 1
Date: Wed, 24 Mar 2010 14:11:21 +0100
From: Ivan Voras <ivo...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: freebsd...@freebsd.org
Message-ID: <hod31p$qlc$1...@dough.gmane.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
On 03/23/10 16:08, John Baldwin wrote:
[snip - looks like a good utility, will probably use it instead of
mergemaster if it gets committed, like the idea about automated updates]
> To that end, I wrote a new tool that I think does a decent job of solving
> these goals.
Since the issue comes around very rarely, I assume there are not many
people who also get the shivers when they see a shell script (and then a
"posixy" /bin/sh shell script) more than a 100 lines long? :)
Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
language interpreter with a syntax that has evolved since the 1970-ies?
(with a side-glance to C that *has* evolved since the K&R style).
There was once Perl in base and even though I personally dislike Perl at
least it was a standard of sorts and guaranteed to be there if needed.
Now there are some fairly large chunks of code written in plain shell
script, like mergemaster, freebsd-update, portsup and adduser. I'm not
specifically against shell scripts but (which might just be my personal
opinion) I think they are even less maintainable in the long term than
Perl scripts. I also think the bus factor on good shell script
programmers must be pretty low.
As a possible alternative, or at least to learn about others' opinion on
the subject, I'd like to suggest Lua (http://www.lua.org/).
The reasons:
1) Very light-weight in terms of system integration. Basically, there
are one or two executables and libraries and the libraries can be
discarded if only the interpreter executable is needed and not the
ability to integrate it into C apps. No "libs directories" needed.
Written in C, designed to be easy to invoked from C (from which the
interpreter executable is built on).
This ability to integrate is important because it allows for some nifty
things like implementing "system" commands through C, e.g. a "sysctl()"
function as a wrapper for sysctl(3), or a "GEOM Class" class that wraps
control of GEOM objects.
The basic interpreter executable and the library are ~~ 150 kB each. The
/bin/sh executable is 130 kB.
2) Easy syntax, which even kind of resembles shell scripts in its flow.
Examples: http://lua-users.org/wiki/SampleCode . Unfortunately, its
error handling is not much better than plain C (no "exceptions"). It has
nice C-like formatting (e.g. "%4.2f, %d":format(3.14, 42)) and goodies
like lexical scoping, foreach and coroutines. It's default OOP
implementation is a bit specific (the "tables" and "metatables" system)
but usable.
3) MIT license. Friendly.
What would be gained? I guess what I'm trying to suggest is that a 3000
line shell script (like portsnap, as a random example) could be more
readable, easier to write and maintain were it a 3000 line Lua script.
Thoughts?
------------------------------
Message: 2
Date: Wed, 24 Mar 2010 10:02:56 -0400
From: John Baldwin <j...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: freebsd...@freebsd.org
Cc: Ivan Voras <ivo...@freebsd.org>
Message-ID: <20100324100...@freebsd.org>
Content-Type: Text/Plain; charset="utf-8"
On Wednesday 24 March 2010 9:11:21 am Ivan Voras wrote:
> On 03/23/10 16:08, John Baldwin wrote:
>
> [snip - looks like a good utility, will probably use it instead of
> mergemaster if it gets committed, like the idea about automated updates]
>
> > To that end, I wrote a new tool that I think does a decent job of solving
> > these goals.
>
> Since the issue comes around very rarely, I assume there are not many
> people who also get the shivers when they see a shell script (and then a
> "posixy" /bin/sh shell script) more than a 100 lines long? :)
>
> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
> language interpreter with a syntax that has evolved since the 1970-ies?
> (with a side-glance to C that *has* evolved since the K&R style).
"You can write Fortran in any language."
If there are specific things in specific scripts that are poorly commented or
implemented then I would work on fixing those. The same is true of the
mountain of C code in the tree. Rewriting them in a different language will
not automatically make them any better.
"Whatever language you write in, your task as a programmer is to do the best
you can with the tools at hand. A good programmer can overcome a poor language
or a clumsy operating system, but even a great programming environment will
not rescue a bad programmer." (Kernighan and Pike)
--
John Baldwin
------------------------------
Message: 3
Date: Wed, 24 Mar 2010 15:49:12 +0100
From: Ivan Voras <ivo...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: freebsd...@freebsd.org
Message-ID: <hod8p8$iuc$1...@dough.gmane.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
On 03/24/10 15:02, John Baldwin wrote:
> On Wednesday 24 March 2010 9:11:21 am Ivan Voras wrote:
>> On 03/23/10 16:08, John Baldwin wrote:
>>
>> [snip - looks like a good utility, will probably use it instead of
>> mergemaster if it gets committed, like the idea about automated updates]
>>
>>> To that end, I wrote a new tool that I think does a decent job of solving
>>> these goals.
>>
>> Since the issue comes around very rarely, I assume there are not many
>> people who also get the shivers when they see a shell script (and then a
>> "posixy" /bin/sh shell script) more than a 100 lines long? :)
>>
>> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
>> language interpreter with a syntax that has evolved since the 1970-ies?
>> (with a side-glance to C that *has* evolved since the K&R style).
>
> "You can write Fortran in any language."
I feel I should quote some saying from a holy book saying "Ah, but some
languages make it harder than the others!" but I don't know of any such :)
> If there are specific things in specific scripts that are poorly commented or
> implemented then I would work on fixing those. The same is true of the
> mountain of C code in the tree. Rewriting them in a different language will
> not automatically make them any better.
C is good enough. I'm after /bin/sh here.
> "Whatever language you write in, your task as a programmer is to do the best
> you can with the tools at hand. A good programmer can overcome a poor language
> or a clumsy operating system, but even a great programming environment will
> not rescue a bad programmer." (Kernighan and Pike)
I'll accept it.
------------------------------
Message: 4
Date: Wed, 24 Mar 2010 09:57:48 -0700
From: Bakul Shah <ba...@bitblocks.com>
Subject: Re: Another tool for updating /etc
To: John Baldwin <j...@freebsd.org>
Cc: hac...@freebsd.org
Message-ID: <201003241657...@mail.bitblocks.com>
On Tue, 23 Mar 2010 11:08:45 EDT John Baldwin <j...@freebsd.org> wrote:
> or 'cvs up'. If the local changes I made do not conflict, then just merge the
> changes automatically (e.g. enabling a serial console in /etc/ttys should not
> conflict with $FreeBSD$ changing when moving from 7.2 to 7.3).
>
> To that end, I wrote a new tool that I think does a decent job of solving
> these goals. It does not force you to read the diffs of any files updated in
> /etc, but there are other tools available for that. However, if you are ok
> with reading UPDATING, commit logs, and/or release notes for that sort of
> info, then this tool may work for you.
>
> It also has a nice feature in that you can generate a 'diff' of your current
> /etc tree against the "stock" tree allowing you to easily see what local
> changes you have made. I have already found this feature to be far more
> useful than I first expected.
>
> The UI is (hopefully) minimalist. The default output looks like the output of
> 'svn up' or 'cvs up'.
>
> If you'd like to give it a shot, you can find the script and manpage at
> http://www.FreeBSD.org/~jhb/etcupdate/ There is a README file that gives a
> brief overview and instructions on how to bootstrap the needed metadata before
> the first update. There is also an HTML version of the manpage.
Looks good!
But I wonder... why not build something like this around cvs?
Basically a three way merge is exactly what we want for /etc,
right? cvs because it is in the base system. I used to
maintain /etc changes in cvs and that was useful in keeping
track of configuration changes on shared machines.
------------------------------
Message: 5
Date: Thu, 25 Mar 2010 06:03:10 +1100
From: Peter Jeremy <peter...@acm.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: Ivan Voras <ivo...@freebsd.org>
Cc: freebsd...@freebsd.org
Message-ID: <20100324190...@server.vk2pj.dyndns.org>
Content-Type: text/plain; charset="us-ascii"
On 2010-Mar-24 14:11:21 +0100, Ivan Voras <ivo...@freebsd.org> wrote:
>Since the issue comes around very rarely, I assume there are not many
>people who also get the shivers when they see a shell script (and then a
>"posixy" /bin/sh shell script) more than a 100 lines long? :)
With the specific exception of GNU configure and related horrors, I
personally don't have anything against shell scripts. You can write
good or bad code in any language.
>Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
>language interpreter with a syntax that has evolved since the 1970-ies?
There's awk (though it's somewhat restricted in its abilities to do
anything more than text manipulation) but in principle, I agree. The
requirements as I see them are (in no particular order):
- BSD-compatible license
- must be compatible with buildworld (primarily, it must be possible
to cross-build)
- contains a critical mass of users in the FreeBSD developer (and
ideally committer) community
- language must be reasonably stable - will a script written today still
work correctly in (say) 5 years.
- must be acceptable to the vast majority of the user base (no religious
wars allowed)
>There was once Perl in base and even though I personally dislike Perl at
>least it was a standard of sorts and guaranteed to be there if needed.
It was removed because it didn't support cross-building (buildworld is
always done as a cross-build) and was evolving at a rate incompatible
with the base system.
>As a possible alternative, or at least to learn about others' opinion on
>the subject, I'd like to suggest Lua (http://www.lua.org/).
As someone who has never used Lua, how well does it meet the
requirements above?
--
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20100324/dbb4f7bc/attachment-0001.pgp
------------------------------
Message: 6
Date: Wed, 24 Mar 2010 13:36:39 -0400
From: John Baldwin <j...@freebsd.org>
Subject: Re: Another tool for updating /etc
To: Bakul Shah <ba...@bitblocks.com>
Cc: hac...@freebsd.org
Message-ID: <20100324133...@freebsd.org>
Content-Type: Text/Plain; charset="iso-8859-15"
On Wednesday 24 March 2010 12:57:48 pm Bakul Shah wrote:
> On Tue, 23 Mar 2010 11:08:45 EDT John Baldwin <j...@freebsd.org> wrote:
> > or 'cvs up'. If the local changes I made do not conflict, then just merge the
> > changes automatically (e.g. enabling a serial console in /etc/ttys should not
> > conflict with $FreeBSD$ changing when moving from 7.2 to 7.3).
> >
> > To that end, I wrote a new tool that I think does a decent job of solving
> > these goals. It does not force you to read the diffs of any files updated in
> > /etc, but there are other tools available for that. However, if you are ok
> > with reading UPDATING, commit logs, and/or release notes for that sort of
> > info, then this tool may work for you.
> >
> > It also has a nice feature in that you can generate a 'diff' of your current
> > /etc tree against the "stock" tree allowing you to easily see what local
> > changes you have made. I have already found this feature to be far more
> > useful than I first expected.
> >
> > The UI is (hopefully) minimalist. The default output looks like the output of
> > 'svn up' or 'cvs up'.
> >
> > If you'd like to give it a shot, you can find the script and manpage at
> > http://www.FreeBSD.org/~jhb/etcupdate/ There is a README file that gives a
> > brief overview and instructions on how to bootstrap the needed metadata before
> > the first update. There is also an HTML version of the manpage.
>
> Looks good!
>
> But I wonder... why not build something like this around cvs?
> Basically a three way merge is exactly what we want for /etc,
> right? cvs because it is in the base system. I used to
> maintain /etc changes in cvs and that was useful in keeping
> track of configuration changes on shared machines.
I wanted something that was version-control agnostic (we use svn at work for
our local version of FreeBSD for example). Also, while having full history
would be nice, that uses up a good bit more disk space and would duplicate
the history that is available in the existing FreeBSD version control (but with
less useful details such as commit log messages). I use CVS for /etc on some
of my machines as well and find it handy, but I tend to not use that to do
full 3-way merges, but just track what is in /etc itself.
--
John Baldwin
------------------------------
Message: 7
Date: Wed, 24 Mar 2010 13:43:14 -0400
From: John Baldwin <j...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: freebsd...@freebsd.org
Cc: Ivan Voras <ivo...@freebsd.org>
Message-ID: <20100324134...@freebsd.org>
Content-Type: Text/Plain; charset="utf-8"
On Wednesday 24 March 2010 10:49:12 am Ivan Voras wrote:
> On 03/24/10 15:02, John Baldwin wrote:
> > On Wednesday 24 March 2010 9:11:21 am Ivan Voras wrote:
> >> On 03/23/10 16:08, John Baldwin wrote:
> >>
> >> [snip - looks like a good utility, will probably use it instead of
> >> mergemaster if it gets committed, like the idea about automated updates]
> >>
> >>> To that end, I wrote a new tool that I think does a decent job of solving
> >>> these goals.
> >>
> >> Since the issue comes around very rarely, I assume there are not many
> >> people who also get the shivers when they see a shell script (and then a
> >> "posixy" /bin/sh shell script) more than a 100 lines long? :)
> >>
> >> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
> >> language interpreter with a syntax that has evolved since the 1970-ies?
> >> (with a side-glance to C that *has* evolved since the K&R style).
> >
> > "You can write Fortran in any language."
>
> I feel I should quote some saying from a holy book saying "Ah, but some
> languages make it harder than the others!" but I don't know of any such :)
>
> > If there are specific things in specific scripts that are poorly commented or
> > implemented then I would work on fixing those. The same is true of the
> > mountain of C code in the tree. Rewriting them in a different language will
> > not automatically make them any better.
>
> C is good enough. I'm after /bin/sh here.
I think if you were to read the source to etcupdate.sh you would find that
it is actually rather close to how I would write it if I were to do it in C
(except it is far less complicated now since I can use tools like diff
directly).
--
John Baldwin
------------------------------
Message: 8
Date: Wed, 24 Mar 2010 14:29:23 -0500
From: Nathan Whitehorn <nwhit...@freebsd.org>
Subject: Review/testing request: changing the detection of
data_addr/text_addr
To: freebsd...@freebsd.org
Message-ID: <4BAA6813...@freebsd.org>
Content-Type: text/plain; CHARSET=US-ASCII; format=flowed
We currently detect the offical "text" and "data" addresses for ELF
files in kern/imgact_elf.c by the heuristic of calling whichever section
contains the executable's entry point the text section and everything
else data. In general, both this concept and the very few things that
use them are obsolete and rare. The data addresses are, however, used in
obreak() to find the current break. On powerpc64, this logic fails,
because the entry point points to a function descriptor in the data
section, causing sbrk() to fail, which in turn breaks profiling.
The patch at http://people.freebsd.org/~nwhitehorn/textseg.diff changes
this algorithm to make the official text area be the largest executable
segment, and all others data. The patch has been tested on sparc64,
amd64, powerpc, and (of course) powerpc64, with no evident bad effects.
I would appreciate any comments, as well as tests on other
architectures. The main symptom of getting these values wrong is that
sbrk() stops working correctly, so I have put a simple test program for
sbrk() at http://people.freebsd.org/~nwhitehorn/sbrktest.c.
-Nathan
------------------------------
Message: 9
Date: Thu, 25 Mar 2010 08:44:30 +1300 (NZDT)
From: Atom Smasher <at...@smasher.org>
Subject: Re: kenv - output needed
To: Garrett Cooper <yane...@gmail.com>
Cc: FreeBSD-Hackers <freebsd...@freebsd.org>, Andrew Thompson
<tho...@freebsd.org>
Message-ID: <1003250827090.40436@smasher>
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
thanks!
if anyone has any input that doesn't behave well, please let me know.
also, this seems to be all x86 variant hardware... does anyone have access
to other hardware platforms?
SHA1 (report_smbios.zsh.gz) = 0afc4c7a5170eaf549b7a1c42d8793861433c654
SHA1 (smbios.txt.gz) = 612353d1c30f7c16f0f0254a21b140721c2a0ed3
1) http://smasher.org/tmp/smbios.txt.gz
2) http://smasher.org/tmp/report_smbios.zsh.gz
(1) is sample i/o with serial numbers and UUIDs removed.
(2) is the script that makes it happen. more comments in the script
there's only one case each (so far) that i need to match strings for
garbage in either "smbios.system.maker" or "smbios.system.product". see
the script and sample input for details.
the logic seems to hold up a a few other system, although the data is
collected differently... dmidecode on linux, sysctl on openbsd/macppc.
here's just the script output based on what i've received:
(ASUSTeK Computer INC., P5BV-C/4L)
(ASUSTeK Computer INC., P5LD2)
(ASUSTeK Computer INC., P5Q-EM)
(ASUSTeK Computer Inc., A8V Deluxe)
(Acer, Navarro, Aspire 5100)
(ConRoe1333-D667..)
(Dell Computer Corporation, OptiPlex GX240)
(Dell Inc., 0H603H, PowerEdge 2950)
(FUJITSU SIEMENS, D1382, SCENIC D)
(FUJITSU SIEMENS, D1382, SCENIC S 2)
(Gigabyte Technology Co., Ltd., GA-MA69VM-S2)
(IBM, eserver xSeries 336 -[883721Y]-)
(Intel Corporation, D865PERL)
(Intel Corporation, DP35DP)
(M7VKD, VIA Technologies, Inc., VT8363x)
(P4i65GV)
(TOSHIBA, Portable PC, Satellite 2415)
(TYAN Computer Corporation, TYAN-Toledo-i3210W-i3200R-S5220)
--
...atom
________________________
http://atom.smasher.org/
762A 3B98 A3C3 96C9 C6B7 582A B88D 52E4 D9F5 7808
-------------------------------------------------
"A Samurai's underwear should be made from the skin
of a badger. This way he will not have lice. In a
long campaign lice are troublesome."
-- Hagakure
------------------------------
Message: 10
Date: Wed, 24 Mar 2010 14:18:35 -0500
From: Nathan Whitehorn <nwhit...@freebsd.org>
Subject: Review/testing request: changing the arguments of
exec_setregs()
To: freebsd...@freebsd.org
Message-ID: <4BAA658B...@freebsd.org>
Content-Type: text/plain; CHARSET=US-ASCII; format=flowed
In order to properly implement exec_setregs() on powerpc64, I must
hand-relocate the function descriptor pointed to by the executable entry
point. To do this, exec_setregs() must be passed the relocation base of
the executable, which would originally come from the image_params struct.
The patch at http://people.freebsd.org/~nwhitehorn/exec_setregs.diff
changes the arguments of exec_setregs to take a pointer to the image's
image_params struct to allow easy future expansion, and removes
redundant arguments (entry and ps_strings). This is similar to the
calling convention in NetBSD. I also take the opportunity to convert
several of the declarations of exec_setregs() from K&R to ANSI C.
This patch is fairly straightforward, but it does touch all
architectures. I have tested the patch on amd64, sparc64, and powerpc,
with no evident problems.
-Nathan
------------------------------
Message: 11
Date: Wed, 24 Mar 2010 16:38:01 -0400
From: John Baldwin <j...@freebsd.org>
Subject: Re: Review/testing request: changing the arguments of
exec_setregs()
To: freebsd...@freebsd.org
Cc: Nathan Whitehorn <nwhit...@freebsd.org>
Message-ID: <20100324163...@freebsd.org>
Content-Type: Text/Plain; charset="iso-8859-1"
On Wednesday 24 March 2010 3:18:35 pm Nathan Whitehorn wrote:
> In order to properly implement exec_setregs() on powerpc64, I must
> hand-relocate the function descriptor pointed to by the executable entry
> point. To do this, exec_setregs() must be passed the relocation base of
> the executable, which would originally come from the image_params struct.
>
> The patch at http://people.freebsd.org/~nwhitehorn/exec_setregs.diff
> changes the arguments of exec_setregs to take a pointer to the image's
> image_params struct to allow easy future expansion, and removes
> redundant arguments (entry and ps_strings). This is similar to the
> calling convention in NetBSD. I also take the opportunity to convert
> several of the declarations of exec_setregs() from K&R to ANSI C.
>
> This patch is fairly straightforward, but it does touch all
> architectures. I have tested the patch on amd64, sparc64, and powerpc,
> with no evident problems.
Looks fine to me, I say go for it.
--
John Baldwin
------------------------------
Message: 12
Date: Wed, 24 Mar 2010 16:42:00 -0400
From: John Baldwin <j...@freebsd.org>
Subject: Re: Review/testing request: changing the detection of
data_addr/text_addr
To: freebsd...@freebsd.org
Cc: Nathan Whitehorn <nwhit...@freebsd.org>
Message-ID: <20100324164...@freebsd.org>
Content-Type: Text/Plain; charset="iso-8859-1"
On Wednesday 24 March 2010 3:29:23 pm Nathan Whitehorn wrote:
> We currently detect the offical "text" and "data" addresses for ELF
> files in kern/imgact_elf.c by the heuristic of calling whichever section
> contains the executable's entry point the text section and everything
> else data. In general, both this concept and the very few things that
> use them are obsolete and rare. The data addresses are, however, used in
> obreak() to find the current break. On powerpc64, this logic fails,
> because the entry point points to a function descriptor in the data
> section, causing sbrk() to fail, which in turn breaks profiling.
>
> The patch at http://people.freebsd.org/~nwhitehorn/textseg.diff changes
> this algorithm to make the official text area be the largest executable
> segment, and all others data. The patch has been tested on sparc64,
> amd64, powerpc, and (of course) powerpc64, with no evident bad effects.
My only comment is that I would keep the existing language about obreak() as
it describes multiple data segments, etc. Instead, I would just use the first
sentence of your new comment to replace the first paragraph in the old
comment.
--
John Baldwin
------------------------------
Message: 13
Date: Wed, 24 Mar 2010 22:32:46 +0100
From: Dag-Erling Sm?rgrav <d...@des.no>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: Ivan Voras <ivo...@freebsd.org>
Cc: freebsd...@freebsd.org
Message-ID: <86mxxxe...@ds4.des.no>
Content-Type: text/plain; charset=utf-8
Ivan Voras <ivo...@freebsd.org> writes:
> C is good enough. I'm after /bin/sh here.
Bourne shell is a perfectly fine programming language if you know how to
use it.
DES
--
Dag-Erling Smørgrav - d...@des.no
------------------------------
Message: 14
Date: Wed, 24 Mar 2010 14:52:36 -0700
From: Garrett Cooper <yane...@gmail.com>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: Ivan Voras <ivo...@freebsd.org>
Cc: freebsd...@freebsd.org
Message-ID:
<7d6fde3d1003241452j2cf...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
On Wed, Mar 24, 2010 at 6:11 AM, Ivan Voras <ivo...@freebsd.org> wrote:
> On 03/23/10 16:08, John Baldwin wrote:
>
> [snip - looks like a good utility, will probably use it instead of
> mergemaster if it gets committed, like the idea about automated updates]
>
>> To that end, I wrote a new tool that I think does a decent job of solving
>> these goals.
>
> Since the issue comes around very rarely, I assume there are not many people
> who also get the shivers when they see a shell script (and then a "posixy"
> /bin/sh shell script) more than a 100 lines long? :)
>
> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
> language interpreter with a syntax that has evolved since the 1970-ies?
> (with a side-glance to C that *has* evolved since the K&R style).
>
> There was once Perl in base and even though I personally dislike Perl at
> least it was a standard of sorts and guaranteed to be there if needed. Now
> there are some fairly large chunks of code written in plain shell script,
> like mergemaster, freebsd-update, portsup and adduser. I'm not specifically
> against shell scripts but (which might just be my personal opinion) I think
> they are even less maintainable in the long term than Perl scripts. I also
> think the bus factor on good shell script programmers must be pretty low.
>
> As a possible alternative, or at least to learn about others' opinion on the
> subject, I'd like to suggest Lua (http://www.lua.org/).
>
> The reasons:
>
> 1) Very light-weight in terms of system integration. Basically, there are
> one or two executables and libraries and the libraries can be discarded if
> only the interpreter executable is needed and not the ability to integrate
> it into C apps. No "libs directories" needed. Written in C, designed to be
> easy to invoked from C (from which the interpreter executable is built on).
>
> This ability to integrate is important because it allows for some nifty
> things like implementing "system" commands through C, e.g. a "sysctl()"
> function as a wrapper for sysctl(3), or a "GEOM Class" class that wraps
> control of GEOM objects.
>
> The basic interpreter executable and the library are ~~ 150 kB each. The
> /bin/sh executable is 130 kB.
>
> 2) Easy syntax, which even kind of resembles shell scripts in its flow.
> Examples: http://lua-users.org/wiki/SampleCode . Unfortunately, its error
> handling is not much better than plain C (no "exceptions"). It has nice
> C-like formatting (e.g. "%4.2f, %d":format(3.14, 42)) and goodies like
> lexical scoping, foreach and coroutines. It's default OOP implementation is
> a bit specific (the "tables" and "metatables" system) but usable.
>
> 3) MIT license. Friendly.
>
> What would be gained? I guess what I'm trying to suggest is that a 3000 line
> shell script (like portsnap, as a random example) could be more readable,
> easier to write and maintain were it a 3000 line Lua script.
Code in general becomes a pain to maintain unless it's well
written when it evolves beyond 500 lines (unless it's C / C++, which
is 1000 lines in my book). So, I don't think that changing sh to Lua
would improve anything (in fact it may make things worse because it's
a less mainstream language). There are a host of other languages out
there that could also serve the needs of the infrastructure, but the
fact remains that the more divergence the project goes in terms of
defacto standard languages, the more of a pain in the ass it's going
to be to maintain longterm because someone decided to write something
in language X.Y.Z.
And there are actually some things which could be done to simplify
the mergemaster and portsnap scripts as well (minor bloat?), but I
digress.
Thanks,
-Garrett
------------------------------
Message: 15
Date: Wed, 24 Mar 2010 23:04:58 +0100
From: Ivan Voras <ivo...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: Peter Jeremy <peter...@acm.org>
Cc: freebsd...@freebsd.org
Message-ID:
<9bbcef731003241504k105...@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On 24 March 2010 20:03, Peter Jeremy <peter...@acm.org> wrote:
> On 2010-Mar-24 14:11:21 +0100, Ivan Voras <ivo...@freebsd.org> wrote:
> There's awk (though it's somewhat restricted in its abilities to do
> anything more than text manipulation) but in principle, I agree. The
> requirements as I see them are (in no particular order):
> - BSD-compatible license
> - must be compatible with buildworld (primarily, it must be possible
> to cross-build)
> - contains a critical mass of users in the FreeBSD developer (and
> ideally committer) community
> - language must be reasonably stable - will a script written today still
> work correctly in (say) 5 years.
> - must be acceptable to the vast majority of the user base (no religious
> wars allowed)
>
>>There was once Perl in base and even though I personally dislike Perl at
>>least it was a standard of sorts and guaranteed to be there if needed.
>
> It was removed because it didn't support cross-building (buildworld is
> always done as a cross-build) and was evolving at a rate incompatible
> with the base system.
>
>>As a possible alternative, or at least to learn about others' opinion on
>>the subject, I'd like to suggest Lua (http://www.lua.org/).
>
> As someone who has never used Lua, how well does it meet the
> requirements above?
I would also like to avoid holy wars so I really want this to be
discussed. I'm more pushing in the direction of "give us something
nicer than sh" than I'm specifically pushing Lua.
To the best of my knowledge, the answers to your questions for Lua are:
> - BSD-compatible license
it's MIT-licensed.
> - must be compatible with buildworld (primarily, it must be possible
> to cross-build)
I'm not sure what you mean by "cross-build". If you mean it needs to
support some specific compiler magic, then I don't know. I do know
this: its src directory is flat (32 .c files, 23 .h), its Makefile
basically says "compile all these .c files into an executable or a
library with CC options varying by platform", and it's ported to at
lest these operating systems: aix ansi bsd freebsd generic linux
macosx mingw posix solaris - the presence of solaris indicates 64-bit
and big endian - aka the big portability test.
I've just verified that a working executable can be compiled with "gcc
-o lua -lm *.c" (if you remove the "luac.c" with duplicates a main()
in addition to the interpreter "lua.c").
> - contains a critical mass of users in the FreeBSD developer (and
> ideally committer) community
> - must be acceptable to the vast majority of the user base (no religious
> wars allowed)
Have no idea. Possibly the biggest single obstacles unless they're
modified to "has the biggest number of people not specifically
objecting to it"
> - language must be reasonably stable - will a script written today still
> work correctly in (say) 5 years.
The Wikipedia entry for it
(http://en.wikipedia.org/wiki/Lua_%28programming_language%29) says
it's been started in 1993. AFAIK it has always been backward
compatible and the recent version is 5.3. One thing I remember it for
is that many Windows games (including some classics like Baldur's Gate
II... I'm sure some people remember it) were scripted by it.
I'd like to summarize why I think Lua is a good idea here:
* Small footprint (150-200k executable). Can be compiled with gcc -o
lua -lm *.c.
* Can be embedded into C apps for scripting, can use C for extending
it (designed to make both directions trivially easy)
* Is kind of like Python (supports OOP, partly functional, etc.
styles) without the extensive libraries. (but has no "significant
whitespace" issue).
------------------------------
Message: 16
Date: Wed, 24 Mar 2010 23:07:06 +0100
From: Ivan Voras <ivo...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: Dag-Erling Sm?rgrav <d...@des.no>
Cc: freebsd...@freebsd.org
Message-ID:
<9bbcef731003241507i137...@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
2010/3/24 Dag-Erling Smørgrav <d...@des.no>:
> Ivan Voras <ivo...@freebsd.org> writes:
>> C is good enough. I'm after /bin/sh here.
>
> Bourne shell is a perfectly fine programming language if you know how to
> use it.
I'll agree that it's fine but only in the abstract - e.g. that it is
Turing complete :)
(But then again, I've heard sendmail.cf is also Turing complete so there...)
------------------------------
Message: 17
Date: Wed, 24 Mar 2010 21:42:12 -0500
From: Nathan Whitehorn <nwhit...@freebsd.org>
Subject: Re: Review/testing request: changing the detection of
data_addr/text_addr
To: John Baldwin <j...@freebsd.org>
Cc: freebsd...@freebsd.org
Message-ID: <4BAACD84...@freebsd.org>
Content-Type: text/plain; CHARSET=US-ASCII; format=flowed
John Baldwin wrote:
> On Wednesday 24 March 2010 3:29:23 pm Nathan Whitehorn wrote:
>
>> We currently detect the offical "text" and "data" addresses for ELF
>> files in kern/imgact_elf.c by the heuristic of calling whichever section
>> contains the executable's entry point the text section and everything
>> else data. In general, both this concept and the very few things that
>> use them are obsolete and rare. The data addresses are, however, used in
>> obreak() to find the current break. On powerpc64, this logic fails,
>> because the entry point points to a function descriptor in the data
>> section, causing sbrk() to fail, which in turn breaks profiling.
>>
>> The patch at http://people.freebsd.org/~nwhitehorn/textseg.diff changes
>> this algorithm to make the official text area be the largest executable
>> segment, and all others data. The patch has been tested on sparc64,
>> amd64, powerpc, and (of course) powerpc64, with no evident bad effects.
>>
>
> My only comment is that I would keep the existing language about obreak() as
> it describes multiple data segments, etc. Instead, I would just use the first
> sentence of your new comment to replace the first paragraph in the old
> comment.
>
That's a good point. I've updated the patch with the new comment. Thanks
for taking a look! If no one objects, I'll commit both of these patches
tomorrow.
-Nathan
------------------------------
Message: 18
Date: Thu, 25 Mar 2010 08:51:14 +0000 (GMT)
From: Robert Watson <rwa...@FreeBSD.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: Ivan Voras <ivo...@freebsd.org>
Cc: freebsd...@freebsd.org
Message-ID: <alpine.BSF.2.00.1...@fledge.watson.org>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
On Wed, 24 Mar 2010, Ivan Voras wrote:
> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
> language interpreter with a syntax that has evolved since the 1970-ies?
> (with a side-glance to C that *has* evolved since the K&R style).
...
> As a possible alternative, or at least to learn about others' opinion on the
> subject, I'd like to suggest Lua (http://www.lua.org/).
I think there are lots of good arguments for Lua in the base, but that
etcmerge is definitely not one of them :-). An important goals for a tool
like etcmerge is a minimal dependency footprint, so that you can use it with
all the existing versions of FreeBSD floating around and upgrade to new
versions. None of those existing versions have lua. Good arguments for lua
in the base might include:
- Moving to Lua as the scripting language for the boot loader
- Improving scripting capabilities in the installer
etcmerge sounds very exciting, especially for shops that want a more automated
upgrade path. It's easy to upgrade web browsers, and they're basically
operating systems at this point, so it would be nice if we could offer FreeBSD
upgrades with similar ease.
Quite a bit of our automated configuration update problem comes down to
configuration file formats and the way diff/patch perform merges. Consider
files like inetd.conf, master.passwd, group, etc: they essentially ensure that
there will be a conflict if you have any local changes and the vendor (us)
makes an upstream change. We used to have this problem with /etc/rc and
/etc/rc.local, but rc.d has basically eliminated the problem by allowing
boot-time custtomization through file insertion rather than file changes.
Choices made in the configuration design for launchd, xinetd, and others avoid
this mistake. Perhaps we shold be considering similar sorts of redesigns,
focusing on how configuration files could be reworked to maximize automated
update support. Where there's a true semantic conflict, an update conflict
requiring resolution is fine, but where there's no semantic conflict (i.e., we
add _anotheruser to the base master.passwd), no upgrade conflict should arise.
(And definitely keeping this mind as we add new configuration files)
Robert
------------------------------
Message: 19
Date: Thu, 25 Mar 2010 10:47:02 +0100
From: Ivan Voras <ivo...@freebsd.org>
Subject: Re: Another tool for updating /etc -- lua||other script
language bikeshed
To: freebsd...@freebsd.org
Message-ID: <hofbep$8p8$1...@dough.gmane.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
On 03/25/10 09:51, Robert Watson wrote:
>
> On Wed, 24 Mar 2010, Ivan Voras wrote:
>
>> Wouldn't it be nice to have a "blessed" (i.e. present-in-base) script
>> language interpreter with a syntax that has evolved since the
>> 1970-ies? (with a side-glance to C that *has* evolved since the K&R
>> style).
> ...
>> As a possible alternative, or at least to learn about others' opinion
>> on the subject, I'd like to suggest Lua (http://www.lua.org/).
>
> I think there are lots of good arguments for Lua in the base, but that
> etcmerge is definitely not one of them :-).
Of course! Maybe I should have worded it better - my post has nothing
specifically to do with etcmerge. At most it could serve some future
developer and/or project.
------------------------------
Message: 20
Date: Thu, 25 Mar 2010 12:38:14 +0100
From: "Thomas Schmitt" <scdb...@gmx.net>
Subject: Solved: How to slow down SATA to 1.5 GBit/s ?
To: freebsd...@freebsd.org
Message-ID: <1057906...@192.168.2.69>
Hi,
i now have running the eSATA attached DVD drive
with 150 MB/s on 8-STABLE and 8-RELEASE.
Done by switching from ata to ahci as described
by Juergen Lock in
http://www.mail-archive.com/freebsd...@freebsd.org/msg70510.html
and the given link to Ivan Vora's
http://ivoras.sharanet.org/blog/tree/2009-11-17.trying-ahci-in-8.0.html
My thanks to both.
The DVD drive at ahcich4 is curbed in
/boot/device.hints by
hint.ahcich.4.sata_rev="1"
while the hard disk still runs on 300 MB/s.
The problems with libburn and ahci have turned
out as suboptimal usage of libcam flags which
worked with ata and usb, but not with ahci.
Thanks to Alexander Motin for telling me the
right gestures of error reply detection.
------------------------------------------------
Little pitfall in Juergen's proposal:
dumpfs /dev/adXsYa | sed -n ...
produces output like
/dev/ufsid/4ba77276 b4a8625
/dev/ufsid/4ba77276 fb8e9567
One has to fill in eventual leading 0s so that
both hex numbers have 8 digits, and one has to
remove the blank:
/dev/ufsid/4ba772760b4a8625
/dev/ufsid/4ba77276fb8e9567
Alternatively one can follow Ivan's proposal
to put into boot/loader.conf:
kern.geom.label.debug=1
and to reboot for seeing the ufsid in the boot
log.
Have a nice day :)
Thomas
------------------------------
End of freebsd-hackers Digest, Vol 365, Issue 4
***********************************************