I have kind of sorted (partly brute-force, partly eye-work) and
came up with the attached splits:
www-servers 48
www-servers-modules 115
www-proxy 27
www-browsers 77
www-apps 26
www-apps-cms 24
www-apps-images 13
www-apps-rss 9
www-apps-wiki 21
www-apps-zope 41
www-devel 11
www-devel-perl 176
www-devel-php 19
www-devel-python 12
www-devel-ruby 11
www-editors 29
www-log 27
www-ml 17
www-spiders 25
www-misc 19
www 0
I'm missing 8 ports somewhere, but the idea is there.
Of course these categories aren't written in stone and I'm not going
to defend them regiously[sp], but it gives an idea what is there.
Maybe even splitting pf www-servers*, www-clients and www-proxy
I am willing to volunteer for this job, but I think it's more the
cvsmeisters which will curse than I am.
Comments? Remarks? Support? Ideas? Hostile takeovers?
Edwin
--
Edwin Groothuis
ed...@freebsd.org
http://www.mavetju.org
This sounds like a good idea to me, But I dont know if it should be
broken down to such an extreme. Maybe just break up the "big ones" and
create a www-misc for everything else. Just my 2 cents.
Regards,
Frank
Portmgr has been, for quite some time now, looking into adding an extra
tier to the ports tree. Assuming that they haven't abandoned the idea,
I'd say it's best to divide www into subcategories, instead of 20 main
categories.
# Adam
--
Adam Weinberger
ad...@magnesium.net || ad...@FreeBSD.org
ad...@vectors.cx || ad...@gnome.org
http://www.vectors.cx
There's even worse categories than www:
/usr/ports# for f in `make -V SUBDIR`; do echo -n "$f\t"; ls $f | wc -l;
done | sort -k 2
[snip dirs with less than 200 entries]
x11-toolkits 211
editors 215
math 249
x11 266
print 270
lang 281
databases 335
japanese 430
misc 441
security 463
audio 471
mail 480
sysutils 511
graphics 526
games 644
textproc 679
www 756
net 796
devel 1404
Why does it matter that some dirs are rather large? Personally I'd rather
be able to guess where a port resides (which is harder with more
categories), than to have an "ls /usr/ports/www" fit onto a single
screen.
There will be 12000 ports soon, imo splitting directories all the time is
not good solution to the problem you're having (getting to that), nor is
adding another level of depth in the file system hierarchy (and having
varying depths is even worse).
Reading between the lines, the problem you're having is that the ports
aren't well enough categorized at the moment. How about borrowing an idea
from some of the knowledge databases, and using keywords to mark ports?
Eg, instead of creating a www-server category, the apache port could be
marked "server www". linux-opera could be market "binary browser client
linux www" or something like that.
The number of keywords that exist as well as the number of keywords
assigned to a single port could then be much more easily changed and
updated. Of course, there'd need to be a program for filtering the ports
by keyword, but it'd be trivial to write one or to put this functionality
into make search.
Just some more ideas from an user :)
Greetings
Benjamin
> You probably would want to only split into a few medium sized
> categories instead of a lot of small ones. Just www, www-client,
> and www-server for example.
Isn't there a rule-of-thumb for this kind of human-oriented
subdivision, e.g. "at least three and no more than ten"?
Robert Huff
> Portmgr has been, for quite some time now, looking into adding an
> extra tier to the ports tree.
Other than a) tradition and b) potential large amount of work
getting all the location pointers retargeted ... what's stopping
them?
Robert Huff
> > Portmgr has been, for quite some time now, looking into adding an
> > extra tier to the ports tree.
>
> Other than a) tradition and b) potential large amount of work
> getting all the location pointers retargeted ... what's stopping
> them?
It's not location pointers, it's reworking the Makefile code. The
assumption that there are only 2 levels is also hardwired into other
code, certainly portsmon and possibly the pointyhat build infrastructure.
There is also the school of thought that we ought to identify what
problem we are trying to solve, and if it's merely a 'search/browse'
type of problem, then we should fix our search/browse interfaces and
leave the directory layout alone.
Unfortunately, this school of thought seems to have only one adherent,
which is myself :-)
mcl
> Reading between the lines, the problem you're having is that the ports
> aren't well enough categorized at the moment. How about borrowing an idea
> from some of the knowledge databases, and using keywords to mark ports?
> Eg, instead of creating a www-server category, the apache port could be
> marked "server www". linux-opera could be market "binary browser client
> linux www" or something like that.
FreeBSD already has "virtual" categories -- categories that aren't used
to physically sort the ports. For example, devel/tcl84/Makefile has:
CATEGORIES= lang tcl84
tcl84 isn't a real folders, but it's listed in a number of port's
$ make search key=tcl84 | grep "Port:" | wc -l
40
One thing I don't see is a way to search or sort by category, but I
admit to not having looking very hard.
--Mike
> Ports/www is too full: 755 ports.
>
I don't see the point for splitting it. The problem I have (for example
still with net and net-mgmt) is, I know what application I want to
install. Problems with such a dividing is.. where the hell is my port in.
I really have to start searching then where I for example found mod_php.
Is it in www-servers-modules, is it in www-devel-php, or... hm, is it in
devel or in lang?
You see what I mean? Now "It has sth. to do with www, so it should be in
www" -> "cd /usr/ports/www/mod_p<tab>" -> "ah there it is"
Having many of subcategories would result in using heavaly make search or
cd /usr/ports/*/mod_php4 which can take some time. and could fail of 1+
match.
Virtual categories are fine imho. But please no pysical ones. What is the
difference of having 700 directories in one dir or 3000?
--
Oliver Lehmann
http://www.pofo.de/
http://wishlist.ans-netz.de/
> There is also the school of thought that we ought to identify
> what problem we are trying to solve, and if it's merely a
> 'search/browse' type of problem, then we should fix our
> search/browse interfaces and leave the directory layout alone.
>
> Unfortunately, this school of thought seems to have only one
> adherent, which is myself :-)
Coung mr in.
It occurs we may be looking at multiple "problems", which may
or may mot have compatible solutions. Someone mentioned finding a
(popular) port with a known name; I often want to look at all
programs which do <foo>, where FOO is far more detailed than the
existing categories support.
Anyone else?
Robert Huff
> Mark Linimon writes:
>
> > There is also the school of thought that we ought to identify
> > what problem we are trying to solve, and if it's merely a
> > 'search/browse' type of problem, then we should fix our
> > search/browse interfaces and leave the directory layout alone.
> >
> > Unfortunately, this school of thought seems to have only one
> > adherent, which is myself :-)
>
> Coung mr in.
Well, I agree too, but I wouldn't try to *stop* somebody from
redefining the directory structure, either. I haven't spoken
up because I think it looks like a bikeshed.
> It occurs we may be looking at multiple "problems", which may
> or may mot have compatible solutions.
Quite true.
> Someone mentioned finding a
> (popular) port with a known name;
If you know the exact name, "make search name=" does a good job right now.
> I often want to look at all
> programs which do <foo>, where FOO is far more detailed than the
> existing categories support.
That's a hard problem, because it can get into semantics. I don't
really see any new approaches to the problem that will do better than
"make search key="
Logical divisions are helpful ... which is why we have directories in
the first place. I agree that splitting for the sake of smaller
directories is the wrong approach, but if a logical division exists in
www, as did in net and (farther back in) x11, should it be resisted
merely to keep the directories large?
At first glance, I don't see a really clear division to the ports in
www. Edwin's initial suggestion sort of works, but I too think that's
far too many directories. Too many things belong in multiple places,
and it would feel silly to `cd /usr/ports/*/someport` just because I
can't remember where something lives. Even splitting into www-server
and www-client doesn't work for many ports.
So until there's a clear reason why a big directory is a problem, I'd
stick with the status quo.
>How about borrowing an idea
> from some of the knowledge databases, and using keywords to mark ports?
> Eg, instead of creating a www-server category, the apache port could be
> marked "server www". linux-opera could be market "binary browser client
> linux www" or something like that.
I really like this idea, and it's at least partly implemented now with a
number of ports (with multiple CATEGORIES; www/gatling, mail/wmbiff,
etc) but implementing this fully is a *much* larger task than merely
applying another logical division to a ports directory.
There are lots of ports that really truly belong in multiple categories;
ftp/wget and ftp/curl are excellent HTTP clients, mail/rlytest verifies
an element of an SMTP security policy, etc. The problem is that having
the port directory named for its "primary" category is just so damned
convenient.
One solution might be to model the packages/ tree. Store all ports in
ports/All/, then have *all* the categories broken out as their own
directories, with symlinks pointing back into ../All/portdir/. Then you
could have multiple symlinks, easily maintained by the meta info in
ports/INDEX.
A single directory with 12000 subdirectories in it may be unruly ... and
we'd have to fix oddities like hydra, jags, replay, etc ... but it would
provide for the most flexible expansion, and the symlink tree would
provide an equivalent interface to folks comfortable with the current
setup.
Thoughts?
--
Paul Chvostek <pa...@it.ca>
Operations / Abuse / Whatever
it.canada, hosting and development http://www.it.ca/
Do you mean something like what OpenBSD does, with, for example, x11/gnome
and x11/kde? Both of those directories contain a group of ports. They're
like the FreeBSD metaports, but they're physical directories with
subdirectories.
Population. cvs (and probably cvsup) do not handle symlinks well. You
would have to have a script populate symlinked ports into the categories
by hand.
--
Robin Schoonover (aka End)
# Just think -- blessed SCSI cables! Do a big enough sacrifice and
# create a +5 blessed SCSI cable of connectivity.
# -- Lionel Lauer
Yes, that's what I was suggesting. A change like this would require
that the symlinks not be part of the distribution, but be built by a
tool like portsdb (nee portupgrade). The downside is that it's a fairly
major architectural change that might break lots of stuff. The upside
is that it's a SMOP, nothing really challenging.
I'm not much of a ruby guy, but I could easily write a tool in awk to
convert from current to revised layout, both in an installed ports tree
and a repository. Making the existing pkgtools and portupgrade happy
would be a bit more work... And we'd have to get everybody to agree on
a process for adding new keywords.
This would eliminate the usefulness of /usr/ports/MOVED, since
everything would be in one place.
If 12000 is just too frickin' many for a directory, perhaps some "titan"
categories could be used to aggregate ports as they are now: app, devel,
sysutil. Whatever would have the least chance of category overlap.
MOVED can then still point to the new location, and cvsup may be a
little happier.
But what is the right design? For the folks who like the directories,
would symlinks be acceptable? If we aggregate, does it make sense to go
ALL the way, or continue to maintain some sort of logical division to
real port homes, albeit fewer (and more atomic)?
Just brainstorming here. Any idea is a good idea until you have to
put it into production.
There is a Perl module in the ports tree named FreeBSD::Ports with
which a port can be identified by contents of comment, if not by
pkg-desrc, besides other things.
Ah, here it is (output is from my own index search tool, parse-index;
perl 5.8.5 has been installed here)...
name: p5-FreeBSD-Ports-0.04
origin: /usr/ports/textproc/p5-FreeBSD-Ports
install-prefix: /usr/local
comment: Perl modules for parsing FreeBSD's Ports INDEX file
description: /usr/ports/textproc/p5-FreeBSD-Ports/package-descr
maintainer: t...@FreeBSD.org
category: textproc
category: perl5
build-dep: perl-5.8.5
run-dep: perl-5.8.5
uri: http://people.FreeBSD.org/~tom/portpm/
extract-dep: perl-5.8.5
patch-dep: perl-5.8.5
- Parv
--
Or All could be divided into prefix directories -- ports/a/p/apache20.
That would still break the number of ports per directory down enough,
leave no confusion as to where the core port directory is, and eliminate
the MOVED file. Then use the script to populate the symlink category
directories.
Jim
--
Jim Trigg, Lord High Everything Else O- /"\
\ / ASCII RIBBON CAMPAIGN
Hostmaster, Huie Kin family website X HELP CURE HTML MAIL
Verger, All Saints Church - Sharon Chapel / \
Ack. Please leave www as it is.
--
,_, | Michael Nottebrock | lo...@freebsd.org
(/^ ^\) | FreeBSD - The Power to Serve | http://www.freebsd.org
\u/ | K Desktop Environment on FreeBSD | http://freebsd.kde.org
I think some of the original Idea can be take. But 'bit by bit'. I
don't think a ports/apache or ports/www-apache may hurt.
--
josemi
KDE has a tool called Barry, not sure if it made it into the distribution but
it should be available somewhere.
--
Matt Douhan
http://www.fruitsalad.org
(http://www.student.uni-oldenburg.de/frerich.raabe/barry/, for those who
don't want to spend 2 minutes grovelling in Google like I just did)
Nice tool. But how does that help with quick window-shopping?
Finer-grained directories help. Using a GUI tool to navigate
coarse-grained directories instead of `cd` and tab-completion doesn't
(especially since I only run X on a single, non-FreeBSD, machine).
-T
--
"Beauty is not diminished by being shared."
-- Robert Heinlein
If we have good logical categories for the ports, and have good
indexing and searching tools, it does not matter where the ports are
physically put. And surely a better interface for browsing can be
built on top of that.
On Tue, 26 Oct 2004 13:47:25 -0600, Tillman Hodgson
<til...@seekingfire.com> wrote:
> On Tue, Oct 26, 2004 at 01:40:09PM -0600, Jie Gao wrote:
> > I agree this. If we have a powerful search utility instead of the
> > simple "make search", it doesn't matter how large the directories are.
>
> Searching and browsing are not the same task.
>
> /me has an image of window-shopping the ports tree ...
>
> -T
>
> --
> Real men use "cat /var/spool/mail/$USER | more" and "telnet $SMTP_HOST 25"
> - Anonymous Unix geek
> "more /var/spool/mail/$USER" <-- don't waste a process, you idiot
> - Second anonymous Unix geek
> _______________________________________________
>
>
> freebs...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-port...@freebsd.org"
>
> > KDE has a tool called Barry, not sure if it made it into the distribution
> > but it should be available somewhere.
>
> (http://www.student.uni-oldenburg.de/frerich.raabe/barry/, for those who
> don't want to spend 2 minutes grovelling in Google like I just did)
sysutils/barry. 'make search key=barry' would have been sufficient to find it.
> Nice tool. But how does that help with quick window-shopping?
A ports tree can't ever be a decent shop window, there's just no way of
displaying enough information about the goods on sale in a filesystem
listing. Thus there's no point in putting more boards into the shelf - if you
want to go window-shopping for software, use freshmeat, then feed your picks
to locate, make search or freshports.
I've created a little python script that entirely feeds off the categories
in the INDEX-5 file.
http://www.maxlor.com/freebsd/files/portsbrowse.py
Usage is simple: "./portsbrowse.py www" will display all www ports.
"./portsbrowse.py www net" will display both www and net ports (logical
OR), while "./portsbrowse.py -and www linux" will display only ports that
are in both categories (logical AND).
How does this work for you, as far as "window-shopping" is concerned?
Sure, you don't have autocomplete, but you do get to grep the scripts
output instead.
Benjamin
I haven't looked at the script, but I suspect that you might have missed
the point of what I was getting at.
Let's say I'm looking for a Apache modules. I'm not looking for anything
in particular, I just want to see what's been ported. Window-shopping.
How does looking at the www directory at the shell, via a python script,
or via a KDE GUI help me?
There's 792 ports in the www tree according to `ls | wc -l`. Of those,
113 are _possibly_ Apache modules (as determined by `ls -d *mod* | wc
-l`). Which of those are really Apache modules, and of those, which are
Apache 1.3.x modules is impossible to easily tell from the output.
`grep`ing for "mod" in the output of some utility would have the same
problem. The fallacy I've fallen into with this example, and the fallacy
that searching tools fall into, is the idea that port names are always
going to be representative of what the port contains (or at least that
the port comment will magically have the right keywords). That's not the
same thing as meta-information like fined-grained categories.
Perhaps virtual categories can do what I'm thinking of, if they were
widely used.
-T
--
[Re : quantum physics]
I can't say I care one way or another. -- Kai Henningsen
That's just because nobody's measured you yet. -- Christian Bauernfeind
> How does this work for you, as far as "window-shopping" is
> concerned? Sure, you don't have autocomplete, but you do get to
> grep the scripts output instead.
I can't speak for the original poster but, as someone who's
more interested in the window shopping part, this doesn't solve my
problem within the status quo.
If I'm looking for (let's say) a replacement mail-reader,
gnawing through 480+ ports is not a win. Given my limited knowledge
of the ports build internals, adding a new layer to the directory
structure, e.g. mail/servers, mail/clients, mail/utilities, and
mail/misc, seems much less painful than editing 12,000 Makefiles.
Robert Huff
I'm the target, and I don't consider it "mean", I think it was true.
Granted, it misses the step of needing to poke into pkg-descr to find
the URL (because make search doesn't display that by itself), but it
*would* have been quicker.
-T
--
"Yield to temptation; it may not pass your way again."
-- Robert Heinlein
> > sysutils/barry. 'make search key=barry' would have been sufficient to
> > find it.
>
> That's mean (but also pretty funny)
Sorry folks, that was meant to be a private reply. (blush)
mcl
> Let's say I'm looking for a Apache modules. I'm not looking for anything
> in particular, I just want to see what's been ported. Window-shopping.
> There's 792 ports in the www tree according to `ls | wc -l`. Of those,
> 113 are _possibly_ Apache modules (as determined by `ls -d *mod* | wc
> -l`). Which of those are really Apache modules, and of those, which are
> Apache 1.3.x modules is impossible to easily tell from the output.
> `grep`ing for "mod" in the output of some utility would have the same
> problem. The fallacy I've fallen into with this example, and the fallacy
> that searching tools fall into, is the idea that port names are always
> going to be representative of what the port contains (or at least that
> the port comment will magically have the right keywords). That's not the
> same thing as meta-information like fined-grained categories.
About the only way I can see for doing this task effectively would be
a google-like keyword search over the contents of the pkg-descr files.
The pkg-descr files generally contain a pretty good summary of what
the port actually contains -- much better than just relying on port
names. Hmmmm... it should be possible to hook up htDig indexing the
README.html files.
Although did you just try typing in 'apache modules' into the search
facility right on the http://www.freebsd.org/ports/ page? You can
even tell it to just search the package descriptions.
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks
Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey Marlow
Tel: +44 1628 476614 Bucks., SL7 1TH UK
Jie Gao wrote:
>They are different if we do not have powerful search utilities. But
>when we have good search tools both searching and browsing will
>benefit from it. Look at the WWW search engines (google, yahoo, ...).
>Almost all of them have web site directories, which are for browsing
>but benefit a lot from the search technologies.
>
>If we have good logical categories for the ports, and have good
>indexing and searching tools, it does not matter where the ports are
>physically put. And surely a better interface for browsing can be
>built on top of that.
>
>
How about freshports?
Regards,
Frank
It would be better, yes, but it would still be playing "hunt the keyword
in the stack of 12000 pkg-descrs". If you picked a good keyword you get
200 hits and still have to browse through them. If you pick a bad
keyword, you get no hits.
Perhaps if pkg-descr had a KEYWORDS: line and it's use was considered
mandatory ... but what are the chances that every port maintainers idea
of a keyword is the same as mine?
I think meta-information like fine-grained categories are different than
searching.
> Although did you just try typing in 'apache modules' into the search
> facility right on the http://www.freebsd.org/ports/ page? You can
> even tell it to just search the package descriptions.
Oh, I agree that there are tools out there that do very cool things with
the ports tree. I'm a regular spelunker (love that word) at
freshports.org. That's not exactly part of the FreeBSD tool set on a
non-networked machine though, so I don't think it's _directly_
relevant to this discussion.
I'm not even advocating the use of finely-grained categories. I'm just
pointing out that searching is not the same as browsing :-). I wouldn't
use Google to select the right chapter of the Handbook to read--I'd use
the table of contents. Searching is useful, and it's very important and
thus worth spending time getting right, but it's a different tool for a
different (albeit related) task.
Here's a better example:
Let's say that rather than being the maintainer of the net/latd port,
I'm some random user who wants the `llogin` program. So I `cd /usr/ports
&& make search key=llogin`. But there's no hits.
There's 989 ports in the net/ category. Searching failed. Browsing that
directory for the port I want is very difficult. But if there was, say,
a dozen sub-categories my task becomes manageable. Maybe in the
net/other-protocols (or whatever) category there's only a score of
ports. Maybe the name "latd" then rings a bell.
Granted, the ability to search pkg-plist would solve this particular
issue. It's easy to find examples where that won't work: if I just want
to see what neat kinds of network protocols that FreeBSD has supported
in it's ports tree, for example.
-T
--
Page 41: Two of the most important Unix traditions are to share and to
help people.
- Harley Hahn, _The Unix Companion_
You should be able to say
make search cat=tcl84
but the search target has a bug that prevents this from working.
I've just submitted a PR; you can use the attached patch in the
meantime.
--
If you cc me or remove the list(s) completely I'll most likely ignore
your message. see http://www.eyrie.org./~eagle/faqs/questions.html
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/73299 :)
-Radek
Thanks, I haven't been paying attention to ports for some time
lately.