Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

The ports collection has some serious issues

244 views
Skip to first unread message

Daniil Berendeev

unread,
Dec 8, 2016, 12:24:49 AM12/8/16
to
Hello guys!

First of all, it's not a hate mail, I appreciate all the work done on
the system and I enjoy using FreeBSD every day.

But after some recent experience I'd like to point out some problems
that make using the ports collection uncomfortable and painful.

Some overview before we start:
* Why I use ports over pkg?
Because, generally, packages are built with poor default options, for
example moc isn't able to play .alac/.mod and that's frustrating.

* Why pkg is still nice?
It is able to update packages with broken ABI, it's fast and easy to
use. Some packages/ports don't have options and can be used via pkg by a
ports user.

I want to contribute to FreeBSD development, so, long story short, I've
decided to move to -CURRENT. Everything went fine except the ports upgrade.

Is it possible to upgrade the ports by hand? Well, it is, but it is not
too comfortable. Ports collection by itself doesn't provide a nice way
to work with port management, so a user needs to use something for port
management. As the handbook advised, I picked portmaster.

And here begin the problems.

1) portmaster is not nice for the user.
If it comes over an error even in one little tiny port that is a
dependency for something bigger , it will abort its work and leave all
the other ports not updated. So, if you try to to do `portmaster -af`,
you should not forget `-m DISABLE_VULNERABILITIES=yes` (we will return
to this one later) and you must pray to God for not coming around a
circular dependency or some port that would fail to deinstall its older
version. You can't leave portmaster for a night to update all the needed
ports and deal with broken ones in the morning, you need to cherry pick
the broken ports and ignore them, and then try to deal with them.

Although portmaster is not releated to the FreeBSD project and is an
outside tool, there aren't any alternatives from the project itself. So
use it or die. Not a nice situation.

2) pkg and ports are not in sync.
pkg appeals to build ports that are from 2xxxQx branches. The promoted
tool for syncing ports (portsnap) always fetches from head. And there is
no way to choose. That gives us the next problem:

3) no integration between ports and packages
There is no clear, easy way to use ports and packages simultaneously. If
I'd like to use some built packages to speed up port updates, I have to
ignore by hand all the packages that I want to be built as ports. It's
easier to stick to only ports or only packages.

4) uncomfortable way of rollback
If I want to rollback, or just choose the branch from where the packages
are built (to stay in sync with pkg), I have to pull the whole svn
repository.

5) svn repository.
I don't want to spark a holy war and I don't belong to those type of
people who are always obsessed that something isn't done in their way.
But guys, svn is not a good tool for ports. Just for one reason,
actually (as for me, I could tolerate anything else, but not this one)
-- size. The size of repository is 20G+ and growing. I don't want to
pull 20G+ in /usr/ports just because I need to use ports. It's just
sick. The repository is so big because, as all ya know, svn is expensive
in branch operations. Since you've began to do those 2xxxQx branches the
size of the repository began to grow rapidly. It's inefficient and
uncomfortable. For such a work something like git or mercurial should be
used, they'd fit in 3-4G.

6) broken ports are pushed to head
Why do we have such a situation, when head contains a handful of broken
ports? Why commit a port that won't build? It's sick.
Ports are broken in a different way. Some fail to build. Some fail to
uninstall their older version (like rust), so that you need to do
`pkg remove -f portname; portmaster portname`. Some have a circular
dependency (d-bus) and will try build until the heat death of the
universe. I just don't get it, why broken ports are pushed to head, if
head is then used by portsnap to update /usr/ports? You leave tons of
users with a broken setup. And there is always a bunch of ports that
won't build. It's not just one, or two, it's a handful of ports.
pkg-f...@FreeBSD.org is overwhelmed with build fails.

7) No way to update ports with broken ABI.
I need to run `pkg update` and then pick the broken ports by hand. Or do
`portmaster -af`.

8) ports with vulnerabilities.
They exist in the tree and on build attempt they shout that they won't
build without DISABLE_VULNERABILITIES=yes. The catch is that there is
always a bunch of ports with vulnerabilities. So if you are doing a
fresh install, you have to install those nasty vulnerable ports anyways.
It causes you to do extra moves and doesn't add no security or safety.
There is no way to pick the latest safe version.

I hope that my mail will produce a productive discussion that will lead
to some good decisions for fixing these problems.

Make ports great again :).
--
Cheers~

PGP key fingerprint:
07B3 2177 3E27 BF41 DC65 CC95 BDA8 88F1 E9F9 CEEF

You can retrieve my public key at pgp.mit.edu.
_______________________________________________
freebs...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-port...@freebsd.org"

Christoph Brinkhaus

unread,
Dec 8, 2016, 12:37:38 PM12/8/16
to
On Thu, Dec 08, 2016 at 09:46:48AM -0500, George Mitchell wrote:
> On 12/08/16 09:14, Kevin P. Neal wrote:
> > On Thu, Dec 08, 2016 at 01:28:02PM +0100, Baptiste Daroussin wrote:
> >> On Thu, Dec 08, 2016 at 05:16:24AM +0000, Daniil Berendeev wrote:

Hello George!
> >>> Hello guys!
> >>> [...]
> >> Have you considered using things like poudriere that would allow you to build
> >> your own repository with your own set of packages and options.
> >
> > Here's a +2 for poudriere.
> > [...]
>
> I've never been able to complete a poudriere run on my build machine
> without panicking because I have "only" four gigabytes of memory. (It
> is likely the machine will get more memory for Christmas, though.) I
> haven't yet had occasion to try synth.
I have two gigabyte only and poudriere works. In the unlucky
constellation of building gcc and dri it requires some swap.
Otherwise the machine hangs, this is true. The CPU is a dual core
type. I have never tried to limit the RAM per jail or number of jobs.
With my configuration poudriere runs two builders.
>
> I love portmaster. My build machine is not a production machine, so
> if the build process goes astray, I can at worst delete all ports and
> rebuild them all. And portmaster will request all options at the
> beginning of the run and then quietly build away until it finishes (or
> breaks).
>
> The worst thing about ports for me is really an NP problem: FreeBSD
> can't possibly test all the option combinations. So I expect to have
> to do some fine tuning on my own. -- George

Happy and successful tuning and enjoy the additional RAM soon!

Kind regards,
Christoph

Mark Millard

unread,
Dec 13, 2016, 4:32:20 AM12/13/16
to
Julian Elischer julian at freebsd.org wrote on Mon Dec 12 16:15:32 UTC 2016:

> The problem I get hit by is that the quarterly packages are deleted
> immediately on the creation of the next quarterly set.

Packages: true --but not true of svn's sources for ports:

https://svnweb.freebsd.org/ports/branches/201*Q*/

svn has 2014 's, 2015 's, and 2016 's sources for each of the
quarters: Q1 , Q2 , Q3 , Q4 .

Things seem to be set up more for source based builds
(or local package builds and distribution based on
such sources for such builds).

(Not that I'm doing anything where I depend on the
issues or distinctions. I just noted the quarterly
history that is available and where it happens to
exist --which may well be obvious to most folks
involved.)

===
Mark Millard
markmi at dsl-only.net

Thomas Mueller

unread,
Dec 17, 2016, 3:52:07 AM12/17/16
to
From John Marino:

> At face value, this doesn't make sense because synth is a tool for building
> everything from source, so your development system is exactly where it should
> be installed.

> So you must be talking about build dependencies of synth (there are no run
> dependencies). While I think the requirement of rebuilding synth from source
> is artificial, I've provided a very reasonable approach to solving this which
> I feel compelled to repeat for the readers of Kevin's post. The solution:

> Starting with a clean system:
> 1) install synth from binary package from official freebsd builder (a single
> package)
> 2) Configure synth if necessary
> 3) command synth to build itself
> 4) pkg delete synth (system is once again clean)
> 5) pkg add -F /path/to/synth/packages/synth-*

> Now you have a system containing s/w built by itself. On an modest system
> less than 4 years old, it might take 30 minutes at most.

> So the "synth has dependencies" detraction is extremely weak. For people that
> trust FreeBSD to provide untainted binaries, it's not an issue at all and for
> the paranoid, it's easily worked around. Saying that the use of Ada limits it
> to the platforms it can run on natively is a valid detraction, but it's BUILD
> dependencies really aren't due to the availability of binary packages, the
> PRIMARY product of the ports tree.

> RE: poudriere, it has no dependencies. It's just as appropriate on the dev
> system and adding a jail and configuring it also takes less than 30 minutes.
> Either is very appropriate for a system that must build everything that is run
> on it.

I believe you could cd $PORTSDIR/ports-mgmt/synth and
make package-recursive |& tee build-12amd64.log (or whatever you want to name the log file; this example if for shell tcsh)?

For a system with pkgng, is there any difference in package format between "make install", portmaster and portupgrade?

If your system already has portmaster, you could portmaster ports-mgmt/synth |& tee synth-12amd64.log?

And then switch from portmaster to synth for all further ports builds/updates?

It would not be necessary to start with a clean system for FreeBSD, as opposed to NetBSD, or am I mistaken here?

First port-upgrading tool I used in FreeBSD was portupgrade. Subsequently I switched to portmaster.

Tom

Jeffrey Bouquet via freebsd-ports

unread,
Dec 17, 2016, 7:42:22 AM12/17/16
to
tacking a slightly off-topic topic onto this one


On 12/15/16 10:31 AM, list-free...@jyborn.se wrote:
> On Thu, Dec 15, 2016 at 01:18:05PM -0500, George Mitchell wrote:
>> On 12/15/16 09:40, Warren Block wrote:
>>> On Thu, 8 Dec 2016, Matt Smith wrote:
>>>
>>>> On Dec 08 05:16, Daniil Berendeev wrote:
>>>>> Although portmaster is not releated to the FreeBSD project and is an
>>>>> outside tool, there aren't any alternatives from the project itself. So
>>>>> use it or die. Not a nice situation.
>>>> People have been trying to get portmaster deprecated and removed from
>>>> the handbook but have met with resistance.
>>> Well, yes. Because it works, has no dependencies, and there is no
>>> equivalent replacement. [...]
>> Warren, you have hit the nail on the head. -- George
> +1
>
> I never have problems with portmaster.
> (But portupgrade could at times be an utter mess,
> I never looked back after switching to portmaster
> many years ago.)
>
> And I'm not at all interested in running poudriere
> or synth, thank you.
>
> Peter
> _______________________________________________
> freebs...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-port...@freebsd.org"
synth here: silently fails, year in year out
synth on a new install equal to this one: good to go
poudriere: inexpert, hundreds of .htm saved for howto
portmanager: MOVED, saved hours debugging portupgrade/portmaster until
it started not working so well.
custom .sh : did most of what portmaster did, sometimes very rarely
in use
edited locally portmaster: fail at coding on my part
portmaster: has worked mostly, gave up recently though. Really would
like it fully pkg compliant
portupgrade: features since /var/db/pkg/DIRECTORIES not front-ended by
sqlite3 expertise required have not worked, pkgdb -F for example
IIRC

All of those restored to 2006 goodness and working together seamlessly?
I cannot wait. Possible with enough systemd-too-complex
systems migrated to FreeBSD someday may happen...

But the reason for this add-on to that topic:
due to gcc gcc49 conflicts I was forced to desinstall math/ised.
Now: if one will pardon xclip pasted in formatting:

pkg-static install ised Updating FreeBSD repository catalogue... FreeBSD
repository is up-to-date. All repositories are up-to-date. pkg-static:
hugin has a missing dependency: autopano-sift-C pkg-static: truecrypt
has a missing dependency: fusefs-kmod Checking integrity... done (2
conflicting) - gcc-4.9.4 conflicts with gcc49-4.9.4_1 on
/usr/local/bin/i386-portbld-freebsd11.0-c++49 - gcc-4.9.4 conflicts with
gcc49-4.9.4_1 on /usr/local/bin/i386-portbld-freebsd11.0-c++49 Checking
integrity... done (0 conflicting) The following 33 package(s) will be
affected (of 0 checked):

Installed packages to be REMOVED: truecrypt-7.1a_3 gcc49-4.9.4_1
ircii-20151120 py34-setuptools_scm-1.10.1 py34-msgpack-python-0.4.7
3dm-2.11.00.021_1,1 py34-llfuse-1.0 xalarm-3.06_3 p5-PDF-Template-0.22_1
pdflib-perl-7.0.5_2 xpi-web_developer-1.2.2 pathneck-1.3 mpack-1.6_3
webcopy-0.98b7 lha-ac-1.14i_10 lv-4.51_3 window-1.0 solarized-1.0.0
freefonts-0.10_7 weedit-2.0.3 ncp-1.2.4 shorten-3.6.1 jail-primer-0.2
lynis-2.4.0 win32-codecs-20110131,1 areca-cli-i386-1.14.7.150519,1
pkg_cleanup-2.1 sscalc-1.0 New packages to be INSTALLED: ised: 2.7.1_1
gcc: 4.9.4 dejavu: 2.37 Installed packages to be REINSTALLED: pkg-1.9.4
opera-12.16_6 (options changed)

Number of packages to be removed: 28
Number of packages to be installed: 3
Number of packages to be reinstalled: 2
The operation will free 103 MiB.
13 MiB to be downloaded.
Proceed with this action? [y/N]: n

Additionally, it does not build. Maybe on the other machine that synth
runs on, I could put it on a thumbdrive and install just
the ised binary...

Also had to deinstall scilab, PDL, py27-game, solarwolf, etc etc. IIRC

Nothing urgent at all but my thoughts on this gcc conflict issue as well
as maybe chiming into the topic of this thread
Sorry to waste anyone's time, may be a local issue to this particular
ports tree, some errant file.

Alphons van Werven

unread,
Dec 17, 2016, 6:16:29 PM12/17/16
to
John Marino wrote:

> maybe you could open one final PR and provide a patch that does this?

Fair enough, will do.

Fonz

--
A.J. "Fonz" van Werven <fre...@skysmurf.nl>
Notice: this e-mail address wil expire on Sat 24 Dec 2016.
signature.asc

John Marino

unread,
Dec 17, 2016, 9:16:31 PM12/17/16
to
On 12/17/2016 19:35, Peter Jeremy wrote:
> $ cd /usr/ports/ports-mgmt/synth/ && make
> [ about an hour of grinding away elided ]
> ===> ini_file_manager-03_2 depends on file: /usr/local/gcc6-aux/bin/ada - not found
> ===> gcc6-aux-20160822 is only for amd64 i386, while you are running armv6.
>
> Overall, a total failure.
>
> OTOH, portmaster installs in a minute or so and runs perfectly well. I fail
> to see why you are so insistant on replacing it with something that doesn't
> work at all.
>

Real smooth there, Slick.

It's been mentioned several times in this thread alone that Ada is only
available for i386 and amd64. I think you already knew that and thus
this is a pure troll.

Use poudriere for non-x86 platforms. armv6 packages are built with
poudriere + QEMU, but I suspect you already knew this as well.

John

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

John Marino

unread,
Dec 19, 2016, 8:45:31 AM12/19/16
to
On 12/19/2016 00:48, Peter Jeremy wrote:
> On 2016-Dec-17 20:16:12 -0600, John Marino <freebsd...@marino.st> wrote:
>> On 12/17/2016 19:35, Peter Jeremy wrote:
>>> $ cd /usr/ports/ports-mgmt/synth/ && make
>>> [ about an hour of grinding away elided ]
>>> ===> ini_file_manager-03_2 depends on file: /usr/local/gcc6-aux/bin/ada - not found
>>> ===> gcc6-aux-20160822 is only for amd64 i386, while you are running armv6.
>>>
>>> Overall, a total failure.
>>>
>>> OTOH, portmaster installs in a minute or so and runs perfectly well. I fail
>>> to see why you are so insistant on replacing it with something that doesn't
>>> work at all.
>>>
>>
>> Real smooth there, Slick.
>>
>> It's been mentioned several times in this thread alone that Ada is only
>> available for i386 and amd64.
>
> Not in this thread, no it hasn't. I went digging and found that it has been
> mentions in some of the other 7 separate "The ports collection has some
> serious issues" that you have started.
>
>> I think you already knew that
>
> Well, I pointed it out to you in February this year and after 10 months,
> nothing has changed, including your persistent desire to get rid of
> portmaster, despite the fact that synth is not a suitable replacement.
>
>> and thus
>> this is a pure troll.
>
> I insist that you retract that insult. In this thread, without any
> qualification, you stated that anyone who used portmaster or portupgrade
> should swap to synth, and gave a process which you know will only work on
> i386 and amd64.
>
>> Use poudriere for non-x86 platforms. armv6 packages are built with
>> poudriere + QEMU, but I suspect you already knew this as well.
>
> I haven't investigated because I haven't had the the need to.
>

I never, not once, tried to "get rid of portmaster". By repeating this
untruth after I already corrected you is trolling. There was a very
small chance you were just ignorant but thanks for admitting you knew
exactly what you were doing and making Dave H. look silly.

What I have (and others) wanted? What would make us happy?

A warning placed on the port (a deprecation message but no expiration
date) and others have suggested the same message at installation time in
the form of a pkg-message.

That's it. Nobody ever tried to remove it.

You are like a die-hard smoker that doesn't want "Cancer kills" stickers
on their cigarette boxes. Ask yourself why you don't want people to be
informed?

Joh

Torsten Zuehlsdorff

unread,
Dec 21, 2016, 6:38:13 AM12/21/16
to

On 21.12.2016 12:32, John Marino wrote:
> On 12/21/2016 01:23, Jim Trigg wrote:
>> Therefore my first
>> assumption was that the problem was the new tool I had just started
>> using. Note: while my phrasing may have been poor, I was not meaning to
>> imply that the tool (poudriere) was necessarily broken, just that I
>> couldn't figure out what was going wrong and that it seemed (based on my
>> data sample) to be poudriere rather than the port. Having now tested
>> using the ports tree directly (make -C
>> /usr/ports/databases/php70-pdo_mysql on a basically clean ports tree
>> with "OPTIONS_SET+= ZTS" in /etc/make.conf) and gotten the same failure
>> as with poudriere, I now have no idea how it worked in portmaster, and
>> acknowledge that it is a problem with the port.
>
> Okay, good.
> Any time you can produce an error with poudriere, it should be easy for
> others to reproduce as well which confirms the error.

The bug was already reported and fixed, but this was not very
transparent for users. I need to dig into the framework to find a
solution (and another user hit by the problem was a little bit faster
than me).

For PHP 7.0 i added a message to make thinks more clear in:
https://svnweb.freebsd.org/changeset/ports/429051

Greetings,
Torsten

George Mitchell

unread,
Dec 26, 2016, 9:02:59 AM12/26/16
to
On 12/26/16 03:11, Thomas Mueller wrote:
> [...]
> What is the current status of portupgrade and portmaster?
>
> Maintained, deprecated or something else?
>
> Tom
> [...]

If "contentious" were an official state, that would be it. Each has
its enthusiastic adherents; others are, shall we say, less enthusiastic.
-- George
0 new messages