How to use Ports under HardenedBSD

[vermad...@gmail.com]

Hi,

the HardenedBSD Handbook states that 'portsnap' should be used to download the Ports tree, I have read that HardenedBSD removed 'portsnap' bucause of its insecurity.

What is the way to use HardenedBSD Ports on HardenedBSD?

Regards.

[Shawn Webb]

On Fri, Dec 08, 2017 at 08:01:22AM -0800, vermad...@gmail.com wrote:
>
> Hi,
>
> the HardenedBSD Handbook states that 'portsnap' should be used to download
> the Ports tree, I have read that HardenedBSD removed 'portsnap' bucause of
> its insecurity.

Good catch. I haven't modified those bits of the Handbook, only added
Chapter 14, the HardenedBSD chapter. I should go through the rest of
the Handbook and make modifications to the necessary bits.

>
> What is the way to use HardenedBSD Ports on HardenedBSD?

For the first time, use the following sequence (from a fresh install):

1. cd ${HOME}
2. fetch --no-verify-peer https://github.com/HardenedBSD/hardenedbsd-ports/archive/master.tar.gz
3. tar -xf master.tar.gz -C /usr/ports --strip-components 1

You now have a ports tree.

Install devel/git or devel/git-lite. Then rm -rf the ports tree and
clone the repo with git:

1. cd /usr/ports/devel/git-lite
2. make install clean BATCH=1
3. cd ${HOME}
4. rm -rf /usr/ports
5. mkdir -p /usr/ports
6. cd /usr/ports
7. git clone https://github.com/HardenedBSD/hardenedbsd-ports.git .

Instead of doing both of those sets of steps, you could just install
git-lite from our pkg repo and skip to step 6 above.

Thanks,

--
Shawn Webb
Cofounder and Security Engineer
HardenedBSD

GPG Key ID: 0x6A84658F52456EEE
GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE

[Joe]

On 12/08/2017 05:06 PM, Shawn Webb wrote:
> On Fri, Dec 08, 2017 at 08:01:22AM -0800, vermad...@gmail.com wrote:

>> What is the way to use HardenedBSD Ports on HardenedBSD?
>
> For the first time, use the following sequence (from a fresh install):
>
> 1. cd ${HOME}
> 2. fetch --no-verify-peer https://github.com/HardenedBSD/hardenedbsd-ports/archive/master.tar.gz
> 3. tar -xf master.tar.gz -C /usr/ports --strip-components 1
>
> You now have a ports tree.
>

Well, depending on what the other people on the wifi felt like serving
you. You might end up with just a meterpreter payload. :-(

Is there no way to get a signed ports tree?

[vermad...@gmail.com]

Hi,

I have tried 'git clone ...' but it failed (two times in a row) like that:

root@hbsd:~ # git clone https://github.com/HardenedBSD/hardenedbsd-ports.git /usr/ports
Cloning into '/usr/ports'...
remote: Counting objects: 4476477, done.
remote: Compressing objects: 100% (179/179), done.
error: index-pack died of signal 9476477), 642.04 MiB | 179.00 KiB/s  
fatal: index-pack failed
root@hbsd:~ # ls /usr/ports/
root@hbsd:~ #

Regards.

[vermad...@gmail.com]

My bad ... or git bad ... the git process (on a 1 GB VM) run out of memory hence the message.

I increased the RAM to 2 GB for that VM and now it works:

root@hbsd:~ # git clone https://github.com/HardenedBSD/hardenedbsd-ports.git /usr/ports
Cloning into '/usr/ports'...

remote: Counting objects: 4476682, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 4476682 (delta 2), reused 1 (delta 1), pack-reused 4476675
Receiving objects: 100% (4476682/4476682), 1.38 GiB | 142.00 KiB/s, done.
Resolving deltas: 100% (2153924/2153924), done.
root@hbsd:~ #

Regards.

[vermad...@gmail.com]

For the record, to update the Ports tree I use 'pull', hope that is the way to update it:

# cd /usr/ports
# git pull

Regards.

[tarj...@gmail.com]

It looks like there is still a problem.

root@rpi3:/usr/ports # git clone https://github.com/HardenedBSD/hardenedbsd-ports.git

Cloning into 'hardenedbsd-ports'...

remote: Counting objects: 4490031, done.

remote: Compressing objects: 100% (6/6), done.

remote: Total 4490031 (delta 2), reused 1 (delta 1), pack-reused 4490024

Receiving objects: 100% (4490031/4490031), 1.39 GiB | 338.00 KiB/s, done.

error: index-pack died of signal 92304)

fatal: index-pack failed

root@rpi3:/usr/ports #

[Shawn Webb]

The RPI3 is so limited in RAM that it cannot use git to clone the
ports repo. The size of the ports repo exceeds the limits of the RPI3.

Thanks,

Shawn

Tor-ified Signal: +1 443-546-8752

[tarj...@gmail.com]

I have added a 4GB page file, so it might be able to complete now.

[tarj...@gmail.com]

The 4GB page file worked fine. It was a bit slow, but the machine muddled through.

I don't know how the git stuff works, but I would think that it would be something that would be suitable for sqlite on small machines perhaps? Somehow I think there is something that is not right if 1GB of memory is not enough for receiving the ports collection.

Greetings

On Sunday, 31 December 2017 17:38:23 UTC+1, Shawn Webb wrote:

[Oliver Pinter]

As workaround we could provide a predownloaded git ports - what is signed, and put them on HardenedBSD.org. This solves the git clone problem on small machines. 

[Goran Mekić]

Untill then try --depth=1 with git clone.
--
FreeB(eer)S(ex)D(drugs) are the real demons

[tarj...@gmail.com]

It seem to work fine.

I see no page file use. And it works much, much faster than it did without the depth parameter.

root@rpi03-01:/usr/ports # git clone --depth=1 https://github.com/HardenedBSD/hardenedbsd-ports.git

Cloning into 'hardenedbsd-ports'...

remote: Counting objects: 167541, done.

remote: Compressing objects: 100% (153676/153676), done.

remote: Total 167541 (delta 10808), reused 114759 (delta 9293), pack-reused 0

Receiving objects: 100% (167541/167541), 63.86 MiB | 306.00 KiB/s, done.

Resolving deltas: 100% (10808/10808), done.

Checking connectivity: 167541, done.

Checking out files: 100% (133040/133040), done.

root@rpi03-01:/usr/ports #

[Goran Mekić]

Research which parameter to pass to pull in the future otherwise you'll pull whole history

[tarj...@gmail.com]

Actually, it is the other way around.

This app completely breaks the principle of "least surprise". Most people that use the app will use it to pull source code that is to be compiled. That means that the needs these people should be the primary driver. Instead the app caters for a tiny minority which will use it to pull sources for development.

The developers seems to assume that there is always enough memory. Which reminds me of Microsoft. The current Azure Windows Server 2016 virtual servers with 2GB memory can't update itself because it runs out of memory. It does not help to add paging memory.