FreeBSD support

[Dave Cottlehuber]

Hi,

I’m very excited by the prospect of FreeBSD support in GCE:

http://googlecloudplatform.blogspot.co.at/2013/12/google-compute-engine-is-now-generally-available.html

However I’ve not found any way of launching one yet, and there’s apparently no docs yet:

  https://developers.google.com/search/results?q=freebsd&p=%2Fcompute

When I list images for my project, I see the usual debian + centos suspects only.

Do I need some special beta tag, or am I jumping the shark here?

A+
Dave

[Paul Rashidi]

Hello Dave and all,

 We've had a few posts regarding getting up and running with FreeBSD over the past few days. The team has put together a best-effort posting that will get most, if not all, of you up and running. That being said, we need to remind you that FreeBSD is being supported on Google Compute Engine by the community. The instructions are being provided as-is and without warranty.

1. Download, build, and install qemu stable-1.6.on your local machine.

Download information can be found here: http://wiki.qemu.org/Download

Note: Version 1.6 or greater is required and we aren't able to support qemu install/setup questions and there are alternative virtual environments with UI tools that can be used in place of qemu.

2. Download FreeBSD.

`wget ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/9.2/FreeBSD-9.2-RELEASE-amd64-disc1.iso`

3. Create qemu disk image.

`qemu-img create disk.raw 10g`

> Formatting 'disk.raw', fmt=raw size=10737418240

Note: the disk format must be raw.

4. Launch a qemu VM and boot from the FreeBSD ISO.

qemu-system-x86_64 --enable-kvm -smp 1 -m 3750m -net nic,model=virtio -net user -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd,physical_block_size=4096 -drive if=none,id=hd,file=disk.raw,cache=none -cdrom FreeBSD-9.2-RELEASE-amd64-disc1.iso

Note: The --enable-kvm parameter allows for increased speed but is not required and should be left off if you are running the qemu command in a virtual environment such as Google Compute Engine.

5. Install FreeBSD.

a) Select ‘Install’ from the initial installer screen.

b) Select ‘No’ for non-default keyboard mapping.

c) Choose a temporary hostname such as `freebsdhost.`

d) Select ‘OK’ to select the default installed packages.

e) Choose ‘Guided’ for partitioning setup.

f) Select ‘Entire disk’ for the partition setup.

g) Click ‘Finish’ for the Partitioning Editor confirmation.

h) Choose ‘Commit’ to start formatting the disk image.

i) Select (and remember) a password for root account. This should obviously be a very strong password.

j) Select "Virtio networking adapter" during Network Configuration.

h) Choose ‘Yes’ to configure IPv4.

i) Choose ‘Yes’ for DHCP.

j) Select ‘No’ for IPv6.

k) Configure DNS resolution to use 169.254.169.254 as your DNS server and `google.internal.` as your search directive.

l) Confirm that your machine clock is UTC.

m) Install sshd and ntpd to run at boot time.

n) Select ‘Yes’ for crash dumps.

o) Select ‘No’ for additional user accounts. We will configure them later.

p) Select ‘Exit’ to finish with the installer and choose ‘Yes’ to fall into a shell for a few more manual steps.

6. Additional system configuration.

Type the following commands to setup a few last items.

Serial console configuration

`echo 'console="comconsole"' >>/etc/rc.conf`

`echo ‘console=”comconsole”’ > /boot/loader.conf`

`sed -I -e “/hostname/d” /etc/rc.conf`

`echo -Dh >/boot.config`

Networking commands:

`cat <<EOF >/etc/dhclient.conf

interface "vtnet0" {

 supersede subnet-mask 255.255.0.0;

}

EOF`

Time server config:

`sed -I -e “/server/d” /etc/ntp.conf`

`echo “server 169.254.169.254” >> /etc/ntp.conf`

Hosts entry:

`echo “169.254.169.254 metadata.google.internal metadata” >> /etc/hosts`

7. Add a first-time login user and enable password based ssh logins.

a) Create a local user , set [and remember] its non-empty password: `adduser`.

b) Add the local user you created to the ‘wheel’ group.

`pw user mod <user> -G wheel`

c) Allow password-based ssh logins.

`sed -I -e “s/#PasswordAuthentication no/PasswordAuthentication yes/” /etc/ssh/sshd_config`

8. Install additional GCE packages

Make sure public DNS works:

`echo “nameserver 8.8.8.8” >> /etc/resolv.conf`

a) Install sudo, python, and wget:

`pkg_add -r sudo`

`pkg_add -r python`

`pkg_add -r wget`

d) Install gsutil (GCS command line tool)

`/usr/local/bin/wget http://storage.googleapis.com/pub/gsutil.tar.gz`

`tar xfz gsutil.tar.gz -C /usr/local/share`

`ln -s /usr/local/share/gsutil/gsutil /usr/local/bin`

e) Install gcutil (GCE command line tool)

`/usr/local/bin/wget  http://dl.google.com/dl/cloudsdk/release/artifacts/gcutil-1.12.0.tar.gz`

`tar xfz gcutil-1.12.0.tar.gz -C /usr/local/share`

`ln -s /usr/local/share/gcutil-1.12.0/gcutil /usr/local/bin`

Remove public DNS server

`sed -I -e “/8.8.8.8/d” /etc/resolv.conf`

9) Shutdown qemu VM Instance.

`poweroff`

10) Create a GCE image on your local machine.

a.) Install Cloud SDK onto your local machine using these instructions.

b) Package the qemu image

`tar -Szcf freebsd.tar.gz disk.raw`

c) Copy the tarball to GCS.

`gsutil mb gs://<my cloud storage bucket>`

`gsutil cp freebsd.tar.gz gs://<my cloud storage bucket>`

Note: Your cloud storage bucket can be created in Google Cloud Console under the Google Cloud Storage tab of your enabled Project.  Also, you are uploading a complete image and this step's length is dependent on your internet connection. Don't perform this step while tethering. :)

d) Create GCE image.

`gcutil addimage freebsd gs://<my cloud storage bucket>/freebsd.tar.gz`

Note: The creation of the image can take a little bit of time.

11) Start a FreeBSD instance on GCE.

a) Create an instance.

`gcutil addinstance freebsd --image=freebsd --zone=us-central1-a --machine_type=n1-standard-1`

b) Now get the instance external IP.

`gcutil getinstance freebsd | grep external-ip`

> |   external-ip          | A.B.C.D

c) ssh into your instance.

`ssh <user>@A.B.C.D`

The following steps would also need to be performed in order for the FreeBSD image to operate in a similar manner to the Debian and Centos Images that many use on Google Compute Engine. If you'd like to help we do take contributions.

Update the following to support FreeBSD and Install onto image in Step 8. 

https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/google-startup-scripts

https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/google-daemon  

https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/gcimagebundle

Feel free to post here if you have questions but do remember this is being offered in a best-effort fashion.

Thank you and happy computing!

Paul Rashidi

[Bri Hatch]

Wait... does this mean we should be able to try (completely unsupported, I understand!) running images with our own arbitrary kernels? I'd love to get our ExtraHop device up and running so we can start monitoring our gce servers properly.

--
© 2013 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/22daa5ed-3a40-4848-b629-ee6af57296e0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Bri Hatch <b...@extrahop.com>, Director of EHIT.

Nothing so improves your company that the lack of it.

[Paul Rashidi]

Hi Bri,

  As part of our GA launch on 12/3/13 Image-supplied-kernels are the preferred option. You'll need to use a newer version of gcutil or the v1 API. More details in our release notes and docs. 

Thanks,

Paul

[Dave Cottlehuber]

Thanks Paul & GCE team!

While I waited for my 10Gb image to upload, I wouldn't be a hacker if I didn't immediately skip all the instructions and try a different approach ;-).

A few gotchas for folk:

1. Use gnu-tar or fail miserably:

ERROR: INVALID_IMAGE_TAR_INCORRECT_FILE_NAME: The file inside the tar archive is not named 'disk.raw'.

- the naming of the file inside the tarball MUST be disk.raw

- you MUST use gnu-tar above, or the uploaded image will be rejected. For FreeBSD or OSX folk you may need to install it. OSX doesn't seem to support sparse for example. 

2. Use the right filenames, or fail miserably:

Error: Invalid value for field 'resource.name': 'mfsbsd-se-92-RELEASE-amd64'.  Must be a match of regex '[a-z]([-a-z0-9]{0,61}[a-z0-9])

ERROR: INVALID_FIELD_VALUE: Invalid value for field 'rawDisk.source': 'http://storage.googleapis.com/**/mfsbsd-92-RELEASE-amd64.tar.gz'.

- the uploaded tarball, and the destination file blob, MUST match a regex (SRSLY?)  [a-z]([-a-z0-9]{0,61}[a-z0-9]) 

Finally, as mfsbsd is 160MiB, it finished uploading long before the 10Gb image even finished tar. It seems to boot successfully, but is not accessible over network yet. You can see the boot log https://gist.github.com/8032171 suggestions welcomed, including ways to make gnu-tar faster...

[1]: http://mfsbsd.vx.sk/

[2]:  https://gist.github.com/8032171

--

© 2013 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 

[Dave Cottlehuber]

Hey folks,

More exploring & feedback.

1. This also works with a full zfs-based installation (boot direct to zfs). A fringe benefit is that you can also get r/w access to a zpool via http://zfsonlinux.org/ useful for reading failed logs, and updating any mistakes you made to a boot image. Install zol via http://zfsonlinux.org/debian.html

2. I've been unable to access the FreeBSD instance from *outside* GCE no matter what I try. Externally, ssh always fails at SSH2_MSG_KEXINIT:

```shell

$ ssh -vo UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no ans...@173.255.113.12

OpenSSH_6.2p2, OpenSSL 0.9.8y 5 Feb 2013

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Connecting to 173.255.113.12 [173.255.113.12] port 22.

debug1: Connection established.

debug1: permanently_set_uid: 0/0

debug1: identity file /root/.ssh/id_rsa type -1

debug1: identity file /root/.ssh/id_rsa-cert type -1

debug1: identity file /root/.ssh/id_dsa type -1

debug1: identity file /root/.ssh/id_dsa-cert type -1

debug1: identity file /root/.ssh/id_ecdsa type -1

debug1: identity file /root/.ssh/id_ecdsa-cert type -1

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_6.2_hpn13v11 FreeBSD-20130515

debug1: Remote protocol version 2.0, remote software version OpenSSH_6.2_hpn13v11 FreeBSD-20130515

debug1: match: OpenSSH_6.2_hpn13v11 FreeBSD-20130515 pat OpenSSH*

debug1: SSH2_MSG_KEXINIT sent

```

ssh from the same freebsd instance to its own external IP also fails (identical to above).

While the above is from a FreeBSD 9.2 client, the same results on ubuntu 13.10 client also.

At this point I'm speculating wildly, perhaps some subtle key exchange thing I'm not aware of? I'm assuming if we get as far as matching ssh protocol versions, we have a working TCP conn in place & presumably no clever firewalls blocking that, but that doesn't match up with the inability to ssh to the external IP of the same instance.

Ideas?

A+

Dave

[Leif Pedersen]

I've been unable to access the FreeBSD instance from *outside* GCE no matter what I try. Externally, ssh always fails at SSH2_MSG_KEXINIT:

```shell

$ ssh -vo UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no ans...@173.255.113.12

OpenSSH_6.2p2, OpenSSL 0.9.8y 5 Feb 2013

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Connecting to 173.255.113.12 [173.255.113.12] port 22.

debug1: Connection established.

debug1: permanently_set_uid: 0/0

debug1: identity file /root/.ssh/id_rsa type -1

debug1: identity file /root/.ssh/id_rsa-cert type -1

debug1: identity file /root/.ssh/id_dsa type -1

debug1: identity file /root/.ssh/id_dsa-cert type -1

debug1: identity file /root/.ssh/id_ecdsa type -1

debug1: identity file /root/.ssh/id_ecdsa-cert type -1

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_6.2_hpn13v11 FreeBSD-20130515

debug1: Remote protocol version 2.0, remote software version OpenSSH_6.2_hpn13v11 FreeBSD-20130515

debug1: match: OpenSSH_6.2_hpn13v11 FreeBSD-20130515 pat OpenSSH*

debug1: SSH2_MSG_KEXINIT sent

```

After much tinkering, I figured this out. GCE uses a nonstandard MTU on its network. For example, in /etc/rc.local, add

ifconfig vtnet0 mtu 1460

Also, don't forget that GCE's DHCP server is misconfigured. Their network uses a netmask of 255.255.0.0, but the DHCP server offers 255.255.255.255. Paul included this work-around in his configuration in /etc/dhclient.conf:

interface "vtnet0" {

 supersede subnet-mask 255.255.0.0;

}

Many thanks for the help, Paul.

[Sergei Iakhnin]

One thing I've learned about this process today, is that GCE only accepts GNU tar archives for images. If you happen to use a BSD tarball the step of adding your image from the Google Storage bucket will fail with a very uninformative HTTP 503 error code:

 | error-code | 503 |
| error-message | SERVICE UNAVAILABLE |

You can check if you have a GNU tarball by running "file my_image.tar". You should see the word GNU in the output:

ubuntu_12_04.tar: POSIX tar archive (GNU)

Sergei.

[Dave Cottlehuber]

Hi,

This is a Work In Progress but should already be useful enough for
others to play with.

Current Status:
- mfsbsd works in GCE (boots up, is accessible)
- can't yet create a freebsd install via zfsinstall (various geom
related errors, not sure yet why)

I started including all of the requirements from Paul (and Leif thanks
for the mtu tip!) for an msfbsd-based[1] ramdisk live image[5]. This
allows you to boot into a fully ramdisk-based FreeBSD, and then
install FreeBSD or variants directly to a GCE persistent disk with
familiar BSD tools, without needing the qemu steps and upload hassle
in the original approach.

Here's my changes to source [2] and instructions [3], with sample
commit of the GCE settings [4]. Note that it uses a root ssh key[6]
from the vagrant project unless you build the image yourself. I
haven't included the python tools in the image yet, as I'm not sure
how to use new pkgng system within mfsbsd yet.

[1]: http://mfsbsd.vx.sk/
[2]: https://github.com/jsonified/mfsbsd
[3]: https://github.com/jsonified/mfsbsd/blob/master/BUILD
[4]: https://github.com/jsonified/mfsbsd/commit/a99319154f86b2c346217bfaf3ecab89d24e899b
[5]: http://commondatastorage.googleapis.com/jsonified/mfsbsd10-prerelease-amd64-20131230-r260064.tar.gz
[6]: https://github.com/mitchellh/vagrant/blob/master/keys/vagrant

To use my image (not that I'm recommending it, for security reasons,
but your choice):

# make a bootable image from my tar.gz
IMG=mfsbsd10-prerelease-amd64-20131230-r260064
gcutil addimage $IMG gs://jsonified/$IMG.tar.gz

# make an instance off that image; optionally attach another disk for
a future installation
gcutil addinstance --machine_type=g1-small --image=$IMG freebsd10pre
# repeat next line until ssh daemon is ready
gcutil getserialportoutput freebsd10pre

# get the ip
gcutil listinstances
# connect and play:
ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o
StrictHostKeyChecking=no -i /path/to/vagrant/key -A -p 22
root@ip_address

Basic steps to roll your own image:

- on amd64 FreeBSD 9.2 with cdrtools, gtar, freebsd source & ports
tree available
- checkout source [2] on GCE branch and modify ./conf and ./files as required
- either mount an appropriate release .iso to /cdrom, or build a
custom kernel/world
- I used a FreeBSD10.0-PRERELEASE as the target as for some reason
mfsbsd doesn't like 9.2-STABLE.
- `make gce` should produce a nice .tar.gz file for you quite quickly
- upload that into GS & turn it into an image as above

Feedback welcomed, esp if you think this is useful enough to finish it
up properly.

A+
Dave

> --
> © 2013 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
>
> Email preferences: You received this email because you signed up for the
> Google Compute Engine Discussion Google Group
> (gce-dis...@googlegroups.com) to participate in discussions with other
> members of the Google Compute Engine community and the Google Compute Engine
> Team.
> ---
> You received this message because you are subscribed to the Google Groups
> "gce-discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gce-discussio...@googlegroups.com.
> To post to this group, send email to gce-dis...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gce-discussion/01eed99e-af76-4405-957f-83663ba5b55b%40googlegroups.com.

[Jimmy Kaplowitz]

Several of you on this thread may be interested in our recommendations on building Google Compute Engine images:

https://developers.google.com/compute/docs/building-image

The GNU tar format and MTU requirements are documented in that page, along with other useful guidance.

Some of the details listed are particular to GNU/Linux, but all the concepts apply to other OSes such as FreeBSD, even if the specifics may vary.

Feedback on our documentation is always welcome.

- Jimmy Kaplowitz

jkapl...@google.com

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/CAL%2BY1nsXjfpaSeGJ6Ei777uW_jBzOvhaGVbs76dZV8eBhtEFJw%40mail.gmail.com.

[Julian Stecklina]

I've tried 10.0 RC4 on a GCE f1-micro instance and have seen two issues:

FreeBSD selects TSC-low as timecounter and this gives me wrong time (time runs at approx. 1/3 of normal). Setting kern.timecounter.hardware=ACPI-safe in /boot/loader.conf fixes this.

I have seen an UFS panic on two different instances. I am right now trying to figure out whether I have done something wrong or whether there is a bug...

HTH, Julian

[Leif Pedersen]

I was able to install FreeBSD on GCE in a very direct way, without the hurdles of Qemu, gcutil, and the FreeBSD installer. All are great tools, but I think my apporach is somewhat more suitable for GCE, or at least much simpler. Much of this is well-known practice for installing manually, with some fluff for managing it as a disk image, as well as the few odd networking requirements of GCE (MTU and netmask).

My basic approach is to:

* Use my FreeBSD machine at home to build a self-hosting "rescue" disk image for GCE.

* Start a temporary Linux machine in GCE from a provided image.

* Copy my image from home to a raw block device.

* Start a FreeBSD machine in GCE using that block device as the root.

Once bootstrapped, I used this "rescue" machine to turn up more machines, then shut down this machine. I kept this "rescue" disk image handy for future rescues and turn-ups. Naturally, this is the bare minimum; you'll still want to install gcutil and stuff once you have your space bootstrapped. Although I typically prefer ZFS, I used UFS here for a smaller memory footprint and because the data on this filesystem is not precious. Either GPT or legacy partitioning schemes work.

1. On your FreeBSD machine at home, create a disk image:

# truncate -s 2G /tmp/rescue.img

# mdconfig -a -t vnode -f /tmp/rescue.img

[adjust below if it says other than md0]

2a. Partition it with gpart:

# gpart create -s gpt /dev/md0

# gpart add -s 222 -t freebsd-boot -l rescue-boot md0

# gpart add -t freebsd-ufs -l rescue-root md0

# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 md0

# newfs -U /dev/md0p2

# mount /dev/md0p2 /mnt

2b. OR partition it the legacy way:

# fdisk -B -b /boot/boot0 /dev/md0

# bsdlabel -w -B /dev/md0s1 

# newfs -U /dev/md0s1a

# mount /dev/md0s1a /mnt

3. Install the base OS:

# tar -C /mnt -xpf base.txz

# tar -C /mnt -xpf doc.txz

# tar -C /mnt -xpf games.txz

# tar -C /mnt -xpf kernel.txz

# tar -C /mnt -xpf lib32.txz

[Add ports.txz and/or src.txz if you prefer.]

4. Configure stuff:

# chroot /mnt csh

# newaliases

# passwd root

[Set a password.]

# mkdir /root/.ssh

# cat > /root/.ssh/authorized_keys << EOF

[your key]

EOF

# echo '/dev/da0p2 / ufs rw,noatime,suiddir 1 1' > /etc/fstab  [change da0p1 to da0s1a if old style partition tables]

# echo -Dh > /boot.config

# echo 'console="comconsole"' > /boot/loader.conf

# cat > /etc/rc.conf << EOF

console="comconsole"

hostname="rescue"

ifconfig_vtnet0="DHCP"

ntpd_enable="YES"

ntpd_sync_on_start="YES"

sshd_enable="YES"

EOF

# cat > /etc/ssh/sshd_config << EOF

PasswordAuthentication no

PermitRootLogin yes

UseDNS no

UsePAM no  # Otherwise PAM will allow password auth.

Subsystem sftp /usr/libexec/sftp-server

EOF

# cat > /etc/ntp.conf << EOF

server 169.254.169.254 burst iburst

EOF

# cat > /etc/dhclient.conf << EOF

interface "vtnet0" {

  supersede subnet-mask 255.255.0.0;

}

EOF

# cat > /etc/rc.local << EOF

ifconfig vtnet0 mtu 1460

EOF

# exit

(You may also want to change your timezone from UTC.)

5. Dismount the image:

# umount /mnt

# mdconfig -d -u md0

6. Start a Linux machine on GCE. If you'll be subsequently creating machines with ZFS, choose one with enough RAM.

7. Copy rescue.img to it. (Remember, it must be dismounted first.)

8. Create a blank disk for FreeBSD and attach it to your Linux machine. I suggest naming the disk Rescue.

9. cat rescue.img > /dev/sdb [assuming your FreeBSD disk is sdb]

10. Detach Rescue from your Linux machine.

11. Create a new machine with Rescue attached as root.

12. After allowing time for it to boot, you should be able to ssh in as root.

13. Destroy the Linux machine to save expense, if desired.

[mmitchel]

FreeBSD 10 needs an addition to /etc/rc.conf

local_enable="YES" # process /etc/rc.local, /etc/rc.shutdown

either that or the mtu parameter needs to be supplied in a different manner :)

[Dave Cottlehuber]

Thanks Michael!

To clarify, you mean that IFF local_enable="YES" is set, then MTU
discovery will work correctly?

For 9.2-RELEASE I needed to hardwire this in via
https://github.com/jsonified/mfsbsd/blob/fdf0e534759c1ec4f9bbc5e6f081318a737cf330/conf/rc.conf#L10
which is a total hack.

Have you had any luck with picking up the subnet mask, or did you also
need custom settings?

https://github.com/jsonified/mfsbsd/blob/fdf0e534759c1ec4f9bbc5e6f081318a737cf330/files/etc/dhclient.conf#L2

A+
Dave

> --
> © 2013 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
>
> Email preferences: You received this email because you signed up for the
> Google Compute Engine Discussion Google Group
> (gce-dis...@googlegroups.com) to participate in discussions with other
> members of the Google Compute Engine community and the Google Compute Engine
> Team.
> ---
> You received this message because you are subscribed to the Google Groups
> "gce-discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gce-discussio...@googlegroups.com.
> To post to this group, send email to gce-dis...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gce-discussion/8a8e8fbc-c973-4188-9dcf-fb703fb847a5%40googlegroups.com.

[mmitchel]

freebsd-10 doesnt pull in /etc/rc.local "legacy" without the "local_enable" entry to rc.conf. it took me about an hour to figure that out -- the handling structure at boot and shutdown really wants to have specific scripts located in /etc/rc.d now.

--mdm

[Leif Pedersen]

For the list archive, he used:

ifconfig_vtnet0="SYNCDHCP mtu 1460"

That's much better than my messy rc.local recommendation.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/CAL%2BY1ntUOM%3DJAj%3DuniMY8_VeR6cqhT6PQMAZ5HBchuHQQnkE4Q%40mail.gmail.com.

[mmitchel]

i find it kind of interesting now that I have had a little bit of play time on using 10-RELEASE that I am

now having an issue trying to update to 10-STABLE.

1. svnlite update http://svn.freebsd.org/base/stable/10 : works

2. cd /usr/src; mergemaster -p; make buildkernel : works

3. make installkernel; reboot : does not work as expected

the console output indicates the kernel is not probing successfully for da0. however

the kernel does boot, then hang at trying to mount the root filesystem.

any suggestions?

[Otto Bretz]

I'm have run into some problems while testing FreeBSD on GCE. I cannot communicate between instances and sometimes all network access to instances are lost. I think the proper routes are not picked up correctly.

Long description here: https://groups.google.com/d/msg/gce-discussion/gWNAsJ-_9-s/iSvUE9nBEEcJ

Have anyone else seen this?

[Leif Pedersen]

Andy corrected me about the netmask on another thread. (Thanks, Andy.) For those finding this thread with the other FreeBSD details, here's the better answer.

On Fri, May 2, 2014 at 11:27 AM, Andy Carrel <w...@google.com> wrote:

The DHCP server is actually advertising the correct netmask (255.255.255.255), the thing that is going wrong for FreeBSD users appears to be the routing necessary to reach the gateway. Luckily we can fix this in the dhclient.conf. If you're using the default network configuration, 10.240.0.0/16, gateway 10.240.0.1, dhclient.conf should contain this:

interface "vtnet0" {
  default classless-routes 32, 10, 240, 0, 1, 0, 0, 0, 0, 0, 10, 240, 0, 1;
}

This causes the /sbin/dhclient-script to install two static routes. The first route, from the [32, 10, 240, 0, 1, 0, 0, 0, 0] part, is saying that 10.240.0.1/32 is available directly on the network interface (RFC 3442 describes this as "Local Subnet Routes"). The second route, from the [0, 10, 240, 0, 1] part, is saying that the route to 0/0, the default route, is through the gateway 10.240.0.1.

On most Linux distributions, the /sbin/dhclient-script automatically installs the x.x.x.x/32 route for the default router if the netmask for the interface is 255.255.255.255, which is why this problem hasn't come up with that software.

I've got 10-STABLE booting, running, and communicating between VMs and the outside world successfully in this configuration. More information about the classless-routes DHCP option can be found in RFC 3442.

Hope this helps.
-- wac

[Leif Pedersen]

It looks like FreeBSD recently fixed this. Adjusting the routing seems unnecessary in FreeBSD 10. I forgot to on a new instance and everything worked fine, and I see the /32 route specifying that the default route is on the local link.

--

As implied by email protocols, the information in this message is not confidential.  Any middle-man or recipient may inspect, modify, copy, forward, reply to, delete, or filter email for any purpose unless said parties are otherwise obligated.  As the sender, I acknowledge that I have a lower expectation of the control and privacy of this message than I would a post-card.  Further, nothing in this message is legally binding without cryptographic evidence of its integrity.

http://bilbo.hobbiton.org/wiki/Eat_My_Sig

[Andy Carrel]

FreeBSD's behavior hasn't changed. What you're observing is actually a recent improvement in Google Compute Engine that adds classless-route information to the DHCP replies that are sent to instances. We hope you find it helpful. :-)

[Leif Pedersen]

Oh, very nice! Thank you.

[jan winter]

Hi. I having massive time drift problems, with freebsd release 10. I didn't try any other version at the moment.

The time drifts are so big that the base ntpd and openntpd are not fast enough to correct the time.

Anyone else see that problem? This instance is f1-micro.

  # /usr/local/sbin/ntpd -s -v -d

ntp engine ready

reply from 169.254.169.254: offset 4.471439 delay 0.000100, next query 5s

set local clock to Sat Jun 28 11:37:08 BST 2014 (offset 4.471439s)

reply from 169.254.169.254: offset 11.283267 delay 0.000080, next query 5s

reply from 169.254.169.254: offset 22.590066 delay 0.000083, next query 8s

peer 169.254.169.254 now valid

reply from 169.254.169.254: offset 40.637657 delay 0.000086, next query 5s

reply from 169.254.169.254: offset 51.931316 delay 0.000113, next query 6s

reply from 169.254.169.254: offset 65.555604 delay 0.000087, next query 9s

reply from 169.254.169.254: offset 85.870822 delay 0.000093, next query 32s

reply from 169.254.169.254: offset 158.053123 delay 0.000079, next query 31s

adjusting local clock by 158.053123s

reply from 169.254.169.254: offset 69.536857 delay 0.000108, next query 34s

[Venkatesh Srinivas]

Which timecounter are you using? ('sysctl -a | grep timecounter' ;
look for kern.timecounter.hardware and kern.timecounter.choice).

-- vs;

[jan winter]

timecounter.hardware was set to TSC-low. I switch it to ACPI-safe.

That look much better. Thanks a lot

ntp engine ready

reply from 169.254.169.254: offset 22558.032794 delay 0.000514, next query 6s

set local clock to Sat Jun 28 20:41:39 BST 2014 (offset 22558.032794s)

reply from 169.254.169.254: offset 0.000073 delay 0.000321, next query 5s

reply from 169.254.169.254: offset 0.000167 delay 0.000388, next query 5s

peer 169.254.169.254 now valid

reply from 169.254.169.254: offset 0.000253 delay 0.000326, next query 8s

reply from 169.254.169.254: offset 0.000407 delay 0.000314, next query 6s

reply from 169.254.169.254: offset 0.000586 delay 0.000425, next query 5s

reply from 169.254.169.254: offset 0.000749 delay 0.000304, next query 34s

reply from 169.254.169.254: offset 0.003783 delay 0.000309, next query 31s

adjusting local clock by 0.000749s

reply from 169.254.169.254: offset 0.003761 delay 0.000309, next query 30s

[Andy Carrel]

I'm saw some pretty substantial drift from ACPI-safe on an n1-highcpu-8 that was CPU bound. i8254 seems more stable in a quick bit of testing I've done.

--
© 2014 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---

You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/YWoa3Aa_49U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussio...@googlegroups.com.

To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/dcda369e-37e6-402f-b06f-26adf0affd0c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

wac@google.com

[Peter Jeremy]

On Monday, 30 June 2014 05:49:12 UTC+10, Andy Carrel wrote:

I'm saw some pretty substantial drift from ACPI-safe on an n1-highcpu-8 that was CPU bound. i8254 seems more stable in a quick bit of testing I've done.

 

I have been trying a 10-RELEASE kernel in a f1-micro instance.  With timecounter.hardware=i8254, the time would just stop after 15-20m, irrespective of load (date(1) reports the same time).  With ACPI-safe, the ciock is stepping occasionally under load but at least it stays running.

The clock failure with i8254 is presumably due to:

Jul 20 08:52:02 instance-1 kernel: attimer0: <AT timer> at port 0x40 on isa0

Jul 20 08:52:02 instance-1 kernel: Timecounter "i8254" frequency 1193182 Hz quality 0

Jul 20 08:52:02 instance-1 kernel: attimer0: Can't map interrupt.

but I haven't tracked down the reason for that yet.

[d...@jsonified.com]

FYI the latest mfsBSD[1] image based off FreeBSD-10.0 amd64 works
brilliantly in GCE now, with no changes needed.

- download the mfsbsd image as disk.raw
- gnu-tar it
- upload to your google storage
- register the upload as a disk image
- spin up an mfsbsd image directly in GCE

```
curl -o disk.raw
http://mfsbsd.vx.sk/files/images/10/amd64/mfsbsd-se-10.0-RELEASE-amd64.img
gtar -Szcf mfsbsd-se-10.0-RELEASE-amd64.tar.gz disk.raw
gsutil cp mfsbsd-se-10.0-RELEASE-amd64.tar.gz gs://skunkwerks-images/
gcutil addimage mfsbsd10-0
gs://skunkwerks-images/mfsbsd-se-10.0-RELEASE-amd64.tar.gz
```

And then your usual:

```
gcloud compute instances create --zone $ZONE --project $PROJECT \
--image mfsbsd10-0 <new-instance>
gcloud compute instances get-serial-port-output <new-instance>
```

The serial output will of course stop partway through the mfsbsd boot
process when it switches consoles, if this worries you, it's still
possible to roll your own image.

You can then use a custom `zfsinstall` script (mine is
http://people.apache.org/~dch/configs/mfsbsd/zfsinstall ) and
deploy FreeBSD directly to a new disk image, with any customisations you
require. This is absolutely brilliant.

NB my image above isn't publically accessible as it's so easy to create
your own one now.

A+
Dave

[1]: http://mfsbsd.vx.sk/

[bthur...@gmail.com]

I know this post is old but I have tried the info to install FreeBSD and everytime I only get cant find XXXX.iso, When I get to step 4 it says cannot find ISO file. I currently have the ISO file copied to a DVD disk but it does not find it. If it is my understanding it has to be the raw ISO Image? not expanded.
Any help would be appreciated.

B. Thurman

Hello Dave and all,

interface "vtnet0" {

 supersede subnet-mask 255.255.0.0;

}

EOF`

The following steps would also need to be performed in order for the FreeBSD image to operate in a similar manner to the Debian and Centos Images that many use on Google Compute Engine. If you'd like to help we do take <a href="https://github.com/GoogleCloudPlatform/compute-image-packages/blob/

...

[Adrián Barbáchano]

Hi,

You don't need to put the ISO in the DVD, you just need to follow the steps provided and the qemu command will use the .iso file located in your current directory.

Once you have installed the OS you can remove the .iso from your computer.

Regards,

Adrián.

[Brian Mitchell]

Thanks Dave!

I've been able to reproduce part of this but I'm having trouble accessing my instance. My reply is inline.

On Monday, September 8, 2014 5:58:36 AM UTC-4, Dave Cottlehuber wrote:

FYI the latest mfsBSD[1] image based off FreeBSD-10.0 amd64 works
brilliantly in GCE now, with no changes needed.

- download the mfsbsd image as disk.raw
- gnu-tar it
- upload to your google storage
- register the upload as a disk image
- spin up an mfsbsd image directly in GCE

```
curl -o disk.raw
http://mfsbsd.vx.sk/files/images/10/amd64/mfsbsd-se-10.0-RELEASE-amd64.img
gtar -Szcf mfsbsd-se-10.0-RELEASE-amd64.tar.gz disk.raw
gsutil cp mfsbsd-se-10.0-RELEASE-amd64.tar.gz gs://skunkwerks-images/
gcutil addimage mfsbsd10-0
gs://skunkwerks-images/mfsbsd-se-10.0-RELEASE-amd64.tar.gz
```

This seems to work well for me. I've made sure to gtar it and the image shows up in my images list.

 

And then your usual:

```
gcloud compute instances create --zone $ZONE --project $PROJECT \
  --image mfsbsd10-0  <new-instance>
gcloud compute instances get-serial-port-output <new-instance>
```

The serial output will of course stop partway through the mfsbsd boot
process when it switches consoles, if this worries you, it's still
possible to roll your own image. 

Using the equivalent gcutil command like:

    gcutil addinstance --project <project_name> --zone us-central1-f --image mfsbsd10-0 --machine_type f1-micro <instance_name>

It seems to boot the instance and then retrieving the shell output after giving it a minute or two for boot shows a that it got to the login prompt.

The next step I try is to access the shell itself and I can't seem to do that via SSH, the gcutil ssh, nor the web interface. Am I missing something

here or is it possible that this VM isn't working as it should?

 

Brian.

[vigyanl...@gmail.com]

hi all 

I'm not able to get the ssh to my freebsd after i followed the steps given here 

can anybody help

...

[Steve Wills]

Since this seems to be one of the first hits in Google search for anything FreeBSD and Google Compute Engine related, it seems like a good place to post that I've been working on a script to build an image to run FreeBSD on Google Compute Engine. See:

https://plus.google.com/112202779615695172291/posts/eYajb8JKerY

for more details.

Thanks,

Steve

[moul...@gmail.com]

Hello Paul,

I am looking for a person who is able to install and run PFSense on Google Compute Engine (GCE).

If you are interested, please let me know the price to complete this job. If we are satisfied with you, we may depend on you to configure the PFSense as per our requirement for additional amount.

Thank you.
Chandra.

[Paul Nash]

Hi Joseph, the post you're replying to is pretty old. More recently we've worked with the FreeBSD maintainers to implement most of the steps necessary to create a well-functioning pre-built image, which I believe is the one you mentioned on a different thread. I'm replying to address this open question, but it's probably easier to continue discussing the network issue on that other thread.

Thx,

-P

On Sun, Jul 17, 2016 at 1:14 AM, Joseph McFall <en...@canvas.social> wrote:

I've been using the gcloud method to generate a freebsd instance. However I'm setting up nginx and can't get the welcome to nginx page to display. Usually when working with D.O. I just install nginx and go to http://ip_address and it works. I've read the linux tutorial on this and it seems like the same process pretty straight forward. Is their an entirely different process for FreeBSD? 

Hello Dave and all,

The following steps would also need to be performed in order for the FreeBSD image to operate in a similar manner to the Debian and Centos Images that many use on Google Compute Engine. If you'd like to help we do take contributions.

Update the following to support FreeBSD and Install onto image in Step 8. 

https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/google-startup-scripts

https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/google-daemon  

https://github.com/GoogleCloudPlatform/compute-image-packages/tree/master/gcimagebundle

Feel free to post here if you have questions but do remember this is being offered in a best-effort fashion.

Thank you and happy computing!

Paul Rashidi

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---

You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.

To post to this group, send email to gce-dis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/eba06c3b-7f41-47d4-a3fd-9f9dd8bfed7f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Paul R. Nash |

Product Manager, Compute Engine |

paul...@google.com |

206-876-1620

[Andrew Hingston]

Hi, 

Did you ever get PFSense runnng on GCE? 

Thanks,