r1995 - Revamp to redirect to a statically served vnfs once cached. Cleanup headers

[com...@warewulf.lbl.gov]

Author: bsallen
Date: 2016-06-19 22:30:44 -0700 (Sun, 19 Jun 2016)
New Revision: 1995

Modified:
trunk/provision/cgi-bin/vnfs.pl
trunk/provision/etc/warewulf-httpd.conf.in
Log:
Revamp to redirect to a statically served vnfs once cached. Cleanup headers

Modified: trunk/provision/cgi-bin/vnfs.pl
===================================================================
--- trunk/provision/cgi-bin/vnfs.pl 2016-06-20 01:03:02 UTC (rev 1994)
+++ trunk/provision/cgi-bin/vnfs.pl 2016-06-20 05:30:44 UTC (rev 1995)
@@ -74,17 +74,10 @@
my $cache_in_progress;

#&nprint("Sending VNFS '$vnfs_name' to node '$node_name'\n");
- $q->print("Content-Type: application/octet-stream; name=\"vnfs.img\"\r\n");
- if (my $size = $obj->size()) {
- $q->print("Content-length: $size\r\n");
- }
- $q->print("Content-Disposition: attachment; filename=\"vnfs.img\"\r\n");
- $q->print("\r\n");

if (! $vnfs_nocache) {
if (-f "$vnfs_cachedir/$vnfs_name/image.$vnfs_checksum") {
&dprint("Found VNFS cache\n");
- # Perhaps do a better job checking here... one day.
$use_cache = 1;
} else {
&dprint("Building VNFS cache\n");
@@ -132,62 +125,44 @@

if ($use_cache) {
&dprint("Sending cached VNFS\n");
- my $cache_fh;
- if (open($cache_fh, "$vnfs_cachedir/$vnfs_name/image.$vnfs_checksum")) {
- my $buffer;
- while($buffer = <$cache_fh>) {
- $q->print($buffer);
- }
- close($cache_fh);
- } else {
- &eprint("Can't open VNFS cache!\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 500\r\n");
- $q->print("\r\n");
- }
+ print $q->redirect("/WW/vnfs_cache/$vnfs_name/image.$vnfs_checksum");

} elsif(! $cache_in_progress) {
&dprint("Sending VNFS from the data store\n");
+ $q->header(-type=>'application/octet-stream',
+ -status=>'200',
+ -attachment=>'vnfs.img');
+ if (my $size = $obj->size()) {
+ $q->header(-Content_length=>$size);
+ }
my $binstore = $db->binstore($obj->get("_id"));
while(my $buffer = $binstore->get_chunk()) {
$q->print($buffer);
}
} else {
&eprint("VNFS is being cached via a different request, try again.\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 503\r\n");
- $q->print("\r\n");
+ $q->header( -status => '503 Service Unavailable' );
}

} else {
&eprint("VNFS request for an unknown VNFS (VNFSID: $vnfsid)\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 404\r\n");
- $q->print("\r\n");
+ $q->header( -status => '404 Not Found' );
}
} else {
&eprint("$node_name has no VNFS set\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 404\r\n");
- $q->print("\r\n");
+ $q->header( -status => '404 Not Found' );
}
} else {
&eprint("VNFS request for an unknown node (HWADDR: $hwaddr)\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 404\r\n");
- $q->print("\r\n");
+ $q->header( -status => '404 Not Found' );
}
} else {
&eprint("VNFS request for a bad hwaddr\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 404\r\n");
- $q->print("\r\n");
+ $q->header( -status => '404 Not Found' );
}
} else {
&eprint("VNFS request without a hwaddr\n");
- $q->print("Content-Type: application/octet-stream\r\n");
- $q->print("Status: 404\r\n");
- $q->print("\r\n");
+ $q->header( -status => '404 Not Found' );
}

# vim: filetype=perl:syntax=perl:expandtab:ts=4:sw=4:

Modified: trunk/provision/etc/warewulf-httpd.conf.in
===================================================================
--- trunk/provision/etc/warewulf-httpd.conf.in 2016-06-20 01:03:02 UTC (rev 1994)
+++ trunk/provision/etc/warewulf-httpd.conf.in 2016-06-20 05:30:44 UTC (rev 1995)
@@ -8,7 +8,8 @@
# This is disabled as RHEL6 perl_mod seems to be missing this support
#PerlPreConnectionHandler Apache2::Reload

-Alias /WW/static /usr/share/warewulf/www
+Alias /WW/static @fulldatadir@/warewulf/www
+Alias /WW/vnfs_cache /var/tmp/warewulf_cache

ScriptAlias /WW/file @fulllibexecdir@/warewulf/cgi-bin/file.pl
ScriptAlias /WW/script @fulllibexecdir@/warewulf/cgi-bin/script.pl
@@ -30,6 +31,16 @@
AllowOverride None
Order allow,deny
Allow from all
+ <IfVersion >= 2.4>
+ Require all granted
+ </IfVersion>
</Directory>

-
+<Directory /var/tmp/warewulf_cache>
+ AllowOverride None
+ Order allow,deny
+ Allow from all
+ <IfVersion >= 2.4>
+ Require all granted
+ </IfVersion>
+</Directory>

[Jason Stover]

Ben,

> print $q->redirect("/WW/vnfs_cache/$vnfs_name/image.$vnfs_checksum");

( ... )
> Alias /WW/vnfs_cache /var/tmp/warewulf_cache

At the top of vnfs.pl we're defining $vnfs_cachedir .... I normally
point this to a tmpfs (which maybe systemd does for /var/tmp now???)
for my file/vnfs cache and build directories.[1] So, instead keep what
you have, move to instead change the warewulf-httpd.conf to be
pointing at where we want the cache stored instead?

-J

[1] - Like /tmp/warewulf/{build,files,vnfs} ... where /tmp is mounted
tmpfs. rc.local on provisioner builds out the structure and sets
permissions on a reboot.

> --
> You received this message because you are subscribed to the Google Groups "Warewulf-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf-deve...@lbl.gov.
>

[Allen, Benjamin S.]

The httpd systemd unit file in RHEL7 has PrivateTmp enabled. This creates a /var/tmp/systemd-private-<lots of text>-httpd.<pid?>/tmp. Systemd bind mounts this to /var/tmp and /tmp under the httpd's private mount namespace, i.e. httpd sees this systemd-private/tmp directory as /tmp and /var/tmp. The systemd-private directory appears to be just a new directory in /var/tmp not a tmpfs mount. httpd cannot see anything you create in the normal /tmp or /var/tmp.

cgi-bin/vnfs.pl had /var/tmp/warewulf_cache hard coded, which is why I stuck with it. If you wanted to use a different directory you'd need to change *both* $vnfs_cachedir in vnfs.pl and the new alias and directory stanza in warewulf-httpd.conf.

Note, if you have selinux enabled on the webserver you'll likely have an issue with using a path outside of /var/tmp, /tmp, or /var/www.

Also in rhel7 checkout systemd-tmpfiles for creating tmpfs mounts on boot.

Any chance you have time to test this out on rhel6, sl6, or centos6?

Ben