URL rewrites and Reposado

[Greg Neagle]

Apple's software update server has the ability to offer the "correct" catalog to a client that requests simply "index.sucatalog". This ability was first added with OS X Server 10.6.6, and involved the use of Apache mod_rewrite.

Lion Server has a similar capability, but this is done via a CGI instead.

Since Reposado doesn't handle the web-serving part of offering Apple software updates, Reposado itself cannot provide a similar feature: instead you must configure your web server to do URL rewrites, or write your own CGI to provide this functionality.

I wanted to do URL rewrites, but also wanted to keep things modular. Since I'm using Apache2 as my web server, it seemed that using an .htaccess file at the root of my Reposado repo would do the trick.

I'm currently testing these .htaccess file rewrite rules at the root of my reposado repo:

RewriteEngine On
Options FollowSymLinks
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} Darwin/8
RewriteRule ^index(.*)\.sucatalog$ content/catalogs/index$1.sucatalog [L]
RewriteCond %{HTTP_USER_AGENT} Darwin/9
RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-leopard.merged-1$1.sucatalog [L]
RewriteCond %{HTTP_USER_AGENT} Darwin/10
RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-leopard-snowleopard.merged-1$1.sucatalog [L]
RewriteCond %{HTTP_USER_AGENT} Darwin/11
RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-lion-snowleopard-leopard.merged-1$1.sucatalog [L]
RewriteCond %{HTTP_USER_AGENT} Darwin/12
RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1$1.sucatalog [L]

(Note that the format for these rules is specific for use in an .htaccess file. The rules would need to be changed if in the main Apache config file.)
(Other web servers such as Nginx support URL rewriting; the specifics are slightly different, but the general concepts are similar.)

These RewriteRules for Apache2 seem to do the job. I can set the CatalogURL on a client to "http://my.sus.example.com/index_testing.sucatalog" and the correct OS-specific branch catalog is returned.

To help verify that the catalog file I was getting via URL rewrites was the correct one, I made a small change in reposado. When catalogs are written out to disk, a key named "_CatalogName" is added to the plist with the base filename of the catalog. This allows you to verify that the catalog being returned is the one you expect.

For example, I want to test that I'm getting the Lion catalog:

% curl --user-agent "Darwin/11.4.0" http://su.example.com/index_testing.sucatalog > /tmp/testing
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1024k 100 1024k 0 0 25.1M 0 --:--:-- --:--:-- --:--:-- 32.2M

% tail -5 /tmp/testing
</dict>
<key>_CatalogName</key>
<string>index-lion-snowleopard-leopard.merged-1_testing.sucatalog</string>
</dict>
</plist>

The _CatalogName is "index-lion-snowleopard-leopard.merged-1_testing.sucatalog", which is what I expect.

I can repeat the test for Snow Leopard:

% curl --user-agent "Darwin/10.8.0" http://su.example.com/index_testing.sucatalog > /tmp/testing
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 912k 100 912k 0 0 29.7M 0 --:--:-- --:--:-- --:--:-- 31.8M

% tail -5 /tmp/testing
</dict>
<key>_CatalogName</key>
<string>index-leopard-snowleopard.merged-1_testing.sucatalog</string>
</dict>
</plist>

You'll need to git pull or otherwise get the latest reposado code from GitHub to get this change.

-Greg

[Jesse Peterson]

Here's a corresponding configuration snippet for lighttpd. The "mod_rewrite" module needs to be loaded in the server:

server.reject-expect-100-with-417 = "disable"

# reposado rewrites for single-url/multi-platform usage

$HTTP["useragent"] =~ "Darwin/8" {

url.rewrite-once = ( "^/index(.*)\.sucatalog$" => "content/catalogs/index$1.sucatalog" )

}

else $HTTP["useragent"] =~ "Darwin/9" {

url.rewrite-once = ( "^/index(.*)\.sucatalog$" => "content/catalogs/others/index-leopard.merged-1$1.sucatalog" )

}

else $HTTP["useragent"] =~ "Darwin/10" {

url.rewrite-once = ( "^/index(.*)\.sucatalog$" => "content/catalogs/others/index-leopard-snowleopard.merged-1$1.sucatalog" )

}

else $HTTP["useragent"] =~ "Darwin/11" {

url.rewrite-once = ( "^/index(.*)\.sucatalog$" => "content/catalogs/others/index-lion-snowleopard-leopard.merged-1$1.sucatalog" )

}

else $HTTP["useragent"] =~ "Darwin/12" {

url.rewrite-once = ( "^/index(.*)\.sucatalog$" => "content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1$1.sucatalog" )

}

Thanks,

- Jesse

[Jesse Peterson]

Might also add that if these rewrites are catching too much then a different regexp might want to be used. E.g. clients using symlinks in the root of the webserver to specific catalogs.

This regexp is a little stricter in that it finds only index.sucatalog or index_foo.sucatalog, but not index-foo.sucatalog: 

^\/index(_)?(?(-1)(.*))\.sucatalog$

Be sure to changed the matched $1 to $2 if using this regexp.

Thanks,

- Jesse

[mallory...@gmail.com]

I'm having trouble implementing URL rewrites as described above and in the documentation on git. I'm sure it's something simple but here's the steps I took so far:

-my base URL for the reposado repo is something like "http://example.domain.com/reposado/html" as my primary server website stores site files in my munki_repo directory and i put reposado in munki_repo/reposado (I realize this isn't ideal)

-checked /etc/apache2/httpd.conf to make sure LoadModule rewrite_module and LoadModule php5_module were uncommented

-changed AllowOverride set to All (tried multiple directories including /, /Library/WebServer/Documents, and creating a new directory with my reposado repo)

-created .htaccess file as above (no changes)

-Ran the test commands and here's what I get:

% curl --user-agent "Darwin/11.4.0" http://example.domain.com/reposado/html/index_testing.sucatalog > /tmp/testing

% tail -5 /tmp/testing

<h1>Not Found</h1>

<p>The requested URL /reposado/html/index_testing.sucatalog was not found on this server.</p>

<hr>

<address>Apache/2.2.22 (Unix) DAV/2 mod_ssl/2.2.22 OpenSSL/0.9.8r Server at example.domain.com Port 80</address>

</body></html>

Thanks,

Mallory

On Wednesday, June 27, 2012 11:45:28 AM UTC-7, Greg Neagle wrote:

[Greg Neagle]

Since setting up Apache rewrites is not Reposado-specific, I'm going to:

1) Point you to Apache documentation: http://httpd.apache.org/docs/2.0/misc/rewriteguide.html

and

2) Let others in the community chime in if they can help.

-Greg

--
You received this message because you are subscribed to the Google Groups "reposado" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reposado+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Joe Wollard]

Mallory,

Where are you storing your .htaccess file stored? You probably want it to live in /Library/WebServer/Documents/munki_repo/reposado/.htaccess.

To see if you have mod_write and AllowOverrides properly configured, try a simple redirect before you try to map all of the reposado URLs. Something like this:

RewriteEngine On

RewriteRule ^/(.*) https://www.google.com?q=$1 [L,R]

If that works, you can focus on mapping your reposado urls. If not, you know you need to keep looking into a server configuration issue. A goo place to start with the latter would be the various log files in /var/log/apache2/

--

You received this message because you are subscribed to the Google Groups "reposado" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reposado+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Joe Wollard

[Joe Wollard]

Wow, that was an embarrassing number of typos. Hopefully the "meat" of the idea still made it across the wire!

--

Joe Wollard

[mallory...@gmail.com]

Looks like it's an issue with apache so I'll keep looking through their docs. Thanks