How to serve large files with jPlayer and keeping them safe from hotlinking

[Marc Dingena]

Hello everyone. I wanted to serve large files with jPlayer while keeping these secure and have done a lot of research on the web on how this is possible. Now that I've successfully transferred over 58 Terabytes of music on my website since its launch, I wanted to share my knowledge with you guys.

The problem

Do you have one of the following problems?

• My streams / downloads get cut off after a while
• My files are exposed to the public! Can I protect them with htaccess / PHP / <insert other methods>?

Then this guide will help you!

Requirements

To implement this solution, you will need the following:

• A server that supports the X-Sendfile module, an Apache mod that handles file transfers for you
• Access to the file system that holds your website. More specifically, you need to be able to create a folder outside your domain folder. This will hold your files out of reach of the public.
• Some PHP knowledge. Most of the things explained here contain comments that help you comprehend the code, but knowing some PHP will drastically help you understand what the thing is doing.

The solution

The solution is a co-operation of the following elements:

• A PHP script that finds the requested file in your hidden folder
• Using X-Sendfile HTTP Headers that push the file to the public instead of PHP functions like fread()

Lets start with creating a folder outside your domain folder. You can best achieve this by using an FTP program (something you are probably already using to upload content to your website). I highly recommend FileZilla in case you don't have one yet. It's free and easy to use. You can also achieve creating a folder by using web-based FTP, provided that your hosting company has a control panel that allows you to do this.

Locate your domain's folder, and navigate up one level. This is usually your root directory, but it depends on your hosting provider. It's possible that your domain sits inside a "html-public" or "wwwroot" folder, or something similarly named. It depends on the server setup, and it could indicate that you are not on an Apache web server. If this is the case, you may want to double check what kind of server your host is running. Chances are that it's not an Apache server, which could mean that it does not have the X-Sendfile mod to begin with. You will need this to serve large files.

Create a new folder here, name it something clever like "audio" or "hidden", depending on your needs. It should now look something like this:

These red folders are located outside the domain folder, which contains your website.

Inside your hidden folder, you can arranges your files anyway you need; you do not have to keep a flat file structure!

Now that your hidden folder is created, we can continue with creating the script that locates the files in your hidden folder.

A basic script would be:

<?php

// test arguments

if(!isset($_REQUEST['file']) || empty($_REQUEST['file'])) 

{

// invalid argument, abort script

header("HTTP/1.0 400 Bad Request");

die();

}

// set variables

$file_requested = $_GET['file'];

$file_name = strip_tags($file_requested);

$file_path = '/home/tjoonz/audio/techno/';

$file_full = $file_path.$file_name;

 

if (file_exists($file_full)) // test if requested mixtape exists

{

// send the file

header("X-SENDFILE: ".$file_full);

header("Content-Type: application/octet-stream");

header("Content-Disposition: attachment; filename=".basename($file_full));

header("Content-Length: ".filesize($file_full));

header("Pragma: public");

header("Expires: 0");

header("Cache-Control: public, must-revalidate, post-check=0, pre-check=0");

header("Content-Description: File Transfer");

header("Content-Transfer-Encoding: binary");

}

else // couldn't find that mixtape

{

header('HTTP/1.0 404 Not Found');

  die();

}

?>

Let's break down the above script:

1. First, check if the script was passed a file argument, and if so, it must not be empty. If this check is failed, return a HTTP Header with code 400 Bad Request. Finally, die(), aborting execution of the rest of the script.
2. If we make it this far (step 1 succeeds), continue with script by setting a couple of variables. A variable is a piece of memory with a name. We fill the memory with some values so that we don't have to type the whole values every time we want to use it. Save the value of our file argument in a variable called file_requested.
3. While we are setting variables, we essentially 'sanitize' it. It is very important to sanitize your user-input, because you prevent unauthorized users accessing your file system. To clarify, by blindly accepting any user input, malicious users can enter requests to files in folders that you do not want them to access. For example, if they requested the file with a leading slash (/), they could jump to the root of your hosting, and from there access any other files. We sanitize our file by prefixing it with an absolute path as seen from the root. This eliminates malicious users from attempting to trick the script to access a different folder.
4. Now that we have reconstructed the full absolute path to the file, we need to check if this is actually valid. There are multiple ways to check this. I chose file_exists(), but you could also use is_file() for example, depending on your needs. When this check fails, it jumps down to the else statement, which returns HTTP Header 404 File Not Found. If the check succeeds, the script continues.
5. The final step of the script is to initiate the file transfer. According to the X-Sendfile documentation, it is not necessary to send various other headers, like Content-Length. My script may feature some over-the-top headers, but I've had some experiences with certain browsers not finding the Content-Length (which is needed to tell jPlayer what the total duration of a file is). I am sure that the community of jPlayer here on Google Groups will point out alternatives of doing essentially the same thing. I am also still open for suggestions and improvement. Please note that aside from browser support, it may also be your server's configuration of the X-Sendfile mod that may require these additional headers to be set up. Just remember that for the basic functionality of sending the file, you only need the first header: header("X-SENDFILE: ".$file_full);

The final step to this solution is to ensure that X-Sendfile is activated on our domain. You will have to place a .htaccess file in the root of your website folder. It can contain anything, but at the first 2 lines you have to enter the following code:

XSendFile On

XSendFileAllowAbove On

These statements enable X-Sendfile, and allows it to travel to folders higher than your website's folder. This is essential because that is where our hidden folder is located.

Also PLEASE NOTE that if you don't correctly type your htaccess file OR if your server does not have X-Sendfile OR if X-Sendfile is not enabled for your domain, the above lines in the htaccess file will cause your entire website to throw a 500 Internal Server Error. Don't fear, remove the lines and it's gone. Again, ensure you are able to use X-Sendfile FIRST!

But, why must I use X-Sendfile? Can't I just use PHP to send the file?

You can use fread(), or other PHP solutions (like chunked transfers and such). Let me elaborate why PHP can be counterproductive for you, depending on your needs.

First of all, depending on your server's configuration, PHP scripts can execute only for a certain amount of time. Your server may be configured to kill any running PHP process that has not finished after 30 seconds. Most shared hosting environments do not allow you to change the timeout for this behavior. It's possible that even if you use PHP commands to temporarily disable this timeout, your shared hosting provider is guarded against it. Now, this may not be a problem if you are serving files that are transferred within 30 seconds (or whatever your server's timeout is), but as I mentioned at the beginning, this is problematic when you want to send large files. It can also be problematic for small files, if your users have a slow connection. That's why the PHP solution may work for you, when you are testing your scripts, but it may not work for the majority of your users. I learned this the hard way, having unsatisfied visitors for while!!

The reason X-Sendfile is so damn good to use is because it uses a much more logical way to interact with the client for sending files. I will not dig into the technicalities of that, but what is important to know is that once the X-Sendfile HTTP Header is output by your PHP script, that's it! Your script finishes, so the PHP process ends on the server. Meanwhile X-Sendfile instructed the server to fully handle the file transfer without the need of any PHP interaction!

Getting X-Sendfile to work!

As said before, it depends on your server. It can be hard to determine if you are on a shared server, it's a good idea to ask your hosting's support for it.

For example, my websites using this script run on a Dreamhost shared server. It's good to know that while X-Sendfile is available here on shared hosting, a Dreamhost employee has to specifically enable it for your domain(s). Once it is enabled for one domain, it's not automatically enabled for the other domain. At Dreamhost, I can simply create a support ticket asking for X-Sendfile activation.

If you have the privilege and money to run on a private server, you will need to install X-Sendfile yourself. As I have no experience with this, I cannot help you with that. Sorry!

Example of my full download script, which also counts the number of plays & downloads

You are free to (ab)use the below script to your needs. I do not care for credit, I just hope you learn something new =)

The following script is used in a WordPress environment. The link to start a download, or play a file in jPlayer looks as follows:

<a class="jplayer-playnow" href="/<?php echo $category[0]->category_nicename; ?>.php?id=<?php the_ID(); ?>&file=<?php echo the_slug(); ?>.mp3">Play it!</a>

For visitors on the front-end, it looks like this (for a post in the Techno category in WordPress):

<a class="jplayer-playnow" href="/techno.php?id=3214&file=larry-larsson-underground-vinyl-sessions-episode-1.mp3">Play it!</a>

The script below then uses the id and file arguments to increment the counter and serve the audio to jPlayer (or download, depending what the user clicked on the site):

<?php // Tjoonz.com download script, courtesy of Marc Dingena

// test arguments

if(!isset($_REQUEST['id']) || empty($_REQUEST['id']) || !isset($_REQUEST['file']) || empty($_REQUEST['file'])) 

{

// invalid argument(s), abort script

header("HTTP/1.0 400 Bad Request");

die();

}

// set variables

$current_time = time();

$user_ip = $_SERVER['REMOTE_ADDR'];

$post_id = $_GET['id'];

$file_requested = $_GET['file'];

$file_name = strip_tags($file_requested);

$file_path = '/home/tjoonz/audio/techno/';

$locked = false;

// test if cookie 'downloadsAllowed' is set

// this prevents hotlinking on other domains for users who haven't visited Tjoonz.com yet

if ($_COOKIE["downloadsAllowed"] == "tj00nz")

{

$file_full = $file_path.$file_name;

if (file_exists($file_full)) // test if requested mixtape exists

{

// establish connection to database for Play Counter test

$user="YOUR_USERNAME";

$password="YOUR_PASSWORD";

$database="YOUR_DATABASE";

$host="YOUR_HOSTNAME";

mysql_connect($host,$user,$password);

mysql_select_db($database) or die(mysql_error());

// set timerange to 2 hours ago (right now minus 7200 miliseconds)

$timerange = $current_time - 7200;

// get all records from 'playcount_lock' table

$locks = mysql_query("SELECT * FROM playcount_lock WHERE request_time > $timerange") or die(mysql_error()); 

// test if current IP address has already accessed this mixtape in the last 2 hours

while ($lock = mysql_fetch_object($locks))

{

// set variables with data from record

$lock_ip = $lock->ip_address;

$lock_id = $lock->post_id;

$lock_time = $lock->request_time;

// compare record data with current user data

if($lock_ip == $user_ip && $lock_id == $post_id && ($lock_time + 7200) >= $current_time)

{

// match found, break out of while loop and continue with script

$locked = true;

break;

}

}

if (!$locked) // if playcounter is not locked, update the database

{ 

// add a lock entry for the next two hours

mysql_query("INSERT INTO `playcount_lock` (`request_time`, `post_id`, `ip_address`) VALUES ('$current_time', '$post_id', '$user_ip')") or die(mysql_error());

// update the play counter for this mixtape

$post_meta = mysql_query("SELECT meta_value FROM `wp_postmeta` WHERE `meta_key` = '_played' AND `post_id` = ".$post_id);

$count = @mysql_result($post_meta, 0);

if($count == FALSE)

{

// if no plays are present, insert the metadata into table

mysql_query("INSERT INTO wp_postmeta (post_id, meta_key, meta_value) VALUES (".$post_id.",'_played',1)");

}

else

{

// otherwise increase the existing number of plays by 1

$newCount = mysql_result($post_meta, 0) + 1;

mysql_query("UPDATE `wp_postmeta` SET meta_value = ".$newCount." WHERE `meta_key` = '_played' AND `post_id` = ".$post_id);

}

}

// we're done with the database, kill the connection

mysql_close();

// finally, send the file

header("X-SENDFILE: ".$file_full);

header("Content-Type: application/octet-stream");

header("Content-Disposition: attachment; filename=".basename($file_full));

header("Content-Length: ".filesize($file_full));

header("Pragma: public");

header("Expires: 0");

header("Cache-Control: public, must-revalidate, post-check=0, pre-check=0");

header("Content-Description: File Transfer");

header("Content-Transfer-Encoding: binary");

}

else // couldn't find that mixtape

{

header('HTTP/1.0 404 Not Found');

  die();

}

}

else // cookie 'downloadsAllowed' not set, redirecting user to mixtape page

{

$file_redirection = substr($file_name,0, -4);

header("Location: http://www.tjoonz.com/techno/" . $file_redirection . "/");

}

?>

Closing words

I hope this little guide was of good use to you. It should answer a couple of questions that most users seem to have regarding serving large files to jPlayer with PHP.

I'm open to suggestions and improvement. Please comment =)

Thank you.

[Mark Panaghiston]

Thank you for this. I'll have to look through it in more detail tomorrow.

My thoughts from a quick skim of the doc...

This line looks like it will cause problems:
header("Content-Type: application/octet-stream");

Don't just go by the w3c media element spec. Most browsers have not implemented that new rule that if a MIME type is not given assume application/octet-stream and that browsers should try and play media with that mime type... And give the error code if the media decoding fails.

Currently, IE9 seems to work if you do not give a mime type at all, but fails if it is application/octet-stream. Best to give the correct MIME type though in IE9.
Firefox and Opera would also fail (unless they change that since I last checked) if the ogg or webm mime type is incorrect.

[Mark Panaghiston]

This is an old test I made for mime type support.
http://thewormlab.com/jquery/jplayer/audioMimeTester/

I should update it with the application/octet-stream mime type for each. (The mp3 and ogg file.)

[Dexter Rutecki]

I see you don't have header('Accept-Ranges: bytes') or any of the other code that handles partial requests from the smartread function that jplayer offers (Found here  ). Do you not need it with this new method? Thanks for sharing this, it looks like just what I need.

[Dexter Rutecki]

FYI-

XSendFileAllowAbove On did not work for me, I had to have our host add my directory via 

XSendFilePath, which can't be set via htaccess

[Dexter Rutecki]

It looks like you should still use the smart read function with this to server partial content. I inserted header("X-Sendfile: ".$location); right above the header for the mimetype and it seems to work.

[Marc Dingena]

The issue is that although jPlayer may accept ranges, X-Sendfile cannot send range requests. An updated module, cleverly called X-Sendfile2 is able to handle range-requests, but my host does not support this module and therefore I cannot test it or give a guide on it.

If you managed to use ranges and X-Sendfile, I would love to hear about it!

[Marc Dingena]

I'm interested in seeing how you implemented what you are saying. Do you have a more complete script I can look at?

I have attempted to create a resumable download approach with X-Sendfile, but I'm pretty sure you'd need X-Sendfile2 to support byte-range.

I had a X-Sendfile version based on this fread() script I found, but I couldn't get it to work:

<?php

// get the file request, throw error if nothing supplied

if(!isset($_REQUEST['file']) || empty($_REQUEST['file'])) 

{

header("HTTP/1.0 400 Bad Request");

exit;

}

// sanitize the file request, keep just the name and extension

// also, replaces the file location with a preset one ('./myfiles/' in this example)

$file_request = $_REQUEST['file'];

$path_parts   = pathinfo($file_request);

$file_name    = $path_parts['basename'];

$file_ext     = $path_parts['extension'];

$file_path    = '/home/tjoonz/audio/dubstep/';

$file_full    = $file_path . $file_name;

// make sure the file exists

if (is_file($file_path))

{

$file_size  = filesize($file_path);

if (file_exists($file_full))

{

// set the headers, prevent caching

header("Pragma: public");

header("Expires: -1");

header("Cache-Control: public, must-revalidate, post-check=0, pre-check=0");

header("Content-Disposition: attachment; filename=\"$file_name\"");

// set the mime type based on extension, add yours if needed.

switch($file_ext)

{

 case "zip":

header("Content-Type: application/zip");

break;

 default:

header("Content-Type: application/octet-stream");

}

//check if http_range is sent by browser (or download manager)

if(isset($_SERVER['HTTP_RANGE']))

{

list($size_unit, $range_orig) = explode('=', $_SERVER['HTTP_RANGE'], 2);

if ($size_unit == 'bytes')

{

//multiple ranges could be specified at the same time, but for simplicity only serve the first range

//http://tools.ietf.org/id/draft-ietf-http-range-retrieval-00.txt

list($range, $extra_ranges) = explode(',', $range_orig, 2);

}

else

{

$range = '';

header('HTTP/1.1 416 Requested Range Not Satisfiable');

exit;

}

}

else

{

$range = '';

}

//figure out download piece from range (if set)

list($seek_start, $seek_end) = explode('-', $range, 2);

//set start and end based on range (if set), else set defaults

//also check for invalid ranges.

$seek_end   = (empty($seek_end)) ? ($file_size - 1) : min(abs(intval($seek_end)),($file_size - 1));

$seek_start = (empty($seek_start) || $seek_end < abs(intval($seek_start))) ? 0 : max(abs(intval($seek_start)),0);

 

//Only send partial content header if downloading a piece of the file (IE workaround)

if ($seek_start > 0 || $seek_end < ($file_size - 1))

{

header('HTTP/1.1 206 Partial Content');

header('Content-Range: bytes '.$seek_start.'-'.$seek_end.'/'.$file_size);

header('Content-Length: '.($seek_end - $seek_start + 1));

header("X-SENDFILE: ".$file_full);

}

else

{

header("Content-Length: $file_size");

header("X-SENDFILE: ".$file_full);

}

header('Accept-Ranges: bytes');

    

// file save was a success

exit;

}

else 

{

// file couldn't be opened

header("HTTP/1.0 500 Internal Server Error");

exit;

}

}

else

{

// file does not exist

header("HTTP/1.0 404 Not Found");

exit;

}

?>