I struggled with this too... but actually it's fairly easy.
View the source code by right-clicking on the page and choose 'View Source'.
From there you can see that they include a line to load the playlist javascript file:
This needs to be inserted after the line that loads the standard jplayer javascript file in your own html page.
Then further down in the source, after the command
$(document).ready(function(){
you can see the code necessary to create the playlist. Copy this into your page making sure that it ends with )};
Your page should then show the same playlist as on the demo page (albeit with a different skin, if you have chosen another), and you can just change the song paths, titles etc. for your own.
Easy :-/