JSdom append to body and render as an ejs view

Roy Mor

unread,

Dec 12, 2015, 12:54:15 PM12/12/15

to nodejs

I wanted to consult on what will be the best approach to achieve the following:

I'm requesting a webpage via the request module, then I throw the body to the

jsdom.env()

function.

What I want in the end is to add a few div, script, and link elements to the document,

and in the end render it to the user. (basically a proxy with modifying the requested remote server html)

What I initially did was to make an ejs that looks as follow:

<!DOCTYPE html>
<html>
<head>
    <%- head %>
    <link rel='stylesheet' href='/stylesheets/style.css' />
    <link rel='stylesheet' href='https://code.jquery.com/ui/1.11.3/themes/smoothness/jquery-ui.css' />
</head>
<body>
    <%- body %>
    <div id="context-menu-container">
        <div id="context-menu-header"></div>
        <div id="context-menu-options"></div>
        <div id="context-menu-steps"></div>
    </div>
    <script src="https://code.jquery.com/jquery-2.1.4.min.js"></script>
    <script src="https://code.jquery.com/ui/1.11.3/jquery-ui.min.js"></script>
    <script src="/javascripts/test.js"></script>
</body>
</html>

I render this view from my route as follows:

var express = require('express');
var router = express.Router();

var jsdom = require('jsdom');
var request = require('request');
var jar = request.jar();


/* GET users listing. */
router.get('/', function(req, res, next) {
    var url = req.param('url');
    if (url) {
        request({
            url: url,
            jar: jar
        }, function (error, response, body) {
            if (!error && response.statusCode == 200) {
                jsdom.env({
                    html: body,
                    scripts: [
                        'https://code.jquery.com/jquery-2.1.4.min.js'
                    ],
                    done: function (err, window) {
                        var $ = window.jQuery;
                        res.render('testenv.ejs', { head: $('head').html(), body: $('body').html() })
                    }
                });
            }
        });
    }
    else {
        res.send("No url param specified...");
    }
});

module.exports = router;

The problem I have with this approach is that /stylesheets/style.css and /javascripts/test/js are not downloaded to the client, they're being requested from the remote server and not from the node server.

I tried to fiddle with the order of the javascript and css files I include with in my view and noticed that If I put the css files above the <%- head> part they're begin loaded correctly.

As to the javascript files I had to move them to the head as well (something I do not with to do for obvious reasons like delaying the rendering of the page) but as to the javascript files I encountered another problem.

After moving them to the head the /javascripts/test/js was being loaded correctly because it was requested from the node server and not from the remote server anymore but the problem is that it depends on the jquery-ui,

which for some reason was being loaded but a function of the library was not recognized as if I loaded the /javascripts/test/js only after the jquery-ui library.

Beside the problems I described, I've thought about a scenario where I request a webpage that has the same css/js file name with the ones I try to load or maybe same framework used (bootstrap/jquery but with different revision) which could lead to conflicts.

It's possible to change the name of the css/js files I load to something unique, but I don't want to leave it to the hands of luck because maybe some other website will have the same file names by chance.

That lead me to think that my approach isn't bulletproof and not the best, so I want to ask what you guys think I should do to achieve my goal.

Message has been deleted

Roy Mor

unread,

Dec 16, 2015, 8:10:56 PM12/16/15

to nodejs

Still struggling with this problem, looking for someone who could shed a light and help me u derstand.

Jimb Esser

unread,

Dec 18, 2015, 12:02:37 PM12/18/15

to nodejs

I don't know anything about jsdom, but from what you said it sounds like you're trying to set up a server, to which you can request http://myhost/?url=otherhost.com/page.html and it returns that page, and then when that pages fetches style.css, it's failing. You need to rewrite what is returned so that on the very first request instead of letting otherhost.com/page.html reference "style.css", it instead references a valid URL that will get what it needs, e.g. a mapped url that goes through your server: http://myhost/?url=otherhost.com/style.css (or, probably just "/?url=otherhost.com/style.css"). I have no idea how you'd do that with jsdom, but it probably involves changing all hrefs to relative paths to be prefixed with "/?url=otherhost.com/" or something. Without remapping those paths, it's trying to fetch the stylesheet from your server, not forwarding the request on appropriately.

Hope this helps!

Jimb Esser

Roy Mor

unread,

Dec 19, 2015, 12:04:33 AM12/19/15

to nodejs

I see. I thought maybe of serving the remote server files from my node server. Do you know maybe a way to download the remote website to a local directory on the node server and I'll edit the html so that my css and js files will be relative to where the website directory is stored?

Also if I do such thing, and serving the files from my node server, I would also need to deal with CORS. Do you know of a way I can intercept the any http OPTION request that is done by the browser and change the origin header to set it to the remote server?

בתאריך יום שישי, 18 בדצמבר 2015 בשעה 19:02:37 UTC+2, מאת Jimb Esser:

Reply all

Reply to author

Forward