GM_xmlhttpRequest or frames?
Shreevatsa
I'm trying to write a script that gathers and displays some data from
several pages (on the same domain). I'm not sure what the best way of
doing this is:
* Using GM_xmlhttpRequest, the other pages' contents are got as a
string, in responseDetails.responseText, so before accessing elements
using DOM functions, I will have to parse the string into a DOM tree.
(Using innerHTML? range.createContextualFragment?)
* Alternatively, I can add an invisible iframe that contains the other
page, and use iframe.contentDocument. This works, but the problem is
that Greasemonkey runs on the other page as well (which is intended,
when the page is loaded by itself, so changing @include rules is not
possible). Is there a way to prevent this?
* Is there some other better way?
Also, is there any way of profiling a Greasemonkey script?
Thanks,
Shreevatsa
esquifit
<shreevat...@gmail.com> wrote:
> I'm trying to write a script that gathers and displays some data from
> several pages (on the same domain). I'm not sure what the best way of
> doing this is:
>
> * Using GM_xmlhttpRequest, the other pages' contents are got as a
> string, in responseDetails.responseText, so before accessing elements
> using DOM functions, I will have to parse the string into a DOM tree.
> (Using innerHTML? range.createContextualFragment?)
At least the latter is extensively used; you can easily find may
examples in [1], for instance.
> * Alternatively, I can add an invisible iframe that contains the other
> page, and use iframe.contentDocument.
This approach has been discussed a couple of times here. As you point
out below, it works. Two things to have in mind when using it are:
1) The iframe must be visible; if this is not desired (and most of the
time it isn't) the iframe size must be set to 1x1, or heigh=0, or
something like that. Setting the style to display='none' does not
work.
2) A shortcoming of inserting an iframe is that the browser also loads
all embedded resources (external scripts, stylesheets, and most
importantly, images and the like), thus causing an undesired overload
since you are only interested in the root document. I don't know
whether this is also the case with createContextualFragment.
> This works, but the problem is
> that Greasemonkey runs on the other page as well (which is intended,
> when the page is loaded by itself, so changing @include rules is not
> possible). Is there a way to prevent this?
One way would be testing whether the document is a subordinate iframe
(window != window.top). Of course, this only works in the simple case
where the created iframe is directly inserted in the root document and
not into some subframe, but this should cover 99% of the cases at this
time, since frames are almost a deprecated feature of 20th century's
design style (well, don't take this judgment too seriously).
Alternatively, you can give the iframe a window.name and test for it
in your script. This property of the window object is readably by
both the parent and the child document.
> * Is there some other better way?
If you are certain that the document you want to parse is indeed XML
and not just HTML, you can use DOMParser object:
var parser = new DOMParser();
dom = parser.parseFromString(response.responseText, "text/xml");
Unfortunately, this does not work with text/html.
An apparently unexplored possibility would be to include a XUL
document in the iframe. XUL allows you to use browser object into
which you can load the document. The advantage of it over a normal
HTML iframe is that the browser object posses a webNavigation property
that allows you to switch off the loading of images, external scripts,
and so on. For a reference see [2], but let me say it again, I'm not
sure if this can work, I've never tried it.
[1] http://userscripts.org/scripts/search?q=createContextualFragment
[2] http://developer.mozilla.org/en/docs/Code_snippets:HTML_to_DOM#Using_a_hidden_XUL_iframe_.28complete_example.29
RodMcguire
On Aug 16, 6:38 pm, Shreevatsa <shreevatsa.pub...@gmail.com> wrote:
> ...
> that Greasemonkey runs on the other page as well (which is intended,
> when the page is loaded by itself, so changing @include rules is not
> possible). Is there a way to prevent this?
bound to an <a> element, and you are opening a.href, instead open
href = a.href + '#myGMIgnore'
Then your GM script can look at document.location.hash and decide to
run or not.
Anthony Lieuallen
> href = a.href + '#myGMIgnore'
>
> Then your GM script can look at document.location.hash and decide to
> run or not.
Or, just "@exclude *#myGMIgnore".
Shreevatsa
For the frames method, both the idea of checking window.top and the
more general trick of appending '#myGMIgnore' work perfectly. My
scripts happen to work even with style.display='none', but I'll keep
in mind that having iframes invisible might be problematic.
As you said, a shortcoming with iframes is that they load images and
everything else. The GM_xmlhttpRequest method is probably better in
this regard, but it doesn't seem possible to use its "output" (the
responseText to the request) as an actual document: getElementById and
getElementsByName, specifically. Is there a way?
Johan Sundström
> As you said, a shortcoming with iframes is that they load images and
> everything else. The GM_xmlhttpRequest method is probably better in
> this regard, but it doesn't seem possible to use its "output" (the
> responseText to the request) as an actual document: getElementById and
> getElementsByName, specifically. Is there a way?
I've written a little @require lib that does serious industrial smoke
and mirrors with xhr, dynamically created iframes or divs loaded with
the string content from the xhr object, and a hogwash of trickstery
with load callbacks and the like, to get as close to the DOM tree of
the loaded page as possible (if still not quite).
The main goal of it is to get something you feed an url, and get a DOM
tree from, or even feed an URL and an XPath expression, and get a
specific DOM node (or array of DOM nodes) from that page, sliced up
from the XPath query. All modes work in async mode, so you provide a
callback to be invoked once the data is parsed. Basic usage:
// @require http://ecmanaut.googlecode.com/svn/trunk/lib/gm/wget.js
wget(url, gotDOM);
function gotDOM(doc, url, xhr) {
alert(url + " has "+ doc.links.length +" links!");
}
wget$x(url, gotNodeArray, '//a[@href]');
function gotNodeArray(nodes, url, doc, xhr) {
alert(url + " has "+ nodes.length +" links!");
}
wget$X(url, gotNode, '//body'); // picks up the first body tag only
function gotNode(node, url, doc, xhr) {
alert(url + " has "+ node.getElementsByTagName("a").length +" links!");
}
The above three (untested) examples all achieve almost (competence
points scored for knowing which differs how from the rest) the same
thing, demonstrating how to do quite a basic link count from the
target document. There are optional extra args "runGM" and "div", both
booleans, that state whether the target document should instead be
loaded by url rather than content into the iframe and have GM scripts
that would apply for that url run (useful for splitting apart a script
into more basic, separate, cooperating producer / consumer type
component oriented scripts), and if you can live with having the DOM
generated in a div instead of a whole frame (the div will break stuff
like the html, head and body tags, but uses substantially less
resources, so if you just want content, it might be the way to go).
It's free software under sort of a "no-questions-asked" licence (but
feel free to ask, I just do not promise I will answer), it's a power
tool you can very easily make very ill behaved scripts with, and it
caches pages loaded, so you probably won't want to run it a lot on
very long lived pages.
But it lets you do stuff that *should* take ten minutes to whip up, in
ten minutes, instead of a few days (and after having spent a decade or
so getting apt at web development).
--
/ Johan Sundström, http://ecmanaut.blogspot.com/
Shreevatsa
> On Wed, Aug 20, 2008 at 4:02 AM, Shreevatsa <shreevat...@gmail.com> wrote:
> > As you said, a shortcoming with iframes is that they load images and
> > everything else. The GM_xmlhttpRequest method is probably better in
> > this regard, but it doesn't seem possible to use its "output" (the
> > responseText to the request) as an actual document: getElementById and
> > getElementsByName, specifically. Is there a way?
>
> I've written a little @require lib that does serious industrial smoke
> and mirrors with xhr, dynamically created iframes or divs loaded with
> the string content from the xhr object, and a hogwash of trickstery
> with load callbacks and the like, to get as close to the DOM tree of
> the loaded page as possible (if still not quite).
> booleans, that state whether the target document should instead be
> loaded by url rather than content into the iframe and have GM scripts
> that would apply for that url run (useful for splitting apart a script
> into more basic, separate, cooperating producer / consumer type
> component oriented scripts), and if you can live with having the DOM
> generated in a div instead of a whole frame (the div will break stuff
> like the html, head and body tags, but uses substantially less
> resources, so if you just want content, it might be the way to go).
One observation is that with "div" set to true, things like
document.getElementByName don't work (as noted with the
GM_xmlhttpRequest method), while with div set to false, it loads all
external resources of the requested pages, like the iframe method.
So besides the XPath stuff, the main advantage, over simply appending
an iframe and using its contentDocument, is that it caches pages, is
that right?