Module for loading JSON or other text file in browser or node

[Ryan Schmidt]

My library needs to load text files (some of which might be JSON) on demand. In the browser I could use XMLHttpRequest; in node I could use the built-in fs module. Does anybody know an npm module that wraps all this up for me in a single function I can use and have it just work in both environments, much like I use canvas-browserify to give me a Canvas object that just works in both the browser and in node?

[Azer Koçulu]

You can require JSON files; require('package.json') I'm not sure if
browserify supports it, but OneJS does...

On Sun, Jul 28, 2013 at 8:19 PM, Ryan Schmidt
<googl...@ryandesign.com> wrote:
> My library needs to load text files (some of which might be JSON) on demand. In the browser I could use XMLHttpRequest; in node I could use the built-in fs module. Does anybody know an npm module that wraps all this up for me in a single function I can use and have it just work in both environments, much like I use canvas-browserify to give me a Canvas object that just works in both the browser and in node?
>

> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Ryan Schmidt]

On Jul 28, 2013, at 22:29, Azer Koçulu wrote:

> You can require JSON files; require('package.json') I'm not sure if

> browserify supports it, but OneJS does…

In browserify, things that are require()d are copied verbatim into the final file. I don't want these text files to be included in my library file. Rather, I want them to be dynamically loaded, from their original locations, as needed. Some of these files are large lookup tables that are only needed occasionally. Others are files the user supplies to the library which I don't know about in advance.

[Ryan Schmidt]

On Jul 28, 2013, at 22:41, Ryan Schmidt wrote:

> I want them to be dynamically loaded, from their original locations, as needed.

I guess what I want is:

loadFile(file, function(err, contents) {
if (err) {
console.log(err);
} else {
//do something with contents
}
});

where the browser implementation of loadFile (which I've written) deals with all the XMLHttpRequest mechanics, and the server implementation could wrap fs.readFile. So far I'm unsure how to deal with character encoding.

I wanted to avoid the problem that I've seen many times on this group already, namely I didn't want to write and publish and announce such a module only to have people refer to me to the existing module that did exactly what I wanted.

I'm just surprised nobody has published such a module yet. Nobody else needs to load text files into their server- and client-side JavaScript libraries?

[raisch]

Ryan,

From the browser side, this is exactly what jQuery.get() and jQuery.getJSON() do. Both are shortcuts for jQuery.ajax(). The only difference between them is that get() allows you to retrieve any datatype (XML,JSON,etc.) while getJSON() retrieves JSON encoded text files and automagically converts them into a JavaScript Object.

The server side is already handled by node's express module, which allows you to specify the MIME-type of the data it sends back in response to an HTTP GET request.

So, from the client-side, you'd:

$.get('/url-to-text-file',function(data,textStatus,jqXHT){

    if(!textStatus.match(/^200/)){

        // do something with the error

        return;

    }

    // do something with the data

},'text');

See the docs for express (http://expressjs.com/) for how to specify the MIME type of returned data.

One thing to keep in mind client-side is that all Ajax calls are limited by the "same origin" policy, where you can only retrieve resources from the same domain/host/port combination as the requesting page. There are ways around this restriction which a search for "JavaScript same origin" will reveal.

All best.

/rr

[Ryan Schmidt]

On Jul 30, 2013, at 16:46, raisch wrote:

> From the browser side, this is exactly what jQuery.get() and jQuery.getJSON() do. Both are shortcuts for jQuery.ajax(). The only difference between them is that get() allows you to retrieve any datatype (XML,JSON,etc.) while getJSON() retrieves JSON encoded text files and automagically converts them into a JavaScript Object.

I am developing a standalone JavaScript library. It must not depend on jQuery, Prototype, or other monolithic library. That is why I am looking for an npm module that does just this specific function that I can embed in my library using browserify.

> The server side is already handled by node's express module, which allows you to specify the MIME-type of the data it sends back in response to an HTTP GET request.

On the server side, I also don't want to depend on express or other big system. On the server side, I just need to load a file on the local filesystem; I don't need to involve any HTTP layer.

> One thing to keep in mind client-side is that all Ajax calls are limited by the "same origin" policy, where you can only retrieve resources from the same domain/host/port combination as the requesting page. There are ways around this restriction which a search for "JavaScript same origin" will reveal.

I am fine with being restricted to the same origin.

[raisch]

>On the server side, I also don't want to depend on express or other big system. On the server side, I just need to load a file on the local filesystem; I don't need to involve any HTTP layer.

How would you expect to get the data to the client without using HTTP? 

[raisch]

You may wish to study XMLHttpRequest Call - Ajax Patterns which presents both the mechanisms used by Ajax as well as a pared-down implementation of something like jQuery's get() called 'AjaxCaller'

All best.

/rr

[Michael Ryan]

For the server-side portion, you just need to use a static-files module.  As to file encoding, I would recommend having all your text files in UTF-8 without a Byte Order Marker (BOM), many utils blow up without them, and many editors in windows by default inject them for new files, so that's something to be aware of.

SERVER: See connect.static() for an example.

http://www.senchalabs.org/connect/static.html

CLIENT: As to loading your files, jQuery's .get() method, which should give you what you are wanting...

http://api.jquery.com/jQuery.get/

> Subject: Re: [nodejs] Module for loading JSON or other text file in browser or node
> From: googl...@ryandesign.com
> Date: Tue, 30 Jul 2013 16:25:28 -0500
> To: nod...@googlegroups.com

[Ryan Schmidt]

On Jul 30, 2013, at 16:59, raisch <rai...@gmail.com> wrote:

>> On the server side, I also don't want to depend on express or other big system. On the server side, I just need to load a file on the local filesystem; I don't need to involve any HTTP layer.
>
> How would you expect to get the data to the client without using HTTP?

Perhaps I should have said "using node" instead of "on the server". The node process needn't be a server. There needn't be a client connecting to it.

On Jul 30, 2013, at 17:00, raisch <rai...@gmail.com> wrote:

> You may wish to study XMLHttpRequest Call - Ajax Patterns which presents both the mechanisms used by Ajax as well as a pared-down implementation of something like jQuery's get() called 'AjaxCaller'

I know how to use XMLHttpRequest and have already written a loadFile function using it. But that's only necessary when running in a browser. I have found XMLHttpRequest implementations for node in npm which I could include, but all I really want when running in node is for loadFile to load the file from the local filesystem using fs.readSync or equivalent.

On Jul 30, 2013, at 17:04, Michael Ryan <mygr...@tracker1.info> wrote:

> For the server-side portion, you just need to use a static-files module.

I think I've confused things by using the word "server". I do not need to serve static files.

Let me explain again. I am writing a JavaScript library. I converts text files in a particular format into images. It is designed to be used in the browser, but now I am enhancing it so that it can also work in node. The user can acquire the contents of the text file using whatever method and feed it to my library. But I also want to offer a convenience function that will accept a filename. If the library is running in the browser, it should use XMLHttpRequest or equivalent to download the file from the server (same origin restriction is fine). If running in node, it should use fs.readSync or similar to read the file from the local filesystem. This seems like an extremely simple and natural thing to want to do (what could be simpler than wanting to read in a text file?), therefore I expected to find a module already written ages ago to do that.

> As to file encoding, I would recommend having all your text files in UTF-8 without a Byte Order Marker (BOM), many utils blow up without them, and many editors in windows by default inject them for new files, so that's something to be aware of.

I would also recommend UTF-8, and I am considering mandating that. However, ideally I want to support all of the features that the text file format I am using allows, and one of those features is that the text file could be in either UTF-8 or ISO-8859-1 encoding, as indicated by a particular line in the file (much like an HTML file might contain a <meta charset="…"> line).

With fs.readSync, there's an options object which could include an encoding flag, but that assumes that I know in advance what the file's encoding is, and I do not since the file is provided by the user of my library. So I could leave off the encoding, and get back a raw buffer, and deal with that somehow…

With XMLHttpRequest, I'm not sure how it handles encoding; I haven't tested it yet. Perhaps it assumes the server sends the correct charset in the Content-Type header, but I do not believe that will be reliable for my use case. Users will probably just upload these text files to their web space, and their web server won't know anything about this particular text file format so it won't know what charset to send.

[Michael Ryan]

> Perhaps I should have said "using node" instead of "on the server". The node process needn't be a server. There needn't be a client connecting to it.

>
> ...

>
> I know how to use XMLHttpRequest and have already written a loadFile function using it. But that's only necessary when running in a browser. I have found XMLHttpRequest implementations for node in npm which I could include, but all I really want when running in node is for loadFile to load the file from the local filesystem using fs.readSync or equivalent.

So, why not use fs.readSync?

[Ryan Schmidt]

On Jul 31, 2013, at 11:56, Michael Ryan wrote:

>> I know how to use XMLHttpRequest and have already written a loadFile function using it. But that's only necessary when running in a browser. I have found XMLHttpRequest implementations for node in npm which I could include, but all I really want when running in node is for loadFile to load the file from the local filesystem using fs.readSync or equivalent.
>
> So, why not use fs.readSync?

In node, I probably will.

In the browser, there is no fs module and indeed no filesystem. The compatibility modules for fs that I've found use the new HTML5 local storage, which is not what I'm after.

[mar...@svenskasakerhetsdorrar.se]

Why not use JSON.parse(fs.readFileSync(file))?

Trying to make a module "standalone" is generally a waste of time if you have to even implement file system access and basic json parsing yourself.. 

[mar...@svenskasakerhetsdorrar.se]

For browser: 

function mycallback(data) {

   //do something funky with 'data'

}

var oHead = document.getElementsByTagName('HEAD').item(0);

var oScript= document.createElement("script");

oScript.type = "text/javascript";

oScript.src="http://somedomain.com/&callback=mycallback";

oHead.appendChild( oScript);

[mar...@svenskasakerhetsdorrar.se]

Use this instead: 

        var oHead = document.getElementsByTagName('HEAD').item(0);

var oScript= document.createElement("script");

oScript.type = "text/javascript";

oScript.src="http://domain.com/jsonfile.json";

oScript.onload = function(){

alert("Done"); 

}

oHead.appendChild( oScript);

On Wednesday, July 31, 2013 10:42:03 PM UTC+2, ryandesign wrote:

[Arnout Kazemier]

Please, do tell us how you imagine that to work? How would you get the contents of pure JSON file

when you load in your browser through a external script tag.

[Martin Schröder]

You get the file parsed. I presumed that was what was asked. Why would you even want it in text form? And if you DO want it in text form then you can always convert the resulting object into json again.

In your json file you write:

var some_global_object = {<json data>}

.. then in your code of the callback you can simply reference that object like you normally would.

AND in case you want to simply parse json data then jQuery is your friend. Nothing simpler than $.getJSON('ajax/test.json', function(data) { do something with data });

Personally I see no reason why you would not want to use jquery for this, since you will probably have very good use for jquery in other parts of your application as well.

You received this message because you are subscribed to a topic in the Google Groups "nodejs" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nodejs/Vi8gCST1PSk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nodejs+un...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Med vänliga hälsningar / Best regards

Martin K. Schröder

Försäljning, Säkra Dörren AB

mar...@svenskasakerhetsdorrar.se

Direkt: +46 8 500 91 997

Mobil: +46 733 38 76 94

FX Säkra Dörren AB

Org nr: 556882-8544 | VAT: SE556882854401

Postadress: Box 55598, 10204, Stockholm

Dörrar: http://svenskasakerhetsdorrar.se

Butiken: http://sakradorren.se

[Arnout Kazemier]

if you do var_global = { object } then it's not a JSON document anymore but a JavaScript file. There's a big difference in that. You just can't load a JSON file through a

script tag. You need to use a XHR request. or post process it on the server.

[Martin Schröder]

Another solution could be this:

in html:
<iframe id="frmFile" src="test.txt" onload="LoadFile();" style="display: none;"></iframe>

in javascript
function LoadFile() {
    var oFrame = document.getElementById("frmFile");
    var strRawContents = oFrame.contentWindow.document.body.childNodes[0].innerHTML;

    var obj = JSON.parse(strRawContents);

}

However they are all really ugly.. simplest solution is to use jQuery.