json-templater api in node/express

[Lucky Jindal]

Hi, 

I am fairly new to node stack and still learning.  I am new to this forum and I hope someone could help me fixing my problem here.

Right now, I am working on backend (express.js) where I need to deal to lot of JSON for writing restful apis. 

Application - Front end sends form data in JSON format ( data model. json) and it is sent to backend. Backend router accepts POST request with form data and does something with it, transforms it into some another JSON format, say, object.json and sends it over to external server via POST and gets 200 OK. Now, I am dealing with huge and nested JSONs and many calls over http to 3rd party server.  

My problem here: Since both JSONs, data model JSON and object. json are in completely different format, transforming is hard part. Since they are written by different teams, it kind of hard to have matching fields/keys.

Example . 

1. "data.json"  -> Coming from Angular front end, it has form data and sent when form submitted like-

{ name : "ASC", id : "101".....}

2. In backend, I have router methods, which gets this POST call and 

~ accept data.json (with data)

~ read  another json, "object.json" which is more like a template. This is in the format accepted by restful api on external server. So we have to use this, fill in it with data from data.json and send it over.

~ "object.json "

 {username : "", userId : "".... }

So Idea is, I read data from data.json and fill in template.json and send it over. 

I tried 2 methods  ---

1. Traverse over keys in object.json one by one and fill in manually, which seems lot of work and these are huge jsons. This was working until I realized, its getting cumbersome.

2. Using some external templating API.  One I tried was json-templater.

Now,  my template.json

~ {username : "{{uname}}", userId : "{{uid}}" } . Values in {{}} will be filled up by this new api.

~

var object = require (../json-templater) 

return object( require(object.json, {uname :"ABC"}}

It worked partially.  

 

Results returned by 'return' are same as I want. But something is changed, format. Because results are rejected by external server.  If I comment this call, and hardcode same results, it works ok.

Since I am just learning and trying to make it work and trying to find out what happened, I hard coded all the data in "object.json" and still run object function in json-template and it breaks.

Only way it works is, not passing it through object function in json-templater. 

Any ideas to make this work or new API or any other approach. Suggestions welcome.

Thanks

[nat...@burchill.io]

Hey Lucky. Welcome to Node!

I'm not sure what language you were using before trying JavaScript but one thing I want to mention is that JavaScript is very much non-opinionated. It doesn't matter if you want to use a functional design paradigm or an object oriented paradigm. The reason I mention this is because how you solve this problem will probably depend on how you want to design your applications. I personally prefer OOP so I will start with solving this problem that way.

To start lets create a class for your request and export it so you can use it elsewhere.

SomeRequest.js

module.exports = class SomeRequest {
  constructor (requestObj) {
    this.uuid = requestObj.id
    this.username = requestObj.name
  }
}

Now to use this class you would do the following

someOtherFile.js

const SomeRequest = require('./SomeRequest.js')
const request = new SomeRequest(originalRequest)

The request object should be converted as you like and is ready to use. As someone from a Java shop this makes the most sense to some of my coworkers since it looks more like what they are used to.

Now if you want to do it functionally like you are I wouldn't bother using a JSON templater. It's another dependency that you don't control and I don't think it does enough to justify it's use. Instead I would try something like the following.

convertSomeRequest.js

module.exports = (requestObj) => {
  return {
    uuid: requestObj.id,
    username: requestObj.name
  }
}

And to use it in another file...

const convertSomeRequest = require('./convertSomeRequest.js')
const request = convertSomeRequest(originalRequest)

One last thing. Looking at this post it seems you are trying to work with raw JSON all the time. It's easier to work with data if you are dealing with an actual object vs a string. If your using a framework like express or koa then the JSON should automatically be converted to an object and then back to JSON when you respond. If not you will need to convert the responses yourself. Javascript has a nice JSON library built in to deal with this.

JSON.stringify(object) // converts an object to a JSON string.
JSON.parse(jsonString) // converts a JSON formatted string into an object.

This should make things easier to work with for you.

[kallem suresh]

Hi,

I might resolve this issue and lets connect with me so I will take remote control and resolve the issue.

Thanks,

Suresh K

[Lucky Jindal]

Hi Suresh, 

What is your idea to resolve this issue? You thinking of writing regex based util to fill in placeholders?

[Lucky Jindal]

Thanks Nat,

My data JSON which I want to fill in with values coming from front end and template json which I want to fill in, both are highly complex. template json It has many nested fields. Right now, I have one static JSON which I know, but in future it might change.  But I know that It makes sense in Java env. 

For example, in my template.json,  with field class, values are again nested into single value or multiple value. 

{  id: "",

   name: "",

   .....,

   fields: [

      field :{ 

     id: 110, type: String, enabled: true, value : { id: 101, name: "Doctor" , hidden : false}

        },    

 field :{

    id: 110, type: String, enabled: true, multivalue : {  value:  [{ id: 101, name: "Doctor" , hidden : false},  { id: 101, name: "Doctor" , hidden : false}]}}

},

    field :{},    field :{},......    field :{},

     ]

}

Can you suggest how will I achieve fields array in my template class (following your approach). I create another field class but then we have values which can be of differnt types.

[Lucky Jindal]

I was able to resolve this issue using Nat's suggestion and classes. Thanks

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to a topic in the Google Groups "nodejs" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nodejs/RNGhWS46P1s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nodejs+unsubscribe@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/4765ecaa-739d-4a11-9465-119b96b0eef7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[kai zhu]

hi lucky,

here's a standalone script templateRender.js (~200 sloc) that can solve your needs in both browser/nodejs env.

cheers,

-kai

```javascript

/*

 * templateRender.js

 * solution to [nodejs] json-templater api in node/express

 * from discussion https://groups.google.com/forum/#!topic/nodejs/RNGhWS46P1s

 * solution to Subject: [nodejs] json-templater api in node/express

 * using code from https://github.com/kaizhu256/node-utility2/blob/2018.1.13/lib.utility2.js#L5922

 *

 * feel free to edit it to suit your needs

 */

/*jslint

    bitwise: true,

    browser: true,

    maxerr: 8,

    maxlen: 100,

    node: true,

    nomen: true,

    regexp: true,

    stupid: true

*/

'use strict';

var local;

local = {};

local.templateRender = function (template, dict, options) {

/*

 * this function will render the template with the given dict

 * example usage:

 * local.templateRender('foo {{aa.bb}}', { aa: { bb: 'bar' } } -> ‘foo bar’

 */

    var argList, getValue, match, renderPartial, rgx, tryCatch, value;

    dict = dict || {};

    options = options || {};

    getValue = function (key) {

        argList = key.split(' ');

        value = dict;

        if (argList[0] === '#this/') {

            return;

        }

        // iteratively lookup nested values in the dict

        argList[0].split('.').forEach(function (key) {

            value = value && value[key];

        });

        return value;

    };

    renderPartial = function (match0, helper, key, partial) {

        switch (helper) {

        case 'each':

        case 'eachTrimRightComma':

            value = getValue(key);

            value = Array.isArray(value)

                ? value.map(function (dict) {

                    // recurse with partial

                    return local.templateRender(partial, dict, options);

                }).join('')

                : '';

            // remove trailing-comma from last element

            if (helper === 'eachTrimRightComma') {

                value = value.trimRight().replace((/,$/), '');

            }

            return value;

        case 'if':

            partial = partial.split('{{#unless ' + key + '}}');

            partial = getValue(key)

                ? partial[0]

                // handle 'unless' case

                : partial.slice(1).join('{{#unless ' + key + '}}');

            // recurse with partial

            return local.templateRender(partial, dict, options);

        case 'unless':

            return getValue(key)

                ? ''

                // recurse with partial

                : local.templateRender(partial, dict, options);

        default:

            // recurse with partial

            return match0[0] + local.templateRender(match0.slice(1), dict, options);

        }

    };

    tryCatch = function (fnc, message) {

    /*

     * this function will prepend the message to errorCaught

     */

        try {

            return fnc();

        } catch (errorCaught) {

            errorCaught.message = message + errorCaught.message;

            throw errorCaught;

        }

    };

    // render partials

    rgx = (/\{\{#(\w+) ([^}]+?)\}\}/g);

    template = template || '';

    for (match = rgx.exec(template); match; match = rgx.exec(template)) {

        rgx.lastIndex += 1 - match[0].length;

        template = template.replace(

            new RegExp('\\{\\{#(' + match[1] + ') (' + match[2] +

                ')\\}\\}([\\S\\s]*?)\\{\\{/' + match[1] + ' ' + match[2] +

                '\\}\\}'),

            renderPartial

        );

    }

    // search for keys in the template

    return template.replace((/\{\{[^}]+?\}\}/g), function (match0) {

        var markdownToHtml, notHtmlSafe;

        notHtmlSafe = options.notHtmlSafe;

        return tryCatch(function () {

            getValue(match0.slice(2, -2));

            if (value === undefined) {

                return match0;

            }

            argList.slice(1).forEach(function (arg) {

                switch (arg) {

                case 'alphanumeric':

                    value = value.replace((/\W/g), '_');

                    break;

                case 'decodeURIComponent':

                    value = decodeURIComponent(value);

                    break;

                case 'encodeURIComponent':

                    value = encodeURIComponent(value);

                    break;

                case 'jsonStringify':

                    value = JSON.stringify(value);

                    break;

                case 'jsonStringify4':

                    value = JSON.stringify(value, null, 4);

                    break;

                case 'markdownSafe':

                    value = value.replace((/`/g), '\'');

                    break;

                case 'markdownToHtml':

                    markdownToHtml = true;

                    break;

                case 'notHtmlSafe':

                    notHtmlSafe = true;

                    break;

                // default to String.prototype[arg]()

                default:

                    value = value[arg]();

                    break;

                }

            });

            value = String(value);

            // default to htmlSafe

            if (!notHtmlSafe) {

                value = value.replace((/["&'<>]/g), function (match0) {

                    return '&#x' + match0.charCodeAt(0).toString(16) + ';';

                });

            }

            if (markdownToHtml && typeof local.marked === 'function') {

                value = local.marked(value);

            }

            return value;

        }, 'templateRender could not render expression ' + JSON.stringify(match0) + '\n');

    });

};

// init test-template

/* jslint-ignore-begin */

local.template = '{\n\

    "username": "{{uname}}",\n\

    "userId" : "{{uid}}",\n\

    "myList": [\n\

    {{#eachTrimRightComma myList}}\n\

        {{#if aa.bb}}\n\

        {\n\

            "aa2": {\n\

                "bb2": {{aa.bb jsonStringify}}\n\

            }\n\

        },\n\

        {{/if aa.bb}}\n\

    {{/eachTrimRightComma myList}}\n\

    ]\n\

}';

/* jslint-ignore-end */

// render test-template

local.dict = {

    uname: 'john1234',

    uid: 1234,

    // test nested objects in a list

    myList: [

        { aa: { bb: { cc: 'dd' } } },

        { aa: { bb: 'cc' } },

        // ignored by templater because missing nested item bb

        null,

        // ignored by templater because missing nested item bb

        {},

        // ignored by templater because missing nested item bb

        { aa: {} }

    ]

};

local.result = local.templateRender(local.template, local.dict, { notHtmlSafe: true });

// validate rendered-result is a valid json-object and

local.result = JSON.parse(local.result);

// print template vs rendered-json-result to stderr

console.error(local.template);

console.error('-> ' + JSON.stringify(local.dict) + ' ->');

console.error(JSON.stringify(local.result, null, 4));

// output:

// {

//     "username": "{{uname}}",

//     "userId" : "{{uid}}",

//     "myList": [

//     {{#eachTrimRightComma myList}}

//         {{#if aa.bb}}

//         {

//             "aa2": {

//                 "bb2": {{aa.bb jsonStringify}}

//             }

//         },

//         {{/if aa.bb}}

//     {{/eachTrimRightComma myList}}

//     ]

// }

// -> {"uname":"john1234","uid":1234,"myList":[{"aa":{"bb":{"cc":"dd"}}},{"aa":{"bb":"cc"}},null,{},{"aa":{}}]} ->

// {

//     "username": "john1234",

//     "userId": "1234",

//     "myList": [

//         {

//             "aa2": {

//                 "bb2": {

//                     "cc": "dd"

//                 }

//             }

//         },

//         {

//             "aa2": {

//                 "bb2": "cc"

//             }

//         }

//     ]

// }

```

You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.

To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CANti%2BeZaw77EmbhWzWmwnT24fqkECEEW5hVZsJUZcKFeBfmbqg%40mail.gmail.com.