escape char interpretation issue

[anandva]

Hi,

Hope this is normal behavior question and answered already.I got below
data from api as valid JSON, but getting error as missing encoding
while returning data.
1. Is there any way i could encode data on api fetch?

data = { "fname" : "fname","lname":"\"lname\""};

return data;

2. if the returned data has null value for one of the data elements
that is also causing issue?

data = { "fname" : "fname","lname":null};

return data;

data returned is a json representation of java object, i am exploring
to see we could transform based on needs for clients.

Thanks,
Vivek.

[Subbu Allamaraju]

Both are due to some limitations/bugs in the underlying PEG.js parser generator. For the first case, the work-around is to use single quotes as

data = { "fname" : "fname","lname":'\"lname\"'};

For the second issue of null, I'll check in a work around today.

Subbu

[anandva]

Thanks for the response Subbu.

i did bit more reading on features, i think for both issues i could
handle through MonkeyPatching, where i could write a local script to
transform values for the keys which has escape chars or null's.

-Vivek.

[Subbu Allamaraju]

ql.io-c...@0.5.3 has the fix for nulls. For the escaped quote issue, I'm still waiting for a resolution/work-around from PEG.js.

Subbu

[anandva]

Thanks Suddu, i will give it a try after the update later today. Thx.

On Apr 10, 6:22 pm, Subbu Allamaraju <su...@subbu.org> wrote:
> ql.io-compi...@0.5.3 has the fix for nulls. For the escaped quote issue, I'm still waiting for a resolution/work-around from PEG.js.

[anandva]

I should have refreshed only ql.io-compiler module instead i did make
clean install on myapp dir , getting following error, any idea?

1. ENOENT, lstat '/..../node_modules/ql.io-app/node_modules/ql.io-
console/node_modules/validator/index.js'
2.
npm ERR! EEXIST, mkdir '/.../node_modules/ql.io-app/node_modules/ql.io-
console/node_modules/connect/lib'
npm ERR! File exists: /..../node_modules/ql.io-app/node_modules/ql.io-
console/node_modules/connect/lib
npm ERR! Move it away, and try again.

tried deleting the folders but those are not existing.

[Subbu Allamaraju]

Looks like an npm issue. Could you try deleting ~/.npm dir and redo make clean install?

[anandva]

That fixed issue on my mac able to proceed,

While trying to setup on CentOS5.5 do we need "expat libexpat1-dev"
modules and tried without those
getting different errors deleting and not deleting npm folder, here
are the details,
https://github.com/isaacs/npm/issues/2359

Appreciate your help Subbu.

-Vivek.

[Subbu Allamaraju]

I'm not sure what native libs are needed for expat on CentOS - but here is a quick way to figure out.

Try "npm install expat" in some temp dir. If that goes through you've the right dependencies installed. If not, expat and expat-devel are the libs to look for.

[anandva]

Subbu,

i managed to install expat and expat-devel packages, after npm clean
and reinstall went successfully but got into below error on startup
--Error: Cannot find module '../build/default/node-expat' >> tried ln -
s Release default now am getting below error
-bash-3.2$ bin/start.sh
Error: Unable to load shared library /home/vapuri/mhwork/qlio/
node_modules/ql.io-app/node_modules/ql.io-console/node_modules/
xml2json/node_modules/node-expat/build/Release/node-expat.node
at Object..node (module.js:472:11)
at Module.load (module.js:348:31)
at Function._load (module.js:308:12)
at Module.require (module.js:354:17)
at require (module.js:370:17)
at Object.<anonymous> (/home/vapuri/mhwork/qlio/node_modules/ql.io-
app/node_modules/ql.io-console/node_modules/xml2json/node_modules/node-
expat/lib/node-expat.js:6:14)
at Module._compile (module.js:441:26)
at Object..js (module.js:459:10)
at Module.load (module.js:348:31)
at Function._load (module.js:308:12)

As suggested in another thread tried deleting combinations of npm or
sudo or perm, none of them is helping...

-Vivek.

[Subbu Allamaraju]

Which version of node/npm are you running?

I just built a CentOS 6.0 x86 VM using VirtualBox. I installed gcc, openssl, expat, expat-devel, and then node-0.6.14. After that I created a new ql.io app and it started fine.

On some distros, some folks ran into this issue. Could you try the fix in the link below?

http://archive.cnblogs.com/a/2178519

Subbu

[anandva]

here's env details, soln in the link didn;t help much,

node -v >> v0.6.15
npm -v >> 1.1.16
centOS >> CentOS (Final) 5.5 x86_64

here is the error without symblink to Release from default,

Error: Cannot find module '../build/default/node-expat'
at Function._resolveFilename (module.js:332:11)
at Function._load (module.js:279:25)

at Module.require (module.js:354:17)
at require (module.js:370:17)
at Object.<anonymous> (/home/vapuri/mhwork/qlio/node_modules/ql.io-
app/node_modules/ql.io-console/node_modules/xml2json/node_modules/node-
expat/lib/node-expat.js:6:14)
at Module._compile (module.js:441:26)
at Object..js (module.js:459:10)
at Module.load (module.js:348:31)
at Function._load (module.js:308:12)
at Module.require (module.js:354:17)

-Vivek.

[Subbu Allamaraju]

On Mac, do you see the same dir structure? I also did not follow the comment "error without symblink to Release from default".

Subbu

[anandva]

yes the dir structures are same, file perm are also same.

I also did not follow the comment "error without symblink to Release
from default".

>> i have created a symblink following this thread, http://groups.google.com/group/qlio/browse_thread/thread/d5efdd5dcdbe2cf7/ef13c1043abf0ce8?lnk=gst&q=expat#ef13c1043abf0ce8
then the error was "unable to load ..." rather than Cannot find the
file.

Could it be that ql.io is unable to load node.js dependent
functionality (javascript, C++ code)..?

-Vivek.

[Subbu Allamaraju]

Got it.

ql.io uses a xml2json which in turn uses node-expat. node-expat is built on expat which is a native lib on most operating systems. Since such modules are not binary portable, npm uses node-waf to build them locally. That step is failing in your case. As I mentioned below, I'm able to run ql.io on a fresh CentOS 6.0 VM last night without any issues.

However, this issue has bugged us enough that I would like to get to the bottom of this.

Let me setup a new CentOS instance on EC2 and share it with you to compare.

Subbu

[Subbu Allamaraju]

I just setup a new EC2 CentOS 5.5 instance and tried and still could not reproduce this problem.

yum groupinstall 'Development Tools'
yum install openssl
yum install openssl-devel
yum install expat-devel
rpm -Uvh http://repo.webtatic.com/yum/centos/5/latest.rpm
yum install --enablerepo=webtatic git-core
Download, configure, make and make install node
Create a new ql.io app
bin/start.sh

I will send you the key to access this instance privately.

Subbu

[anandva]

Thanks for you help Subbu. and sorry its keeps dragging a bit ..

just i did the same, i have created a brand new vm on my side clean
with CentoOS 5.5 Final and i finally able to run qlio app
successfully.

i found the issue its while make build of nodejs,
1. when the vm is created i don;t have perm/rights to create files
under /usr/local/lib, /bin, man1 etc, so i cerated them manually
changed ownership to myself, this went well
2. while installing nodejs got error that "OSError: [Errno 13]
Permission denied", and that needed sudo access, just for install i
used "sudo make install", installed qlioapp

thats it and everything is fine now.

-Vivek.

On Apr 12, 2:03 pm, Subbu Allamaraju <su...@subbu.org> wrote:
> I just setup a new EC2 CentOS 5.5 instance and tried and still could not reproduce this problem.
>
> yum groupinstall 'Development Tools'
> yum install openssl
> yum install openssl-devel
> yum install expat-devel

> rpm -Uvhhttp://repo.webtatic.com/yum/centos/5/latest.rpm

> yum install --enablerepo=webtatic git-core
> Download, configure, make and make install node
> Create a new ql.io app
> bin/start.sh
>
> I will send you the key to access this instance privately.
>
> Subbu
>
> On Apr 12, 2012, at 1:34 PM, Subbu Allamaraju wrote:
>
>
>
> > Got it.
>
> > ql.io uses a xml2json which in turn uses node-expat. node-expat is built on expat which is a native lib on most operating systems. Since such modules are not binary portable, npm uses node-waf to build them locally. That step is failing in your case. As I mentioned below, I'm able to run ql.io on a fresh CentOS 6.0 VM last night without any issues.
>
> > However, this issue has bugged us enough that I would like to get to the bottom of this.
>
> > Let me setup a new CentOS instance on EC2 and share it with you to compare.
>
> > Subbu
>
> > On Apr 12, 2012, at 12:40 PM, anandva wrote:
>
> >> yes the dir structures are same, file perm are also same.
>
> >> I also did not follow the comment "error without symblink to Release
> >> from default".

> >>>> i have created a symblink following this thread,http://groups.google.com/group/qlio/browse_thread/thread/d5efdd5dcdbe...

[Subbu Allamaraju]

Awesome. Happy to know :)

[Subbu Allamaraju]

The fix to support escaped quotes in quoted strings and escaped apostrophe's in single quoted strings is in the pull req https://github.com/ql-io/ql.io/pull/442. Will make it to npm shortly.