how to solve Unhandled error: URIError: URI malformed

[Neeraj Bodhe]

i am coming across the below exception while using rest api for attaching a image file to a jira issue. This isn't consistent but it occurs 2 in 3 times.

Unhandled error: URIError: URI malformed
at encodeURIComponent (native)
at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:244:27)
at stringifyObject (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:300:16)
at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:240:12)
at stringifyArray (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:275:14)
at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:238:12)
at stringifyObject (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:300:16)
at Object.exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:240:12)
at Request.form (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/request/request.js:989:20)
at Request.init (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/request/request.js:247:10)

The second line of the exception says (native), i'm not sure what it means.
How do i resolve this exception.

my set of code

var httpClient = addon.httpClient({
clientKey: req.session.clientKey,
userKey: req.body.userKey,
appKey: addon.key
});
httpClient.post({
url: '/rest/api/2/issue/' + req.body.issueKey + '/attachments',
headers: {
'X-Atlassian-Token': 'nocheck',
'Content-Type': 'application/json',
'Accept': 'application/json'
},
form: {
file: [
(uploadData), // filedata
{
filename: (filename),
contentType: 'image/png'
}
]
}
}, function(err, callbackRes, body) {
});

[Peter Brownlow]

Hi Neeraj,

Have you logged the malformed URIs?

Are you generating the same URI every single time or different URIs?

Does a given URI fail or pass consistently, or alternatively does a given URI sometimes pass and sometimes fail?

-Peter

[Neeraj Bodhe]

hi peter, a given specific URI sometimes passes or fails.

[Peter Brownlow]

Can you tell us what this URL is? :)

[Neeraj Bodhe]

the url is

 url: '/rest/api/2/issue/' + req.body.issueKey + '/attachments',

 url: '/rest/api/2/issue/TEST-1/attachments',

baseurl is 

http://localhost:2990/jira

[Peter Brownlow]

Neeraj,

When I generate a new ACE project using "atlas-connect new my-project-name", then "npm i", and then inspect node_modules/atlassian-connect-express/node_modules/request/request.js:989 it appears to be calling qs.stringify(form). Checking node_modules/atlassian-connect-express/node_modules/qs/index.js:240 verifies this.

The failure is happening when processing the form, not the URI. This makes sense of a given URI only sometimes being associated with an error.

You need to check the content of the form and perhaps sanitise it.

-Peter

[Patrick Streule]

Neeraj,

According to the JIRA docs, you will need to send a multipart/form-data POST, not a application/x-www-form-urlencoded POST.

A Java example can be found here:

http://stackoverflow.com/questions/18631361/add-attachment-to-jira-via-rest-api

For doing the same in ACE, see the request.js docs about Forms:

https://github.com/mikeal/request

Regards,

Patrick

[Neeraj Bodhe]

hi patrick and peter,
 
i tried to use the code of request lib.

var r = request.post('http://service.com/upload', function optionalCallback (err, httpResponse, body) {
  if (err) {
    return console.error('upload failed:', err);
  }
  console.log('Upload successful!  Server responded with:', body);
})
var form = r.form()
form.append('my_field', 'my_value')
form.append('my_buffer', new Buffer([1, 2, 3]))
form.append('my_file', fs.createReadStream(path.join(__dirname, 'doodle.png')))
form.append('remote_file', request('http://google.com/doodle.png'))

The form that is created here for upload is through the method form() from the request object r.
should we create a new request object and try to integrate with httpClient.post here or something else needs to be done to get this going?

[Peter Brownlow]

Hi Neeraj,

Since you are using ACE you can let it take care of some of the details for you. Here is some code from an add-on I'm working on that successfully uploads attachments to JIRA.

var httpClient = addon.httpClient({

            clientKey: clientKey,

            userKey: userKey,

            appKey: addon.key

        });

httpClient.post( {

                url: '/rest/api/2/issue/' + issueKey + '/attachments',

                headers: {

                    'X-Atlassian-Token': 'nocheck'

                },

                form: {

                    file: [

                        attachmentContentString,

                        {

                            filename: filename,

                            contentType: 'text/plain'

                        }

                    ]

                }

            }, function(err, res) {
                // check err and res

            });

-Peter

[Neeraj Bodhe]

hi peter,
             Presently i am usingg the same code that you have suggested and i am facing the error in this code itself.

What i am trying to do is make the upload in multipart way as suggested by patrick here.

Hi patrick,

           Can you pls look into my previous comment and suggest me how to go around this multipart file upload and not just passing the content of the file in the form..

[Peter Brownlow]

What error are you seeing?

[Neeraj Bodhe]

hi peter, this is the error i see in the nodejs console

Unhandled error: URIError: URI malformed
at encodeURIComponent (native)
at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:244:27)
at stringifyObject (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:300:16)
at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:240:12)
at stringifyArray (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:275:14)
at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:238:12)
at stringifyObject (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:300:16)
at Object.exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:240:12)
at Request.form (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/request/request.js:989:20)
at Request.init (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/request/request.js:247:10)

at new Request (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/request/request.js:98:8)

[Neeraj Bodhe]

hi patrick,

I think the cause of the problem is the multipart upload
can pls you post a elaborated code snippet which can be included in the jira rest api to overcome this problem as i am finding difficulty to include it there.

[Peter Brownlow]

Hi Neeraj,

I think that you should now be unblocked using the workaround that I posted on thread https://groups.google.com/forum/#!topic/atlassian-connect-dev/OX-tOQksMro .

-Peter

[Neeraj Bodhe]

hi peter,

         

I tried your work around suggested in thread https://groups.google.com/forum/#!topic/atlassian-connect-dev/OX-tOQksMro It worked fine there for the txt file

But here i am using a png file and the error is related to nodejs. Can you pls look into this and provide a work around.

[Neeraj Bodhe]

hi peter 

My code snippet for the one you suggested and i even tried the form-data component

               var FormData = require('form-data');

               var form = new FormData(); 

               form.append("file", fileData);

                

               httpClient.post({

                    url: '/rest/api/2/issue/' + req.body.issueKey + '/attachments',

                    headers: {

                        'X-Atlassian-Token': 'nocheck',

                        'content-type': 'multipart/form-data'

                    },

                    form: {

                        file: [

                                form,

                             {

                                 filename:  "image1.png",

                                 contentType: 'image/png'

                             }

                        ]

                    }

                }, function(err, callbackRes, body) {

                });

Also i tried a different approach of appending empty text to it which you suggested, but i think this problem might not be related to that as it occurs in any size of image.

                  if (fileData.length > 0 && fileData.length < 200) {

                       fileData += '\n                                                                                                                     \n';

                  }  

                   headers: {

                        'X-Atlassian-Token': 'nocheck',

                        'content-type': 'multipart/form-data'

                    },

                    form: {

                        file: [

                                fileData,

                             {

                                 filename:  "image1.png",

                                 contentType: 'image/png'

                             }

                        ]

                    }

The error on nodejs console persists

Unhandled error: URIError: URI malformed
    at encodeURIComponent (native)
    at exports.stringify (/home/addteq/onDemand/jira-ondemand-sequence-generator/node_modules/atlassian-connect-express/node_modules/qs/index.js:244:27)

[Neeraj Bodhe]

hi peter,

i have also raised this as a bug in ecosystem.atlassian.net
I have provided the code and data to reproduce this
Pls take a look at it.

Link: https://ecosystem.atlassian.net/browse/AC-1274

[Patrick Streule]

Neeraj,

Where is 'fileData' coming from in your case? 

The following works flawlessly for me, regardless of file type and file size (I tested it with a 6KB jpeg and a very small text file):

      var filePath = path.join(__dirname, '../public/test.jpeg');

      fs.readFile(filePath, function (err, data) {

        httpClient.post({

            uri: "/rest/api/2/issue/TEST-1/attachments",

            headers: {

              'X-Atlassian-Token': 'nocheck'

            },

            form: {

              file: [

                data,

                {

                  filename: 'someothername.jpeg'

                }

              ]

            }

          },

          function (err, httpResponse, body) {

            // handle errors

          });

      });

Patrick

[Patrick Streule]

Actually, with the particular image you attached to AC-1274 I get the same error... 

I'll look into it to see whether this could be a bug in ACE or request.js

Regards,

Patrick

[Neeraj Bodhe]

Thanks patrick,

If you need any more image examples which result in that error then pls do let me know.

[Patrick Streule]

Neeraj,

I could confirm that this is actually an issue in ACE. The fix will be tracked in AC-1274 that you created.

Regards,

Patrick

[Neeraj Bodhe]

hi patrick, 

       For now i took the code from the bitbucket repo where AC-1274 is merged as npm wasn't updating the current changes.

i can see that "options.form is deprecated: please use options.multipartFormData" 

Thanks, my problem is solved for now.

[Patrick Streule]

Hi Neeraj,

Correct. The fix is pushed, but we haven't released a new version of ACE to npm yet (this should happen soon).

Regards,

Patrick