Re: Vugen Scripting - Handling Binary data in request body

[James Pulley]

Your first stop should be the developer of the application as they are they ones most familiar with the algorithm and mechanism for encode/decode.     If you have not yet spoken with the developer then give them a ring/drop them an email on this issue.

'Pulley

On Sunday, November 11, 2012 10:48:17 PM UTC-5, Abhi wrote:

Hi-

Has anyone has any success working with the 'Binary data' in web_custom_request() ?   My Vugen scripts captures all the client server request in a 'Binary format' in the body with an encryption Type as : "EncType=application/octet-stream". Do you guys have any suggestion on how to read these data and/or convert them to readable text ? I am using http protocol with Vugen 11.04 for a smartclient built in .NET 3.5.  A sample request body is as below -

 

web_custom_request("appserver.aspx_106",
  "URL=http://SERVER/app/appserver.aspx",
  "Method=POST",
  "TargetFrame=",
  "Resource=0",
  "RecContentType=text/html",
  "Referer=",
  "Snapshot=t107.inf",
  "Mode=HTML",
  "EncType=application/octet-stream",
  "BodyBinary=PK\\x03\\x04-\\x00\t..............................................)

 

thanks,

Abhi

[Mayank Mishra]

Capture the binary value using web_reg_save_param  and then convert the value using 

web_reg_save_param("ParamCaptureBinaryValue", 

"LB/IC=name=\"__EVENTVALIDATION\" id=\"__EVENTVALIDATION\" value=\"", 

"RB/IC=\" />", 

"Ord=1", 

"Search=Body", 

"RelFrameId=1.2", 

LAST);

web_convert_param("Param12", 

"SourceString={ParamCaptureBinaryValue}", 

"SourceEncoding=HTML", 

"TargetEncoding=URL", 

LAST);

Pass Param12 in Binary body  "BodyBinary=PK\\x03\\x04-\\x00\t..............................................)

  "BodyBinary={Param12}

Try this.. it should work i did the same 

On Mon, Nov 12, 2012 at 9:18 AM, Abhi <abhiji...@gmail.com> wrote:

Hi-

Has anyone has any success working with the 'Binary data' in web_custom_request() ?   My Vugen scripts captures all the client server request in a 'Binary format' in the body with an encryption Type as : "EncType=application/octet-stream". Do you guys have any suggestion on how to read these data and/or convert them to readable text ? I am using http protocol with Vugen 11.04 for a smartclient built in .NET 3.5.  A sample request body is as below -

 

web_custom_request("appserver.aspx_106",
  "URL=http://SERVER/app/appserver.aspx",
  "Method=POST",
  "TargetFrame=",
  "Resource=0",
  "RecContentType=text/html",
  "Referer=",
  "Snapshot=t107.inf",
  "Mode=HTML",
  "EncType=application/octet-stream",
  "BodyBinary=PK\\x03\\x04-\\x00\t..............................................)

 

thanks,

Abhi

--
 
 

[James Pulley]

Assuming you are testing an on house developed application and not a COTS product, the developers will always be the best source of information on internal algorithms used which would require implementation in your code

Who aid anything about encryption?   SSL is handled at the http protocol layer.   You are looking for a way to understand the content of the octet stream on the off chance that you need to correlate or parameterize some information inside the file

On Nov 13, 2012, at 1:21 AM, Abhi <abhiji...@gmail.com> wrote:

Thanks James. That might be a painful process for me as we are testing a Product and divulging product related info might be a bit of challenge.  I will reach out to them anyway. But even if they come and provide me with the encryption information, is there an easy way to get to the bottom of it ?

 

Thanks,

Abhijit

--
 
 

[James Pulley]

"There are hundreds....."

Whoever old you this job was simple or easy lied to you.  This is why you (should) get paid the big bucks 

On Nov 13, 2012, at 1:26 AM, Abhi <abhiji...@gmail.com> wrote:

Thanks Mayank - I will give it a go and see.

 

My issue is :  there are hundreds of such web_custom_request() been recorded, even for a very small part of the Business process. So I might have to do them manually or create some rule.

 

-Abhijit

--
 
 

[James Pulley]

Use your programming skills to decompress the file on the fly and pull content out that you need. 

Refer to your mentor for concerts of wan friendly applications related to the number of turns an application makes and the sizes of the data flows

If would appear that your application architects and developers may want to consider some inbuilt features of web servers to compress data on the fly using the GZIP algorithm (gZip header).    If they have compression enabled on the web server operating on an already compressed file then they are burning CPU cycles with almost zero benefit.   

On Nov 14, 2012, at 11:55 PM, Abhi <abhiji...@gmail.com> wrote:

Thanks Andre. The issue is Binary body is not a static text so not sure if we could use lr_save_var()

 

But you made an ineteresting point about network latency; why network latency will have an impact on the request body - end of the day, it is just a http request from client. isn't it ?

 

Thanks,

Abhijit

On Wednesday, November 14, 2012 11:55:39 PM UTC+11, André Luyer wrote:

> "BodyBinary=PK\\x03\\x04-\...
The first four characters indicate that you are POSTing a zip file. That should be easy to unzip it to make it readable.

Parameters are binary save. So after:
   lr_save_var("Some\0binary\0\7\xFFdata", 18, 0, "BinaryData");
you can use it this way:
  web_custom_request(...
  "EncType=application/octet-stream",
  "Body={BinaryData}",

> My issue is :  there are hundreds of such web_custom_request() been recorded, even for a very small part of the Business process.

So this application is extremely sensitive for network latency... stop here and have that fixed first.

André

On Monday, November 12, 2012 4:48:17 AM UTC+1, Abhi wrote:

Hi-

Has anyone has any success working with the 'Binary data' in web_custom_request() ?   My Vugen scripts captures all the client server request in a 'Binary format' in the body with an encryption Type as : "EncType=application/octet-stream". Do you guys have any suggestion on how to read these data and/or convert them to readable text ? I am using http protocol with Vugen 11.04 for a smartclient built in .NET 3.5.  A sample request body is as below -

 

web_custom_request("appserver.aspx_106",
  "URL=http://SERVER/app/appserver.aspx",
  "Method=POST",
  "TargetFrame=",
  "Resource=0",
  "RecContentType=text/html",
  "Referer=",
  "Snapshot=t107.inf",
  "Mode=HTML",
  "EncType=application/octet-stream",
  "BodyBinary=PK\\x03\\x04-\\x00\t..............................................)

 

thanks,

Abhi

--
 
 

[chaitanya bhatt]

More than often Octet Stream MIME type is used when this is a necessity for uploading attachments in the form of binary array to the server. If you are uploading exe's, pdfs, compressed files etc. don't worry about decoding the binary array from the request, as you are not going to gain anything out of it. Also, if you are working on a thick client which is serializing client objects into binary array then it's highly likely that you don't have to correlate anything in it. And, if the developer is using the MIME type Octet-Strem for metadata exchange then ask him to go back to school as marshaling and un-marshalling byte arrays is a super expensive process.

Thanks

Chaitanya M Bhatt

http://www.PerformanceCompetence.com

[James Pulley]

You have a c development environment for web virtual users 

There exists c libraries for the compression and decompression of files in a zip format

Follow the implied path

Yes, this has been done before successfully. I can even recall one instance where two layers of data modification were necessary.   First the Zipped file.   Them the content had been converted to base64, so that then needed to be reversed.  Then you had raw data that you could see.  On the send, first the reconvert to base64 and then zipped up.   

If your c skills are weak to middling them go grab the beat c programmer you can find to help you on this one.  There will be a lot of dynamic memory allocation and deal location so you will be in a model where it's quite easy ton introduce a fatal memory leak

On Nov 16, 2012, at 12:34 AM, Abhi <abhiji...@gmail.com> wrote:

 

Just to give you more context, we are doing it for a thick client that is implemented in an n-tier architecture ( and it might be serializing client data) but theer are still dyanmic values that we would like to handle.  The web components still works on the .NET viewstate and is fine to work with.

 

 A format, PKZIP matches our pattern (according to the structure of the PKZip file "https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html"), but no decompression method is available unfortunately.

 

  Site - "http://i-tools.org/gzip" provides online method to decrypt gzip format of encoding type UTF - 6.  Is there a similar tool available for UTF-16 ?

 

Thanks,

Abhijit

--
 
 

[James Pulley]

You have code.  You have file.   You have a hypothesis that this algorithm will not work and that another algorithm is required.   This seems to be a testable hypothesis with the items you have on hand.  Give it a shot and see what happens.

On Monday, November 19, 2012 11:48:22 PM UTC-5, Abhi wrote:

There are quite a few compress/ decompress algorithm available but probably none for PKZip. Has anyone got a definitive algorithm for PKZIP decompression ?

 

I got one at : http://www.opensource.apple.com/source/gnuzip/gnuzip-28/gzip/unzip.c     but not sure if it is a right one.

 

Thanks,

Abhijit

[gopi krishna]

U can actually use the link provided(http://meyerweb.com/eric/tools/dencoder/) to decode this value.

search the decoded value in the server response and correlate it. This should help you in resolving the issue. 

Regards/

On Wed, Nov 28, 2012 at 3:44 AM, André Luyer <an...@luyer.nl> wrote:

Base64.

For (de-)compression: look up the lr_load_dll function...

André

On Monday, November 26, 2012 3:37:23 AM UTC+1, Abhi wrote:

Well..guys, no amount of coding and algorithm did help us in decrypting the encrupted compressed request and finally we turned off the compression at a server level ( knowing that has performance implication).  Finally, we got a cpatured code from LR that are better off than earlier and quite readable but the problem does not seem to go away. 

The http request has an XML recorded as part of the http body and within that XML, a certain part is encrypted again which is probably being used to pass on the lgin and other session specific information. I did not see that in any of the response from the previous requests. So a corelation would not help, I believe.

 

Here is a sample request.. ( cut down version). Anyone has got any clue how to decode that portion ?

 

 web_custom_request("appserver.aspx_3",
  "URL=http://MYSERVER/SMINT/app/appserver.aspx",
  "Method=POST",
  "RecContentType=text/html",
  "Mode=HTTP",
  "EncType=application/octet-stream",
  "Body=U<BusObjAssName>T1.Tb</BusObjAssName><BusObjData>
<?xml version=\"1.0\" encoding=\"utf-16\"?>\r\n"
"<LoginVO xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" Dummy=\"Hi\" /></BusObjData><BusObjDataOrigSize>0</BusObjDataOrigSize><BusObjMethod>DoLogin</BusObjMethod><BusObjTypeName>T1.Tb.AppServer.Server.LoginVO</BusObjTypeName><BusObjCompressed>False</BusObjCompressed><DebugFlag>False</DebugFlag><LogDebugPriority>0</LogDebugPriority><StatisticsOnly>False</StatisticsOnly><MDCInfo></MDCInfo>
<Seasame>B+6iPRxhi1GyFnVWbRG1kIgCK7ljSvDNEmABPakyX1jhsLmYpSDMpm9zycYIiv7teayCneSy+f5aJOvBpHDyempwi2GhmhqjDd1u5ekb7LJycSIGuwnv7SsFLecu1jy8ZVSC3i7Ejto8EuLE.................cpKoOJq+jTSiXGqNvkHFLcYpVrU2/0WtKldi4A1ZwaV3XAobrjEK3jR8HeNUkWW3lWX1Rux3tjyBNMM+xR+Mds0fTnthLHDYKmCJN+szOXTfn1iaiTFdTnwXw3YBRxIWODNrS+MiNjWwGFm8kI5pu4WiToiXFX5F19IsULboAeLBtWjSVMFs3gQHshRonBz4/NU9Vw==</Seasame>
<ReqNo>3</ReqNo><PayLoad><?xml version=\"1.0\" encoding=\"utf-16\"?>\r\n"
"<DoRunMethodPayload xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\">\r\n"
"  <Items />\r\n" "</DoRunMethodPayload></PayLoad><SerialisationMode>XML</SerialisationMode><ServiceId>T1.Tb.AppServer.Server.LoginVO</ServiceId><SqlFlag>False</SqlFlag><LogDetailedSql>False</LogDetailedSql><UiContext></UiContext><CheckMaintenanceMode>False</CheckMaintenanceMode><CheckSchemaVersion>False</CheckSchemaVersion>",
LAST);

 

 

Many thanks,

Abhi

--
 
 

[Ajit Lal]

LR version is 11.0

On Tue, Nov 5, 2013 at 9:54 PM, Ajit Lal <ajit...@gmail.com> wrote:

Hi,

I am encountered with a similar situation as mentioned by Abhi....

I created the script successfully on one version of application using web (http/html) protocol. The developers used the compression technology for the client-server communication in the next version of application. I tried recording on next version of application and got the body binary contents captured in binaries throughout the script with no value captured in EncType, i.e, "EncType=".

Please suggest, how to rectify such situations and how to decode the binary data.

Regards,

Ajit

[Ajit Lal]