"bzm - Random CSV Data Set Config" not working with UTF-8 encoded CSV-Files with non ASCII-characters

[Slim Zghal]

The random CSV Data Set Config is not capable of correctly dealing with UTF-8 encoded CSV-files with non "standard" characters (with characters requiring multibyte encoding). 

I attached such a CSV-file. Everything works fine as long as "random order" is not checked (which defaults to the standard csv-config element's behavior). As soon as it is checked, testing the CSV reading fails and show wrong results.

Tested with JMeter 3.3 on Windows 7 64bit (System property: file.encoding=Cp1252)

[Andrey Pokhilko]

Thanks for reporting this issue. Fixed version 0.2 of the plugin has been released.

Andrey Pokhilko

06.11.2017 16:20, Slim Zghal пишет:

The random CSV Data Set Config is not capable of correctly dealing with UTF-8 encoded CSV-files with non "standard" characters (with characters requiring multibyte encoding). 

I attached such a CSV-file. Everything works fine as long as "random order" is not checked (which defaults to the standard csv-config element's behavior). As soon as it is checked, testing the CSV reading fails and show wrong results.

Tested with JMeter 3.3 on Windows 7 64bit (System property: file.encoding=Cp1252)

--
You received this message because you are subscribed to the Google Groups "jmeter-plugins" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jmeter-plugin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Slim Z.]

Thank you for fixing it! Great work!

Slim Z.

[Slim Z.]

Unfortunately, I found out that the config element is still buggy when shared between threads. To reproduce it, just set the thread count in the Thread Group to something >1 and share the element between threads (uncheck "independent list per thread").

You'll then get plenty of "java.lang.RuntimeException: Cannot get next record from csv file: malformed input" and "java.io.UTFDataFormatException: malformed input" errors. This is the case whether "random order" is set or not.

Am Dienstag, 7. November 2017 18:36:46 UTC+1 schrieb Andrey Pokhilko:

[Andrey Pokhilko]

Thanks for testing it! The bug was fixed and version 0.3 were released. Please check it out.

Andrey Pokhilko

08.11.2017 03:31, Slim Z. пишет:

[Slim Z.]

Hi Andrey,

thumbs up, but still some work to do:

First, the filechosser is set to Directories only:

browseButton.addActionListener(new BrowseAction(filenameField, true)); (RandomCSVDataSetConfigGui.java line 66, should be set to false)

Second, the implementation is very slow, quite a lot of time is spent on DataInput.readByte() (see attached JProfiler screenshots). Threads are blocked most of the time (see comparison screenshot between stadard CSV-Config-Element and Random CSV Data Set config-Element). As I am not a developer, I can't help on this issue unfortunately. May be looking at something like https://github.com/lveci/nest/blob/master/beam/beam-core/src/main/java/org/esa/nest/dataio/EnhancedRandomAccessFile.java or https://github.com/Unidata/thredds/blob/b731bcb45b6e10b7e6102e97a9ef35e9fef43c93/cdm/src/main/java/ucar/unidata/io/RandomAccessFile.java would be helpful?

[Andrey Pokhilko]

Thanks again for your testing. New version 0.4 has been released to fix this issue, please check it out.

Andrey Pokhilko

08.11.2017 18:26, Slim Z. пишет:

[heaji...@gmail.com]

Hi,  I am using "Random CSV data set config' v0.6 but still does not work for me. Use the same CSV data file, Regular CSV Data Set config works but Random CSV data set config fail. Basically I am trying to login to web application.

Any suggestion?

[Artem Fedorov]

Hi

If you use the same .csv file from this topic, please, recheck that  your Jmeter lib/ext folder contains just the latest version of this plugin. (Remove previous plugin's versions if you'll find them) 

Thanks,

Artem

3 мар. 2018 г. 5:07 пользователь <heaji...@gmail.com> написал:

To unsubscribe from this group and stop receiving emails from it, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[Heajin Kim]

Hi, Artem

  I double checked as you mentioned, it is v0.6 as you can see in the screenshot. Also I am using jmeter v4.0. I hope that was NOT mistaken.

Any other suggestion why it is not working?

--
You received this message because you are subscribed to a topic in the Google Groups "jmeter-plugins" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jmeter-plugins/r6KtmN2TxzA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[artem....@blazemeter.com]

So, 

And you said, that you use the same .csv file (from the first message) in this topic, is it correct? (I attached it again)

From my side I know, that this plugin have unit test with this file https://github.com/Blazemeter/jmeter-bzm-plugins/blob/master/random-csv-data-set/src/test/java/com/blazemeter/csv/RandomCSVReaderTest.java#L97 

I'll try to reproduce it with on windows and jmeter 4.0. Which Java version do you use? 

Thanks,
Artem

понедельник, 5 марта 2018 г., 21:49:55 UTC+3 пользователь Heajin Kim написал:

--

You received this message because you are subscribed to a topic in the Google Groups "jmeter-plugins" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jmeter-plugins/r6KtmN2TxzA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[artem....@blazemeter.com]

Maybe you can clarify what exactly does not work?

What is your input and output? Please, attache jmeter.log, it can helps me

Thanks,
Artem

понедельник, 5 марта 2018 г., 21:49:55 UTC+3 пользователь Heajin Kim написал:

Hi, Artem

--

You received this message because you are subscribed to a topic in the Google Groups "jmeter-plugins" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jmeter-plugins/r6KtmN2TxzA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[Heajin Kim]

Hi, Artem

  yes, I am using same csv file, just simple login user file (username, userid, etc)
my java version is 8, detail is,

java version "1.8.0_161"

Java(TM) SE Runtime Environment (build 1.8.0_161-b12)

Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

Appreciate your help.

Heajin.

To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[artem....@blazemeter.com]

What result output do you have? Please, attache jmeter.log, it can helps me

понедельник, 5 марта 2018 г., 22:44:03 UTC+3 пользователь Heajin Kim написал:

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[Heajin Kim]

Hi, Artem

  I just re-test again with same csv file (Users.csv - 10 users), still failed 9 out of 10 with unauthorized  error which does not make any sense because it is authorized (auth key exists) Also if the auth key is not working, it should failed on 10 users and if normal csv data config, it works.

I attached in here with zip file with test result tree, jmeter.log and other reports just in case it might help you to debug the issue.

Appreciate your help.

Heajin.

To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[Heajin Kim]

Hi, Artem

  I made a mistake on configuration  so the test result in the previous result is not correct. I should set for 1 Thread not 10 Threads (concurrent).

I did re-test with 1 Thread 1 Second, 10 loop - because 10 users but it is not working so it does not matter, the test result is attached in here. 

For jmeter.log file, you should check the log after timestamp = "2018-03-05 15:29" which is restarting time of test.

My apologies for inconvenient and/or wasting your time if any

Thank you.

Heajin.

[artem....@blazemeter.com]

I analyzed your jmeter.log file and I can say, that plugin found 10 records in your .csv file. I think you have some error with variables in your TestPlan. 

I suggest your to add DebugSampler and ViewResultsTree in your TestPlan. (I attached it). And after execution in ViewResultTree select  DebugSampler -> Response data. In this response data you can see all variables from your csv file.

Thanks,
Artem

вторник, 6 марта 2018 г., 3:07:15 UTC+3 пользователь Heajin Kim написал:

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[Heajin Kim]

Thank you for your help Artem. 

Yes I already user DebuggerSampler. I do not see any data on "respond data" section if I use 'Random CSV data set config' but I see the data on 'respond data' section if I use 

"CSV data set config".

I think I should explain a little bit more detail about the application I am testing...

Here is the steps I need it

1) Accessing Auth Token generation URL with userid + userName which are from CSV data file

2) Pop up message to enter base admin user id / pwd to generate auth token, after enter the base admin user / pwd, it will generate the auth token for the specific user

for example - 92skdi729jsa&26272

3) Generate a new URL with Auth Token from step 1 URL, for example

https://www.abcde.com/template/authkey= 92skdi729jsa&26272

So I can access the application I need it. 

Here is a thing, on the step 1, when you access the URL, it generates an auth token but website shows the auth token only, no tag, no id, no variable. I use the debugger to find anything to parameterized, nothing. It looks like this

So I use the regular expression extractor to get the auth token. get everything, like this.

I am not sure but the auth token generation url might causes the issue. 

Just fyi, I am not able to do concurrent test either, for example, thread config with like "2 threads, 60 secs ramp-up, 5 loops" is failing 

Appreciate any suggestion / recommendation I should do it.

Thank you.

Heajin Kim.

To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[artem....@blazemeter.com]

My recommendation is simplify your test. Let's start from Random CSV Data Set and Debug Sampler. If you don't see any Response Data in Debug sampler it means, that we do something wrong. 

What do you see after click 'Test CSV Reading' button in RandomCSV config element?

https://www.blazemeter.com/blog/introducing-the-random-csv-data-set-config-plugin-on-jmeter

Thanks,
Artem

вторник, 6 марта 2018 г., 21:16:49 UTC+3 пользователь Heajin Kim написал:

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[Heajin Kim]

This is what I see if I click om 'Test CSV Reading' button ..

Reading CSV successfully finished, 10 records found:

${clientId} = 11111

${name} = AbcdEfgh

${userName} = 943ff5323d17b

${locale} = en_us

${userId} = 122333

${corporateId} = 11112

------------

it looks like plugin reads properly based on what I see ... That is why I am so puzzled and not sure what's going on.

To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[artem....@blazemeter.com]

And in ViewResultTree ->Debug Sample -> Response Data you can see the same output now?

вторник, 6 марта 2018 г., 21:56:37 UTC+3 пользователь Heajin Kim написал:

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[Heajin Kim]

I just re-test, this is what I found it

first 4 random users are failed, request URL is correct, I manually validate (cp from request tab & paste to FF, it works as should be). The respond data shows the below message 

<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.41 - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 401 - </h1><div class="line"></div><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>This request requires HTTP authentication.</u></p><hr class="line"><h3>Apache Tomcat/8.0.41</h3></body></html>

5th random generate user is succeed and move to next steps. Also 6th random generate uer works too.... 

The thing I do not under why HTTP 401 error which is UNAUTHORIZED error. 

I set Thread Group with # of Threads = 1, Ram-Up periods = 1, Loop Count = 10.

To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[artem....@blazemeter.com]

I think that failed authorization doesn't depends from this plugin)
If you already have some users, that successfully authorize it means, that plugin works normal.

My suggestion to you is simplify your test plan (disable HTTP request, and test it with Debug Sampler where you can see variables values)

Thanks,
Artem

вторник, 6 марта 2018 г., 22:23:04 UTC+3 пользователь Heajin Kim написал:

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.

[Heajin Kim]

Thank you for your support and recommendation.

Do you have any idea why first 4 random generate users are failing ... I've been testing multiple times (3 times actually), it always failed on first 4 random generated users and after that it works fine... just curious ... any idea? 

To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.

[artem....@blazemeter.com]

Maybe first 4 records doesn't exists on your server or something like this. You can use Step-by-step debbuger for debug your test plan. Maybe it can be helpful

Thanks,
Artem

вторник, 6 марта 2018 г., 22:54:07 UTC+3 пользователь Heajin Kim написал:

To unsubscribe from this group and all its topics, send an email to jmeter-plugin...@googlegroups.com.