400 - Bad request error first time running Taurus on RHEL 7

[scott.h...@gmail.com]

After installing Taurus on RHEL 7 behind a corporate firewall, I  ran it for the first time, and while attempting to download JMeter I get the following error:

[2016-12-01 15:55:00,981 ERROR Engine.jmeter.JMeter] Error while downloading https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.0.zip: Unsuccessful download from //archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.0.zip: 400 - Bad Request

I have validated that the system https_proxy setting is correct on RHEL and validated it with other external requests.

Is there a straightforward way to debug this problem so I an see the exact nature of the Bad Request?

Is there a known problem in Taurus executing behind a corporate firewall?

Any help will be appreciated.

Regards,

Scott

[ta...@blazemeter.com]

Hi, Scott.

I couldn't reproduce your problem. Looks like taurus uses https_proxy variable well and jmeter archive is working too. Please run taurus in verbose mode (-v) and share bzt.log with us.

If you are experienced in python look at exception caught in _download function (utils.py) for debug purpose.

[scott.h...@gmail.com]

I have done some more experimentation.

I was able to manually download JMeter with wget using the URL that taurus was trying.

This means that the system proxy environment variable is pointing to the correct internal proxy.

I then created the .bzt/jmeter-taurus directory and expanded the zip.

Doing this I was able to start taurus, and ran the quick_test.yml test file.

However, as I suspected, the problem has not gone away and all of the jmeter threads could not connect to the external blazedemo site.

I terminated the script execution after a while as there was no data being generated and the error.jtl file shows that all socket connections to the server fail.

I am running on 1.7.4 on RHEL 7.

I am not sure where/how to post the two bzt.log files, one from the failed JMeter installation and one from the failed threads connection to blazedemo as part of the quick_test run.and the error.jtl file

I am fairly competent in Python. 

If you can tell me how to configure the taurus execution to capture the python failure I will do that ..

-- Scott

[ta...@blazemeter.com]

1. how did you set up the https_proxy variable? Have you exported it? 

2. just pack the first bzt.log (failed installation) and attach to the message.

---

Taras

[scott.h...@gmail.com]

I set up the proxy as specified by Red Hat http://www.putorius.net/2012/06/how-to-configure-system-proxy-settings.html

It is specified in my .bash_profile and exported in that file.

This works correctly as I can wget resources on the external public network.

As I said I am on 1.7.4 which is fairly new.

I am using the web client which does not seem to allow file attachments to group posts.

Since the file is small I will post here: red highlighting is mine

[2016-12-02 09:44:31,796 INFO root] Taurus CLI Tool v1.7.4

[2016-12-02 09:44:31,796 DEBUG root] Command-line options: {'verbose': True, 'no_system_configs': None, 'quiet': None, 'log': 'bzt.log', 'option': None, 'aliases': []}

[2016-12-02 09:44:31,796 DEBUG root] Python: CPython 2.7.5

[2016-12-02 09:44:31,797 DEBUG root] OS: ('Linux', 'vc2coma2171826n', '3.10.0-327.18.2.el7.x86_64', '#1 SMP Fri Apr 8 05:09:53 EDT 2016', 'x86_64', 'x86_64')

[2016-12-02 09:44:31,797 INFO root] Starting with configs: ['quick_test.yml']

[2016-12-02 09:44:31,797 INFO Engine] Configuring...

[2016-12-02 09:44:31,797 DEBUG Engine] Reading machine configs from: /etc/bzt.d

[2016-12-02 09:44:31,797 INFO Engine] No personal config: /home/a123456/.bzt-rc

[2016-12-02 09:44:31,797 DEBUG Engine.Configuration] Configs: ['/etc/bzt.d/10-base.json', '/etc/bzt.d/99-installID.yml']

[2016-12-02 09:44:31,797 DEBUG Engine.Configuration] Reading /etc/bzt.d/10-base.json as JSON

[2016-12-02 09:44:31,798 DEBUG Engine.Configuration] Reading /etc/bzt.d/99-installID.yml as YAML

[2016-12-02 09:44:31,799 DEBUG Engine.Configuration] Configs: ['quick_test.yml']

[2016-12-02 09:44:31,799 DEBUG Engine.Configuration] Reading quick_test.yml as YAML

[2016-12-02 09:44:31,800 DEBUG root] Configs: ['quick_test.yml']

[2016-12-02 09:44:31,801 DEBUG root] Reading quick_test.yml as YAML

[2016-12-02 09:44:31,802 DEBUG Engine] Requesting updates info: http://gettaurus.org/updates/?version=1.7.4&installID=163e096224

[2016-12-02 09:44:32,113 DEBUG Engine] Result: {"latest":"1.7.4","needsUpgrade":false}

[2016-12-02 09:44:32,113 DEBUG Engine] Installation is up-to-date

[2016-12-02 09:44:32,114 INFO Engine] Artifacts dir: /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065

[2016-12-02 09:44:32,114 DEBUG Engine] New artifact filename: /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/effective

[2016-12-02 09:44:32,115 DEBUG Engine.Configuration] Dumping YAML config into /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/effective.yml

[2016-12-02 09:44:32,131 DEBUG Engine.Configuration] Dumping JSON config into /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/effective.json

[2016-12-02 09:44:32,132 DEBUG Engine] New artifact filename: /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/merged.yml

[2016-12-02 09:44:32,133 DEBUG root] Dumping YAML config into /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/merged.yml

[2016-12-02 09:44:32,134 DEBUG Engine] New artifact filename: /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/merged.json

[2016-12-02 09:44:32,134 DEBUG root] Dumping JSON config into /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/merged.json

[2016-12-02 09:44:32,134 DEBUG Engine] Add existing artifact (move=False): quick_test.yml

[2016-12-02 09:44:32,134 DEBUG Engine] Copying quick_test.yml to /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/quick_test.yml

[2016-12-02 09:44:32,135 INFO Engine] Preparing...

[2016-12-02 09:44:32,135 DEBUG Engine] Module config: consolidator defaultdict(None, {u'ignore-labels': [u'ignore'], u'percentiles': [0.0, 50.0, 90.0, 95.0, 99.0, 99.9, 100.0], u'class': u'bzt.modules.aggregator.ConsolidatingAggregator'})

[2016-12-02 09:44:32,135 DEBUG root] Importing module: bzt.modules.aggregator

[2016-12-02 09:44:32,137 DEBUG root] Loading class: 'ConsolidatingAggregator' from <module 'bzt.modules.aggregator' from '/usr/lib/python2.7/site-packages/bzt/modules/aggregator.pyc'>

[2016-12-02 09:44:32,137 DEBUG Engine.consolidator] Exception in dehumanize_time(inf)

[2016-12-02 09:44:32,137 DEBUG Engine.consolidator] Buffer scaling setup: percentile 95.0 from [0.0, 50.0, 90.0, 95.0, 99.0, 99.9, 100.0] selected

[2016-12-02 09:44:32,137 DEBUG Engine] Module config: monitoring defaultdict(None, {u'class': u'bzt.modules.monitoring.Monitoring'})

[2016-12-02 09:44:32,138 DEBUG root] Importing module: bzt.modules.monitoring

[2016-12-02 09:44:32,142 DEBUG root] Loading class: 'Monitoring' from <module 'bzt.modules.monitoring' from '/usr/lib/python2.7/site-packages/bzt/modules/monitoring.pyc'>

[2016-12-02 09:44:32,142 DEBUG Engine] Module config: local defaultdict(None, {u'class': u'bzt.modules.provisioning.Local'})

[2016-12-02 09:44:32,142 DEBUG root] Importing module: bzt.modules.provisioning

[2016-12-02 09:44:32,143 DEBUG root] Loading class: 'Local' from <module 'bzt.modules.provisioning' from '/usr/lib/python2.7/site-packages/bzt/modules/provisioning.pyc'>

[2016-12-02 09:44:32,143 DEBUG Engine] Module config: jmeter defaultdict(None, {u'path': u'~/.bzt/jmeter-taurus/', u'plugins': [u'jpgc-casutg', u'jpgc-dummy', u'jpgc-ffw', u'jpgc-fifo', u'jpgc-functions', u'jpgc-json', u'jpgc-perfmon', u'jpgc-prmctl', u'jpgc-tst'], u'properties': defaultdict(None, {u'jmeter.save.saveservice.autoflush': u'true', u'jmeterengine.force.system.exit': u'true', u'jmeter.save.saveservice.connect_time': u'true', u'summariser.name': u'', u'mode': u'Stripped'}), u'class': u'bzt.modules.jmeter.JMeterExecutor'})

[2016-12-02 09:44:32,143 DEBUG root] Importing module: bzt.modules.jmeter

[2016-12-02 09:44:32,152 DEBUG root] Loading class: 'JMeterExecutor' from <module 'bzt.modules.jmeter' from '/usr/lib/python2.7/site-packages/bzt/modules/jmeter.pyc'>

[2016-12-02 09:44:32,152 DEBUG Engine.local] Preparing executor: jmeter/21581008

[2016-12-02 09:44:32,153 DEBUG Engine] New artifact filename: /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/jmeter.log

[2016-12-02 09:44:32,153 DEBUG Engine.jmeter] Checking if port 4445 is free

[2016-12-02 09:44:32,153 DEBUG Engine.jmeter] Port 4445 is free

[2016-12-02 09:44:32,153 DEBUG Engine.jmeter] Using port 4445 for management

[2016-12-02 09:44:32,153 DEBUG Engine.jmeter.JavaVM] Trying JavaVM: ['java', '-version']

[2016-12-02 09:44:32,228 DEBUG Engine.jmeter.JavaVM] JavaVM output: java version "1.7.0_101"

OpenJDK Runtime Environment (rhel-2.6.6.1.el7_2-x86_64 u101-b00)

OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

[2016-12-02 09:44:32,229 DEBUG Engine.jmeter.TclLibrary] We don't need to check tcl library on this platform

[2016-12-02 09:44:32,229 INFO Engine.jmeter.JMeter] Will install JMeter into /home/a123456/.bzt/jmeter-taurus

[2016-12-02 09:44:32,230 DEBUG Engine.jmeter.JMeter.JMeterMirrorsManager] Retrieving mirrors from page: https://jmeter.apache.org/download_jmeter.cgi

[2016-12-02 09:44:32,329 ERROR Engine.jmeter.JMeter.JMeterMirrorsManager] Can't fetch https://jmeter.apache.org/download_jmeter.cgi

[2016-12-02 09:44:32,329 DEBUG Engine.jmeter.JMeter.JMeterMirrorsManager] Total mirrors: 1

[2016-12-02 09:44:32,329 INFO Engine.jmeter.JMeter] Downloading: https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.0.zip

[2016-12-02 09:44:32,417 ERROR Engine.jmeter.JMeter] Error while downloading https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.0.zip: Unsuccessful download from //archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.0.zip: 400 - Bad Request

[2016-12-02 09:44:32,418 ERROR root] Internal Error: JMeter download failed: No more links to try

[2016-12-02 09:44:32,418 INFO Engine] Post-processing...

[2016-12-02 09:44:32,418 DEBUG Engine.consolidator] Consolidator buffer[0]: []

[2016-12-02 09:44:32,420 DEBUG Engine.Configuration] Dumping YAML config into /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/effective.yml

[2016-12-02 09:44:32,440 DEBUG Engine.Configuration] Dumping JSON config into /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/effective.json

[2016-12-02 09:44:32,441 INFO root] Artifacts dir: /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065

[2016-12-02 09:44:32,441 WARNING root] Done performing with code: 1

[2016-12-02 09:44:32,442 DEBUG Engine] Add existing artifact (move=True): bzt.log

[2016-12-02 09:44:32,442 DEBUG Engine] Moving bzt.log to /home/a123456/taurus_scripts/2016-12-02_09-44-32.114065/bzt.log

--Scott

[ta...@blazemeter.com]

Hi. 

thanks for your respond. 

I asked about export due to some difference between tools: it's enough just set up proxy variable for wget ($ var=val) but not enough for taurus (cos it runs new shell)

I see correct connection to http in the log. Is http filtered in your organization? Have you any login/pass for proxy?

I'll try to reproduce your problem again and tell you if I find out anything.

---

Taras

[scott.h...@gmail.com]

I appreciate you looking at this.

I think that the running of the python program (bzt) is not picking up the shell proxy environment variables.

I know that when I have to use pip I need to use a syntax "sudo -E pip install ..." so that pip can see my local environment.proxy settings.

If I do not use the '-E' option pip cannot access the external network.

(I think that there may also be an explicit --proxy option on pip for this purpose).

Perhaps you can print out the taurus environment once it starts up to see if it inherits the local shell environment.

-- Scott

[scott.h...@gmail.com]

It seems that there are two places that the proxy needs to be set.

- In the bzt process (which will allow the download of the jameter runtime) and

- In the generated jmeter JMX file.

Using the modified_requests.jmx file as generated by taurus all socket connections failed.

Even with adding a proxy section to the quick-test.yml all socket connections failed.

By adding the proxy values into the modified_requests.jmx file and running the embedded jmeter tool with this jmx the run was successful.

          <stringProp name="HTTPSampler.proxyHost">http.proxy.myserver.com</stringProp>

          <stringProp name="HTTPSampler.proxyPort">8000</stringProp>

-- Scott

[ta...@blazemeter.com]

Hi, Scott. 

It's a good idea about variables checking.

I added next line at the start of Engine.prepare() method in engine.py file 

self.log.info('vars: %s', os.environ)

and saw all variables that I sent to taurus.
1. can you do the same and check if variables are received by taurus?

2. if #1 fails (no https_proxy in the log ) please add any variable, export it manually and check again.

3. Does standalone jmeter work without adding the proxy values into jmx?

4. in any case, please add to message name of your shell.

---

Taras

[scott.h...@gmail.com]

1. Looks like http_proxy and https_proxy variables are received by taurus (I inserted the above python code into the engine.prepare method).

[2016-12-04 15:55:26,536 INFO Engine] vars: {'http_proxy': 'http://http.proxy.myserver.com:8000', 'LESSOPEN': '||/usr/bin/lesspipe.sh %s', 'SSH_CLIENT': '10.25.65.139 51732 22', 'LOGNAME': 'a123456', 'USER': 'a123456', 'PATH': '/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/chef/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/a123456/.local/bin:/home/a123456/bin', 'HOME': '/home/a123456', 'LANG': 'en_US.UTF-8', 'TERM': 'cygwin', 'SHELL': '/bin/bash', 'GENAUTH': '', 'SHLVL': '1', 'HISTSIZE': '1000', 'https_proxy': 'http://https.proxy.myserver.com:8000', 'XDG_RUNTIME_DIR': '/run/user/29001', 'KRB5CCNAME': 'FILE:/tmp/krb5cc_29001_o9WglO', 'XDG_SESSION_ID': '6286', '_': '/usr/bin/bzt', 'SSH_CONNECTION': '10.25.65.139 51732 10.240.136.29 22', 'SSH_TTY': '/dev/pts/0', 'OLDPWD': '/home/a123456', 'HOSTNAME': 'vc2coma2171826n', 'HISTCONTROL': 'ignoredups', 'PWD': '/home/a123456/taurus_scripts', 'MAIL': '/var/spool/mail/a123456', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:'}

[2016-12-04 15:55:26,537 DEBUG Engine] Module config: consolidator defaultdict(None, {u'ignore-labels': [u'ignore'], u'percentiles': [0.0, 50.0, 90.0, 95.0, 99.0, 99.9, 100.0], u'class': u'bzt.modules.aggregator.ConsolidatingAggregator'})

3. jmeter does not connect to external server without explicitly adding the proxy fields into the JMX file. When you do the embedded Jmeter does work and connect.

-- Scott

[ta...@blazemeter.com]

Hi.

It looks strange.

1. As jmeter can't read environment variable too, it's probably the problem of your shell, isn't? 

2. so what is the name of your shell and what type of login do you use? (console, kdm/gdm, ...)

---

Taras

[Andrey Pokhilko]

'TERM': 'cygwin'

This might mean CYGWIN that we don't support.

Andrey Pohilko Chief Scientist P: +7 (909) 631-21-69

--
You received this message because you are subscribed to the Google Groups "codename-taurus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/7b9543c6-e6b8-4653-848a-eda21d59212c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[scott.h...@gmail.com]

Andrey,

Thanks for looking at this.

I have done some investigation and this is what I have found.

I am running a terminal program on Windows into a Red Hat Enterprise 7 VM cloud instance, so all interaction is through the command line.

I see the same behavior whether I am running a 'cygwin' terminal or an 'xterm' terminal.

There are two places that the proxy settings need to be correct:

1. the bzt instance (so that it can install JMeter if necessary)

2. the JMeter instance that bzt starts

1. I have set the appropriate 'http_proxy' and 'https_proxy' environment values in my .bash_profile.

This allows me to access content outside of my corporate network. 

These values are inherited by the bzt python process as seen by the additional logging that was enabled.

However executing bzt -v quick_test.yml results in the Bad request 400 error.

I am not sure why the urllib library that you are using to get the downloaded content is not using the proxy environment variables, 

but on RHEL 7 they are not. (Python 2.7.5)

Perhaps this can be done explicitily as shown in the last code sample on 

"http://stackoverflow.com/questions/3168171/how-can-i-open-a-website-with-urllib-via-proxy-in-python"

2. I have manually installed the JMeter runtime and while it works within taurus, it cannot process requests outside of my corporate environment, 

so all requests fail, even though the ASCII monitor display comes up and waits to display data that never is available .

Of all of the places that it is possible to set proxy values, the only place that the 'http_proxy' setting is visible in the logs is 

when I set it in the .bzt-rc file where I get an output like

[2016-12-06 12:41:35,165 INFO Engine] proxy settings: defaultdict(None, {'address': 'http://http.mycompany.com:8000'})

[2016-12-06 12:41:35,165 DEBUG Engine] Using proxy settings: SplitResult(scheme='http', netloc='http.mycompany.com:8000', path='', query='', fragment='')

Even with this setting all requests from JMeter fail.

There are no proxies set on the JMeter command line nor are they part of the generated JMX file.

[2016-12-06 12:41:35,732 DEBUG bzt.utils] Executing shell: [u'/home/a123456/.bzt/jmeter-taurus/bin/jmeter.sh', '-n', '-t', '/home/a123456/taurus_scripts/2016-12-06_12-41-35.170153/modified_requests.jmx', '-j', '/home/a598184/taurus_scripts/2016-12-06_12-41-35.170153/jmeter.log', '-q', '/home/a123456/taurus_scripts/2016-12-06_12-41-35.170153/jmeter-bzt.properties', '-S', '/home/a123456/taurus_scripts/2016-12-06_12-41-35.170153/system.properties']

When i add them to the modified_requests.jmx file and run JMeter I can get data back.

My conclusion:

Taurus relies on implicit inheritance of proxy settings from environment variables to propagate these settings to bzt and JMeter.

I believe that this implicit inheritance is not always successful for all platforms and python versions.

There are explicit ways of setting the proxy values that should be implemented to make sure that the correct proxy setting are available at each stage of the processing.

Please let me know if I can be any further help in debugging or resolving this issue, 

Scott

[ta...@blazemeter.com]

Scott, thanks for your detailed report.

1. I'm not expert in RedHat virtualisation tecnologies, so could you explain what means

> I am running a terminal program on Windows into a Red Hat Enterprise 7 VM cloud instance, so all interaction is through the command line.

and how exactly did you get this environment?

---

Taras

[Scott Snyder]

I am running Taurus on a RHEL 7 VM that is being provided to me by my employer.

I SSH into the VM from a Windows terminal.

There is nothing special or Red Hat specific about this set up.

All access outside of our internal network has to go through an HTTP proxy and that is why I am having problems.

When I try and execute a ".yml" taurus test file all connections to the "blazedemo.com" site fail, but when I run taurus and execute the generated JMX (JMeter) file that that has the proper proxy settings added everything works. 

I will continue to run my JMX file directly in taurus instead of having the JMX file generated for me so I can manually add the proxy settings until this is resolved.

-- Scott

[ta...@blazemeter.com]

Hi, Scott.

Sorry, but we have no idea about this problem. Please let us know if you resolve it.

---

Taras