Variable substitution in YAML files
1,150 views
Skip to first unread message
dade...@gmail.com
Dec 20, 2016, 10:12:54 AM12/20/16
to codename-taurus
Hi,
I'm executing this config with the following command:
Secondly, the value of modules.jmeter.properties.user.classpath doesn't seem to be replaced either. In the affective config it look like:
My questions are how this substitution works and how I can pass values for this variables in the command line?
My apologies if this question has been asked and answered before but I cannot find a topic with such discussion.
I am new to Taurus and I'm trying to run a simple JMeter test in the command line. This is what my config looks like:
---
execution:
- executor: jmeter
concurrency: 100
ramp-up: 2s
iterations: 10
scenario: sample
scenarios:
sample:
script: my-jmeter-test.jmx
timeout: 500ms
keepalive: true
reporting:
- module: final_stats
- module: console
modules:
jmeter:
path: ~/.bzt/jmeter-taurus/bin/jmeter
version: 3.1
download-link: https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-{version}.zip
gui: false
properties:
log_level: DEBUG
user.classpath: {additional-jars}
console:
disable: false
I'm executing this config with the following command:
bzt -v -o additional-jars=/tmp/libs/ stress.yaml
First of all, when I execute this file I get the following error:
[2016-12-20 14:43:15,662 DEBUG Engine.Configuration] Reading /usr/local/etc/bzt.d/10-base.json as JSON
[2016-12-20 14:43:15,663 DEBUG Engine.Configuration] Reading /usr/local/etc/bzt.d/99-installID.yml as YAML
[2016-12-20 14:43:15,664 DEBUG Engine.Configuration] Configs: ['./src/main/resources/stress.yaml']
[2016-12-20 14:43:15,665 DEBUG Engine.Configuration] Reading ./src/main/resources/stress.yaml as YAML
[2016-12-20 14:43:15,670 DEBUG root] Configs: ['./src/main/resources/stress.yaml']
[2016-12-20 14:43:15,670 DEBUG root] Reading ./src/main/resources/stress.yaml as YAML
[2016-12-20 14:43:15,676 DEBUG Engine] Requesting updates info: http://gettaurus.org/updates/?version=1.7.4&installID=34363bd12f80
[2016-12-20 14:43:16,137 DEBUG Engine] Result: {"latest":"1.7.4","needsUpgrade":false}
[2016-12-20 14:43:16,137 DEBUG Engine] Installation is up-to-date
[2016-12-20 14:43:16,137 DEBUG ConfigOverrider] Applying additional-jars=/tmp/test-suite/lib/
[2016-12-20 14:43:16,137 DEBUG ConfigOverrider] Applying: [additional-jars]=/tmp/test-suite/lib/
[2016-12-20 14:43:16,138 DEBUG ConfigOverrider] Parsed override value: '/tmp/test-suite/lib/' -> '/tmp/test-suite/lib/' (<type 'str'>)
[2016-12-20 14:43:16,138 INFO Engine] Artifacts dir: /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694
[2016-12-20 14:43:16,139 DEBUG Engine] New artifact filename: /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/effective
[2016-12-20 14:43:16,140 DEBUG Engine.Configuration] Dumping YAML config into /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/effective.yml
[2016-12-20 14:43:16,160 DEBUG Engine.Configuration] Dumping JSON config into /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/effective.json
[2016-12-20 14:43:16,162 DEBUG Engine] New artifact filename: /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/merged.yml
[2016-12-20 14:43:16,162 DEBUG root] Dumping YAML config into /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/merged.yml
[2016-12-20 14:43:16,165 DEBUG Engine] New artifact filename: /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/merged.json
[2016-12-20 14:43:16,166 DEBUG root] Dumping JSON config into /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/merged.json
[2016-12-20 14:43:16,166 DEBUG Engine] Add existing artifact (move=False): ./src/main/resources/stress.yaml
[2016-12-20 14:43:16,166 DEBUG Engine] Copying ./src/main/resources/stress.yaml to /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/stress.yaml
[2016-12-20 14:43:16,167 INFO Engine] Preparing...
[2016-12-20 14:43:16,167 DEBUG Engine] Module config: consolidator defaultdict(None, {u'ignore-labels': [u'ignore'], u'percentiles': [0.0, 50.0, 90.0, 95.0, 99.0, 99.9, 100.0], u'class': u'bzt.modules.aggregator.ConsolidatingAggregator'})
[2016-12-20 14:43:16,168 DEBUG root] Importing module: bzt.modules.aggregator
[2016-12-20 14:43:16,171 DEBUG root] Loading class: 'ConsolidatingAggregator' from <module 'bzt.modules.aggregator' from '/usr/local/lib/python2.7/site-packages/bzt/modules/aggregator.pyc'>
[2016-12-20 14:43:16,171 DEBUG Engine.consolidator] Exception in dehumanize_time(inf)
[2016-12-20 14:43:16,171 DEBUG Engine.consolidator] Buffer scaling setup: percentile 95.0 from [0.0, 50.0, 90.0, 95.0, 99.0, 99.9, 100.0] selected
[2016-12-20 14:43:16,171 DEBUG Engine] Module config: monitoring defaultdict(None, {u'class': u'bzt.modules.monitoring.Monitoring'})
[2016-12-20 14:43:16,171 DEBUG root] Importing module: bzt.modules.monitoring
[2016-12-20 14:43:16,209 DEBUG root] Loading class: 'Monitoring' from <module 'bzt.modules.monitoring' from '/usr/local/lib/python2.7/site-packages/bzt/modules/monitoring.pyc'>
[2016-12-20 14:43:16,210 DEBUG Engine] Module config: local defaultdict(None, {u'class': u'bzt.modules.provisioning.Local'})
[2016-12-20 14:43:16,210 DEBUG root] Importing module: bzt.modules.provisioning
[2016-12-20 14:43:16,210 DEBUG root] Loading class: 'Local' from <module 'bzt.modules.provisioning' from '/usr/local/lib/python2.7/site-packages/bzt/modules/provisioning.pyc'>
[2016-12-20 14:43:16,211 DEBUG Engine] Module config: jmeter defaultdict(None, {'gui': False, u'properties': defaultdict(None, {u'jmeter.save.saveservice.autoflush': u'true', u'jmeter.save.saveservice.connect_time': u'true', 'user.classpath': defaultdict(None, {'additional-jars': None}), u'summariser.name': u'', u'jmeterengine.force.system.exit': u'true', u'mode': u'Stripped', 'log_level': 'DEBUG'}), 'version': 3.1, 'download-link': 'https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-{version}.zip', u'plugins': [u'jpgc-casutg', u'jpgc-dummy', u'jpgc-ffw', u'jpgc-fifo', u'jpgc-functions', u'jpgc-json', u'jpgc-perfmon', u'jpgc-prmctl', u'jpgc-tst'], u'path': '~/.bzt/jmeter-taurus/bin/jmeter', u'class': u'bzt.modules.jmeter.JMeterExecutor'})
[2016-12-20 14:43:16,211 DEBUG root] Importing module: bzt.modules.jmeter
[2016-12-20 14:43:16,228 DEBUG root] Loading class: 'JMeterExecutor' from <module 'bzt.modules.jmeter' from '/usr/local/lib/python2.7/site-packages/bzt/modules/jmeter.pyc'>
[2016-12-20 14:43:16,228 DEBUG Engine.local] Preparing executor: jmeter/4428551504
[2016-12-20 14:43:16,228 DEBUG Engine] New artifact filename: /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/jmeter.log
[2016-12-20 14:43:16,228 DEBUG Engine.jmeter] Checking if port 4445 is free
[2016-12-20 14:43:16,231 DEBUG Engine.jmeter] Port 4445 is free
[2016-12-20 14:43:16,232 DEBUG Engine.jmeter] Using port 4445 for management
[2016-12-20 14:43:16,232 DEBUG Engine.jmeter.JavaVM] Trying JavaVM: ['java', '-version']
[2016-12-20 14:43:16,330 DEBUG Engine.jmeter.JavaVM] JavaVM output: java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
[2016-12-20 14:43:16,330 DEBUG Engine.jmeter.TclLibrary] We don't need to check tcl library on this platform
[2016-12-20 14:43:16,331 INFO Engine.jmeter.JMeter] Will install JMeter into /Users/someuser/.bzt/jmeter-taurus
[2016-12-20 14:43:16,331 INFO Engine.jmeter.JMeter] Downloading: https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-{version}.zip
[2016-12-20 14:43:16,792 ERROR Engine.jmeter.JMeter] Error while downloading https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-{version}.zip: Unsuccessful download from //archive.apache.org/dist/jmeter/binaries/apache-jmeter-%7Bversion%7D.zip: 404 - Not Found
100% [ ] ETA: --:--:--
[2016-12-20 14:43:16,794 ERROR root] Internal Error: JMeter download failed: No more links to try
[2016-12-20 14:43:16,794 INFO Engine] Post-processing...
[2016-12-20 14:43:16,794 DEBUG Engine.consolidator] Consolidator buffer[0]: []
[2016-12-20 14:43:16,796 DEBUG Engine.Configuration] Dumping YAML config into /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/effective.yml
[2016-12-20 14:43:16,816 DEBUG Engine.Configuration] Dumping JSON config into /Users/someuser/dev/workspace/test-suite/stress-tests/2016-12-20_14-43-16.138694/effective.json
The substitution of {version} in download-link does not work for me - this example was taken from the docs.
I took a look at the effective.yml which looks like this:
---additional-jars: /tmp/hive-test-suite/lib/cli-aliases: cloud: provisioning: cloud detach: modules: cloud: detach: true func: settings: aggregator: functional-consolidator gui: modules: jmeter: gui: true install-tools: services: - install-checker local: provisioning: local locations: modules: cloud: dump-locations: true provisioning: cloud report: reporting: - blazemeter sequential: modules: local: sequential: trueexecution:- concurrency: 100 executor: jmeter iterations: 10 ramp-up: 2s scenario: stress-hiveinstall-id: 34363bd12f80modules: ab: class: bzt.modules.ab.ApacheBenchmarkExecutor blazemeter: class: bzt.modules.blazemeter.BlazeMeterUploader capturehar: class: bzt.modules.blazemeter.ServiceStubCaptureHAR chrome-metric-reporter: class: bzt.modules.chrome.MetricReporter chrome-profiler: class: bzt.modules.chrome.ChromeProfiler processors: trace: class: bzt.modules.chrome.TraceProcessor extractors: - bzt.modules.chrome.TabNameExtractor - bzt.modules.chrome.MemoryMetricsExtractor cloud: check-interval: 5s class: bzt.modules.blazemeter.CloudProvisioning default-location: us-central1-a use-deprecated-api: true console: class: bzt.modules.console.ConsoleStatusReporter disable: false consolidator: buffer-multiplier: 2 buffer-scale-choice: 0.5 class: bzt.modules.aggregator.ConsolidatingAggregator generalize-labels: false ignore-labels: - ignore max-buffer-len: .inf min-buffer-len: 2 percentiles: - 0.0 - 50.0 - 90.0 - 95.0 - 99.0 - 99.9 - 100.0 final_stats: class: bzt.modules.reporting.FinalStatus functional-consolidator: class: bzt.modules.functional.FunctionalAggregator gatling: class: bzt.modules.gatling.GatlingExecutor properties: gatling.data.file.bufferSize: 256 grinder: class: bzt.modules.grinder.GrinderExecutor path: ~/.bzt/grinder-taurus/lib/grinder.jar properties: grinder.useConsole: false install-checker: class: bzt.modules.services.InstallChecker jmeter: class: bzt.modules.jmeter.JMeterExecutor gui: false path: ~/.bzt/jmeter-taurus/bin/jmeter plugins: - jpgc-casutg - jpgc-dummy - jpgc-ffw - jpgc-fifo - jpgc-functions - jpgc-json - jpgc-perfmon - jpgc-prmctl - jpgc-tst properties: jmeter.save.saveservice.autoflush: 'true' jmeter.save.saveservice.connect_time: 'true' jmeterengine.force.system.exit: 'true' log_level: DEBUG mode: Stripped summariser.name: '' user.classpath: additional-jars: null shutdown-port: 4445 version: 3.1 junit-xml: class: bzt.modules.reporting.JUnitXMLReporter local: class: bzt.modules.provisioning.Local locust: class: bzt.modules.locustio.LocustIOExecutor monitoring: class: bzt.modules.monitoring.Monitoring passfail: class: bzt.modules.passfail.PassFailStatus pbench: class: bzt.modules.pbench.PBenchExecutor proxy2jmx: class: bzt.modules.proxy2jmx.Proxy2JMX screenshoter: class: bzt.modules.blazemeter.ServiceStubScreenshoter selenium: class: bzt.modules.selenium.SeleniumExecutor selenium-tools: junit: hamcrest-core: ~/.bzt/selenium-taurus/tools/junit/hamcrest-core.jar jar-name: compiled.jar json-jar: ~/.bzt/selenium-taurus/tools/junit/json.jar path: ~/.bzt/selenium-taurus/tools/junit/junit.jar selenium-server: ~/.bzt/selenium-taurus/selenium-server.jar working-dir: classes mocha: tools-dir: ~/.bzt/selenium-taurus/mocha nose: working-dir: classes rspec: interpreter: ruby testng: hamcrest-core: ~/.bzt/selenium-taurus/tools/testng/hamcrest-core.jar jar-name: compiled.jar json-jar: ~/.bzt/selenium-taurus/tools/testng/json.jar path: ~/.bzt/selenium-taurus/tools/testng/testng.jar selenium-server: ~/.bzt/selenium-taurus/selenium-server.jar working-dir: classes shellexec: class: bzt.modules.shellexec.ShellExecutor siege: class: bzt.modules.siege.SiegeExecutor tsung: class: bzt.modules.tsung.TsungExecutor unpacker: class: bzt.modules.services.Unpackerprovisioning: localreporting:- final_stats- console- module: final_stats- module: consolescenarios: stress-hive: keepalive: true script: stress-hive.jmx timeout: 500msservices:- local: - metrics: - cpu - mem - bytes-sent - bytes-recv - disk-read - disk-write - disk-space - engine-loop module: monitoring run-at: localsettings: aggregator: consolidator artifacts-dir: '%Y-%m-%d_%H-%M-%S.%f' check-interval: 5s check-updates: true default-executor: jmeter proxy: {}version: 1.7.4Secondly, the value of modules.jmeter.properties.user.classpath doesn't seem to be replaced either. In the affective config it look like:
additional-jars: /tmp/hive-test-suite/lib/
...
modules
jmeter:
properties: user.classpath: additional-jars: nullMy questions are how this substitution works and how I can pass values for this variables in the command line?
I appreciate your help with this issue.
Thanks,
Daniel
ta...@blazemeter.com
Dec 21, 2016, 3:59:50 AM12/21/16
to codename-taurus, dade...@gmail.com
Hi, Daniel.
It looks like documentation mistake. Of course, variable substitution requires ${...} form - you have to add '$' before the left bracket.
---
Taras
Reply all
Reply to author
Forward
0 new messages