Chronos job does not run, stays in queued state (in chronos 2.5)
129 views
Skip to first unread message
siya
Oct 12, 2017, 1:40:04 PM10/12/17
to chronos-scheduler
I have set up chronos 2.5 and while testing by creating a dummy job, the chronos task does not run at all. Please help me troubleshoot it.
the mesos logs does not show any error. When i have docker image of chronos 3 it works.
Is there something wrong on my settings ?
My chronos settings are:
Chronos JOB:
chronos logs:
the mesos logs does not show any error. When i have docker image of chronos 3 it works.
Is there something wrong on my settings ?
My chronos settings are:
{
"id": "/chronos",
"cmd": "/usr/bin/chronos run_jar --http_port 8082 --zk_hosts 10.150.4.57:2181,10.150.4.90:2181,10.150.4.188:2181 --master zk://10.150.4.57:2181,10.150.4.90:2181,10.150.4.188:2181/mesos --mail_from x...@xx.com --mail_server xx.xx.com:25 --zk_path /chronos/state",
"cpus": 0.5,
"mem": 512,
"disk": 0,
"instances": 1,
"acceptedResourceRoles": [
"*"
],
"container": {
"type": "DOCKER",
"volumes": [],
"docker": {
"image": "mesosphere/chronos:chronos-2.5.0-0.1.20170628182950.ubuntu1404-mesos-1.3.0",
"network": "HOST",
"portMappings": [],
"privileged": false,
"parameters": [],
"forcePullImage": false
}
},
"env": {
"PORT0": "8082",
"PORT1": "8079"
},
"portDefinitions": [
{
"port": 10031,
"protocol": "tcp",
"labels": {}
}
]
}
Chronos JOB:
---
- hosts: localhost
connection: local
gather_facts: no
roles:
- role: chronos_task
chronos_url: 'http://10.150.4.132:8082'
chronos_tasks:
- type: iso8601
app:
name: test-1
onwer:
command: "ls"
schedule: "R/2017-10-13T11:00Z/PT24H"
scheduleTimeZone: "America/new_york"
epsilon: "PT05M"
async: "false"
~chronos logs:
I1011 23:27:36.693475 2568 exec.cpp:162] Version: 1.3.0 I1011 23:27:36.697681 2570 exec.cpp:237] Executor registered on agent 92821a94-2aaf-4ba1-9ec3-4dcd5973155a-S0 /usr/bin/chronos: line 17: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory 2017-10-11 23:27:39,481:7(0x7f1bc9383700):ZOO_INFO@log_env@726: Client environment:zookeeper.version=zookeeper C client 3.4.8 2017-10-11 23:27:39,481:7(0x7f1bc9383700):ZOO_INFO@log_env@730: Client environment:host.name=ip-10-150-4-132 2017-10-11 23:27:39,481:7(0x7f1bc9383700):ZOO_INFO@log_env@737: Client environment:os.name=Linux 2017-10-11 23:27:39,481:7(0x7f1bc9383700):ZOO_INFO@log_env@738: Client environment:os.arch=3.10.0-514.26.2.el7.x86_64 2017-10-11 23:27:39,481:7(0x7f1bc9383700):ZOO_INFO@log_env@739: Client environment:os.version=#1 SMP Tue Jul 4 15:04:05 UTC 2017 2017-10-11 23:27:39,481:7(0x7f1bc9383700):ZOO_INFO@log_env@747: Client environment:user.name=(null) 2017-10-11 23:27:39,482:7(0x7f1bc9383700):ZOO_INFO@log_env@755: Client environment:user.home=/root 2017-10-11 23:27:39,482:7(0x7f1bc9383700):ZOO_INFO@log_env@767: Client environment:user.dir=/ 2017-10-11 23:27:39,482:7(0x7f1bc9383700):ZOO_INFO@zookeeper_init@800: Initiating client connection, host=10.150.4.57:2181,10.150.4.90:2181,10.150.4.188:2181 sessionTimeout=10000 watcher=0x7f1bfdcedc90 sessionId=0 sessionPasswd=<null> context=0x7f1bb0001eb0 flags=0 2017-10-11 23:27:39,484:7(0x7f1bc637d700):ZOO_INFO@check_events@1728: initiated connection to server [10.150.4.57:2181] 2017-10-11 23:27:39,488:7(0x7f1bc637d700):ZOO_INFO@check_events@1775: session establishment complete on server [10.150.4.57:2181], sessionId=0x15da95fbe8e00eb, negotiated timeout=10000 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@726: Client environment:zookeeper.version=zookeeper C client 3.4.8 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@730: Client environment:host.name=ip-10-150-4-132 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@737: Client environment:os.name=Linux 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@738: Client environment:os.arch=3.10.0-514.26.2.el7.x86_64 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@739: Client environment:os.version=#1 SMP Tue Jul 4 15:04:05 UTC 2017 I1011 23:28:24.053308 68 sched.cpp:232] Version: 1.3.0 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@747: Client environment:user.name=(null) 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@755: Client environment:user.home=/root 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@log_env@767: Client environment:user.dir=/ 2017-10-11 23:28:24,053:7(0x7f1bc737f700):ZOO_INFO@zookeeper_init@800: Initiating client connection, host=10.150.4.57:2181,10.150.4.90:2181,10.150.4.188:2181 sessionTimeout=10000 watcher=0x7f1bfdcedc90 sessionId=0 sessionPasswd=<null> context=0x7f1bb0003cb0 flags=0 2017-10-11 23:28:24,054:7(0x7f1b83dfd700):ZOO_INFO@check_events@1728: initiated connection to server [10.150.4.188:2181] 2017-10-11 23:28:24,059:7(0x7f1b83dfd700):ZOO_INFO@check_events@1775: session establishment complete on server [10.150.4.188:2181], sessionId=0x35cf15997360124, negotiated timeout=10000 I1011 23:28:24.059999 36 group.cpp:340] Group process (zookeeper-group(1)@10.150.4.132:34617) connected to ZooKeeper I1011 23:28:24.060040 36 group.cpp:830] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0) I1011 23:28:24.060083 36 group.cpp:418] Trying to create path '/mesos' in ZooKeeper I1011 23:28:24.061589 36 detector.cpp:152] Detected a new leader: (id='16') I1011 23:28:24.061717 36 group.cpp:699] Trying to get '/mesos/json.info_0000000016' in ZooKeeper I1011 23:28:24.062949 33 zookeeper.cpp:262] A new leading master (UPID=mas...@10.150.4.57:5050) is detected I1011 23:28:24.063050 33 sched.cpp:336] New master detected at mas...@10.150.4.57:5050 I1011 23:28:24.063235 33 sched.cpp:352] No credentials provided. Attempting to register without authentication 2017-10-12 07:30:25,174:7(0x7f1bc637d700):ZOO_WARN@zookeeper_interest@1570: Exceeded deadline by 18ms 2017-10-12 09:46:30,697:7(0x7f1bc637d700):ZOO_WARN@zookeeper_interest@1570: Exceeded deadline by 38ms 2017-10-12 14:43:25,336:7(0x7f1b83dfd700):ZOO_WARN@zookeeper_interest@1570: Exceeded deadline by 14ms 2017-10-12 14:48:33,762:7(0x7f1bc637d700):ZOO_WARN@zookeeper_interest@1570: Exceeded deadline by 11msThe mesos logs:
[2017-10-12 17:24:25,047] INFO Calling next for stream: R/2017-10-13T11:00Z/PT24H, jobname: test-1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:509) [2017-10-12 17:24:26,651] INFO 10.83.125.144 - - [12/Oct/2017:17:24:26 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:26,654] INFO 10.83.125.144 - - [12/Oct/2017:17:24:26 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 23 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:34,630] INFO 10.83.125.144 - - [12/Oct/2017:17:24:34 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:34,631] INFO 10.83.125.144 - - [12/Oct/2017:17:24:34 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 23 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:39,977] INFO Manually triggering job:test-1 (org.apache.mesos.chronos.scheduler.api.JobManagementResource:153) [2017-10-12 17:24:39,977] INFO JobNotificationObserver does not handle JobQueued(ScheduleBasedJob(R/2017-10-13T11:00Z/PT24H,test-1,ls,PT5M,0,0,,,,2,,,,,,false,0.1,256.0,128.0,false,0,ListBuffer(),ListBuffer(),false,root,null,America/new_york,ListBuffer(),true,ListBuffer(),false,false,ListBuffer()),ct:1507829079977:0:test-1:,0) (org.apache.mesos.chronos.scheduler.jobs.JobsObserver$:27) [2017-10-12 17:24:39,978] INFO Updating state for job (test-1) to queued (org.apache.mesos.chronos.scheduler.jobs.stats.JobStats:64) [2017-10-12 17:24:39,978] INFO 10.83.125.144 - - [12/Oct/2017:17:24:39 +0000] "PUT /scheduler/job/test-1 HTTP/1.1" 204 0 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:43,771] INFO 10.83.125.144 - - [12/Oct/2017:17:24:43 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:43,771] INFO 10.83.125.144 - - [12/Oct/2017:17:24:43 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:51,766] INFO 10.83.125.144 - - [12/Oct/2017:17:24:51 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:51,770] INFO 10.83.125.144 - - [12/Oct/2017:17:24:51 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:59,766] INFO 10.83.125.144 - - [12/Oct/2017:17:24:59 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:24:59,773] INFO 10.83.125.144 - - [12/Oct/2017:17:24:59 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:07,791] INFO 10.83.125.144 - - [12/Oct/2017:17:25:07 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:07,792] INFO 10.83.125.144 - - [12/Oct/2017:17:25:07 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:15,777] INFO 10.83.125.144 - - [12/Oct/2017:17:25:15 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:15,781] INFO 10.83.125.144 - - [12/Oct/2017:17:25:15 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:23,777] INFO 10.83.125.144 - - [12/Oct/2017:17:25:23 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:23,777] INFO 10.83.125.144 - - [12/Oct/2017:17:25:23 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:25,049] INFO Size of streams: 1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:488) [2017-10-12 17:25:25,049] INFO Checking schedules with time horizon:PT60S (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:480) [2017-10-12 17:25:25,049] INFO Calling next for stream: R/2017-10-13T11:00Z/PT24H, jobname: test-1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:509) [2017-10-12 17:25:31,778] INFO 10.83.125.144 - - [12/Oct/2017:17:25:31 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:31,780] INFO 10.83.125.144 - - [12/Oct/2017:17:25:31 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:39,783] INFO 10.83.125.144 - - [12/Oct/2017:17:25:39 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:39,792] INFO 10.83.125.144 - - [12/Oct/2017:17:25:39 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:47,793] INFO 10.83.125.144 - - [12/Oct/2017:17:25:47 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:47,794] INFO 10.83.125.144 - - [12/Oct/2017:17:25:47 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:55,772] INFO 10.83.125.144 - - [12/Oct/2017:17:25:55 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:25:55,781] INFO 10.83.125.144 - - [12/Oct/2017:17:25:55 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:03,764] INFO 10.83.125.144 - - [12/Oct/2017:17:26:03 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:03,775] INFO 10.83.125.144 - - [12/Oct/2017:17:26:03 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:11,781] INFO 10.83.125.144 - - [12/Oct/2017:17:26:11 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:11,786] INFO 10.83.125.144 - - [12/Oct/2017:17:26:11 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:19,818] INFO 10.83.125.144 - - [12/Oct/2017:17:26:19 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:19,818] INFO 10.83.125.144 - - [12/Oct/2017:17:26:19 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:25,050] INFO Size of streams: 1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:488) [2017-10-12 17:26:25,051] INFO Checking schedules with time horizon:PT60S (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:480) [2017-10-12 17:26:25,051] INFO Calling next for stream: R/2017-10-13T11:00Z/PT24H, jobname: test-1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:509) [2017-10-12 17:26:27,821] INFO 10.83.125.144 - - [12/Oct/2017:17:26:27 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:27,821] INFO 10.83.125.144 - - [12/Oct/2017:17:26:27 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:35,811] INFO 10.83.125.144 - - [12/Oct/2017:17:26:35 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:35,813] INFO 10.83.125.144 - - [12/Oct/2017:17:26:35 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:43,758] INFO 10.83.125.144 - - [12/Oct/2017:17:26:43 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:43,815] INFO 10.83.125.144 - - [12/Oct/2017:17:26:43 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:51,801] INFO 10.83.125.144 - - [12/Oct/2017:17:26:51 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:51,801] INFO 10.83.125.144 - - [12/Oct/2017:17:26:51 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:59,761] INFO 10.83.125.144 - - [12/Oct/2017:17:26:59 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:26:59,815] INFO 10.83.125.144 - - [12/Oct/2017:17:26:59 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:07,792] INFO 10.83.125.144 - - [12/Oct/2017:17:27:07 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:07,798] INFO 10.83.125.144 - - [12/Oct/2017:17:27:07 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:15,794] INFO 10.83.125.144 - - [12/Oct/2017:17:27:15 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:15,798] INFO 10.83.125.144 - - [12/Oct/2017:17:27:15 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:23,801] INFO 10.83.125.144 - - [12/Oct/2017:17:27:23 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:23,804] INFO 10.83.125.144 - - [12/Oct/2017:17:27:23 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:25,052] INFO Size of streams: 1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:488) [2017-10-12 17:27:25,053] INFO Checking schedules with time horizon:PT60S (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:480) [2017-10-12 17:27:25,053] INFO Calling next for stream: R/2017-10-13T11:00Z/PT24H, jobname: test-1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:509) [2017-10-12 17:27:31,804] INFO 10.83.125.144 - - [12/Oct/2017:17:27:31 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:31,804] INFO 10.83.125.144 - - [12/Oct/2017:17:27:31 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:39,795] INFO 10.83.125.144 - - [12/Oct/2017:17:27:39 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:39,805] INFO 10.83.125.144 - - [12/Oct/2017:17:27:39 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:47,791] INFO 10.83.125.144 - - [12/Oct/2017:17:27:47 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:47,800] INFO 10.83.125.144 - - [12/Oct/2017:17:27:47 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:55,812] INFO 10.83.125.144 - - [12/Oct/2017:17:27:55 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:27:55,812] INFO 10.83.125.144 - - [12/Oct/2017:17:27:55 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:03,816] INFO 10.83.125.144 - - [12/Oct/2017:17:28:03 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:03,818] INFO 10.83.125.144 - - [12/Oct/2017:17:28:03 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:11,787] INFO 10.83.125.144 - - [12/Oct/2017:17:28:11 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:11,799] INFO 10.83.125.144 - - [12/Oct/2017:17:28:11 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:19,807] INFO 10.83.125.144 - - [12/Oct/2017:17:28:19 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:19,811] INFO 10.83.125.144 - - [12/Oct/2017:17:28:19 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:25,054] INFO Size of streams: 1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:488) [2017-10-12 17:28:25,054] INFO Checking schedules with time horizon:PT60S (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:480) [2017-10-12 17:28:25,054] INFO Calling next for stream: R/2017-10-13T11:00Z/PT24H, jobname: test-1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:509) [2017-10-12 17:28:27,812] INFO 10.83.125.144 - - [12/Oct/2017:17:28:27 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:27,816] INFO 10.83.125.144 - - [12/Oct/2017:17:28:27 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:35,809] INFO 10.83.125.144 - - [12/Oct/2017:17:28:35 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:35,833] INFO 10.83.125.144 - - [12/Oct/2017:17:28:35 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:43,821] INFO 10.83.125.144 - - [12/Oct/2017:17:28:43 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:43,825] INFO 10.83.125.144 - - [12/Oct/2017:17:28:43 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:51,786] INFO 10.83.125.144 - - [12/Oct/2017:17:28:51 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:51,836] INFO 10.83.125.144 - - [12/Oct/2017:17:28:51 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:59,821] INFO 10.83.125.144 - - [12/Oct/2017:17:28:59 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:28:59,825] INFO 10.83.125.144 - - [12/Oct/2017:17:28:59 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:07,791] INFO 10.83.125.144 - - [12/Oct/2017:17:29:07 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:07,843] INFO 10.83.125.144 - - [12/Oct/2017:17:29:07 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:15,798] INFO 10.83.125.144 - - [12/Oct/2017:17:29:15 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:15,836] INFO 10.83.125.144 - - [12/Oct/2017:17:29:15 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:23,842] INFO 10.83.125.144 - - [12/Oct/2017:17:29:23 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:23,842] INFO 10.83.125.144 - - [12/Oct/2017:17:29:23 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:25,056] INFO Size of streams: 1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:488) [2017-10-12 17:29:25,056] INFO Checking schedules with time horizon:PT60S (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:480) [2017-10-12 17:29:25,056] INFO Calling next for stream: R/2017-10-13T11:00Z/PT24H, jobname: test-1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:509) [2017-10-12 17:29:32,692] INFO 10.83.125.144 - - [12/Oct/2017:17:29:32 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:32,692] INFO 10.83.125.144 - - [12/Oct/2017:17:29:32 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:40,694] INFO 10.83.125.144 - - [12/Oct/2017:17:29:40 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:40,695] INFO 10.83.125.144 - - [12/Oct/2017:17:29:40 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:48,661] INFO 10.83.125.144 - - [12/Oct/2017:17:29:48 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:48,699] INFO 10.83.125.144 - - [12/Oct/2017:17:29:48 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:56,692] INFO 10.83.125.144 - - [12/Oct/2017:17:29:56 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:29:56,694] INFO 10.83.125.144 - - [12/Oct/2017:17:29:56 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:04,689] INFO 10.83.125.144 - - [12/Oct/2017:17:30:04 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:04,692] INFO 10.83.125.144 - - [12/Oct/2017:17:30:04 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:12,688] INFO 10.83.125.144 - - [12/Oct/2017:17:30:12 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:12,696] INFO 10.83.125.144 - - [12/Oct/2017:17:30:12 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:21,687] INFO 10.83.125.144 - - [12/Oct/2017:17:30:21 +0000] "GET /scheduler/jobs HTTP/1.1" 200 572 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:21,694] INFO 10.83.125.144 - - [12/Oct/2017:17:30:21 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 25 "http://10.150.4.132:8082/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog:15) [2017-10-12 17:30:25,057] INFO Size of streams: 1 (org.apache.mesos.chronos.scheduler.jobs.JobScheduler:488)
Craig Wickesser
Apr 19, 2018, 7:15:31 AM4/19/18
to chronos-scheduler
Did you ever figure out what was going on? Seeing similar behavior with 3.0.0 on one cluster but not another.
Judith Malnick
Apr 20, 2018, 6:17:23 PM4/20/18
to Craig Wickesser, Gastón Kleiman, chronos-scheduler
+Gastón in case this fell off his radar :)
--
You received this message because you are subscribed to the Google Groups "chronos-scheduler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chronos-scheduler+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Craig Wickesser
Apr 23, 2018, 9:08:41 AM4/23/18
to chronos-scheduler
One thing I noticed on this new cluster is each host has gpus...whereas the other cluster, where chronos works does not. I just modified the "resources" for a mesos-agent to explicitly exclude gpus:
--resources gpus(*):0
...when I went to chronos to kick off a job, it ran! I'm not sure why chronos won't receive offers from servers that have GPUs yet.
--resources gpus(*):0
...when I went to chronos to kick off a job, it ran! I'm not sure why chronos won't receive offers from servers that have GPUs yet.
To unsubscribe from this group and stop receiving emails from it, send an email to chronos-schedu...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Craig Wickesser
Apr 23, 2018, 9:48:55 AM4/23/18
to chronos-scheduler
Turns out Mesos wont send offers for agents that have GPUs if a framework doesn't have the "GPU" capability. A ticket, with pull request fixes this in Chronos but hasn't been merged or discussed in over a year :(
https://github.com/mesos/chronos/issues/764
https://github.com/mesos/chronos/issues/764
Craig Wickesser
Apr 23, 2018, 12:35:28 PM4/23/18
to chronos-scheduler
But it also turns out mesos has a flag to get around this, look for filter_gpu_resources, if you set it to "--no-filter_gpu_resources" on the mesos masters chronos will work with offers that include GPUs (though Chronos can't use the gpu resources): http://mesos.apache.org/documentation/latest/configuration/master/
Reply all
Reply to author
Forward
0 new messages