Rabbitmq Shovel plugin continuously logs same error on failed start

[Sasa Trifunovic]

I'm trying to start Shovel plugin on source host with command:

 

sudo rabbitmqctl set_parameter shovel my-shovel '{"src-uri": "amqp://someusername:somepassword@localhost:5671/%2fcolor%2fblue", "src-queue": "test_shovel", "dest-uri": "amqp://someusername:somepassword@secondhost:5671/%2fcolor%2fblue", "dest-queue": "test_shovel"}'

On stdout i get:

Setting runtime parameter "my-shovel" for component "shovel" to "{\"src-uri\": \"amqp://someusername:somepassword@localhost:5671/%2fcolor%2fblue\", \"src-queue\": \"test_shovel\", \"dest-uri\": \"amqp://someusername:somepassword@secondhost:5671/%2fcolor%2fblue\", \"dest-queue\": \"test_shovel\"}"

But when i open rabbitmq log i'm repeatedly getting:

 

 

   =ERROR REPORT==== 28-Jul-2017::16:27:46 ===



    ** Generic server <0.15680.1> terminating

    ** Last message in was {'$gen_cast',init}

    ** When Server state == {state,undefined,undefined,undefined,undefined,

                                {<<"/">>,<<"my-shovel">>},

                                dynamic,

                                {shovel,

                                    {endpoint,

                                        ["amqp://someusername:somepassword@localhost:5671/%2fcolor%2fblue"],

                                        #Fun<rabbit_shovel_parameters.4.22841904>},

                                    {endpoint,

                                        ["amqp://someusername:somepassword@secondhost:5671/%2fcolor%2fblue"],

                                        #Fun<rabbit_shovel_parameters.5.22841904>},

                                    1000,on_confirm,

                                    #Fun<rabbit_shovel_parameters.6.22841904>,

                                    #Fun<rabbit_shovel_parameters.7.22841904>,

                                    <<"test_shovel">>,1,never},

                                undefined,undefined,undefined,undefined,undefined}

    ** Reason for termination

    ** {{badmatch,{error,{socket_closed_unexpectedly,'connection.start'}}},

        [{rabbit_shovel_worker,make_conn_and_chan,2,

                               [{file,"src/rabbit_shovel_worker.erl"},{line,238}]},

         {rabbit_shovel_worker,handle_cast,2,

                               [{file,"src/rabbit_shovel_worker.erl"},{line,63}]},

         {gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,1048}]},

         {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}

Like it's stuck in some kind of loop. Only thing that stops logs from filling up is to stop shovel plugin with :

rabbitmqctl eval 'application:stop(rabbitmq_shovel).'

When i try to query for status with:

rabbitmqctl eval 'rabbit_shovel_status:status().'

I get:

[{{<<"/">>,<<"my-shovel">>},
  dynamic,
  {terminated,
      {{badmatch,{error,{socket_closed_unexpectedly,'connection.start'}}},
       [{rabbit_shovel_worker,make_conn_and_chan,2,
            [{file,"src/rabbit_shovel_worker.erl"},{line,238}]},
        {rabbit_shovel_worker,handle_cast,2,
            [{file,"src/rabbit_shovel_worker.erl"},{line,63}]},
        {gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,1048}]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}},
  {{2017,7,30},{16,24,47}}}]

 

rabbitmqctl status returns:

[{pid,3498},
 {running_applications,
     [{rabbitmq_shovel,"Data Shovel for RabbitMQ","3.6.10"},
      {rabbitmq_management,"RabbitMQ Management Console","3.6.10"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.6.10"},
      {cowboy,"Small, fast, modular HTTP server.","1.0.4"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.6.10"},
      {rabbit,"RabbitMQ","3.6.10"},
      {ranch,"Socket acceptor pool for TCP protocols.","1.3.0"},
      {ssl,"Erlang/OTP SSL application","8.2"},
      {public_key,"Public key infrastructure","1.4.1"},
      {asn1,"The Erlang ASN1 compiler version 5.0","5.0"},
      {cowlib,"Support library for manipulating Web protocols.","1.0.2"},
      {amqp_client,"RabbitMQ AMQP Client","3.6.10"},
      {rabbit_common,
          "Modules shared by rabbitmq-server and rabbitmq-erlang-client",
          "3.6.10"},
      {syntax_tools,"Syntax tools","2.1.2"},
      {xmerl,"XML parser","1.3.15"},
      {crypto,"CRYPTO","4.0"},
      {compiler,"ERTS  CXC 138 10","7.1"},
      {os_mon,"CPO  CXC 138 46","2.4.2"},
      {inets,"INETS  CXC 138 49","6.4"},
      {mnesia,"MNESIA  CXC 138 12","4.15"},
      {sasl,"SASL  CXC 138 11","3.0.4"},
      {stdlib,"ERTS  CXC 138 10","3.4"},
      {kernel,"ERTS  CXC 138 10","5.3"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:64] [kernel-poll:true]\n"},
 {memory,
     [{total,185931872},
      {connection_readers,1963648},
      {connection_writers,74176},
      {connection_channels,275920},
      {connection_other,3452664},
      {queue_procs,1295264},
      {queue_slave_procs,0},
      {plugins,3362888},
      {other_proc,23285800},
      {mnesia,333792},
      {metrics,562480},
      {mgmt_db,1803232},
      {msg_index,102400},
      {other_ets,2746248},
      {binary,111017552},
      {code,24762394},
      {atom,1041593},
      {other_system,10411493}]},
 {alarms,[]},
 {listeners,
     [{clustering,25672,"::"},{'amqp/ssl',5671,"::"},{https,15672,"::"}]},
 {vm_memory_high_watermark,{absolute,"2048MiB"}},
 {vm_memory_limit,2147483648},
 {disk_free_limit,50000000},
 {disk_free,25670946816},
 {file_descriptors,
     [{total_limit,924},
      {total_used,90},
      {sockets_limit,829},
      {sockets_used,43}]},
 {processes,[{limit,1048576},{used,1004}]},
 {run_queue,0},
 {uptime,5432},
 {kernel,{net_ticktime,60}}]

I'm not sure what is wrong with the way i'm trying to start shovelling and why logs are being filled with the same error on first failed attempt. 

Any insight on this problem would be greatly appreciated! 

P.S:

I originally tried sending HTTP request to broker:

 curl -i -H "content-type:application/json" -X PUT -d '{"value":{"src-uri":  "amqp://someusername:somepassword@firsthost:5671/%2fcolor%2fblue", "src-queue":  "test_shovel", "dest-uri": "amqp://someusername:somepassword@secondhost:5671/%2fcolor%2fblue", "dest-queue": "test_shovel"}}' https//firsthost:15672/api/parameters/shovel/%2f/my-shovel

But 401 was the only response i was getting, no matter what privileges "someusername" user had.

Thanks,

Sasha

[Michael Klishin]

It tries to connect and fails with "socket_closed_unexpectedly". Something, e.g. a proxy or load balancer,
closes TCP connection prematurely.

See http://www.rabbitmq.com/heartbeats.html as it is often highly relevant.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Sasa Trifunovic]

Problem was in src-uri and dest-uri. They contained port which was using SSL,  so proper src-uri should be  "amqps://someusername:somepassword@localhost:5671/%2fcolor%2fblue" and dest-uri "amqp://someusername:somepassword@secondhost:5671/%2fcolor%2fblue". 

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.