CP 2.0.0 won't install via yum

[Jake Robb]

Hi all,

I'm new to this list, new to Confluent/Kafka, and new to yum, so I apologize if I'm asking total n00b questions here. :)

I've been getting things up and running with Kafka for a couple days now. I had a working three-node test installation of CP 1.0.1, using the bundled command-line clients to confirm that everything was working. Since CP 2.0 came out today and we expect to need SSL, I wanted to upgrade right away. My servers run RHEL, so I used the yum installation instructions.

Here's what happens when I follow the instructions:

[root@kafka01 ~]# rpm --import http://packages.confluent.io/rpm/2.0/archive.key

[root@kafka01 ~]# vi /etc/yum.repos.d/confluent.repo

[root@kafka01 ~]# yum install confluent-platform-2.11.7

confluent-1.0                                                                         | 2.9 kB     00:00     

confluent-2.0                                                                         | 2.9 kB     00:00     

confluent-2.0/primary_db                                                              |  11 kB     00:00     

epel/metalink                                                                         |  12 kB     00:00     

epel                                                                                  | 4.3 kB     00:00     

epel/primary_db                                                                       | 5.8 MB     00:01     

public_ol6_UEKR3_latest                                                               | 1.2 kB     00:00     

public_ol6_latest                                                                     | 1.4 kB     00:00     

public_ol6_latest/primary                                                             |  53 MB     00:06     

public_ol6_latest                                                                                32672/32672

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package confluent-platform-2.11.7.noarch 0:2.0.0-1 will be installed

--> Processing Dependency: librdkafka1 >= 0.9.0_confluent2.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Processing Dependency: confluent-schema-registry >= 2.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Processing Dependency: confluent-kafka-rest >= 2.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Processing Dependency: confluent-kafka-connect-jdbc >= 2.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Processing Dependency: confluent-kafka-connect-hdfs >= 2.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Processing Dependency: confluent-kafka-2.11.7 >= 0.9.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Processing Dependency: confluent-camus >= 2.0.0 for package: confluent-platform-2.11.7-2.0.0-1.noarch

--> Running transaction check

---> Package confluent-camus.noarch 0:1.0.1-2 will be updated

---> Package confluent-camus.noarch 0:2.0.0-1 will be an update

---> Package confluent-kafka-2.11.7.noarch 0:0.9.0.0-1 will be installed

---> Package confluent-kafka-connect-hdfs.noarch 0:2.0.0-1 will be installed

---> Package confluent-kafka-connect-jdbc.noarch 0:2.0.0-1 will be installed

---> Package confluent-kafka-rest.noarch 0:1.0.1-2 will be updated

---> Package confluent-kafka-rest.noarch 0:2.0.0-1 will be an update

---> Package confluent-schema-registry.noarch 0:1.0.1-2 will be updated

---> Package confluent-schema-registry.noarch 0:2.0.0-1 will be an update

---> Package librdkafka1.x86_64 0:0.9.0_confluent2.0.0-1.fc20 will be installed

--> Processing Dependency: openssl-libs for package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64

--> Processing Dependency: libstdc++.so.6(GLIBCXX_3.4.15)(64bit) for package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64

--> Processing Dependency: libc.so.6(GLIBC_2.14)(64bit) for package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64

--> Processing Dependency: libsasl2.so.3()(64bit) for package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64

--> Finished Dependency Resolution

Error: Package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64 (confluent-2.0)

           Requires: openssl-libs

Error: Package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64 (confluent-2.0)

           Requires: libsasl2.so.3()(64bit)

Error: Package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64 (confluent-2.0)

           Requires: libstdc++.so.6(GLIBCXX_3.4.15)(64bit)

Error: Package: librdkafka1-0.9.0_confluent2.0.0-1.fc20.x86_64 (confluent-2.0)

           Requires: libc.so.6(GLIBC_2.14)(64bit)

 You could try using --skip-broken to work around the problem

 You could try running: rpm -Va --nofiles --nodigest

[root@kafka01 ~]# 

As I said, I'm new to yum, so the "you could try" stuff at the end there is mysterious and I'm not confident about what difference those arguments would make, but it sure seems like the best I could hope for would be a healthy install of CP2 but without the SSL support I need. My best guess is that Confluent's yum repo has a few invalid dependency references. 

Can anyone offer any advice?

Thanks!

Jake

[Michael Noll]

Jake,

thanks for the report.  I'm sorry to hear you've run into this issue while giving the new release a spin.

The commands you used (quickstart for rpm/yum) work fine on e.g. Fedora 20 and CentOS 7.  Which version of RHEL are you running exactly?

Best,
Michael

[Ismael Juma]

Hi Jake,

One other thing (maybe unrelated, but good to try to be sure), the upgrade instructions recommend removing 1.0 before installing 2.0:

http://docs.confluent.io/2.0.0/upgrade.html#rpm-packages-via-yum

It looks like you did not do this based on:

---> Package confluent-camus.noarch 0:1.0.1-2 will be updated

---> Package confluent-camus.noarch 0:2.0.0-1 will be an update

---> Package confluent-kafka-2.11.7.noarch 0:0.9.0.0-1 will be installed

---> Package confluent-kafka-connect-hdfs.noarch 0:2.0.0-1 will be installed

---> Package confluent-kafka-connect-jdbc.noarch 0:2.0.0-1 will be installed

---> Package confluent-kafka-rest.noarch 0:1.0.1-2 will be updated

---> Package confluent-kafka-rest.noarch 0:2.0.0-1 will be an update

---> Package confluent-schema-registry.noarch 0:1.0.1-2 will be updated

---> Package confluent-schema-registry.noarch 0:2.0.0-1 will be an update

Best,

Ismael

[Jake Robb]

We're running Oracle Enterprise Linux 6.5 with the Red Hat Enterprise Linux 6.5 Kernel.

[Jake Robb]

Good point. I'll try those.

I might have found them on my own if there was a link from either the blog post or the installation instructions. :D

-Jake

[Ismael Juma]

On Tuesday, 8 December 2015 16:43:38 UTC, Jake Robb wrote:

Good point. I'll try those.

Thanks Jake. It seems unlikely that this will help (it looks like librdkafka depends on one or more packages that are not available in RHEL 6.x).

In the meantime, do you intend to use librdkafka? If not, an option may be to install the packages you care about individually (after removing CP 1.0).

Best,

Ismael

[Jake Robb]

Nope, we don't need any C support. We'll be using the Java and possibly REST interfaces.

[Jake Robb]

Just to confirm: I got the same errors at the end of the process after removing 1.0 first and trying again.

Reading more closely over the output, I think I see that openssl-libs is only a dependency of librdkafka (the C client), which makes me think that other clients might still be able to use SSL even if I skip that part. True?

Jake

[Ismael Juma]

On Tuesday, 8 December 2015 17:12:09 UTC, Jake Robb wrote:

Just to confirm: I got the same errors at the end of the process after removing 1.0 first and trying again.

Thanks for confirming.

Reading more closely over the output, I think I see that openssl-libs is only a dependency of librdkafka (the C client), which makes me think that other clients might still be able to use SSL even if I skip that part. True?

That's right. The Java client relies on the JDK's SSL implementation at this point.

Ismael

[Jon Guerin]

I received the same error on CentOS 6.6.

However, I was able to yum install the individual components I wanted . . .

% sudo rpm --import http://packages.confluent.io/rpm/2.0/archive.key

% sudo yum install confluent-kafka-2.11.7

% sudo yum install confluent-kafka-rest

% sudo yum install confluent-schema-registry

[Michael Noll]

FYI:  Today we are pushing an update to the CP 2.0.0 documentation that adds a note for RHEL 6.x based distros as well as the current workaround -- which is, as Jon Guerin said earlier in this discussion, to install the components individually and, if needed, to install librdkafka from the tar or zip archives.

Sorry for this inconvenience!

-Michael

[Ewen Cheslack-Postava]

Frizzi,

It looks like the hostname that is automatically looked up (KafkaNode0.domain.local) is not valid. If your hostname configuration isn't valid everywhere, you may need to explicity set it by configuring the `listeners` setting (and maybe `advertised.listeners` setting) in your Kafka configuration.

-Ewen

On Mon, Feb 1, 2016 at 2:59 PM, Frizzi San Roman <friz...@gmail.com> wrote:

Hi,
I installed the individual packages and started the services: first zookeeper and then kafka. When I try to start schema-registry service appear this error:
Do you have any idea about it?
I need to do other additional configuration?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/0737a608-e2ab-41c5-beec-7e8ba050e254%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen

[Steve Hoffman]

Also appears to get the same error on AMZN linux.  Is there a github issue list or something to post these things to?

[David Tucker]

It looks like you’re using RedHat 6.x.   Did you follow the instructions at 

http://docs.confluent.io/2.0.1/installation.html#installation-yum

You’ll need to pull in the librdkafka shared library from the zip files, and use yum simply for the core packages :

yum install -y confluent-kafka-2.11.7

yum install -y confluent-schema-registry 

— David

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/33f1b87f-e554-4918-a841-c5669f4bd245%40googlegroups.com.