Error while installing Confluent 3.0 RPM package

474 views
Skip to first unread message

Avinash Deshmukh

unread,
Aug 11, 2016, 2:49:02 AM8/11/16
to Confluent Platform
Hi All,

I am new to Confluent platform and currently trying to install it on my local environment.
I choose to use the RPM based installation of confluent and followed the steps.

After downloading the RPM package, I unzipped it and run the ./install.sh script. 

I am getting an error message about some dependencies. Below is the error message -

Installing libserdes
error: Failed dependencies:
        libavro.so.23.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el6.x86_64
        libavrocpp.so.1.8.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el6.x86_64
        libjansson.so.4()(64bit) is needed by confluent-libserdes-3.0.0-1.el6.x86_64
        libavro.so.23.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64
        libavrocpp.so.1.8.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64
        libc.so.6(GLIBC_2.14)(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64
        libjansson.so.4()(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64

How I can resolve these dependency issues?

Thanks

Avinash Deshmukh

Michael Noll

unread,
Aug 11, 2016, 3:16:28 AM8/11/16
to confluent...@googlegroups.com
What's your operating system and its exact version Avinash?

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/54c61b41-079c-4d67-81dc-9c6f4ed8ac9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

Avinash Deshmukh

unread,
Aug 18, 2016, 6:35:52 AM8/18/16
to Confluent Platform
Hi Michael,

Thanks for your reply and sorry for late response.

The operating system that I am using is Cent OS Linux and the version that I am using is confluent-3.0.0-2.11.


On Thursday, August 11, 2016 at 12:46:28 PM UTC+5:30, Michael Noll wrote:
What's your operating system and its exact version Avinash?

On Thu, Aug 11, 2016 at 8:49 AM, Avinash Deshmukh <d.avinas...@gmail.com> wrote:
Hi All,

I am new to Confluent platform and currently trying to install it on my local environment.
I choose to use the RPM based installation of confluent and followed the steps.

After downloading the RPM package, I unzipped it and run the ./install.sh script. 

I am getting an error message about some dependencies. Below is the error message -

Installing libserdes
error: Failed dependencies:
        libavro.so.23.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el6.x86_64
        libavrocpp.so.1.8.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el6.x86_64
        libjansson.so.4()(64bit) is needed by confluent-libserdes-3.0.0-1.el6.x86_64
        libavro.so.23.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64
        libavrocpp.so.1.8.0.0()(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64
        libc.so.6(GLIBC_2.14)(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64
        libjansson.so.4()(64bit) is needed by confluent-libserdes-3.0.0-1.el7.x86_64

How I can resolve these dependency issues?

Thanks

Avinash Deshmukh

-- 


Michael Noll

unread,
Aug 18, 2016, 7:10:11 AM8/18/16
to confluent...@googlegroups.com
And which version of CentOS?  6.0?  7.1? ...?

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/0bf1ed33-8637-4d58-942f-17bfe064d3d6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Avinash Deshmukh

unread,
Aug 18, 2016, 7:19:33 AM8/18/16
to Confluent Platform
It is CentOS 6.7 final.
To post to this group, send email to confluent...@googlegroups.com.

Michael Noll

unread,
Aug 18, 2016, 9:26:53 AM8/18/16
to confluent...@googlegroups.com
Thanks Avinash,

could you also share the following information:

1. The exact command (e.g. yum install ...) that triggered the error message in your first email.
2. Your yum repository setup (see http://docs.confluent.io/3.0.0/installation.html#rpm-packages-via-yum), notably the contents of your `/etc/yum.repos.d/confluent.repo` file.

The reason I am asking for (2) is that your installation command shows both EL6 and EL7 packages for libserdes being requested for installation.



To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/9d8daa31-938e-4022-98d0-58fcf84c80ac%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Avinash Deshmukh

unread,
Aug 19, 2016, 1:35:39 AM8/19/16
to Confluent Platform
Hi Michael,

Initially I tried with yum installation, but it was giving me error (right now, not able to recollect the error message for yum installation).

After that I tried with 'DEB/RPM packages with installer script' where I have downloaded the tar.gz file.

Following are the steps that I followed with this installation.

1. I used 'wget http://packages.confluent.io/archive/3.0/confluent-3.0.0-2.11.tar.gz' command to download the .tar.gz file.
2. Extracted it by using command 'tar -xzvf confluent-3.0.0-2.11.tar.gz' 
3. And then I run the ./install.sh file to do the required installation.

These steps perform the required Kafka components for me, including Kafka broker, zookeeper etc. However, when it starts with 'libserdes' installation, it gives me above error.

For you information - I am using Sandbox machine which is provided by Hortonworks. It comes with CentOS v 6.7 and all installed Hadoop packages.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

Dustin Cote

unread,
Aug 19, 2016, 10:04:20 AM8/19/16
to confluent...@googlegroups.com
Hi Avinash,

To install with the installer script, you'll have to remove the packages for EL7 from the directory.  There are different native dependencies between the two versions, so you'll have to do 'rm *el7*' since you are running on centos 6.

Once you've done that, I found a couple of bugs in our install.sh script that I'll be cleaning up for next release.  You can modify the script directly to get going with your install.  Here's what I changed:
1) Install the EPEL repo "sudo yum install epel-release"
2) Install jansson (required by libserdes/avro-c) "sudo yum install jansson-devel"
3) Change line 8 of install.sh to this --> PACKAGES="common rest-utils schema-registry kafka-rest camus kafka-connect-hdfs kafka-connect-jdbc librdkafka avro-c avro-cpp libserdes"
4) Change the if/else block at line 45 to -->     

# TODO: non-confluent prefixed packages require special treatment
    if [[ "$PACKAGE" =~ ^(avro-c|avro-cpp) ]]; then
      eval "${COMMAND} ${PATH_PREFIX}${PACKAGE}-1${COMMAND_EXT}"
    elif [[ "$PACKAGE" =~ ^(librdkafka) ]]; then
      eval "${COMMAND} ${PATH_PREFIX}${PACKAGE}1${COMMAND_EXT}"
    else
      eval "${COMMAND} ${PATH_PREFIX}confluent-${PACKAGE}${COMMAND_EXT}"
    fi

That should get it installed for you and I'll work on getting these changes in the next release.  Thanks for pointing this out to us.



To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/62484189-c75c-4bfa-874f-ab1906d621f2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

Avinash Deshmukh

unread,
Aug 22, 2016, 5:58:46 AM8/22/16
to Confluent Platform
Hi Dustin,

Thanks for your solution.

This solution works and now the error related to 'libserdes' is resolved.
However, this time getting another error as 'error: File not found by glob: /opt/confluent/confluent-3.0.0/avro-c1*.rpm'.

I tried to uninstall the confluent from my local-environment. But getting an error message as 

'Removing libserdes
error: package confluent-libserdes is not installed'

I have attached the modified 'install' shell script for your reference.

Though, currently, this is not show-stopper for us, I can work with other Kafka related stuff. However, I think this will help you to resolve all possible issues.

Thanks,

Avinash Deshmukh


To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
install.sh

Dustin Cote

unread,
Aug 22, 2016, 8:24:00 AM8/22/16
to confluent...@googlegroups.com
Hi Avinash,

This line is missing the "-" before the "1".  It should be like this:
 if [[ "$PACKAGE" =~ ^(avro-c|avro-cpp) ]]; then
      eval "${COMMAND} ${PATH_PREFIX}${PACKAGE}-1${COMMAND_EXT}"

The error your are seeing is because libserdes is not installed.  It depends on the avro-c library, so it failed to install in the first place.  If you fix up the install.sh you attached with the line I've posted above, you should be good to go.

Regards,


To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/64153cb1-5efe-4658-b001-56a09776c5ac%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Avinash Deshmukh

unread,
Aug 23, 2016, 2:02:08 AM8/23/16
to Confluent Platform
Hi Dustin,

Thanks for your reply.

As mentioned in below step I modified the script and run it again to install the avro components.

It was working fine and I was able to install the avro-c libraries.

But this time I am getting another error. This time it is related to avro-cpp libraries.

Below is the error message.

==================================================================
Installing avro-cpp
error: Failed dependencies:
        boost-devel is needed by avro-cpp-1.8.0_confluent3.0.0-1.el6.x86_64
        libboost_filesystem-mt.so.5()(64bit) is needed by avro-cpp-1.8.0_confluent3.0.0-1.el6.x86_64
        libboost_iostreams-mt.so.5()(64bit) is needed by avro-cpp-1.8.0_confluent3.0.0-1.el6.x86_64
        libboost_program_options-mt.so.5()(64bit) is needed by avro-cpp-1.8.0_confluent3.0.0-1.el6.x86_64
        libboost_system-mt.so.5()(64bit) is needed by avro-cpp-1.8.0_confluent3.0.0-1.el6.x86_64
warning: group mockbuild does not exist - using root
warning: group mockbuild does not exist - using root
warning: group mockbuild does not exist - using root
=============================================================================

Requesting you to let me know are there any changes required.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

Avinash Deshmukh

unread,
Aug 23, 2016, 2:59:51 AM8/23/16
to Confluent Platform
Hi Dustin,

There is one more update would like to give you and this is about installation of Confluent by using 'RPM packages via YUM'.

On my local environment, I deleted the sandbox virtual machine and imported it again, so that I can get a complete fresh machine setup for Confluent.
I used the following steps to install the confluent platform by using Yum.

1. Installed the Conflulent's public key by using 'sudo rpm --import http://packages.confluent.io/rpm/3.0/archive.key'
2. Created confluent.repo file under /etc/yum.repos.d folder and copied the content for CentOS 6 version.
3. Cleared all yum cache.
4. Started the installation by using 'sudo yum install confluent-platform-2.11'

I was able to install Confluent components except 'Confluent Control Center' and 'Confluent Schema Registry'. I am getting below error messages while installing with YUM.

===============================================================================================================
Trying other mirror.
Trying other mirror.

Error Downloading Packages:
  confluent-control-center-3.0.0-1.noarch: failure: confluent-control-center-3.0.0-1.noarch.rpm from Confluent: [Errno 256] No more mirrors to try.
  confluent-schema-registry-3.0.0-1.noarch: failure: confluent-schema-registry-3.0.0-1.noarch.rpm from Confluent: [Errno 256] No more mirrors to try.
=============================================================================================================
Please let me know are there any additional steps required to perform this installation by using YUM.

Dustin Cote

unread,
Aug 23, 2016, 8:08:53 AM8/23/16
to confluent...@googlegroups.com
Hi Avinash,

It looks like you are missing C++ boost library for the avro-cpp error.  You'll need to have that installed to install avro-cpp (yum install boost-devel).  

For the latter question, it looks like you have a slow internet connection to the confluent platform repository.  You can probably just try it again until you are able to reach the repository since you were able to install the other components.  

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/b4a9b0f3-0d4e-4fa8-b4cc-7f08548b71df%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Avinash Deshmukh

unread,
Aug 24, 2016, 2:05:43 AM8/24/16
to Confluent Platform
Hi Dustin,

Thank you very much for your help. 

Finally I was able to complete the Confluent installation on my local environment successfully.

About the RPM based on yum install, I think this might not be due to slow internet connection as I was able to install other packages.
This could be due to slow performance of CentOS machine as I am running this on Windows environment by using Virtual Box.

After trying 2-3 times, I was finally able to complete the confluent installation on my local env, by using both install script as well as by using yum installation.

Thanks again for your help.

Avinash Deshmukh
=======================================================================
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Michael G. Noll
Product Manager | Confluent
Follow us: Twitter | Blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.



--
Dustin Cote
Customer Operations Engineer | Confluent
Follow us: Twitter | blog

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages