what is client_id used for?

[Team Ubest]

Hi, I'm new to kafka-python

config options: client_id is used for the server to distinguish consumer instances?

consider the message distribution semantic on server side:

produce:

send(message) to topic

save (message) to partition

consume:

broadcast(message) to groups

dispatch(this_group.message) to this_group.consumer(client_id,partition)

 

[Dana Powers]

ClientId is part of the kafka request protocol. Protocol docs describe it as:

```

This is a user supplied identifier for the client application. The user can use any identifier they like and it will be used when logging errors, monitoring aggregates, etc. For example, one might want to monitor not just the requests per second overall, but the number coming from each client application (each of which could reside on multiple servers). This id acts as a logical grouping across all requests from a particular client.

```

see https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Requests

I have never looked specifically into how the server might use it. kafka-python doesn't do anything with it other than comply w/ the protocol spec (and set 'kafka-python' as a default).

-Dana

--
You received this message because you are subscribed to the Google Groups "kafka-clients" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kafka-client...@googlegroups.com.
To post to this group, send email to kafka-...@googlegroups.com.
Visit this group at http://groups.google.com/group/kafka-clients.
To view this discussion on the web visit https://groups.google.com/d/msgid/kafka-clients/407e1d31-2b57-4ac0-93e4-e481ca97f85e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joe Stein]

client id should be used to distinguish each running app against kafka. its a pass through to the logs. it should be set by every instance running for best fine grained monitoring, etc

~ Joe Stein

- - - - - - - - - - - - - - - - - - -
     

  http://www.elodina.net

    http://www.stealth.ly

- - - - - - - - - - - - - - - - - - -

To view this discussion on the web visit https://groups.google.com/d/msgid/kafka-clients/CAAndJ_yAEMJgJ8-V5OjNUX4CeUwji%3D-UfEz3KOj9vnMoSi%2BPoQ%40mail.gmail.com.

[Dana Powers]

Joe: what's your recommendation for fine-grained monitoring tools? You've built some yourself, right? We're still rocking home-grown jmx->graphite + grafana stats that we threw together early last year.

-Dana

[Team Ubest]

thanks, I got it 

[Joel Koshy]

Worth adding that client-id will be used for quota enforcement:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas
https://issues.apache.org/jira/browse/KAFKA-2083

> >> dispatch(this_group.message) to this_group.consumer(*client_id*
> >> ,partition)

> >>
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "kafka-clients" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an

> >> email to kafka-client...@googlegroups.com <javascript:>.

> >> To post to this group, send email to kafka-...@googlegroups.com

> >> <javascript:>.

> >> Visit this group at http://groups.google.com/group/kafka-clients.
> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/kafka-clients/407e1d31-2b57-4ac0-93e4-e481ca97f85e%40googlegroups.com

> >> <https://groups.google.com/d/msgid/kafka-clients/407e1d31-2b57-4ac0-93e4-e481ca97f85e%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> >
>

> --
> You received this message because you are subscribed to the Google Groups "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kafka-client...@googlegroups.com.
> To post to this group, send email to kafka-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/kafka-clients.

> To view this discussion on the web visit https://groups.google.com/d/msgid/kafka-clients/6108df22-a9cb-4f20-bfd1-70175586d6e7%40googlegroups.com.

[kant kodali]

Why not use consumer group.id instead of client-id if it is meant for logical grouping? what is difference between client-id and group.id?