java_kafkasamples/docs/06 Consuming messages with Kafka Consumers and Consumer groups

# 06 Consuming messages with Kafka Consumers and Consumer Groups

Very similar to producer.

## Creating a consumer

- Properties:
    - `bootstrap.servers`
    - `key.serializer`
    - `value.serializer`
    - and others
    - => `KafkaProducer`
    - [Consumer config doc](http://kafka.apache.org/documentation.html#consumerconfigs)
- See [KafkaConsumerApp](../src/main/java/KafkaConsumerApp.java)

## (Un)Subscribing

- A single consumer can subscribe to any number of topics
- subscribe takes either a list of strings or a regex string,
    - each call replaces the previous subscription list
- They can also unsubscribe all topics at once.

## Subscribing/assigning
### consumer.subscribe()

- automatic/dynamic partition assignment
    - that single consumer listens to every partition on every listed topic
    - entirely managed by the `KafkaConsumer`
        - will get newly added partitions to subscribed topics (`SubscriptionState`)
- now needs a valid `group.id` == `ConsumerConfig.GROUP_ID_CONFIG`

### consumer.assign()

- used to subscribe to explicit partitions
    - one or more partitions, regardless of topic
    - manual, self-administering mode
- more of an advanced case
- does not needs a `group.id`
- blocks instead of erroring if one of the partitions does not exist, in 
  `updateFetchPositions.updateFetchPositions()`.

## The poll loop

- primary function of the `KafkaConsumer`
- continuously poll the brokers for data
- single API for handling all Consumer <-> Broker interactions
    - a lot of interactions beyond message retrieval