[Python] How auto-commit works in case of batch processing messages from kafka? [closed]

Let's consider a following python snippet code:

from confluent_kafka import Consumer

conf = {
"bootstrap.servers": "servers",
"group.id": "group_id",
}
consumer = Consumer(conf)

while True:
messages = consumer.consume(num_messages=100, timeout=1.0)
events = process(messages)


I call it like batch-manner consumer of kafka. Let's consider a following questions/scenarios:

How auto-commit works in this case? I can find information about auto-commit with poll call, however I have not managed to find information about consume method. It is possible that auto-commit happend even before touching message (let's say the last one in batch). It means that we acked message we have not seen never. It can lead to message loss.

Continue reading...