查看“︁Kafka data lose”︁的源代码

= Will kafka lose data? =
Kafka’s event hubs are designed to be resilient. When certain failures occur, they should be able to recover. However, there are still risks that you have to consider<ref>https://www.kannika.io/kannika-stories/scenarios-for-data-loss-in-kafka</ref>. 

== Incorrect configuration ==
* Retention time too short: Kafka allows you to configure how long messages are kept before deletion. If the retention.ms (retention time in milliseconds) is set too short, messages will be deleted sooner than expected. A simple mistake like missing a few zeros can have significant consequences. For instance, setting it to 60,000 means messages are only stored for 60 seconds.
* Retention bytes set too low: Kafka can also delete messages based on topic size. If retention.bytes is configured too low, Kafka may delete older messages prematurely to free up space. A misstep here can lead to unintended data loss.
* Incorrect Use of the Compact Cleanup Policy: Kafka's log compaction feature allows old records to be removed once newer ones with the same key arrive. If a topic is wrongly configured with cleanup.policy=compact instead of cleanup.policy=delete, Kafka will only keep the latest message for each key. This can result in losing older messages if the intention was to preserve all versions.

== App bugs  == 

* A bug may cause an application to mistakenly produce events to the wrong Kafka topic or partition.
* Kafka producers automatically retry sending messages when failures occur. If the retry logic is misconfigured, events that were never successfully written to the topic may be lost.
* If the producer is set with acks=0, there is no confirmation that messages were successfully written to Kafka. A safer configuration like acks=1 or acks=all could prevent data loss, but if misconfigured, data may be dropped.


[[Category:Kafka]]
[[Category:MQ]]