providers.mongokafka.PersistentKafkaPublisher
A Kafka event publisher that uses a MongoDB DAO to store stateless events as-is.
Usage
providers.mongokafka.PersistentKafkaPublisher()This class should be used for events that do not represent a stateful object, such as user info, but rather stateless information. This includes things like notifications, completed actions, file processing results, etc.
Methods
| Name | Description |
|---|---|
| __init__() | Please do not call directly! Should be called by the construct method. |
| construct() | Setup and teardown KafkaEventPublisher instance with some config params. |
| publish_pending() | Publishes all non-published events. |
| republish() | Republishes all stored events independent of whether they have |
__init__()
Please do not call directly! Should be called by the construct method.
Usage
__init__(*, event_publisher, dao, compacted_topics, topics_not_stored)
construct()
Setup and teardown KafkaEventPublisher instance with some config params.
Usage
construct(
*,
config,
dao_factory,
compacted_topics=None,
topics_not_stored=None,
collection_name="",
kafka_producer_cls=AIOKafkaProducer
)Parameters
config: MongoKafkaConfig-
Config parameters needed for connecting to Apache Kafka.
compacted_topics: set[str] | None = None-
A set of topics that should be compacted. For these topics, only the latest event for a given key will be republished. Prior events for a given key in a compacted topic are replaced by new events, so only one event should be stored at a given time per key per topic.
topics_not_stored: set[str] | None = None-
A set of topics which should not be stored in the database. Events for these topics will be published identically as they would be in the KafkaEventPublisher class.
collection_name: str = ""-
The name of the MongoDB collection in which to store events.
dao_factory: MongoDbDaoFactory-
A MongoDbDaoFactory instance that can be used to create a DAO.
kafka_producer_cls: type[KafkaProducerCompatible] = AIOKafkaProducer- Overwrite the used Kafka Producer class. Only intended for unit testing.
publish_pending()
Publishes all non-published events.
Usage
publish_pending()
republish()
Republishes all stored events independent of whether they have
Usage
republish()already been published or not.