Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion core/common/src/traits/partitioner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,19 @@ use crate::types::message::IggyMessage;
use std::fmt::Debug;

/// The trait represent the logic responsible for calculating the partition ID and is used by the `IggyClient`.
/// This might be especially useful when the partition ID is not constant and might be calculated based on the stream ID, topic ID and other parameters.
///
/// Iggy uses a hierarchical model for append-only logs. A stream contains topics which hold partitions. Each partition is an append-only log.[^note]
/// A producer of messages such as an [`IggyProducer`], that appends messages to the log, may want to choose which partition to write the messages into.
/// To do that, a producer can take a type that implements this trait.
/// This is especially useful when computing the partition ID requires some client side info, i.e. stream ID, topic ID and/ or [`IggyMessage`] attributes.
///
/// Note the difference between [`Partitioning`] and [`Partitioner`]. [`Partitioning`] is a type used to set the _partitioning strategy_ for a producer.
/// If you use both, the [`Partitioner`] overwrites the strategy, sets it to [`PartitioningKind::PartitionID`] and the partition ID is
/// calculated with with the logic implemented in [`Partitioner::calculate_partition_id()`].
///
/// [^note]: [Website docs on how Iggy organizes data.](https://iggy.apache.org/docs/#how-iggy-organizes-data)
pub trait Partitioner: Send + Sync + Debug {
/// Calculate a partition ID.
fn calculate_partition_id(
&self,
stream_id: &Identifier,
Expand Down
16 changes: 13 additions & 3 deletions core/common/src/types/message/partitioning.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,18 @@ use std::{
hash::{Hash, Hasher},
};

/// `Partitioning` is used to specify to which partition the messages should be sent.
/// It has the following kinds:
/// A type that defines a what strategy the server should choose to partition the messages.
///
/// Iggy uses a hierarchical model for append-only logs. A stream contains topics which hold partitions. Each partition is an append-only log.[^note]
/// A producer of messages such as an [`IggyProducer`], that appends messages to the log can choose between three partitioning strategies.
/// - `Balanced` - the partition ID is calculated by the server using the round-robin algorithm.
/// - `PartitionId` - the partition ID is provided by the client.
/// - `MessagesKey` - the partition ID is calculated by the server using the hash of the provided messages key.
/// - `PartitionId` - the partition ID is provided by the client.
///
/// Note, that using a [`Partitioner`] on top of [`Partitioning`] sets the strategy to [`PartitioningKind::PartitionId`]. The value is then computed
/// based on your concrete implementation of [`Partitioner::calculate_partition_id()`].
///
/// [^note]: [Website docs on how Iggy organizes data.](https://iggy.apache.org/docs/#how-iggy-organizes-data)
#[serde_as]
#[derive(Debug, Serialize, Deserialize, PartialEq, Eq, Clone)]
pub struct Partitioning {
Expand Down Expand Up @@ -160,6 +167,9 @@ impl Partitioning {
}

/// Maximum size of the Partitioning struct
/// NOTE(haubur) I think this can be removed as it seems to be superseded
/// by implementing Sizeable/ get_size_bytes().
#[doc(hidden)]
pub const fn maximum_byte_size() -> usize {
2 + 255
}
Expand Down
Loading
Loading