to handle properly complex and custom data pipeline, the data pipeline needs to be more customizabled, by decoupling loading data, and preprocessing operations, adding dataclass configuration to data module, bring the data factory inside the data module, and adding data callbacks.
to handle properly complex and custom data pipeline, the data pipeline needs to be more customizabled, by decoupling loading data, and preprocessing operations, adding dataclass configuration to data module, bring the data factory inside the data module, and adding data callbacks.