Skip to content

How to handle delta dictionaries and finish_preserve_values with nested builders #9600

@adamreichold

Description

@adamreichold

Which part is this question about
How to use the API of arrow_ipc and arrow_array for delta dictionaries when nested builders are involved.

Describe your question
I have implemented delta dictionaries based on the example at https://docs.rs/arrow-ipc/latest/arrow_ipc/writer/struct.StreamWriter.html#example---efficient-delta-dictionaries and this works for top-level columns in my schema by calling column.finish_preserve_values() instead of column.finish() when collecting the arrays for the next RecordBatch.

However, I also have nested builders of lists of structs which contain dictionary fields. The type of the top-level build is then ListBuilder<Box<dyn ArrayBuilder>> which does not have a finish_preserve_values method.

How can I produce the top-level array while instructing the StringDictionaryBuilder hidden inside the StructBuilder inside the ListBuilder to preserve its values?

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions