-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Which part is this question about
How to use the API of arrow_ipc and arrow_array for delta dictionaries when nested builders are involved.
Describe your question
I have implemented delta dictionaries based on the example at https://docs.rs/arrow-ipc/latest/arrow_ipc/writer/struct.StreamWriter.html#example---efficient-delta-dictionaries and this works for top-level columns in my schema by calling column.finish_preserve_values() instead of column.finish() when collecting the arrays for the next RecordBatch.
However, I also have nested builders of lists of structs which contain dictionary fields. The type of the top-level build is then ListBuilder<Box<dyn ArrayBuilder>> which does not have a finish_preserve_values method.
How can I produce the top-level array while instructing the StringDictionaryBuilder hidden inside the StructBuilder inside the ListBuilder to preserve its values?