Repository navigation
Add streaming of :many via chan and/or callback func #1218
Description
I noticed that :many queries collect result objects in slices. That could be inefficient for large number of rows. I was wondering if you'd be willing to add/allow support for some sort of :stream annotation, which produces functions like :many, but they return a <-chan T instead. The channel can be read from in a different goroutine while the first goroutine does the row scan. It closes the channel when done. The channel should definitely be buffered, but I've got no idea how to set the size. Could be a field in Queries{}.
Another idea is a :callback annotation. A query function takes a func(T) callback as parameter, which handles the scanned T one by one. No separate goroutine, but also no buffering.
Activity
[-]Add streaming of :many via chan[/-][+]Add streaming of :many via `chan`[/+]cameronpm commentedon Sep 27, 2021
A
:callbackannotation is cleaner and if only one of the two approaches were chosen, much simpler when you want to read rows synchronously (while allowing you to implement:streamyourself). It also allows the caller an extra means of requesting cancellation by returning a bool (iefunc(T) (ok bool)) as opposed to cancelling the context.cloneable commentedon Sep 27, 2021
True, the callback would allow one to implement streaming. I thought it's a bit much ceremony at the moment to do it in every callback, with
selecting onchan<-and<-ctx.Done()and all. Needs to be done correctly and the code generator would be ideal. However, I realized with the upcoming support for generics a simple generic helper function could take care of this, too.[-]Add streaming of :many via `chan`[/-][+]Add streaming of :many via `chan` and/or callback `func`[/+]