| Re: [nosql-dev] Support for iterables #200 |
On the topic of working with iterables instead of lists: I like having either iterable or stream as the default internally, and then having the intermediary code able to convert between them. The nice thing there is that it could end up being whatever is most efficient for the driver... in Darwino, the internal Cursor maps pretty closely to Stream operations, so it could be very efficient implementing that directly
Maybe ideally it'd be up to the driver: default to Iterable as the standard mechanism of exchange and use StreamSupport conversion as the default, but also have the option for drivers to produce customized Streams
Ok, I checked several drivers and yes.
Usually, they return as Iterable.
I have on my mind two solutions both it will change the result to all search results.We can change the result to Iterable itself, e.g.:
Iterable<DocumentEntity> query(String query);What I propose it to have an interface to represents this result as iterable, therefore:
- We can have a Stream default method to make easier to the final user.
- We can create a default supplier where give the entities result from database and a converter, to translate the entities to Jakarta NoSQL API, it will create a default Result to the drives, so make to driver support as well.
import java.util.Objects; import java.util.function.BiFunction; import java.util.function.Function; import java.util.stream.Stream; import java.util.stream.StreamSupport; public interface Result<T> extends Iterable<T> { default Stream<T> toStream() { return StreamSupport.stream(this.spliterator(), false); } static <E, T> Result<T> of(Iterable<E> entities, Function<E, T> converter) { Objects.requireNonNull(converter, "converter is required"); Objects.requireNonNull(entities, "entities is required"); final ResultSupplier<T, E> supplier = ServiceLoaderProvider.get(ResultSupplier.class); return supplier.apply(converter, entities); } interface ResultSupplier<T, E> extends BiFunction<Function<E, T>, Iterable<E>, Result<T>> { } }The new return:
Result<DocumentEntity> query(String query);So:
//database provider side Iterable<DatabaseEntity> entities = ...;//native query Function<DatabaseEntity, ColumnEntity> converter = ...;//the translate logic final Result<ColumnEntity> result = Result.of(entities, converter);//return this value to the user //lazy value for (ColumnEntity columnEntity : result) { System.out.println(""); } //return as Stream final Stream<ColumnEntity> columnEntityStream = result.toStream();On Sat, Aug 17, 2019 at 6:05 AM Otávio Gonçalves de Santana <otaviopolianasantana@xxxxxxxxx> wrote:It’s been on my mental back-burner to try out some lazy-loading options in my driver. I’ve considered if it’d work to make a List implementation that lazy-loads internally, since that’d keep some handy methods that Iterable doesn’t have. On the other hand, maybe Stream would be most idiomatic now, and could have efficient count, etc. operations while also having a stronger implication of lazy-loading.
-JesseOn Fri, Aug 16, 2019 at 10:35 PM Otávio Gonçalves de Santana <otaviopolianasantana@xxxxxxxxx> wrote:DocumentCollectionManager::search produces a list.
Do we assume that result will always comfortably fit in memory?
In NoSQL world, it often doesn't and this prevents the whole (very convenient) API to be used at all.
We need a separate search implementation that returns an iterable, or a stream, or an observable
This would be easy to create from existing search implementation. Looking at drivers, most of the time it does iterable -> stream(iterable).collect(toList)Suggestions?--Otávio Gonçalves de Santana--Otávio Gonçalves de Santana--Otávio Gonçalves de Santana