[ANN] streamly-0.10 with fast binary serialization

Streamly is a concurrent stream and array library supporting a wide range of functionality.

Important features in the new release include: (1) fast binary serialization to arrays with an option to serialize record field names; (2) the array type can now represent pinned as well unpinned memory, unifying the representations used by bytestring, text and vector; and (3) concurrency support for folds.

For more details, see this post: Haskell streamly-0.10 with fast binary serialization - Composewell Blog .

15 Likes

Any plans to stabilize the API?

1 Like

Pasting my answer from the reddit thread:

Stability of the API is very important, we are always working towards it, and we are getting there slowly.

streamly-0.9.0 was an exception because we changed from the so called “magical” concurrency to an explicit concurrency API. Also the package was broken into two streamly-core (with minimal dependencies) and streamly. Both of these were demands raised by the users.

Post that we do not see any major disruption and we are taking all precautions to not break the existing APIs. It should be extremely rare for released APIs to change. Even in internal modules we deprecate the APIs rather than changing the existing ones. But some less used internal APIs may still change.

That said, it is hard to come up with the right abstractions that - cover all use cases, are elegant and perform exceptionally well. To be able to achieve that we have to change the types sometimes after some experimentation which creates a ripple effect in the users of those. You may be affected if you are using unexposed constructors directly, if you use smart constructors or only released API you should mostly be fine.

A large number of abstractions are now almost complete and quite stable. Some new abstractions are coming in 0.11 like composable and concurrent scans. Parsers may see some changes in future for context and error reporting, fused pipes will come in 0.12, and then the basic streaming stuff will be complete and fully stable.

8 Likes

Thanks a lot for this explanation! As an end-user this is reassuring. :slight_smile:

2 Likes