Implement Applicative based API by Shimuuar · Pull Request #522 · haskell/vector

Shimuuar · 2025-01-28T09:16:08Z

Implementation follows plan described in #477 with generateA :: Applicative f => Int -> (Int -> f a) -> f (v a) as primitive. It is not subject to stream fusion.

Writing generateA is not difficult. Problem is doing it efficiently. Current benchmarks are (suggestions for more are very welcome):

Generate vector of random numbers using state monad
Generate vector using IO action
Compute sum of vector using lens
Map vector using lens

First and naive implementation uses list as intermediate data structure. Sum benchmark performs well in this case. Using STA instead brings map benchmark on par with explicit loop and produces slight (5-10%) improvements in state and IO benchmark)

newtype STA v a = STA {  _runSTA :: forall s. Mutable v s a -> ST s (v a) }

Currently sum and map perform on par with explicit loop. State gives 7x slowdown and 8x allocations, IO benchmark 4x slowdown and 8x allocations. We obviously can add rewrite rules for IO/ST but maybe there're more general optimizations.

Fixes #477, #69, #132, #144

konsumlamm

What about for/for_/traverse_ etc?

Shimuuar · 2025-01-28T14:28:50Z

What about for/for_/traverse_ etc?

Good point! I forgot about them. But they're simpler since in this case one isn't in business of constructing vector so it simply reduces to fold.

Shimuuar · 2025-09-03T16:00:39Z

I think that's about as far as reasonably possible for improving performance.

For many PrimMonads we would like to simply create buffer and just write to it. But it's not safe in general case, applicatives with backtracking can break referential transparency. And it's not clear how to express such rewrite rule and whether it's even possible. Also unstreamM suffers from same problem

generateA is used as primitive and all other functions are expressed in it terms. First version goes through intermediate list. This is simplest implementation possible and would serve as baseline for further optimizations

We establish implementation which goes through list as baseline and the we can try to optimize it. Note definition of foldlOf'. It's different from definition in lens<=5.3.3 but it's absolutely necessary to get good perfomance in folds

Does wonders for traversals using Identity

Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>

No performance change in benchmarks

This is clearly not enough. There're many other types that will benefit form same rewrite but we don't know how to do that. unstreamM suffers from same problem. Identity is important since it's used in lens for mapping (over) and rewrite rule does improve performance: 10-20% in microbenchmark.

Relevant for TypeApplication and other function library use same convention

Now it works in the same way as generateM

Shimuuar · 2025-09-05T08:35:01Z

I think PR is ready.

It may perform better for some applicatives

treeowl · 2025-09-09T10:20:16Z

I think that's about as far as reasonably possible for improving performance.

For many PrimMonads we would like to simply create buffer and just write to it. But it's not safe in general case, applicatives with backtracking can break referential transparency. And it's not clear how to express such rewrite rule and whether it's even possible. Also unstreamM suffers from same problem

We definitely can write such rewrite rules for specific cases.

Shimuuar · 2025-09-09T10:30:35Z

And for IO/ST there're such rules already. But it seems there's no way to write such rules compositionally.

If for example we want to add rewrite rules for WriterT/ReaderT/StateT over IO then there're exponentially many types for given stack depth. We probably need some variant of typeclass guided rewrites

lehins

Sorry for not reviewing it earlier. Very busy time for me right now.
Looks great. I only had a few minor comment that don't necessarily need to be addressed

lehins · 2025-09-11T22:31:30Z



-withIndexFirst m f = m (uncurry f) . zip [0..]
+-- withIndexFirst :: (Int -> a -> [a]) -> [a] -> [a]


Forgot a comment

Suggested change

-- withIndexFirst :: (Int -> a -> [a]) -> [a] -> [a]

lehins · 2025-09-11T22:34:04Z

+
+ * [#522](https://github.com/haskell/vector/pull/522) API using Applicatives
+   added: `traverse` & friends.
+   * [#518](https://github.com/haskell/vector/pull/518) `UnboxViaStorable` added.


Looks like an accidental indentation

Suggested change

* [#518](https://github.com/haskell/vector/pull/518) `UnboxViaStorable` added.

* [#518](https://github.com/haskell/vector/pull/518) `UnboxViaStorable` added.

lehins · 2025-09-11T22:37:02Z

-      -- Use fromListN to be more efficient in construction of resulting vector
-      -- Also behaves better with compact regions, preventing runtime exceptions
-      in  Data.Vector.fromListN n Applicative.<$> Traversable.traverse f (toList xs)
+  traverse = traverse


I would have used the imported generic version instead for clarity. Otherwise at first glance it looks like bottom because of a recursive call. It is only when one looks at a qualified import for Traversable it becomes clear that it is not the case.

Suggested change

traverse = traverse

traverse = G.traverse

Shimuuar · 2025-09-12T15:36:34Z

Sorry for not reviewing it earlier. Very busy time for me right now.

No problem. I you want to make a review but don't have time at the moment please leave note. I would wait

konsumlamm reviewed Jan 28, 2025

View reviewed changes

Comment thread vector/src/Data/Vector/Generic.hs Outdated

Comment thread vector/src/Data/Vector/Generic.hs Outdated

Shimuuar and others added 12 commits September 5, 2025 09:30

Implement functions which use Applicatives

3ed0df2

generateA is used as primitive and all other functions are expressed in it terms. First version goes through intermediate list. This is simplest implementation possible and would serve as baseline for further optimizations

Implement STA optimization trick as an optimization

8b1be88

Does wonders for traversals using Identity

Update vector/src/Data/Vector/Generic.hs

a08bf3d

Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>

We don't need to return vector in STA

10d112d

No performance change in benchmarks

Add function which discard result of applicative action

a6bd2a5

Put Applicative in first place in context

9881b63

Relevant for TypeApplication and other function library use same convention

Add reexports

37b674c

Update changelog

701f0a4

Add tests

36f86e3

Fix handling of negative sizes for generateA

341c230

Now it works in the same way as generateM

Shimuuar force-pushed the traverse branch from 6d919a1 to 341c230 Compare September 5, 2025 08:24

Shimuuar marked this pull request as ready for review September 5, 2025 08:33

Use liftA2

cc5725f

It may perform better for some applicatives

Shimuuar force-pushed the traverse branch from 07c1d92 to cc5725f Compare September 5, 2025 09:00

Use new traverse for Traversable instances

e1d2688

Shimuuar merged commit 4b4eade into master Sep 9, 2025
55 of 56 checks passed

This was referenced Sep 9, 2025

add generic traverse method, a la Data.Traversable #69

Closed

Weaken constraint on mapM #132

Closed

Can we get generateA? #144

Closed

Shimuuar deleted the traverse branch September 9, 2025 08:06

lehins reviewed Sep 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Applicative based API #522

Implement Applicative based API #522
Shimuuar merged 14 commits into
masterfrom
traverse

Shimuuar commented Jan 28, 2025

Uh oh!

konsumlamm left a comment

Uh oh!

Uh oh!

Uh oh!

Shimuuar commented Jan 28, 2025

Uh oh!

Shimuuar commented Sep 3, 2025

Uh oh!

Shimuuar commented Sep 5, 2025

Uh oh!

Uh oh!

treeowl commented Sep 9, 2025

Uh oh!

Shimuuar commented Sep 9, 2025

Uh oh!

lehins left a comment

Uh oh!

lehins Sep 11, 2025

Uh oh!

lehins Sep 11, 2025

Uh oh!

lehins Sep 11, 2025

Uh oh!

Shimuuar commented Sep 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants



		withIndexFirst m f = m (uncurry f) . zip [0..]
		-- withIndexFirst :: (Int -> a -> [a]) -> [a] -> [a]

	* [#518](https://github.com/haskell/vector/pull/518) `UnboxViaStorable` added.
	* [#518](https://github.com/haskell/vector/pull/518) `UnboxViaStorable` added.

Conversation

Shimuuar commented Jan 28, 2025

Uh oh!

konsumlamm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Shimuuar commented Jan 28, 2025

Uh oh!

Shimuuar commented Sep 3, 2025

Uh oh!

Shimuuar commented Sep 5, 2025

Uh oh!

Uh oh!

treeowl commented Sep 9, 2025

Uh oh!

Shimuuar commented Sep 9, 2025

Uh oh!

lehins left a comment

Choose a reason for hiding this comment

Uh oh!

lehins Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

lehins Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

lehins Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Shimuuar commented Sep 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants