Programming language for functional parallelism: F# vs Haskell?

If the kind of code you have in mind allocates memory heavily, then you might find that the GHC garbage collector scales better than the . NET garbage collector. There's some anedcodal evidence that the .

NET GC becomes a bottleneck when multiple threads are allocating heavily, and this is also a thorn in the side of most Java collectors too. On the other hand we've paid quite a lot of attention to achieving good locality and scalability in the GHC garbage collector - mainly because we have no choice, most idiomatic Haskell code allocates heavily anyway. I have benchmarks that allocate like crazy and keeping scaling beyond 24 cores In Haskell note that you get a guarantee of determinism from the type system, which you don't get in F You mentioned Data Parallel Haskell: a cautionary note here, it isn't ready for production use at the present time, although the DPH team are expecting that the forthcoming GHC 7.2.1 release will have a stable DPH implementation.

If the kind of code you have in mind allocates memory heavily, then you might find that the GHC garbage collector scales better than the . NET garbage collector. There's some anedcodal evidence that the .

NET GC becomes a bottleneck when multiple threads are allocating heavily, and this is also a thorn in the side of most Java collectors too. On the other hand we've paid quite a lot of attention to achieving good locality and scalability in the GHC garbage collector - mainly because we have no choice, most idiomatic Haskell code allocates heavily anyway. I have benchmarks that allocate like crazy and keeping scaling beyond 24 cores.In Haskell note that you get a guarantee of determinism from the type system, which you don't get in F#.

You mentioned Data Parallel Haskell: a cautionary note here, it isn't ready for production use at the present time, although the DPH team are expecting that the forthcoming GHC 7.2.1 release will have a stable DPH implementation.

2 You've pointed out a very important aspect of parallel programming. I kind of see a similar problem of . NET GC here stackoverflow.Com/questions/2311154/….

I think garbage collection is critical in context of functional parallelism where memory allocation is scattered over the program. From a developer's perspective, what are your suggestions to avoid such problems? – pad Apr 4 at 11:24 2 @Jon: Mutating in-place only doesn't solve the problem.

As you mentioned cache oblivious data structures, the situation is pretty clear for arrays and matrices, but how are you gonna organize recursive data structures (trees) in memory to reduce cache misses? – pad Apr 9 at 20:25 4 @Jon avoiding allocation is not a "solution". If your language implementation forces you to avoid allocation in order to get parallel speedup, that is a serious limitation.

Maybe you should consider a different language :-) – Simon Marlow Apr 11 at 15:23 5 @SimonMarlow: Degrading absolute performance is not a "solution". If your language implementation forces you to sacrifice performance in order to get parallel speedup, that is a serious limitation. – Jon Harrop Apr 12 at 23:19 4 Indeed, it's pretty trivial to get good parallel speedup if you sacrifice enough performance that your serial version is sufficiently slow.

– Jonathan Dursi May 4 at 2:23.

I'm going to get downvoted for this, but let me be the curmudgeon. Functional languages are great. They change the way you think about decomposing problems, and they map incredibly well to certain kinds of problems.

Every programmer should be familar with at least one functional programming language. But "functional languages are inherently good for parallel programming" is probably not the reason why. It's worth noting that what is unquestionably the most successful parallel functional language of all time, Erlang, uses completely bog-standard message passing to implement its parallelism, and the connection between its functional nature and its parallelism is indirect at best.

Twenty-five years ago, there was a huge push for functional languages, because the argument then seemed very compelling as well -- functional languages seemed a natural fit for the increasingly parallel architectures of the time. The argument was that compilers and runtimes would automatically be able to implement parallelism, due to the side-effect free nature of the languages. SISAL, which even then could be compiled into shared- and distributed- memory (!) executables was developed at this time, as was Haskell, as was ML, the the predecessor to Objective CAML and the other lanaguages in the ML family.

This is all just offered a bit of historical perspective. For literally a quarter of a century, functional-language advocates, including some of the brightest minds in the field, have been saying that functional languages' day in the sun was just around the corner, and that it's going to the be applicability to parallelism that is the killer app. And yet, here we are, with no one even having heard of SISAL; and I'm guessing that most readers of this post think of Haskell as a hot new language.It could well be, of course, that now with multicore considerations things have finally gotten so urgent that functional languages are really going to shine, or that this year will be the year that there's some breakthrough I can't even imagine which completely changes the landscape.

This year could well be different than each of the 25 previous years. But it might not, too. The vast, vast, vast majority of parallel and concurrent code extant now, and well into the forseeable future, is not written in functional languages.

If you're looking to learn about parallelism, by all means explore the mechanisms available in F#, Haskell etc; but don't limit yourself to those, is all I'm saying.

7 Simon Peyton Jones dealt with this point head-on in his recent Function Programming eXchange talk, Managing parallelism: embrace diversity, but control side effects. – Simon Marlow Mar 31 at 6:56 7 The killer argument for functional languages isn't parallelism, its the factor of between 4 and 10 reduction in the size of the code. The thing stopping functional languages in the past was that they couldn't compile to efficient code.

That is no longer true. – Paul Johnson Mar 31 at 17:39 6 Just to clarify, Erlang is a language for concurrent programming, not parallelism (i.e. Parallel speedup isn't the focus) – Don Stewart Apr 3 at 21:50 1 Well, ultimately it's the programmers who decide what a language is for, and googling "erlang" "scalability" gives 400k+ hits; so a lot of people are using it for it's ability to make use of lots of processors.(Eg, weak scaling, as vs strong-scaling -- but parallel performance nonetheless).

And performance has always been important to Erlang - the real-time telco stuff does require efficient use of whatever hardware is available. So while I agree with the distinction you're making, I'm not sure it's true that Erlang is "for" concurrent vs parallel programming. – Jonathan Dursi Apr 4 at 3:02 4 @SimonMarlow: Simon Peyton-Jones' lecture had two major flaws.

Firstly, he describes functional programming as "obviously" better in the context of parallelism when it is not obvious at all that massive performance degradation is good in the context of parallel programming. Secondly, he didn't present a single benchmark comparing parallel Haskell code with production-quality imperative code, which is a fundamental flaw in methodology. – Jon Harrop Apr 15 at 8:53.

The simple answer is that because both languages have solid support for parallelism and concurrency, this shouldn't be a factor in your decision on which language to use. I.e. , there are much larger factors to take into consideration for a decision like that.

7 Like "how cheap the programers are" O. O – pst Mar 30 at 22:03.

First of all, I agree with others that there is no objective answer. However, I think that the idea of functional parallelism is a bit overrated. Surely, you can easily find data dependencies in your program and if you're processing lots of data, you can use some data-parallel library to easily and safely parallelize it.

However, this can be done even in C# (using TPL and PLINQ) if you're a bit careful about what you're writing. The problem is, that most of the programs don't need to be parallelized, because they simply don't do enough CPU-intensive work. For example, F# async solves (I think) more important problem of enabling asynchronous I/O, which is the reason for most "hangs" in connected applications.

I think the popularity of Node. Js demonstrates this importance quite nicely. The real value of functional languages is in the expressivity of the language - you can easily define abstractions for your problem, write code in a more succinct way that is easier to understand, reason about and test.

You get this in both F# and Haskell. To answer your specific question about parallelism - I believe that the status of parallelism support in F# is more stable (but then, I'm an F# person). You can choose between async, TPL and (Erlang-inspired) F# agents (which are all quite stable libraries).

On the Haskell side, there is still a lot of evolution going on. The most recent work is just few weeks old. I also find it easier to use parallelism in a language with clearly specified evaluation model, but that may be just my personal preference.

2 Being a complete newbie using your F# book (which is excellent so far BTW) I would add that the elegance of the solutions in functional languages (F#, Haskell) are incomparable to the imperative ones. – Markust Mar 31 at 2:09 3 F# doesn't have STM though, right? That is a major load off the brain when writing certain kinds of concurrent code.

– luqui Mar 31 at 2:54 7 I think you're understating the value here. In a functional language it's much more likely that you can start with an idiomatically written program and parallelise it without mangling the code too much, or equivalently that you can write a parallel program that doesn't look too different from the idiomatic sequental one. Furthermore you're much more likely to get it right (fewer hazards to trip you up).

It's not about absolute performance: it's about the tradeoff between performance, effort, maintainability, and robustness. – Simon Marlow Mar 31 at 7:11 2 I suspect that in a language with pervasive mutation, STM is somewhat low level -- you're either locking everything or only able to synchronize a small number of things directly. In a language where most things are immutable, then there's less of an issue -- TVars and TChans provide a higher level interface than otherwise.

– sclv Mar 31 at 11:30 3 @Jon, I'm not arguing that STM is the best solution for the problem in the question I linked. A nice example of where STM hits its domain is if you want a large balanced tree that can be updated in parallel. Anyway, your argument is reminiscent of novice programmers when they learn about closures."OK, but I have never wanted them, and I can't possibly see how they would be useful.

" It's a new abstraction, and you have to get familiar and comfortable with it before uses start jumping out at you. – luqui Mar 317 at 20:44.

There's no objective answer. For Haskell, there's a large body of active work, and no "one size fits all" approach. Instead, in Haskell, many different tools for achieving parallelism are provided.

What's the status of multicore programming in Haskell?

Cilk style multicore programming seems to be the one size that fits all. Consequently, both Intel and Microsoft adopted its design for their TBB and TPL products. – Jon Harrop Apr 10 at 23:23.

Haskell's purity means that it makes a clear distinction between parallel processing and concurrency. If you are looking to speed up your big data-crunching application by distributing the work over multiple cores then you want parallel processing, which means "par" and its derivatives. By careful use of these constructs you can have your CPU-intensive pure function run N times faster on N cores whilst being sure that you haven't changed the meaning of the original code or introduced non-determinism into your program.

On the other hand if you want your program to interact with multiple entities in the outside world, interleaving communications from different entities but still having some degree of shared resources, then you want concurrency, which means using "fork" and some combination of STM and TVars. STM gives you nice transactional semantics, which goes a long way towards eliminating race conditions and other nasties of concurrency. You do need to pay attention to collision frequency and retry rates though.

This answer seems to be born out of a misconception about what parallelism and concurrency are. – Jon Harrop Apr 8 at 22:57 1 @Jon, This answer accurately reflects the accepted definitions of those two terms in the Haskell community. Are the definitions different in other communities?

– luqui Apr 9 at 17:45 2 Yes, there is much confusion about these terms. Don Syme recently pointed out that fetching URLs at the same time (e.g. Using Async. Parallel) is parallelism and not concurrency although it is not CPU bound.

Concurrent garbage collection is an example of a CPU-bound application that is concurrent but not (necessarily) parallel. I particularly liked DMBarbour's statement correcting Simon Marlow on his blog that enumerated some deterministic forms of concurrency such as – Jon Harrop Apr 10 at 20:27 2 temporal logic programming, functional reactive programming, future passing with single assignment variables, Kahn process networks, concurrent constraint programming. And I liked his definitions of parallelism as "an implementation property that says many calculations are happening at the same time" and concurrency as "a semantic property that says many behaviors are happening at the same time".

– Jon Harrop Apr 10 at 20:30.

Functional programming has immutable data structures and no side effect which are inherently suitable for parallel programming. That is a common misconception. Parallelism is solely about performance and purity degrades performance.So purely functional programming is not a good starting point if your objective is to get decent performance.

In general, purity means more allocation and worse locality. In particular, purely functional data structures replace arrays with trees and that incurs many more allocations and places far more burden on the garbage collector. For example, measure the performance of the elegant purely functional "quicksort" in Haskell.

Last I checked, it was thousands of times slower than the conventional imperative solution on my machine. Also, nobody has ever managed to implement an efficient dictionary data structure (pure or impure) or an efficient purely-functional sort in Haskell and nobody has figured out how to write an asymptotically efficient persistent disjoint set data structure and there is no known way to implement other basic data structures like purely functional weak dictionaries! Moreover, although it is theoretically possible to write impure code in Haskell the GC is heavily optimized for pure code at the expense of the performance of mutation.

For example, GHC's hash table is still 26× slower than . NET's. The performance of mutation was considered so unimportant in Haskell that writing a single pointer to an array was an O(n) operation in GHC for five years.

I investigate how to exploit multicore computation in a functional language, and target production code for some numerical applications. The best way I have found is to learn how to write decent parallel programs for multicores in the imperative style (study Cilk, in particular) and then factor the code using first-class functions and tail call elimination into the impure functional style. This means cache oblivious data structures and algorithms.

Nobody has ever done this in Haskell. Indeed, none of the research published on parallel Haskell to-date has even mentioned the essential concept of cache complexity. Furthermore, although it is widely known that non-strict (aka lazy) evaluation renders space consumption unpredictable, it is not yet widely appreciated that this same problem renders scalability wildly unpredictable on multicores.

F# has Microsoft behind its back, and its parallel constructs such as PLINQ, TPL, Async Workflow have been well-documented and shown some potentials. They are well beyond showing potential. Thousands of commercial applications are built upon those industrial-strength foundations.

However, research about parallelism in Haskell is very active at the moment, and it posseses many nice features which haven't supported by F# yet: Why do you assume they are "nice features"? I suggest reading Simon Marlow's latest paper about this: "...a combination of practical experience and investigation has lead us to conclude that this approach is not without drawbacks.In a nutshell, the problem is this: achieving parallelism with par requires that the programmer understand operational properties of the language that are at best implementation-de� Ned (and at worst unde�

Ned). This makes par dif� Cult to use, and pitfalls abound — new users have a high failure rate..." My question is which language should I choose for functional parallelism?

I'd advise against parallel purely functional code for production because it is a complete wild card. Assuming you're happy to sacrifice some purity in order to attain competitive performance, I use and recommend F#.

6 This post sows confusion in a number of ways. I'll only address the first paragraph. First -- it is true that asymptotically optimal persistent versions of certain algorithms are open questions.

However, Haskell is fully capable of using mutable state when required. More to the point, it is not true that purity, as a rule, degrades performance. The partitioning of side effects, and use of immutable data structures as a default improves reasoning and clarity of algorithms, including parallel ones.

Mutation is available when necessary. – sclv Apr 10 at 20:16 6 Yawn. More fear, uncertainty and doubt.

I don't know what you mean by "intercepts every single pointer," and I'd be surprised if you did either. – sclv Apr 10 at 22:18 3 When you speak of a "write barrier" do you mean the one common to generational garbage collectors in general? You've also mischaracterized the recent changes to GHC arrays, which is surprising since you spent years complaining about them -- all the while, while apparently not understanding what the issues were.

Writes were always O(1) -- the issue was that mutation to an array caused the entire thing to be traversed on GC. But while that was bad and well worth fixing, it only caused O(n) behavior in the degenerate case. – sclv Apr 11 at 2:36 4 The partial quote doesn't instill confidence: Marlow goes on to state that using the predefined abstractions doesn't share this difficulty.In any case, even if one dislikes the strategy approach, the paper you cite brings another approach in.

The implication that people who use Haskell don't ship code and just write papers degrades the quality of this answer. – ScottWest Apr 11 at 11:21 5 @Jon I am not getting into the technical side of this issue, only commenting on the rhetorical style employed in the answer, and how it could be improved. I can't take an answer seriously that takes such a tone, it distracts greatly from the question that was originally posed about purely functional parallelism.

– ScottWest Apr 12 at 11:51.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.