i using scala parallel collections.
val largelist = list.par.map(x => largecomputation(x)).tolist
it blazing fast, have feeling may run out-of-memory issues if run may "largecomputation" in parallel.
therefore when testing, know how many threads parallel collection using , if-need-be, how can configure number of threads parallel collections.
here piece of scaladoc explain how change task support , wrap inside forkjoinpool
. when instantiate forkjoinpool
pass parameter desired parallelism level:
here way change task support of parallel collection: import scala.collection.parallel._ val pc = mutable.pararray(1, 2, 3) pc.tasksupport = new forkjointasksupport(new scala.concurrent.forkjoin.forkjoinpool(2))
so case
val largelist = list.par largerlist.tasksupport = new forkjointasksupport( new scala.concurrent.forkjoin.forkjoinpool(x) ) largerlist.map(x => largecomputation(x)).tolist
Comments
Post a Comment