r - Why is using %do% loop is using multiple processors? Expected sequential loop -
i'm using foreach , reading on e.g.
- https://www.r-bloggers.com/the-wonders-of-foreach/
- https://www.rdocumentation.org/packages/foreach/versions/1.4.3/topics/foreach
my understanding use %dopar%
parallel processing , %do%
sequential.
as happens having issues %dopar%
, while trying debug changed thought sequential loop using %do%
. happened have terminal open , noticed processors running while ran loop.
is expected?
reproducible example:
library(tidyverse) library(caret) library(foreach) # expected see parallel here because caret , xgb train() xgbfit <- train(species ~ ., data = iris, method = "xgbtree", trcontrol = traincontrol(method = "cv", classprobs = true)) iris_big <- do.call(rbind, replicate(1000, iris, simplify = f)) nr <- nrow(iris_big) n <- 1000 # loop on in chunks of 20 pieces <- split(iris_big, rep(1:ceiling(nr/n), each=n, length.out=nr)) lenp <- length(pieces) # did not expect see parallel processing take place when running block below predictions <- foreach(i = seq_len(lenp)) %do% { # prediction preds <- pieces[[i]] %>% mutate(xgb_prediction = predict(xgbfit, newdata = .)) return(preds) } bah <- do.call(rbind, predictions)
my best guess these processes still running previous runs.
it same when using foreach::registerdoseq()
?
my second guess predict
runs in parallel.
Comments
Post a Comment