Parallel Processing in tidyLPA?

54 views
Skip to first unread message

David J. Hoelzel

unread,
Jun 24, 2024, 10:39:28 AM6/24/24
to tidyLPA
Thanks for this great package! I already used it for a somewhat small sample. Now, however, I would like to apply LPA to a sample of millions of cases which takes a lot of time in R. Would it be possible to implement parallel processing into tidyLPA?

David J. Hoelzel

unread,
Jul 24, 2024, 7:42:47 AM7/24/24
to tidyLPA
Just as a short update: Somebody contacted me via private message and suggested to use the furrr-package which is based on futures (Bengtsson 2021); thanks for this tip! From my understanding of the paper: As there are multiple ways of doing parallelization (which partly depend on the OS), developers are advised to not implement a particular parallelization solution into their packages / functions: "A developer who wishes to support multiple parallel backends has to implement support for each of [the end-user's options among parallel frameworks] individually and provide the end-user with a mechanism to choose between them. This approach often results in unwieldy, hard-to-maintain code of conditional statements with low test coverage" (Bengtsson 2021:274). Therefore, I do not expect the tidyLPA developers to implement parallel processing as requested one month ago. Rather, I'll be looking into finding a way of modifying the tidyLPA-function on my own in the upcoming weeks.
Reply all
Reply to author
Forward
0 new messages