What predictionio do or download when running pio build?

[Kim Trang Le]

Dear all,

Does anybody know what predictionio do or download when running pio build?

It took me 45 minutes to build the first algorithm in a cluster. I would like to know what is necessary for build process to make it available in local for a fast build.

Thank you very much.

[Pat Ferrel]

`pio build` took 45 minutes? Did you mean `pio train` took that long? What template?

[Kim Trang Le]

Dear Pat, pio build not pio train, similar product template.

Thank you very much.

[Seongduk Cheon]

Hi Kim

When you run `pio build` for the first time, it takes long time to download all of dependency libraries.

Once it has been completed, It will be faster. Because it uses local cache downloaded before.

Do you mean it will be downloaded every time you run `pio build` ?

- Seongduk

[Kim Trang Le]

Dear Seongduk,

Yes I run pio build, I would like to know dependency libraries to get all of them to my local for fast at the first time as well. 

Thank you very much.

[Kim Trang Le]

I used docker and pio build in a container of clusters. 

Thanks.

[Kim Trang Le]

Hi, I found them

"io.prediction"    %% "core"          % pioVersion.value % "provided",

  "org.apache.spark" %% "spark-core"    % "1.4.1" % "provided",

  "org.apache.spark" %% "spark-mllib"   % "1.4.1" % "provided",

  "org.scalatest"    %% "scalatest"     % "2.2.1" % "test",

  "org.pegdown"      % "pegdown"        % "1.0.2" % "test"

[Kim Trang Le]

Dear all, my question now is can I download all of these dependencies and make them available for predictionio without downloading them via build.sbt

"io.prediction"    %% "core"          % pioVersion.value % "provided",

   "org.apache.spark" %% "spark-core"    % "1.4.1" % "provided",

   "org.apache.spark" %% "spark-mllib"   % "1.4.1" % "provided",

   "org.scalatest"    %% "scalatest"     % "2.2.1" % "test",

   "org.pegdown"      % "pegdown"        % "1.0.2" % "test"

Thank you very much.

[Seongduk Cheon]

As default, ivy home is $HOME/.ivy2

How about this?

1. Change ivy home directory from guest machine to host machine.

2. Mount to the host directory when docker run with `-v /path/to/host/ivy:/path/to/guest/ivy` 

3. pio build --clean --sbt-extra "-Dsbt.ivy.home=/path/to/guest/ivy"

Even If you stop docker, the cache directory in host machine is still alive.

and then you can reuse the ivy directory later.

--

Seongduk

[Kim Trang Le]

Dear Seongduk, thank you very much for your suggestion. We will try it.

Regards.