question to mix and hex.pm (equivalent to python pip ?)

[Derek Cheung]

Hi there,

I am new elixir user with Python background, I though mix is a tool
similar to Python's pip, wonder any one knows an equivalent to pip
install ?

from an empty directory, pip allows to install a package, then I can
start with python interpreter to play this package on command line, it
doesn't require a requirements.txt file or anything,

$ pip install ipython --user            # I'm install  the ipython
package to user's home dir

With elixir mix, I'm not sure if I can install a package to home dir
to play it in a iex shell ?

➸ mix install httpotion
** (Mix) The task "install" could not be found

➸ mix archive.install httpotion
** (Mix) Expected a local file path or a file URL.
Usage: mix archive.install <path or url>

I think hex.pm is an equivalent to python community's pypi index, however

https://pypi.python.org/pypi/ipython from this page I can use browser
download the source code in a tarball or other format

https://hex.pm/packages/httpotion from this page, if without mix tool
how do I download source code ?


Thanks,

[José Valim]

Welcome Derek!

Mix does not have a command to install packages outside the context of a project. Elixir v1.4 will bring some of that functionality but generally speaking you don't have global packages, they always belong to a project.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/a53caa6f-64e3-4a2a-80a5-487a191c63b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[derek]

> I think hex.pm is an equivalent to python community's pypi index, however

> https://pypi.python.org/pypi/ipython from this page I can use browser download the source code in a tarball or other format

> https://hex.pm/packages/httpotion from this page, if without mix tool how do I download source code ?

Then any developer behind hex.pm can comment here? Why designed this
service not allowing people directly download from web ? someone might
say the github for httpotion is exposed you can download from github
for tarball, I'm not sure is that true for all hex.pm packages ? and
for all historical versions like
https://hex.pm/packages/httpotion/2.2.2 is there always a github
release for that? what if the original github user has deleted his
repo?

> You received this message because you are subscribed to a topic in the
> Google Groups "elixir-lang-talk" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elixir-lang-talk/nm3GaX_tdLw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> elixir-lang-ta...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4JRHTEeVi9ugpQ5i%2BygPUZPeLbY%2BUQB97hjXb9-dQnbzw%40mail.gmail.com.

[José Valim]

Then any developer behind hex.pm can comment here? Why designed this
service not allowing people directly download from web ? someone might
say the github for httpotion is exposed you can download from github
for tarball, I'm not sure is that true for all hex.pm packages ? and
for all historical versions like
https://hex.pm/packages/httpotion/2.2.2 is there always a github
release for that? what if the original github user has deleted his
repo?

Oh, now I understand what you have in mind. I originally thought you wanted to install a package so you can use it globally, anywhere from your system. However, you want to download the package so you can see the source. I think that's a very sensible request, I would recommend you open up a discussion / feature requests here: http://github.com/hexpm/hex_web

[derek]

> Mix does not have a command to install packages outside the context of a project. Elixir v1.4 will bring some of that functionality but generally speaking you don't have global packages, they always belong to a project.

The global package installation is definitely a proven useful feature
from other language community, like:

# this install to user's home directory; then I can play with this
module in python shell

$ pip install ipython --user

# nodejs's npm (Node Package Manager) always install to current
directory ./node_modules/..., and doesn't require current directory to
be a nodejs project directory, similarly play this package from nodejs
REPL
# from outside this directory, I can set NODE_PATH=~/node_modules and
play with it
$ npm install async

With Elixir, there are packages like https://hex.pm/packages/benchee
are generally useful package, but not specific to any particular
project, I want to install it globally, instead of keeping a copy in
each of my elixir projects

The workaround I found so far is:

# Download https://github.com/PragTob/benchee/archive/0.2.0.tar.gz and
extract into a temporary directory, cd into it

# build an archive
➸ mix archive.build
Generated archive "benchee-0.2.0.ez" with MIX_ENV=dev

# create my globally installed packages directory
➸ mkdir -vp .mix/libs
mkdir: created directory '.mix/libs'

# copy the archive or make a link
➸ ln -vnf ~/tmp/benchee-0.2.0/benchee-0.2.0.ez .mix/libs
.mix/libs/benchee-0.2.0.ez => ~/tmp/benchee-0.2.0/benchee-0.2.0.ez

# start the shell with Erlang's ERL_LIBS env variable to extend system
library searching path; here I have freedom to play with this package,
no need to maintain a fake elixir project with mix.exs file ...

➸ ERL_LIBS=~/.mix/libs iex
Erlang/OTP 19 [erts-8.0] [source-6dc93c1] [64-bit] [smp:8:8]
[async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (1.3.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Benchee.run()
** (UndefinedFunctionError) function Benchee.run/0 is undefined or
private. Did you mean one of:

* run/1
* run/2

Benchee.run()
iex(1)> list = Enum.to_list(1..10_000)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,
23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42,
43, 44, 45, 46, 47, 48, 49, 50, ...]
iex(2)> map_fun = fn(i) -> [i, i * i] end
#Function<6.52032458/1 in :erl_eval.expr/5>
iex(3)>
nil
iex(4)> Benchee.run(%{time: 3},
...(4)> [{"flat_map", fn -> Enum.flat_map(list, map_fun) end},
...(4)> {"map.flatten",
...(4)> fn -> list |> Enum.map(map_fun) |> List.flatten end}])
Benchmarking flat_map...
Benchmarking map.flatten...

Name ips average
deviation median
flat_map 99.64 10036.02μs
(±2.24%) 9980.50μs
map.flatten 97.07 10302.08μs
(±8.86%) 9931.00μs

Comparison:
flat_map 99.64
map.flatten 97.07 - 1.03x slower
:ok


This shows it is doable but I hope Elixir can have this feature in mix
out of box ...

[Ben Wilson]

Global package installation is also a constant source of problems when they conflict. That isn't to say that there aren't scenarios where it's handy. However great care needs to be taken when it comes to make code globally available as we don't want to end up in dependency hell.

It's also worth pointing out that as a compiled language Elixir is simply a different beast than Python. To my knowledge it is not possible to consolidate protocols outside of a mix project, and without such consolidation there is literally no point in benchmarking anything. If you have a mix project to consolidate protocols, then you may as well just add your benchmarking library as a dev dependency.

[derek]

On Sat, Jul 9, 2016 at 11:29 AM, Ben Wilson <benwil...@gmail.com> wrote:
> Global package installation is also a constant source of problems when they
> conflict. That isn't to say that there aren't scenarios where it's handy.
> However great care needs to be taken when it comes to make code globally
> available as we don't want to end up in dependency hell.
>
> It's also worth pointing out that as a compiled language Elixir is simply a
> different beast than Python. To my knowledge it is not possible to
> consolidate protocols outside of a mix project, and without such
> consolidation there is literally no point in benchmarking anything. If you
> have a mix project to consolidate protocols, then you may as well just add
> your benchmarking library as a dev dependency.

In the company I'm working for there are still lots of components in
pure Erlang, and those Erlang packages are installed globally under
/usr/lib/erlang/lib/... by a Linux package manager like yum; Managing
packages dependencies is not a big deal here I would say, even without
yum; With switching to Elixir doesn't mean we have to rewrite all
components in Elixir, a consensus here is only when rewriting is
proven to have superior benefits; so here why not give user freedom to
manage global packages? not restricting user have to make a fake
project to evaluate something; here benchee is used as a global
package and doesn't belong to any particular project, don't even want
to add into mix.exs

The current requirement of mix to install a package within a project
is just odd to people with different language background

This is the proposal:

$ mix install benchee --save # install the package to
current project, `--save` option is optinal, inspired from `npm
install`, which does add the new dependency to its package.json; for
mix this can be added into mix.exs
$ mix install benchee --user # install into a hidden
directory under user's home, this works for a user with ERL_LIBS
properly set; this is pip's behavior
$ sudo mix install benchee --global # install globally to where
Elixir or Erlang's system libs (that is /usr/lib/erlang/lib to Erlang
usually)

[OvermindDL1]

Honestly that seems horrifying and prone to breakage to me.  What if different elixir apps depend on different version of benchee?  How would you resolve that when all they would do is just 'use the global'?  Global state in general for package management seems utterly horrifying to me...  (Hence why I love docker for my servers)

[Pedro Medeiros]

I currently like the approach of descriptive dependencies at a mix.exs file or maintain a mix.lock.  In the end of day I think this is the best practice to run your project anywhere in a more descriptive way.

But I can also see the need for people that want to test or debug something or another inside a project by not necessary having to install the deps inside the mix.exs. I find useful to have a way to install globally libs but I believe it can brings problems with dependency versions more frequently.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAJctwx42m%2B%2BmzJVV1F0c1ok7WVaO9iGs5PKVT9uxH%2BYqqKCvgQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Pedro Henrique de Souza Medeiros
----------------------------------
Cel: +55 (61) 9197-0993
Email: pedr...@gmail.com

Beautiful is better than ugly,
Explicit is better than implicit,
Simple is better than complex,
Complex is better than complicated.

The Zen of Python, by Tim Peters

[Ben Wilson]

Global dependencies available in a mix project is gonna send us back to the world of needing some equivalent of `bundle exec blah` when we want to run our project with ONLY the mix deps. Dependencies via mix "just work" at the moment and it's awesome. If there's gonna be globally available dependencies at all I'd much prefer if they weren't usable in a mix project at all. Use cases would mostly be to let you do `Poison.decode!` inside a normal `iex` session or something. Global dependencies intermingled with local dependencies is a recipe for problems that plague dependency management in python and the other languages mentioned as models here.

[Pedro Medeiros]

I also dislike handling with this kind of problem. I work mostly with ruby and python . We had these kind in the past a lot, but in the end prohibit to install global dependencies is not the best solution IMO. Is cool to install and just test a library, show something for someone at work, make a small POC... etc.

On other languages like ruby or python they started putting everything globally, then moved to local solutions (virtualenv or bundler). In Elixir we started with local dependencies then people are discussing if there's the necessity to move to global. I don't believe this kind of problem will occur constantly that way. but that's my 2 cents

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/1e9f934a-d4ce-469e-8030-14b466f8f9b9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[José Valim]

Elixir master already supports installing packages globally but only as escripts. You won't be able to use them directly in your Mix projects.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAJbPmJNvOrus_qhFrVuF2qgDBKYE678e6c%2Bk8iKTATey46kwsg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Christopher Adams]

It might be useful for mix new to accept a list of dependencies to
pre-fill the mix.exs template. Something like:

$ mix new --sup --deps benchee

That could reduce the friction for trying out a new package (of course
you still have to deps.get and compile).

But having mix rewrite an existing mix.exs file (like npm --save does)
feels antithetical to the Elixir philosophy.

Christopher

On 07/14/2016 01:47 AM, José Valim wrote:
> Elixir master already supports installing packages globally but only as
> escripts. You won't be able to use them directly in your Mix projects.
>
> On Wednesday, July 13, 2016, Pedro Medeiros <pedr...@gmail.com

> Complexis better than complicated.
>
> The Zen of Python, byTim Peters

>
> --
> You received this message because you are subscribed to the
> Google Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> elixir-lang-ta...@googlegroups.com

> <javascript:_e(%7B%7D,'cvml','elixir-lang-talk%2Bunsu...@googlegroups.com');>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-talk/1e9f934a-d4ce-469e-8030-14b466f8f9b9%40googlegroups.com
> <https://groups.google.com/d/msgid/elixir-lang-talk/1e9f934a-d4ce-469e-8030-14b466f8f9b9%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Pedro Henrique de Souza Medeiros
> ----------------------------------
> Cel: +55 (61) 9197-0993
> Email: pedr...@gmail.com

> <javascript:_e(%7B%7D,'cvml','pedr...@gmail.com');>
>
> Beautifulis better than ugly,
> Explicitis better than implicit,
> Simpleis better than complex,
> Complexis better than complicated.
>
> The Zen of Python, byTim Peters

>
> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to elixir-lang-ta...@googlegroups.com

> <javascript:_e(%7B%7D,'cvml','elixir-lang-talk%2Bunsu...@googlegroups.com');>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-talk/CAJbPmJNvOrus_qhFrVuF2qgDBKYE678e6c%2Bk8iKTATey46kwsg%40mail.gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-talk/CAJbPmJNvOrus_qhFrVuF2qgDBKYE678e6c%2Bk8iKTATey46kwsg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
>
>

> *José Valim*
> www.plataformatec.com.br <http://www.plataformatec.com.br/>

> Skype: jv.ptec
> Founder and Director of R&D
>

> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elixir-lang-ta...@googlegroups.com

> <mailto:elixir-lang-ta...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4Kp%3DtC3oZQeAvfKba_S5cooiYu07PmnLw6eYU%2BK3AwtGA%40mail.gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4Kp%3DtC3oZQeAvfKba_S5cooiYu07PmnLw6eYU%2BK3AwtGA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[derek]

On Wed, Jul 13, 2016 at 10:47 AM, José Valim
<jose....@plataformatec.com.br> wrote:
> Elixir master already supports installing packages globally but only as
> escripts. You won't be able to use them directly in your Mix projects.

I'm interested how did that globally installing packages work? is it
documented somwhere?
http://elixir-lang.org/docs/master/mix/Mix.html (is there an online
version for `mix help` ? from here it seems only find API references
?)

BTW, I just found from this: (I'm not the only one searching for
global install elixir packages ...)
http://stackoverflow.com/questions/33548079/in-elixir-how-do-i-install-packages-globally

that `mix archive.install` looks like what I'm looking for,

`mix archive.install
https://github.com/phoenixframework/phoenix/releases/download/v1.0.3/phoenix_new-1.0.3.ez`

but I wonder is there a central place maintained all installable ez
files ? is there always a compiled ez file maintained for each phoenix
release ? why I can't see the ez file download link from
https://github.com/phoenixframework/phoenix/releases ?

from this comment I wonder does hex.pm saved a compiled ez copy for
each package's each release ?
https://github.com/hexpm/hex_web/issues/379#issuecomment-234240573

>>>>>>>>>>>>>>>>>>>>

About global packages, I wonder if anyone is using elixir to write
useful CLI tools ? or is phonenix and web server the only selling
point?

the question is: since I am not aware of a json parser in standard
elixir library, if I want to write a CLI tool to test JSON API with
some manipulation, probably I'll need to pull
https://hex.pm/packages/poison as dependency, then what's the best way
to deliver such a tool ?

think about this http tool written in python; I can do pip install to
user's home; then I can run this http tool, all dependencies are
managed automatically

https://pypi.python.org/pypi/httpie

➸ pip install httpie -U --user
Collecting httpie
Downloading httpie-0.9.4-py2.py3-none-any.whl (71kB)
100% |████████████████████████████████| 71kB 1.7MB/s
Collecting Pygments>=1.5 (from httpie)
Downloading Pygments-2.1.3-py2.py3-none-any.whl (755kB)
100% |████████████████████████████████| 757kB 1.5MB/s
Requirement already up-to-date: requests>=2.3.0 in
./.local/lib/python2.7/site-packages (from httpie)
Installing collected packages: Pygments, httpie
Found existing installation: Pygments 2.1.1
Uninstalling Pygments-2.1.1:
Successfully uninstalled Pygments-2.1.1
Successfully installed Pygments-2.1.3 httpie-0.9.4

➸ http
usage: http [--json] [--form] [--pretty {all,colors,format,none}]
[--style STYLE] [--print WHAT] [--headers] [--body] [--verbose]
[--all] [--history-print WHAT] [--stream] [--output FILE]
[--download] [--continue]
[--session SESSION_NAME_OR_PATH | --session-read-only
SESSION_NAME_OR_PATH]
[--auth USER[:PASS]] [--auth-type {basic,digest}]
[--proxy PROTOCOL:PROXY_URL] [--follow]
[--max-redirects MAX_REDIRECTS] [--timeout SECONDS]
[--check-status] [--verify VERIFY]
[--ssl {ssl2.3,tls1,tls1.1,tls1.2}] [--cert CERT]
[--cert-key CERT_KEY] [--ignore-stdin] [--help] [--version]
[--traceback] [--debug]
[METHOD] URL [REQUEST_ITEM [REQUEST_ITEM ...]]
http: error: too few arguments

[Eric Meadows-Jönsson]

On Thu, Jul 28, 2016 at 10:02 AM, derek <den...@gmail.com> wrote:

I'm interested how did that globally installing packages work? is it
documented somwhere?
http://elixir-lang.org/docs/master/mix/Mix.html (is there an online
version for `mix help` ? from here it seems only find API references
?)

Check the docs for escript.install and archive.install, http://elixir-lang.org/docs/master/mix/Mix.Tasks.Escript.Install.html and http://elixir-lang.org/docs/master/mix/Mix.Tasks.Archive.Install.html respectively.

On Thu, Jul 28, 2016 at 10:02 AM, derek <den...@gmail.com> wrote:

but I wonder is there a central place maintained all installable ez
files ? is there always a compiled ez file maintained for each phoenix
release ? why I can't see the ez file download link from
https://github.com/phoenixframework/phoenix/releases ?

 

from this comment I wonder does hex.pm saved a compiled ez copy for
each package's each release ?
https://github.com/hexpm/hex_web/issues/379#issuecomment-234240573

 

No, that wouldn't work because it needs to be built on the local machine because of potential version incompatibilities, hex.pm only stores source code. The install tasks will fetch, build and install it for you.

On Thu, Jul 28, 2016 at 10:02 AM, derek <den...@gmail.com> wrote:

About global packages, I wonder if anyone is using elixir to write
useful CLI tools ? or is phonenix and web server the only selling
point? 

the question is: since I am not aware of a json parser in standard
elixir library, if I want to write a CLI tool to test JSON API with
some manipulation, probably I'll need to pull
https://hex.pm/packages/poison as dependency, then what's the best way
to deliver such a tool ?

That's what escripts are for. You host them on hex.pm and use escript.install to install them.

--

Eric Meadows-Jönsson