[erlang-questions] reliably figure out hostname

[Tim Watson]

Is there a way to calculate the hostname reliably across platforms in Erlang? I have a non-distributed node that I wish to become a distributed node. Normally I call net_kernel:start([Name, shortnames]) and this is just fine. It also works with [Name, longnames] *sometimes and on some systems* - but other times it pukes. I've tried looking in the 'domain' or 'search' entries from inet:get_rc/0 but these are not always populated, even when dns config is clearly in place. I've also tried using 'inet_db:get_searchlist/0' but again, sometimes this returns [[]] but net_kernel:start([foobar, longnames]) doesn't work, whereas doing net_kernel:start([foo...@mydomain.com, longnames]) does.

Am I missing something incredibly obvious here? *is* there actually a simple way of determining what the proper fqdn for the machine should be, without breaking out to the os? I had even considered doing inet:gethostbyname/1 but again, the search domains entry seems to be empty, so I'd assume that -name foobar will work whereas in fact, -name foobar@fqdn is required otherwise net_kernel won't start. 

[Tim Watson]

Any takers for this? I can't be the only person who's had to figure this out. The context is a distributed systems testing framework that needs to support resolving 'localhost' to a proper host name so the framework can become a hidden node and interact with the other erlang nodes It is testing against. Having to specify the real host name would make configuring the tool across different machines and development environments unpleasantly complicated so figuring out the right hostname is a boon in terms of keeping the configuration overhead down, but doing this for long names nodes is proving highly awkward.

Can anyone suggest a good portable solution?

[Vlad Dumitrescu]

Hi,

On Thu, Aug 2, 2012 at 10:17 AM, Tim Watson <watson....@gmail.com> wrote:
> Any takers for this? I can't be the only person who's had to figure this
> out. The context is a distributed systems testing framework that needs to
> support resolving 'localhost' to a proper host name so the framework can
> become a hidden node and interact with the other erlang nodes It is testing
> against. Having to specify the real host name would make configuring the
> tool across different machines and development environments unpleasantly
> complicated so figuring out the right hostname is a boon in terms of keeping
> the configuration overhead down, but doing this for long names nodes is
> proving highly awkward.

I'm not a guru for this kind of issues, but even looking at Java's
support for this, the docs say

String java.net.InetAddress.getCanonicalHostName()
Gets the fully qualified domain name for this IP address. Best
effort method, meaning we may not be able to return the FQDN depending
on the underlying system configuration.

String java.net.InetAddress.getHostName()
Gets the host name for this IP address.
If this InetAddress was created with a host name, this host
name will be remembered and returned; otherwise, a reverse name lookup
will be performed and the result will be returned based on the system
configured name lookup service. If a lookup of the name service is
required, call getCanonicalHostName.

So it depends on how the system is configured...

regards,
Vlad

> Can anyone suggest a good portable solution?
>
> On 31 Jul 2012, at 13:43, Tim Watson <watson....@gmail.com> wrote:
>
> Is there a way to calculate the hostname reliably across platforms in
> Erlang? I have a non-distributed node that I wish to become a distributed
> node. Normally I call net_kernel:start([Name, shortnames]) and this is just
> fine. It also works with [Name, longnames] *sometimes and on some systems* -
> but other times it pukes. I've tried looking in the 'domain' or 'search'
> entries from inet:get_rc/0 but these are not always populated, even when dns
> config is clearly in place. I've also tried using 'inet_db:get_searchlist/0'
> but again, sometimes this returns [[]] but net_kernel:start([foobar,
> longnames]) doesn't work, whereas doing
> net_kernel:start([foo...@mydomain.com, longnames]) does.
>
> Am I missing something incredibly obvious here? *is* there actually a simple
> way of determining what the proper fqdn for the machine should be, without
> breaking out to the os? I had even considered doing inet:gethostbyname/1 but
> again, the search domains entry seems to be empty, so I'd assume that -name
> foobar will work whereas in fact, -name foobar@fqdn is required otherwise
> net_kernel won't start.
>
>

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Gustav Simonsson]

{ok, LocalHostName} = inet:gethostname().
inet_res:gethostbyname(LocalHostName).

You get a hostent record back which has the FQDN.

// Gustav Simonsson

Sent from my PC

> doing net_kernel:start([ foo...@mydomain.com , longnames]) does.

>
>
> Am I missing something incredibly obvious here? *is* there actually a
> simple way of determining what the proper fqdn for the machine
> should be, without breaking out to the os? I had even considered
> doing inet:gethostbyname/1 but again, the search domains entry seems
> to be empty, so I'd assume that -name foobar will work whereas in

> fact, -name foobar @fqdn is required otherwise net_kernel won't
> start.

[Raimo Niskanen]

On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote:
>
> {ok, LocalHostName} = inet:gethostname().
> inet_res:gethostbyname(LocalHostName).

That forces using the OTP DNS resolver, which may be in conflict
with the current node configuration. The first approach should be:

{ok, LocalHostName} = inet:gethostname().

inet:gethostbyname(LocalHostName).

/ Raimo Niskanen

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB

[Tim Watson]

On 2 Aug 2012, at 09:40, Gustav Simonsson wrote:

>
> {ok, LocalHostName} = inet:gethostname().
> inet_res:gethostbyname(LocalHostName).
>
> You get a hostent record back which has the FQDN.
>

Cool thanks for pointing that out Gustav, I'll give this a try on the various machines we had troubles with.

Cheers,
Tim

[Tim Watson]

On 2 Aug 2012, at 15:31, Raimo Niskanen wrote:

> On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote:
>>
>> {ok, LocalHostName} = inet:gethostname().
>> inet_res:gethostbyname(LocalHostName).
>
> That forces using the OTP DNS resolver, which may be in conflict
> with the current node configuration. The first approach should be:
>
> {ok, LocalHostName} = inet:gethostname().
> inet:gethostbyname(LocalHostName).
>

Thanks both of you, I'll look at both these and see what comes up on the machines that are causing issues. My fallback position is to make the test operator provide a settings file with their FQDN when using longnames configurations, which at least minimises the impact, but still it'd be nice to do this without forcing the user to intervene. I will post back to the list if/when I find out the best approach for my own particular use case.

Cheers!
Tim

[Tim Watson]

On 2 August 2012 15:31, Raimo Niskanen

<raimo+erlan...@erix.ericsson.se> wrote:
> On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote:
>>
>> {ok, LocalHostName} = inet:gethostname().
>> inet_res:gethostbyname(LocalHostName).
>
> That forces using the OTP DNS resolver, which may be in conflict
> with the current node configuration. The first approach should be:
>
> {ok, LocalHostName} = inet:gethostname().
> inet:gethostbyname(LocalHostName).
>

That's clearly not what net_kernel is doing though:

==============================================

t4@frigg:x-test $ erl -name foobar
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.1 (abort with ^G)
(foo...@frigg.eng.mycorp.com)1>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
(v)ersion (k)ill (D)b-tables (d)istribution
a
t4@frigg:x-test $ erl -name foo...@frigg.eng.mycorp.com
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.1 (abort with ^G)
(foo...@frigg.eng.mycorp.com)1>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
(v)ersion (k)ill (D)b-tables (d)istribution
a
t4@frigg:x-test $ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.1 (abort with ^G)
1> {ok, Host} = inet:gethostname().
{ok,"frigg"}
2> inet:gethostbyname(Host).
{ok,{hostent,"frigg",[],inet,4,[{127,0,0,1}]}}
3>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
(v)ersion (k)ill (D)b-tables (d)istribution
a
t4@frigg:x-test $

==============================================

And the problem I've got is that whilst on this machine (CentOS 6.2,
R15B01-64bit) the network configuration is *somehow* set up so that
net_kernal magically picks up the right domain info, this doesn't work
consistently everywhere else. And neither does calling inet_rc, as
Gustav suggested:

==============================================
t4@frigg:x-test $ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.1 (abort with ^G)
1> {ok, LocalHostName} = inet:gethostname().
{ok,"frigg"}
2> inet_res:gethostbyname(LocalHostName).
{error,nxdomain}
3>

=============================================

On some other machines, inet_res:gethostbyname/1 returns {error, timeout}.

[JD Bothma]

I remember having to add something to the hosts file of a recent
ubuntu release before erlang would find out its own hostname properly.
Not sure if that's related. I'll see if I can dig up more info.

Does a machine necessarily have a FQDN? Is this necessarily exactly
one? Perhaps you'll have to work out what erlang does to get some
hostname and then set the appropriate setting (once) on each relevant
machine to make sure it gets what you expect it to.

JD

[Tim Watson]

On 2 August 2012 17:42, JD Bothma <jbo...@gmail.com> wrote:
> I remember having to add something to the hosts file of a recent
> ubuntu release before erlang would find out its own hostname properly.
> Not sure if that's related. I'll see if I can dig up more info.
>

Thanks, I appreciate that.

> Does a machine necessarily have a FQDN? Is this necessarily exactly
> one? Perhaps you'll have to work out what erlang does to get some
> hostname and then set the appropriate setting (once) on each relevant
> machine to make sure it gets what you expect it to.
>

Well, the issue is that the testing framework knows nothing about this
issue, but at the moment it does 2 things:

1. on startup, it uses net_kernel:start/1 to become a distributed erlang node
2. when (optionally) connecting to remote nodes (i.e., test subjects)
it uses net_kernel:hidden_connect_node/1

This all works fine for long and short names, as long as the machine
configuration is correct, but there's another catch. When you specify
the test nodes you're going to start, you can optionally provide them
as [{localhost, [node1, node2]}] etc. In *this* case, localhost must
be converted to the correct machine name, and as I said that works
*fine* for shortnames, but for longnames it appears to depend on the
configuration of the machine.

Perhaps as Vlad suggested, this is an insurmountable problem because
of the varying configurations that could exist in different
environments.

[JD Bothma]

I like insurmountable problems :)

So I know you have various different environments and setups, but I
still think if it's actually set up in a decent way on each machine,
it should work. At the end of the day if you want things to guess
correctly you need to be sure they're really set up correctly - you
didn't say how you checked so you might have it really correct
already.

For us it was that ubuntu didn't give useful FQDNs unless we set it
properly according to
http://www.leonardoborda.com/blog/127-0-1-1-ubuntu-debian/

The bits to take from it are:
"some applications like GNOME expects that the hostname to be resolved
to an ip address with a canonical fully qualified domain name"
(context is that ubuntu uses a hack with FQDN pointing to 127.0.1.1 in
/etc/hosts)

and

"In order to support that software, it is necessary to ensure that the
system hostname can be resolved. Most often this is done by putting a
line in /etc/hosts containing some IP address and the system hostname.
If your system has a permanent IP address then use that; otherwise use
the address 127.0.1.1."

and a bit more help

"To see whether your system hostname can be resolved to an IP address
with a fully qualified domain name, use the hostname –fqdn command.

Also it could be any ip address in the 127.0.0.1/8 address block since
according to the RFC 1700 , 127.0.0.0/8 addresses are reserved for
loopback purposes."

I think this made things work properly when looking up the domain name
for tests that depended on automatically knowing the FQDN of the
machine as suggested by others in the thread, but I might be totally
on the wrong track :)

{ok, Hostname} = inet:gethostname(),
{ok,{hostent,FullHostname,[],inet,_,[_]}} = inet:gethostbyname(Hostname),

What the article about ubuntu suggests I think is that localhost can
still be 127.0.0.1 and you can have dynamic ips and whatever while the
machine has a hardcoded FQDN that it should call itself (e.g. when
sending a message that need's a hostname for return path). If you want
a machine to automatically pick up the hostname pointing to it (a la
hostname.local), I think that's tricky feature of your DHCP or name
server or something, and then the machines have to play along.

Hope that helps !

JD

[Richard O'Keefe]

On 2/08/2012, at 8:40 PM, Gustav Simonsson wrote:

> {ok, LocalHostName} = inet:gethostname().
> inet_res:gethostbyname(LocalHostName).
>
> You get a hostent record back which has the FQDN.

You *may* do so. Then again, you may *not*.

Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [kernel-poll:false]

Eshell V5.6.3 (abort with ^G)
1> {ok, LocalHostName} = inet:gethostname().
{ok,"oucs1251"}
2> inet_res:gethostbyname(LocalHostName).
{error,timeout}

Tested on an intel Core 2 duo Mac running OSX 10.6.8.

[Ulf Wiger]

On 2 Aug 2012, at 22:58, Richard O'Keefe wrote:

> Eshell V5.6.3 (abort with ^G)
> 1> {ok, LocalHostName} = inet:gethostname().
> {ok,"oucs1251"}
> 2> inet_res:gethostbyname(LocalHostName).
> {error,timeout}
>
> Tested on an intel Core 2 duo Mac running OSX 10.6.8.

This works on my Mac:

hostname() ->
case {inet_db:gethostname(),inet_db:res_option(domain)} of
{H,D} when is_list(D), is_list(H),
length(D)> 0, length(H)>0 ->
H ++ "." ++ D;
Other ->
error({hostname, Other})
end.

BR,
Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com

[Geoff Cant]

From (hazy) memory, longnames are configured using
net_adm:localhost(). This differs from the local host name resolution
methods given in the thread in that it uses the contents of
'resolv.conf' (specifically the domain option) to construct the FQDN
from the hostname.

https://github.com/erlang/otp/blob/master/lib/kernel/src/net_adm.erl#L74


The last time I tried to write a reliable distribution based
start/stop/status control script, I'm pretty sure I ended up inspecting
the machine configuration, then manually starting distribution via
something like:

https://github.com/archaelus/egc_examples/blob/master/rescriptmsh#L6

to ensure that I would get the name I wanted. Unfortunately I don't have
permission yet to release the control script that implements all this.

Cheers,
-Geoff

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions

--
Geoff Cant

[Geoff Cant]

Oh hey, that rabbit-hole goes way deeper. My system (dns/inet_db config is setup depending on os:type(), mac os gets generic unix treatment). There's a whole lot going on in inet_config to configure inet_db which affects name lookup. But the bit that figures out the hostname for my machine seems to boil down to:

5> {ok, U} = gen_udp:open(0, []).
{ok,#Port<0.597>}
6> inet:gethostname(U).
{ok,"Balder.local"}.

Which seems to be more or less the result of calling 'man 3 gethostname'.

Bleh.

You could just cheat and make the hostname part of your longname either a) the hostname of the system you're trying to connect to, or b) a local ip address converted to an atom (i.e. '127.0.0.1') if you make a hidden connection. It's a little non-kosher, but generally works.


Erlang distribution is about the only thing I use on a regular basis that expects DNS to be configured such that:
1. Any machine can construct its own hostname.
2. These hostnames can be passed off to any third party, and they can subsequently use that hostname to connect back to the original host.

In most places I've worked, 1 is dicey but fixable, and 2 prompts weird looks from the ops/network team.

Good luck,
-Geoff

[Tim Watson]

On 2 August 2012 22:47, Ulf Wiger <u...@feuerlabs.com> wrote:
>
> This works on my Mac:
>
> hostname() ->
> case {inet_db:gethostname(),inet_db:res_option(domain)} of
> {H,D} when is_list(D), is_list(H),
> length(D)> 0, length(H)>0 ->
> H ++ "." ++ D;
> Other ->
> error({hostname, Other})
> end.

Sweet, thanks Ulf. I'll try that elsewhere, though it'll mean I've got
to carefully audit new OTP releases just in case the APIs change or
move, but I think pinning the framework to "R15.*" with rebar is
probably sufficient to do so for now.

[Tim Watson]

On 2 August 2012 22:58, Geoff Cant <n...@erlang.geek.nz> wrote:
>
> From (hazy) memory, longnames are configured using
> net_adm:localhost(). This differs from the local host name resolution
> methods given in the thread in that it uses the contents of
> 'resolv.conf' (specifically the domain option) to construct the FQDN
> from the hostname.

Now that looks a bit more promising... But what of systems where
resolv.conf isn't used?

>
> The last time I tried to write a reliable distribution based
> start/stop/status control script, I'm pretty sure I ended up inspecting
> the machine configuration, then manually starting distribution via
> something like:
>
> https://github.com/archaelus/egc_examples/blob/master/rescriptmsh#L6
>

Ok that's interesting, I'll play around with it and see how things
behave across unices. Thanks for the pointer!

> to ensure that I would get the name I wanted. Unfortunately I don't have
> permission yet to release the control script that implements all this.
>

Not to worry. My framework starts as a dist node only so it can make
hidden connections if required. By and large, it just run scripts like
the one you're describing to do the actual work of interfacing with
the various nodes in a system under test, but I wanted rpc support
where it made sense (as it did for the initial use case).

Thanks again for pointing out net_adm - will look into that when I get
back from holiday the week after next! :)

[Tim Watson]

On 2 August 2012 23:28, Geoff Cant <n...@erlang.geek.nz> wrote:
>
> You could just cheat and make the hostname part of your longname either a) the hostname of the system you're trying to connect to, or b) a local ip address converted to an atom (i.e. '127.0.0.1') if you make a hidden connection. It's a little non-kosher, but generally works.
>

This has crossed my mind! The gen_udp based approach doesn't fly, as
you noticed too, for the same reasons that calling 'hostname' via the
OS doesn't always tell you whether or not `-name foo` will work
without the FQDN.

I'm not actually sure how using an IP address plays with short versus
longnames - I'll experiment with that, thanks.

>
> Erlang distribution is about the only thing I use on a regular basis that expects DNS to be configured such that:
> 1. Any machine can construct its own hostname.
> 2. These hostnames can be passed off to any third party, and they can subsequently use that hostname to connect back to the original host.
>
> In most places I've worked, 1 is dicey but fixable, and 2 prompts weird looks from the ops/network team.
>

Yep. The main thing I'm using the test framework for completely
depends on distributed Erlang though, so there's little point in
skirting the issue. I'll probably simply force users to enter the full
hostname for longnames and provide a way to override the resolution of
localhost per machine using a config file, so developers don't have to
keep editing the test setup when they check out from source control.

For those using shortnames, none of this seems to really matter.