[PATCH 1/9] Add c-ares (dns query library) as submodule
116 views
Skip to first unread message
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:16 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Note: this adds my fork of the library, containing changes for
virtual IO functions, required for the lib to be really useful to
scylla.
Should be changed to either fork in the scylladb account, or
to main trunk of library once the above changes are accepted (if).
---
.gitmodules | 3 +++
c-ares | 1 +
2 files changed, 4 insertions(+)
create mode 160000 c-ares
diff --git a/.gitmodules b/.gitmodules
index ac15ad4..fd96ea7 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -4,3 +4,6 @@
[submodule "fmt"]
path = fmt
url = ../fmt
+[submodule "c-ares"]
+ path = c-ares
+ url = https://github.com/elcallio/c-ares.git
diff --git a/c-ares b/c-ares
new file mode 160000
index 0000000..9668918
--- /dev/null
+++ b/c-ares
@@ -0,0 +1 @@
+Subproject commit 966891821b4d5ae622d9ab31f38b740f39650106
--
1.9.1
virtual IO functions, required for the lib to be really useful to
scylla.
Should be changed to either fork in the scylladb account, or
to main trunk of library once the above changes are accepted (if).
---
.gitmodules | 3 +++
c-ares | 1 +
2 files changed, 4 insertions(+)
create mode 160000 c-ares
diff --git a/.gitmodules b/.gitmodules
index ac15ad4..fd96ea7 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -4,3 +4,6 @@
[submodule "fmt"]
path = fmt
url = ../fmt
+[submodule "c-ares"]
+ path = c-ares
+ url = https://github.com/elcallio/c-ares.git
diff --git a/c-ares b/c-ares
new file mode 160000
index 0000000..9668918
--- /dev/null
+++ b/c-ares
@@ -0,0 +1 @@
+Subproject commit 966891821b4d5ae622d9ab31f38b740f39650106
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:16 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
C-ares based DNS query module. Based on fork of the library using virtual
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
Note: currently hosts file entries are _not_ handled, since virtualizing
the file io in c-ares is way ickier than the networking. Can either do
that, or do our own hosts file parsing (latter might be easier), or
simply allow for some synchronous disk io in this case.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
Calle Wilund (9):
Add c-ares (dns query library) as submodule
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
configure.py | 54 +++-
net/dns.hh | 131 ++++++++
net/inet_address.hh | 84 +++++
net/packet.hh | 2 +-
net/socket_defs.hh | 6 +
net/dns.cc | 899 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 148 +++++++++
net/posix-stack.cc | 6 +
net/stack.cc | 9 +
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
13 files changed, 1441 insertions(+), 8 deletions(-)
create mode 100644 net/dns.hh
create mode 100644 net/inet_address.hh
create mode 100644 net/dns.cc
create mode 100644 net/inet_address.cc
create mode 100644 tests/dns_test.cc
create mode 160000 c-ares
--
1.9.1
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
Note: currently hosts file entries are _not_ handled, since virtualizing
the file io in c-ares is way ickier than the networking. Can either do
that, or do our own hosts file parsing (latter might be easier), or
simply allow for some synchronous disk io in this case.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
Calle Wilund (9):
Add c-ares (dns query library) as submodule
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
configure.py | 54 +++-
net/dns.hh | 131 ++++++++
net/inet_address.hh | 84 +++++
net/packet.hh | 2 +-
net/socket_defs.hh | 6 +
net/dns.cc | 899 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 148 +++++++++
net/posix-stack.cc | 6 +
net/stack.cc | 9 +
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
13 files changed, 1441 insertions(+), 8 deletions(-)
create mode 100644 net/dns.hh
create mode 100644 net/inet_address.hh
create mode 100644 net/dns.cc
create mode 100644 net/inet_address.cc
create mode 100644 tests/dns_test.cc
create mode 160000 c-ares
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:17 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
diff --git a/configure.py b/configure.py
index 9890cf1..a1be833 100755
--- a/configure.py
+++ b/configure.py
@@ -155,12 +155,14 @@ modes = {
'sanitize_libs': '-lasan -lubsan',
'opt': '-O0 -DDEBUG -DDEBUG_SHARED_PTR -DDEFAULT_ALLOCATOR -DSEASTAR_THREAD_STACK_GUARDS',
'libs': '',
+ 'cares_opts': '--enable-debug',
},
'release': {
'sanitize': '',
'sanitize_libs': '',
'opt': '-O2',
'libs': '',
+ 'cares_opts': '',
},
}
@@ -608,6 +610,25 @@ if args.dpdk:
if file.endswith('.h') or file.endswith('.c')]
dpdk_sources = ' '.join(dpdk_sources)
+# both source and builddir location
+cares_dir = 'c-ares'
+cares_lib = 'cares-seastar'
+cares_src_lib = cares_dir + '/.libs/libcares.a'
+
+if not os.path.exists(cares_dir) or not os.listdir(cares_dir):
+ raise Exception(cares_dir + ' is empty. Run "git submodule update --init".')
+
+if not os.path.exists(cares_dir + '/configure'):
+ subprocess.check_call('sh -x buildconf', cwd=cares_dir, shell=True)
+
+cares_sources = []
+for root, dirs, files in os.walk('c-ares'):
+ cares_sources += [os.path.join(root, file)
+ for file in files
+ if file.endswith('.h') or file.endswith('.c')]
+cares_sources = ' '.join(cares_sources)
+libs += ' -l' + cares_lib
+
outdir = 'build'
buildfile = 'build.ninja'
os.makedirs(outdir, exist_ok = True)
@@ -641,6 +662,8 @@ with open(buildfile, 'w') as f:
rule protobuf
command = protoc --cpp_out=$outdir $in
description = PROTOC $out
+ rule copy_file
+ command = cp $in $out
''').format(**globals()))
if args.dpdk:
f.write(textwrap.dedent('''\
@@ -657,26 +680,36 @@ with open(buildfile, 'w') as f:
elif modeval['sanitize']:
modeval['sanitize'] += ' -DASAN_ENABLED'
f.write(textwrap.dedent('''\
- cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen
+ cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen -I $builddir/{mode}/c-ares
libs_{mode} = {sanitize_libs} {libs}
rule cxx.{mode}
command = $cxx -MMD -MT $out -MF $out.d $cxxflags_{mode} $cxxflags -c -o $out $in
description = CXX $out
depfile = $out.d
rule link.{mode}
- command = $cxx $cxxflags_{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK $out
pool = link_pool
rule link_stripped.{mode}
- command = $cxx $cxxflags_{mode} -s $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -s -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK (stripped) $out
pool = link_pool
rule ar.{mode}
command = rm -f $out; ar cr $out $in; ranlib $out
description = AR $out
''').format(mode = mode, **modeval))
- f.write('build {mode}: phony {artifacts}\n'.format(mode = mode,
+ f.write('build {mode}: phony $builddir/{mode}/lib{cares_lib}.a {artifacts}\n'.format(mode = mode, cares_lib=cares_lib,
artifacts = str.join(' ', ('$builddir/' + mode + '/' + x for x in build_artifacts))))
+ f.write(textwrap.dedent('''\
+ rule caresmake_{mode}
+ command = make -C build/{mode}/{cares_dir}
+ rule caresconfigure_{mode}
+ command = mkdir -p $builddir/{mode}/{cares_dir} && cd $builddir/{mode}/{cares_dir} && {srcdir}/$in {cares_opts}
+ build $builddir/{mode}/{cares_dir}/Makefile : caresconfigure_{mode} {cares_dir}/configure
+ build $builddir/{mode}/{cares_dir}/ares_build.h : phony $builddir/{mode}/{cares_dir}/Makefile
+ build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
+ build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
+ ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
compiles = {}
ragels = {}
swaggers = {}
@@ -718,7 +751,7 @@ with open(buildfile, 'w') as f:
f.write('build $builddir/{}/{}_g: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
f.write(' extralibs = {}\n'.format(' '.join(extralibs)))
else:
- f.write('build $builddir/{}/{}: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
+ f.write('build $builddir/{}/{}: link.{} {} | {} $builddir/{}/lib{}.a\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps, mode, cares_lib))
for src in srcs:
if src.endswith('.cc'):
@@ -756,7 +789,7 @@ with open(buildfile, 'w') as f:
rule configure
command = python3 configure.py $configure_args
generator = 1
- build build.ninja: configure | configure.py
+ build build.ninja: configure | configure.py {cares_dir}/configure
rule cscope
command = find -name '*.[chS]' -o -name "*.cc" -o -name "*.hh" | cscope -bq -i-
description = CSCOPE
--
1.9.1
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
diff --git a/configure.py b/configure.py
index 9890cf1..a1be833 100755
--- a/configure.py
+++ b/configure.py
@@ -155,12 +155,14 @@ modes = {
'sanitize_libs': '-lasan -lubsan',
'opt': '-O0 -DDEBUG -DDEBUG_SHARED_PTR -DDEFAULT_ALLOCATOR -DSEASTAR_THREAD_STACK_GUARDS',
'libs': '',
+ 'cares_opts': '--enable-debug',
},
'release': {
'sanitize': '',
'sanitize_libs': '',
'opt': '-O2',
'libs': '',
+ 'cares_opts': '',
},
}
@@ -608,6 +610,25 @@ if args.dpdk:
if file.endswith('.h') or file.endswith('.c')]
dpdk_sources = ' '.join(dpdk_sources)
+# both source and builddir location
+cares_dir = 'c-ares'
+cares_lib = 'cares-seastar'
+cares_src_lib = cares_dir + '/.libs/libcares.a'
+
+if not os.path.exists(cares_dir) or not os.listdir(cares_dir):
+ raise Exception(cares_dir + ' is empty. Run "git submodule update --init".')
+
+if not os.path.exists(cares_dir + '/configure'):
+ subprocess.check_call('sh -x buildconf', cwd=cares_dir, shell=True)
+
+cares_sources = []
+for root, dirs, files in os.walk('c-ares'):
+ cares_sources += [os.path.join(root, file)
+ for file in files
+ if file.endswith('.h') or file.endswith('.c')]
+cares_sources = ' '.join(cares_sources)
+libs += ' -l' + cares_lib
+
outdir = 'build'
buildfile = 'build.ninja'
os.makedirs(outdir, exist_ok = True)
@@ -641,6 +662,8 @@ with open(buildfile, 'w') as f:
rule protobuf
command = protoc --cpp_out=$outdir $in
description = PROTOC $out
+ rule copy_file
+ command = cp $in $out
''').format(**globals()))
if args.dpdk:
f.write(textwrap.dedent('''\
@@ -657,26 +680,36 @@ with open(buildfile, 'w') as f:
elif modeval['sanitize']:
modeval['sanitize'] += ' -DASAN_ENABLED'
f.write(textwrap.dedent('''\
- cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen
+ cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen -I $builddir/{mode}/c-ares
libs_{mode} = {sanitize_libs} {libs}
rule cxx.{mode}
command = $cxx -MMD -MT $out -MF $out.d $cxxflags_{mode} $cxxflags -c -o $out $in
description = CXX $out
depfile = $out.d
rule link.{mode}
- command = $cxx $cxxflags_{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK $out
pool = link_pool
rule link_stripped.{mode}
- command = $cxx $cxxflags_{mode} -s $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -s -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK (stripped) $out
pool = link_pool
rule ar.{mode}
command = rm -f $out; ar cr $out $in; ranlib $out
description = AR $out
''').format(mode = mode, **modeval))
- f.write('build {mode}: phony {artifacts}\n'.format(mode = mode,
+ f.write('build {mode}: phony $builddir/{mode}/lib{cares_lib}.a {artifacts}\n'.format(mode = mode, cares_lib=cares_lib,
artifacts = str.join(' ', ('$builddir/' + mode + '/' + x for x in build_artifacts))))
+ f.write(textwrap.dedent('''\
+ rule caresmake_{mode}
+ command = make -C build/{mode}/{cares_dir}
+ rule caresconfigure_{mode}
+ command = mkdir -p $builddir/{mode}/{cares_dir} && cd $builddir/{mode}/{cares_dir} && {srcdir}/$in {cares_opts}
+ build $builddir/{mode}/{cares_dir}/Makefile : caresconfigure_{mode} {cares_dir}/configure
+ build $builddir/{mode}/{cares_dir}/ares_build.h : phony $builddir/{mode}/{cares_dir}/Makefile
+ build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
+ build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
+ ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
compiles = {}
ragels = {}
swaggers = {}
@@ -718,7 +751,7 @@ with open(buildfile, 'w') as f:
f.write('build $builddir/{}/{}_g: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
f.write(' extralibs = {}\n'.format(' '.join(extralibs)))
else:
- f.write('build $builddir/{}/{}: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
+ f.write('build $builddir/{}/{}: link.{} {} | {} $builddir/{}/lib{}.a\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps, mode, cares_lib))
for src in srcs:
if src.endswith('.cc'):
@@ -756,7 +789,7 @@ with open(buildfile, 'w') as f:
rule configure
command = python3 configure.py $configure_args
generator = 1
- build build.ninja: configure | configure.py
+ build build.ninja: configure | configure.py {cares_dir}/configure
rule cscope
command = find -name '*.[chS]' -o -name "*.cc" -o -name "*.hh" | cscope -bq -i-
description = CSCOPE
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:18 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/packet.hh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/packet.hh b/net/packet.hh
index bd0b32e..f6f4db0 100644
--- a/net/packet.hh
+++ b/net/packet.hh
@@ -381,7 +381,7 @@ class packet final {
packet::packet(Iterator begin, Iterator end, deleter del) {
unsigned nr_frags = 0, len = 0;
nr_frags = std::distance(begin, end);
- std::for_each(begin, end, [&] (fragment& frag) { len += frag.size; });
+ std::for_each(begin, end, [&] (const fragment& frag) { len += frag.size; });
_impl = impl::allocate(nr_frags);
_impl->_deleter = std::move(del);
_impl->_len = len;
--
1.9.1
net/packet.hh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/packet.hh b/net/packet.hh
index bd0b32e..f6f4db0 100644
--- a/net/packet.hh
+++ b/net/packet.hh
@@ -381,7 +381,7 @@ class packet final {
packet::packet(Iterator begin, Iterator end, deleter del) {
unsigned nr_frags = 0, len = 0;
nr_frags = std::distance(begin, end);
- std::for_each(begin, end, [&] (fragment& frag) { len += frag.size; });
+ std::for_each(begin, end, [&] (const fragment& frag) { len += frag.size; });
_impl = impl::allocate(nr_frags);
_impl->_deleter = std::move(del);
_impl->_len = len;
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:18 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 1 +
net/inet_address.hh | 76 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 184 insertions(+)
create mode 100644 net/inet_address.hh
create mode 100644 net/inet_address.cc
diff --git a/configure.py b/configure.py
index a1be833..c091318 100755
--- a/configure.py
+++ b/configure.py
@@ -268,6 +268,7 @@ libnet = [
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
new file mode 100644
index 0000000..d19ad50
--- /dev/null
+++ b/net/inet_address.hh
@@ -0,0 +1,76 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+#pragma once
+
+#include <iosfwd>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <stdexcept>
+#include <vector>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+
+namespace seastar {
+namespace net {
+
+class unknown_host : public std::invalid_argument {
+public:
+ using invalid_argument::invalid_argument;
+};
+
+struct inet_address {
+ enum class family {
+ INET = AF_INET, INET6 = AF_INET6
+ };
+
+ family in_family;
+
+ union {
+ ::in_addr in;
+ ::in6_addr in6;
+ };
+
+
+ inet_address();
+ inet_address(::in_addr i);
+ inet_address(::in6_addr i);
+ // NOTE: does _not_ resolve the address. Only parses
+ // ipv4/ipv6 numerical address
+ inet_address(const sstring&);
+ inet_address(inet_address&&) = default;
+ inet_address(const inet_address&) = default;
+
+ inet_address& operator=(const inet_address&) = default;
+ bool operator==(const inet_address&) const;
+
+ size_t size() const;
+
+ operator const ::in_addr&() const;
+ operator const ::in6_addr&() const;
+};
+
+std::ostream& operator<<(std::ostream&, const inet_address&);
+std::ostream& operator<<(std::ostream&, const inet_address::family&);
+
+}
+}
diff --git a/net/inet_address.cc b/net/inet_address.cc
new file mode 100644
index 0000000..89d2770
--- /dev/null
+++ b/net/inet_address.cc
@@ -0,0 +1,107 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+#include <ostream>
+#include <arpa/inet.h>
+
+#include "inet_address.hh"
+
+seastar::net::inet_address::inet_address()
+ : in_family(family::INET) {
+ std::fill(std::begin(in6.s6_addr), std::end(in6.s6_addr), 0);
+}
+
+seastar::net::inet_address::inet_address(::in_addr i)
+ : in_family(family::INET), in(i) {
+}
+
+seastar::net::inet_address::inet_address(::in6_addr i)
+ : in_family(family::INET6), in6(i) {
+}
+
+seastar::net::inet_address::inet_address(const sstring& addr)
+ : inet_address([&addr] {
+ inet_address in;
+ if (::inet_pton(AF_INET, addr.c_str(), &in.in)) {
+ in.in_family = family::INET;
+ return in;
+ }
+ if (::inet_pton(AF_INET6, addr.c_str(), &in.in6)) {
+ in.in_family = family::INET6;
+ return in;
+ }
+ throw std::invalid_argument(addr);
+}())
+{}
+
+bool seastar::net::inet_address::operator==(const inet_address& o) const {
+ if (o.in_family != in_family) {
+ return false;
+ }
+ switch (in_family) {
+ case family::INET:
+ return in.s_addr == o.in.s_addr;
+ case family::INET6:
+ return std::equal(std::begin(in6.s6_addr), std::end(in6.s6_addr),
+ std::begin(o.in6.s6_addr));
+ default:
+ return false;
+ }
+}
+
+seastar::net::inet_address::operator const ::in_addr&() const {
+ return in;
+}
+
+seastar::net::inet_address::operator const ::in6_addr&() const {
+ return in6;
+}
+
+size_t seastar::net::inet_address::size() const {
+ switch (in_family) {
+ case family::INET:
+ return sizeof(in_addr);
+ case family::INET6:
+ return sizeof(in6_addr);
+ default:
+ return 0;
+ }
+}
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
+ char buffer[64];
+ return os << inet_ntop(int(addr.in_family), &addr.in, buffer, sizeof(buffer));
+}
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address::family& f) {
+ switch (f) {
+ case inet_address::family::INET:
+ os << "INET";
+ break;
+ case inet_address::family::INET6:
+ os << "INET6";
+ break;
+ default:
+ break;
+ }
+ return os;
+}
+
--
1.9.1
configure.py | 1 +
net/inet_address.hh | 76 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 184 insertions(+)
create mode 100644 net/inet_address.hh
create mode 100644 net/inet_address.cc
diff --git a/configure.py b/configure.py
index a1be833..c091318 100755
--- a/configure.py
+++ b/configure.py
@@ -268,6 +268,7 @@ libnet = [
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
new file mode 100644
index 0000000..d19ad50
--- /dev/null
+++ b/net/inet_address.hh
@@ -0,0 +1,76 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+#pragma once
+
+#include <iosfwd>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <stdexcept>
+#include <vector>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+
+namespace seastar {
+namespace net {
+
+class unknown_host : public std::invalid_argument {
+public:
+ using invalid_argument::invalid_argument;
+};
+
+struct inet_address {
+ enum class family {
+ INET = AF_INET, INET6 = AF_INET6
+ };
+
+ family in_family;
+
+ union {
+ ::in_addr in;
+ ::in6_addr in6;
+ };
+
+
+ inet_address();
+ inet_address(::in_addr i);
+ inet_address(::in6_addr i);
+ // NOTE: does _not_ resolve the address. Only parses
+ // ipv4/ipv6 numerical address
+ inet_address(const sstring&);
+ inet_address(inet_address&&) = default;
+ inet_address(const inet_address&) = default;
+
+ inet_address& operator=(const inet_address&) = default;
+ bool operator==(const inet_address&) const;
+
+ size_t size() const;
+
+ operator const ::in_addr&() const;
+ operator const ::in6_addr&() const;
+};
+
+std::ostream& operator<<(std::ostream&, const inet_address&);
+std::ostream& operator<<(std::ostream&, const inet_address::family&);
+
+}
+}
diff --git a/net/inet_address.cc b/net/inet_address.cc
new file mode 100644
index 0000000..89d2770
--- /dev/null
+++ b/net/inet_address.cc
@@ -0,0 +1,107 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+#include <ostream>
+#include <arpa/inet.h>
+
+#include "inet_address.hh"
+
+seastar::net::inet_address::inet_address()
+ : in_family(family::INET) {
+ std::fill(std::begin(in6.s6_addr), std::end(in6.s6_addr), 0);
+}
+
+seastar::net::inet_address::inet_address(::in_addr i)
+ : in_family(family::INET), in(i) {
+}
+
+seastar::net::inet_address::inet_address(::in6_addr i)
+ : in_family(family::INET6), in6(i) {
+}
+
+seastar::net::inet_address::inet_address(const sstring& addr)
+ : inet_address([&addr] {
+ inet_address in;
+ if (::inet_pton(AF_INET, addr.c_str(), &in.in)) {
+ in.in_family = family::INET;
+ return in;
+ }
+ if (::inet_pton(AF_INET6, addr.c_str(), &in.in6)) {
+ in.in_family = family::INET6;
+ return in;
+ }
+ throw std::invalid_argument(addr);
+}())
+{}
+
+bool seastar::net::inet_address::operator==(const inet_address& o) const {
+ if (o.in_family != in_family) {
+ return false;
+ }
+ switch (in_family) {
+ case family::INET:
+ return in.s_addr == o.in.s_addr;
+ case family::INET6:
+ return std::equal(std::begin(in6.s6_addr), std::end(in6.s6_addr),
+ std::begin(o.in6.s6_addr));
+ default:
+ return false;
+ }
+}
+
+seastar::net::inet_address::operator const ::in_addr&() const {
+ return in;
+}
+
+seastar::net::inet_address::operator const ::in6_addr&() const {
+ return in6;
+}
+
+size_t seastar::net::inet_address::size() const {
+ switch (in_family) {
+ case family::INET:
+ return sizeof(in_addr);
+ case family::INET6:
+ return sizeof(in6_addr);
+ default:
+ return 0;
+ }
+}
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
+ char buffer[64];
+ return os << inet_ntop(int(addr.in_family), &addr.in, buffer, sizeof(buffer));
+}
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address::family& f) {
+ switch (f) {
+ case inet_address::family::INET:
+ os << "INET";
+ break;
+ case inet_address::family::INET6:
+ os << "INET6";
+ break;
+ default:
+ break;
+ }
+ return os;
+}
+
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:20 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Otherwise there is no way to get them to wake up.
---
net/udp.cc | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/udp.cc b/net/udp.cc
index 598ba71..8fedf31 100644
--- a/net/udp.cc
+++ b/net/udp.cc
@@ -108,6 +108,7 @@ class native_channel : public udp_channel_impl {
}
virtual void close() override {
+ _state->_queue.abort(std::make_exception_ptr(std::system_error(EBADF, std::system_category())));
_reg.unregister();
_closed = true;
}
--
1.9.1
---
net/udp.cc | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/udp.cc b/net/udp.cc
index 598ba71..8fedf31 100644
--- a/net/udp.cc
+++ b/net/udp.cc
@@ -108,6 +108,7 @@ class native_channel : public udp_channel_impl {
}
virtual void close() override {
+ _state->_queue.abort(std::make_exception_ptr(std::system_error(EBADF, std::system_category())));
_reg.unregister();
_closed = true;
}
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:20 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
diff --git a/net/inet_address.cc b/net/inet_address.cc
index 89d2770..f0cec76 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -23,6 +23,7 @@
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
seastar::net::inet_address::inet_address()
: in_family(family::INET) {
@@ -105,3 +106,8 @@ size_t seastar::net::inet_address::size() const {
return os;
}
+std::ostream& operator<<(std::ostream& os, const socket_address& a) {
+ return os << seastar::net::inet_address(a.as_posix_sockaddr_in().sin_addr)
+ << ":" << a.u.in.sin_port
+ ;
+}
diff --git a/net/stack.cc b/net/stack.cc
index 5ef67e8..d7c0a6b 100644
--- a/net/stack.cc
+++ b/net/stack.cc
@@ -148,3 +148,12 @@ void server_socket::abort_accept() {
: socket_address(make_ipv4_address(addr))
{}
+
+bool socket_address::operator==(const socket_address& a) const {
+ // TODO: handle ipv6
+ return std::tie(u.in.sin_family, u.in.sin_port, u.in.sin_addr.s_addr)
+ == std::tie(a.u.in.sin_family, a.u.in.sin_port,
+ a.u.in.sin_addr.s_addr);
+}
+
+
--
1.9.1
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
* Copyright (C) 2016 ScyllaDB.
*/
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
diff --git a/net/inet_address.cc b/net/inet_address.cc
index 89d2770..f0cec76 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -23,6 +23,7 @@
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
seastar::net::inet_address::inet_address()
: in_family(family::INET) {
@@ -105,3 +106,8 @@ size_t seastar::net::inet_address::size() const {
return os;
}
+std::ostream& operator<<(std::ostream& os, const socket_address& a) {
+ return os << seastar::net::inet_address(a.as_posix_sockaddr_in().sin_addr)
+ << ":" << a.u.in.sin_port
+ ;
+}
diff --git a/net/stack.cc b/net/stack.cc
index 5ef67e8..d7c0a6b 100644
--- a/net/stack.cc
+++ b/net/stack.cc
@@ -148,3 +148,12 @@ void server_socket::abort_accept() {
: socket_address(make_ipv4_address(addr))
{}
+
+bool socket_address::operator==(const socket_address& a) const {
+ // TODO: handle ipv6
+ return std::tie(u.in.sin_family, u.in.sin_port, u.in.sin_addr.s_addr)
+ == std::tie(a.u.in.sin_family, a.u.in.sin_port,
+ a.u.in.sin_addr.s_addr);
+}
+
+
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:21 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Or they shall sleep forever more
---
net/posix-stack.cc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/net/posix-stack.cc b/net/posix-stack.cc
index cd02c39..64879a3 100644
--- a/net/posix-stack.cc
+++ b/net/posix-stack.cc
@@ -433,6 +433,9 @@ class posix_udp_channel : public udp_channel_impl {
virtual future<> send(ipv4_addr dst, packet p);
virtual void close() override {
_closed = true;
+ auto p = std::make_exception_ptr(std::system_error(EBADF, std::system_category()));
+ _fd->abort_reader(p);
+ _fd->abort_writer(p);
_fd.reset();
}
virtual bool is_closed() const override { return _closed; }
@@ -476,6 +479,9 @@ class posix_datagram : public udp_datagram_impl {
auto dst = ipv4_addr(_recv._cmsg.pktinfo.ipi_addr.s_addr, _address.port);
return make_ready_future<udp_datagram>(udp_datagram(std::make_unique<posix_datagram>(
_recv._src_addr, dst, packet(fragment{_recv._buffer, size}, make_deleter([buf = _recv._buffer] { delete[] buf; })))));
+ }).handle_exception([p = _recv._buffer](auto ep) {
+ delete[] p;
+ return make_exception_future<udp_datagram>(std::move(ep));
});
}
--
1.9.1
---
net/posix-stack.cc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/net/posix-stack.cc b/net/posix-stack.cc
index cd02c39..64879a3 100644
--- a/net/posix-stack.cc
+++ b/net/posix-stack.cc
@@ -433,6 +433,9 @@ class posix_udp_channel : public udp_channel_impl {
virtual future<> send(ipv4_addr dst, packet p);
virtual void close() override {
_closed = true;
+ auto p = std::make_exception_ptr(std::system_error(EBADF, std::system_category()));
+ _fd->abort_reader(p);
+ _fd->abort_writer(p);
_fd.reset();
}
virtual bool is_closed() const override { return _closed; }
@@ -476,6 +479,9 @@ class posix_datagram : public udp_datagram_impl {
auto dst = ipv4_addr(_recv._cmsg.pktinfo.ipi_addr.s_addr, _address.port);
return make_ready_future<udp_datagram>(udp_datagram(std::make_unique<posix_datagram>(
_recv._src_addr, dst, packet(fragment{_recv._buffer, size}, make_deleter([buf = _recv._buffer] { delete[] buf; })))));
+ }).handle_exception([p = _recv._buffer](auto ep) {
+ delete[] p;
+ return make_exception_future<udp_datagram>(std::move(ep));
});
}
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:22 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Adds seastar::net::dns_resolver with methods to query names/addresses.
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 899 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1142 insertions(+), 1 deletion(-)
create mode 100644 net/dns.hh
create mode 100644 net/dns.cc
create mode 100644 tests/dns_test.cc
diff --git a/configure.py b/configure.py
index c091318..6742554 100755
--- a/configure.py
+++ b/configure.py
@@ -209,6 +209,7 @@ tests = [
'tests/scollectd_test',
'tests/perf/perf_fstream',
'tests/json_formatter_test',
+ 'tests/dns_test',
]
apps = [
@@ -269,6 +270,7 @@ libnet = [
'net/dhcp.cc',
'net/tls.cc',
'net/inet_address.cc',
+ 'net/dns.cc',
]
core = [
@@ -428,6 +430,7 @@ deps = {
'tests/scollectd_test': ['tests/scollectd_test.cc'] + core,
'tests/perf/perf_fstream': ['tests/perf/perf_fstream.cc'] + core,
'tests/json_formatter_test': ['tests/json_formatter_test.cc'] + core + http,
+ 'tests/dns_test': ['tests/dns_test.cc'] + core + libnet,
}
boost_tests = [
@@ -447,6 +450,7 @@ boost_tests = [
'tests/connect_test',
'tests/scollectd_test',
'tests/json_formatter_test',
+ 'tests/dns_test',
]
for bt in boost_tests:
@@ -673,6 +677,7 @@ with open(buildfile, 'w') as f:
build {dpdk_deps} : dpdkmake {dpdk_sources}
''').format(**globals()))
for mode in build_modes:
+ objdeps = {}
modeval = modes[mode]
if modeval['sanitize'] and not do_sanitize:
print('Note: --static disables debug mode sanitizers')
@@ -711,6 +716,7 @@ with open(buildfile, 'w') as f:
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
new file mode 100644
index 0000000..74b9020
--- /dev/null
+++ b/net/dns.hh
@@ -0,0 +1,131 @@
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+ * A c++-esque version of a hostent
+ */
+struct hostent {
+ // Primary name is always first
+ std::vector<sstring> names;
+ // Primary address is also always first.
+ std::vector<inet_address> addr_list;
+};
+
+typedef std::experimental::optional<inet_address::family> opt_family;
+
+/**
+ * A DNS resolver object.
+ * Wraps the query logic & networking.
+ * Can be instantiated with options and your network
+ * stack of choice, though for "normal" non-test
+ * querying, you are probably better of with the
+ * global calls further down.
+ */
+class dns_resolver {
+public:
+ struct options {
+ std::experimental::optional<bool>
+ use_tcp_query;
+ std::experimental::optional<std::vector<inet_address>>
+ servers;
+ std::experimental::optional<std::chrono::milliseconds>
+ timeout;
+ std::experimental::optional<uint16_t>
+ tcp_port, udp_port;
+ std::experimental::optional<std::vector<sstring>>
+ domains;
+ };
+
+ dns_resolver();
+ dns_resolver(dns_resolver&&);
+ dns_resolver(const options&);
+ dns_resolver(::network_stack&, const options& = {});
+ ~dns_resolver();
+
+ dns_resolver& operator=(dns_resolver&&);
+
+ /**
+ * Resolves a hostname to one or more addresses and aliases
+ */
+ future<hostent> get_host_by_name(const sstring&, opt_family = {});
+ /**
+ * Resolves an address to one or more addresses and aliases
+ */
+ future<hostent> get_host_by_addr(const inet_address&);
+
+ /**
+ * Resolves a hostname to one (primary) address
+ */
+ future<inet_address> resolve_name(const sstring&, opt_family = {});
+ /**
+ * Resolves an address to one (primary) name
+ */
+ future<sstring> resolve_addr(const inet_address&);
+
+ /**
+ * Shuts the object down. Great for tests.
+ */
+ future<> close();
+private:
+ class impl;
+ ::shared_ptr<impl> _impl;
+};
+
+namespace dns {
+
+// See above. These functions simply queries using a shard-local
+// default-stack, default-opts resolver
+future<hostent> get_host_by_name(const sstring&, opt_family = {});
+future<hostent> get_host_by_addr(const inet_address&);
+
+future<inet_address> resolve_name(const sstring&, opt_family = {});
+future<sstring> resolve_addr(const inet_address&);
+
+}
+
+}
+}
diff --git a/net/dns.cc b/net/dns.cc
new file mode 100644
index 0000000..5c7bd10
--- /dev/null
+++ b/net/dns.cc
@@ -0,0 +1,899 @@
+ */
+
+#include <chrono>
+#include <experimental/string_view>
+
+#include <c-ares/ares.h>
+
+#include "ip.hh"
+#include "api.hh"
+#include "dns.hh"
+#include "core/sstring.hh"
+#include "core/timer.hh"
+#include "core/reactor.hh"
+#include "core/gate.hh"
+#include "util/log.hh"
+
+static seastar::logger dns_log("dns_resolver");
+
+class ares_error_category : public std::error_category {
+public:
+ constexpr ares_error_category() noexcept : std::error_category{} {}
+ const char * name() const noexcept {
+ return "C-Ares";
+ }
+ std::string message(int error) const {
+ switch (error) {
+ /* Server error codes (ARES_ENODATA indicates no relevant answer) */
+ case ARES_ENODATA: return "No data";
+ case ARES_EFORMERR: return "Form error";
+ case ARES_ESERVFAIL: return "Server failure";
+ case ARES_ENOTFOUND: return "Not found";
+ case ARES_ENOTIMP: return "Not implemented";
+ case ARES_EREFUSED: return "Refused";
+
+ /* Locally generated error codes */
+ case ARES_EBADQUERY: return "Bad query";
+ case ARES_EBADNAME: return "Bad name";
+ case ARES_EBADFAMILY: return "Bad family";
+ case ARES_EBADRESP: return "Bad response";
+ case ARES_ECONNREFUSED :return "Connection refused";
+ case ARES_ETIMEOUT: return "Timeout";
+ case ARES_EOF: return "EOF";
+ case ARES_EFILE: return "File error";
+ case ARES_ENOMEM: return "No memory";
+ case ARES_EDESTRUCTION: return "Destruction";
+ case ARES_EBADSTR: return "Bad string";
+
+ /* ares_getnameinfo error codes */
+ case ARES_EBADFLAGS: return "Invalid flags";
+
+ /* ares_getaddrinfo error codes */
+ case ARES_ENONAME: return "No name";
+ case ARES_EBADHINTS: return "Bad hints";
+
+ /* Uninitialized library error code */
+ case ARES_ENOTINITIALIZED: return "Not initialized";
+
+ /* ares_library_init error codes */
+ case ARES_ELOADIPHLPAPI: return "Load PHLPAPI";
+ case ARES_EADDRGETNETWORKPARAMS: return "Get network parameters";
+
+ /* More error codes */
+ case ARES_ECANCELLED: return "Cancelled";
+ default:
+ return "Unknown error";
+ }
+ }
+};
+
+static const ares_error_category ares_errorc;
+
+static void check_ares_error(int error) {
+ if (error != ARES_SUCCESS) {
+ throw std::system_error(error, ares_errorc);
+ }
+}
+
+struct ares_initializer {
+ ares_initializer() {
+ check_ares_error(ares_library_init(0));
+ }
+ ~ares_initializer() {
+ ares_library_cleanup();
+ }
+};
+
+class seastar::net::dns_resolver::impl
+ : public enable_shared_from_this<impl>
+{
+public:
+ impl(::network_stack& stack, const options& opts)
+ : _stack(stack)
+ , _timeout(opts.timeout ? *opts.timeout : std::chrono::milliseconds(5000) /* from ares private */)
+ , _timer(std::bind(&impl::poll_sockets, this))
+ {
+ static const ares_initializer a_init;
+
+ // this can "block" ever so slightly, because it will
+ // look in resolv.conf etc for query setup. We could
+ // do this ourselves, and instead set ares options
+ // here, but it seems more error prone (me parsing
+ // resolv.conf -> hah!)
+ ares_options a_opts = { 0, };
+ // TODO:
+ // We only allow querying dns server. We ignore hosts
+ // files. Either we need to add virtual file calls
+ // to c-ares (huger!)
+ // or we do our read+(partial)parse outselved for
+ // hosts files.
+ // Big downside here right now that we can essentially
+ // get different answers on a hsotfile based system
+ char buf[2] = "b";
+ a_opts.lookups = buf; // only net
+ // Alsways set the timeout
+ a_opts.timeout = _timeout.count();
+ int flags = ARES_OPT_LOOKUPS|ARES_OPT_TIMEOUTMS;
+
+ if (opts.use_tcp_query && *opts.use_tcp_query) {
+ a_opts.flags = ARES_FLAG_USEVC | ARES_FLAG_PRIMARY;
+ flags |= ARES_OPT_FLAGS;
+ }
+ std::vector<in_addr> addr_tmp;
+ if (opts.servers) {
+ std::transform(opts.servers->begin(), opts.servers->end(), std::back_inserter(addr_tmp), [](const inet_address& a) {
+ if (a.in_family != inet_address::family::INET) {
+ throw std::invalid_argument("Servers must be ipv4 addresses");
+ }
+ in_addr in = a;
+ return in;
+ });
+ a_opts.servers = addr_tmp.data();
+ a_opts.nservers = int(addr_tmp.size());
+ flags |= ARES_OPT_SERVERS;
+ }
+ std::vector<const char *> dom_tmp;
+ if (opts.domains) {
+ std::transform(opts.domains->begin(), opts.domains->end(), std::back_inserter(dom_tmp), [](const sstring& s) {
+ return s.data();
+ });
+ a_opts.domains = const_cast<char **>(dom_tmp.data());
+ a_opts.ndomains = int(dom_tmp.size());
+ flags |= ARES_OPT_DOMAINS;
+ }
+ if (opts.tcp_port) {
+ a_opts.tcp_port = *opts.tcp_port;
+ flags |= ARES_OPT_TCP_PORT;
+ }
+ if (opts.udp_port) {
+ a_opts.udp_port = *opts.udp_port;
+ flags |= ARES_OPT_UDP_PORT;
+ }
+
+ check_ares_error(ares_init_options(&_channel, &a_opts, flags));
+
+ static auto get_impl = [](void * p) { return reinterpret_cast<impl *>(p); };
+ static const ares_socket_functions callbacks = {
+ [](int af, int type, int protocol, void * p) { return get_impl(p)->do_socket(af, type, protocol); },
+ [](ares_socket_t s, void * p) { return get_impl(p)->do_close(s); },
+ [](ares_socket_t s, const struct sockaddr * addr, socklen_t len, void * p) { return get_impl(p)->do_connect(s, addr, len); },
+ [](ares_socket_t s, void * dst, size_t len, int flags, struct sockaddr * addr, socklen_t * alen, void * p) {
+ return get_impl(p)->do_recvfrom(s, dst, len, flags, addr, alen);
+ },
+ [](ares_socket_t s, const struct iovec * vec, int len, void * p) {
+ return get_impl(p)->do_sendv(s, vec, len);
+ },
+ };
+
+ ares_set_socket_functions(_channel, &callbacks, this);
+
+ // just in case you need printf-debug.
+ //dns_log.set_level(seastar::log_level::trace);
+ }
+ ~impl() {
+ _timer.cancel();
+ if (_channel) {
+ ares_destroy(_channel);
+ }
+ }
+
+ future<inet_address> resolve_name(sstring name, inet_address::family family) {
+ return get_host_by_name(std::move(name), family).then([](hostent h) {
+ return make_ready_future<inet_address>(h.addr_list.front());
+ });
+ }
+
+ future<hostent> get_host_by_name(sstring name, inet_address::family family) {
+ auto p = new promise<hostent>();
+ auto f = p->get_future();
+
+ dns_log.debug("Query name {} ({})", name, family);
+
+ dns_call call(*this);
+
+ ares_gethostbyname(_channel, name.c_str(), int(family), [](void* arg, int status, int timeouts, ::hostent* host) {
+ auto p = reinterpret_cast<promise<hostent> *>(arg);
+
+ switch (status) {
+ default:
+ dns_log.debug("Query failed: {}", status);
+ p->set_exception(std::system_error(status, ares_errorc));
+ break;
+ case ARES_SUCCESS:
+ p->set_value(make_hostent(*host));
+ break;
+ }
+
+ delete p;
+ }, reinterpret_cast<void *>(p));
+
+
+ poll_sockets();
+
+ return f.finally([this] {
+ end_call();
+ });
+ }
+
+ future<hostent> get_host_by_addr(inet_address addr) {
+ auto p = new promise<hostent>();
+ auto f = p->get_future();
+
+ dns_log.debug("Query addr {}", addr);
+
+ dns_call call(*this);
+
+ ares_gethostbyaddr(_channel, &addr.in, addr.size(), int(addr.in_family), [](void* arg, int status, int timeouts, ::hostent* host) {
+ auto p = reinterpret_cast<promise<hostent> *>(arg);
+
+ switch (status) {
+ default:
+ dns_log.debug("Query failed: {}", status);
+ p->set_exception(std::system_error(status, ares_errorc));
+ break;
+ case ARES_SUCCESS:
+ p->set_value(make_hostent(*host));
+ break;
+ }
+
+ delete p;
+ }, reinterpret_cast<void *>(p));
+
+
+ poll_sockets();
+
+ return f.finally([this] {
+ end_call();
+ });
+ }
+
+ future<sstring> resolve_addr(inet_address addr) {
+ return get_host_by_addr(addr).then([](hostent h) {
+ return make_ready_future<sstring>(h.names.front());
+ });
+ }
+
+ future<> close() {
+ _closed = true;
+ ares_cancel(_channel);
+ dns_log.trace("Shutting down {} sockets", _sockets.size());
+ for (auto & p : _sockets) {
+ do_close(p.first);
+ }
+ dns_log.trace("Closing gate");
+ return _gate.close();
+ }
+private:
+ enum class type {
+ none, tcp, udp
+ };
+ struct dns_call {
+ dns_call(impl & i)
+ : _i(i)
+ , _c(++i._calls)
+ {}
+ ~dns_call() {
+ // If a query does not immediately complete
+ // it might never do so, unless data actually
+ // comes back to us and a waiting recv promise
+ // is fulfilled.
+ // We need to add a timer to do polling at ~timeout
+ // ms later, so the ares logic can detect this and
+ // tell us we're over.
+ if (_c == 1 && _i._calls != 0) {
+ _i._timer.arm_periodic(_i._timeout);
+ }
+ }
+ impl& _i;
+ uint64_t _c;
+ };
+
+ void end_call() {
+ if (--_calls == 0) {
+ _timer.cancel();
+ }
+ }
+ void poll_sockets() {
+ fd_set readers, writers;
+ int n = 0;
+
+ dns_log.trace("Poll sockets");
+
+ do {
+ // Retrieve the set of file descriptors that the library wants us to monitor.
+ FD_ZERO(&readers);
+ FD_ZERO(&writers);
+
+ n = ares_fds(_channel, &readers, &writers);
+
+ dns_log.trace("ares_fds: {}", n);
+
+ if (n == 0) {
+ break;
+ }
+
+ n = 0;
+
+ for (auto & p : _sockets) {
+ auto & e = p.second;
+ auto fd = p.first;
+ auto r = FD_ISSET(p.first, &readers);
+ auto w = FD_ISSET(p.first, &writers);
+ auto ra = e.avail & POLLIN;
+ auto wa = e.avail & POLLOUT;
+
+ dns_log.trace("fd {} {}{}/{}{}", fd, (r ? "r" : ""),
+ (w ? "w" : ""), (ra ? "r" : ""),
+ (wa ? "w" : ""));
+
+ if (!wa) {
+ FD_CLR(fd, &writers);
+ }
+ if (!ra) {
+ FD_CLR(fd, &readers);
+ }
+ if (FD_ISSET(fd, &writers) || FD_ISSET(fd, &readers)) {
+ ++n;
+ }
+ }
+
+ ares_process(_channel, &readers, &writers);
+ } while (n != 0);
+ }
+
+ static hostent make_hostent(const ::hostent& host) {
+ hostent e;
+ e.names.emplace_back(host.h_name);
+ auto np = host.h_aliases;
+ while (*np != 0) {
+ e.names.emplace_back(*np++);
+ }
+ auto p = host.h_addr_list;
+ while (*p != nullptr) {
+ switch (host.h_addrtype) {
+ case AF_INET:
+ assert(size_t(host.h_length) >= sizeof(in_addr));
+ e.addr_list.emplace_back(*reinterpret_cast<const in_addr*>(*p));
+ break;
+ case AF_INET6:
+ assert(size_t(host.h_length) >= sizeof(in6_addr));
+ e.addr_list.emplace_back(*reinterpret_cast<const in6_addr*>(*p));
+ }
+
+ dns_log.debug("Query success: {}/{}", e.names.front(), e.addr_list.front());
+
+ return e;
+ }
+ // We need to partially ref-count our socket entries
+ // when we have pending reads/writes, so we don't erase the
+ // entry to early.
+ void use(ares_socket_t fd) {
+ _gate.enter();
+ auto& e = _sockets.at(fd);
+ ++e.pending;
+ }
+ void release(ares_socket_t fd) {
+ auto& e = _sockets.at(fd);
+ dns_log.trace("Release socket {} -> {}", fd, e.pending - 1);
+ if (--e.pending < 0) {
+ _sockets.erase(fd);
+ dns_log.trace("Released socket {}", fd);
+ }
+ _gate.leave();
+ }
+ ares_socket_t do_socket(int af, int type, int protocol) {
+ if (_closed) {
+ return -1;
+ }
+ int fd = next_fd();
+ switch (type) {
+ case SOCK_STREAM:
+ _sockets.emplace(fd, connected_socket());
+ dns_log.trace("Created tcp socket {}", fd);
+ break;
+ case SOCK_DGRAM:
+ _sockets.emplace(fd, _stack.make_udp_channel());
+ dns_log.trace("Created udp socket {}", fd);
+ break;
+ default: return -1;
+ }
+ return fd;
+ }
+ void do_close(ares_socket_t fd) {
+ dns_log.trace("Close socket {}", fd);
+ auto& e = _sockets.at(fd);
+
+ // Mark as closed.
+ if (std::exchange(e.closed, true)) {
+ return;
+ }
+
+ _gate.enter(); // "leave" is done in release(fd)
+
+ switch (e.typ) {
+ case type::tcp:
+ {
+ dns_log.trace("Close tcp socket {}, {} pending", fd, e.pending);
+ future<> f = make_ready_future();
+ if (e.tcp.in) {
+ f = e.tcp.socket.shutdown_input().then([fd] {
+ dns_log.trace("Closed tcp socket {} input", fd);
+ });
+ }
+ if (e.tcp.out) {
+ f = f.then([&e] {
+ return e.tcp.out->close();
+ }).then([fd] {
+ dns_log.trace("Closed tcp socket {} output", fd);
+ });
+ }
+ f = f.finally([me = shared_from_this(), fd] {
+ me->release(fd);
+ });
+ break;
+ }
+ case type::udp:
+ e.udp.channel.close();
+ release(fd);
+ break;
+ default:
+ // should not happen
+ _gate.leave();
+ break;
+ }
+ }
+ socket_address sock_addr(const sockaddr * addr, socklen_t len) {
+ if (addr->sa_family != AF_INET) {
+ throw std::invalid_argument("No ipv6 yet");
+ }
+ auto in = reinterpret_cast<const sockaddr_in *>(addr);
+ return *in;
+ }
+ int do_connect(ares_socket_t fd, const sockaddr * addr, socklen_t len) {
+ if (_closed) {
+ return -1;
+ }
+ try {
+ auto& e = get_socket_entry(fd);
+ auto sa = sock_addr(addr, len);
+
+ dns_log.trace("Connect {}({})->{}", fd, int(e.typ), sa);
+
+ assert(e.avail == 0);
+
+ e.avail = POLLOUT|POLLIN; // until we know otherwise
+
+ switch (e.typ) {
+ case type::tcp: {
+ auto f = _stack.connect(sa);
+ if (!f.available()) {
+ dns_log.trace("Connection pending: {}", fd);
+ e.avail = 0;
+ use(fd);
+ f.then_wrapped([me = shared_from_this(), &e, fd](future<connected_socket> f) {
+ try {
+ e.tcp.socket = f.get0();
+ dns_log.trace("Connection complete: {}", fd);
+ } catch (...) {
+ dns_log.debug("Connect {} failed: {}", fd, std::current_exception());
+ }
+ e.avail = POLLOUT|POLLIN;
+ me->poll_sockets();
+ me->release(fd);
+ });
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+ e.tcp.socket = f.get0();
+ break;
+ }
+ case type::udp:
+ // we do not have udp connect, so just keep
+ // track of the destination
+ e.udp.dst = sa;
+ break;
+ default:
+ return -1;
+ }
+ return 0;
+ } catch (...) {
+ return -1;
+ }
+ }
+ ssize_t do_recvfrom(ares_socket_t fd, void * dst, size_t len, int flags, struct sockaddr * from, socklen_t * from_len) {
+ if (_closed) {
+ return -1;
+ }
+ try {
+ auto& e = get_socket_entry(fd);
+ dns_log.trace("Read {}({})", fd, int(e.typ));
+ // check if we're already reading.
+ if (!(e.avail & POLLIN)) {
+ dns_log.trace("Read already pending {}", fd);
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+ for (;;) {
+ switch (e.typ) {
+ case type::tcp: {
+ auto & tcp = e.tcp;
+ if (!tcp.indata.empty()) {
+ dns_log.trace("Read {}. {} bytes available", fd, tcp.indata.size());
+ len = std::min(len, tcp.indata.size());
+ std::copy(tcp.indata.begin(), tcp.indata.begin() + len, reinterpret_cast<char *>(dst));
+ tcp.indata.trim_front(len);
+ return len;
+ }
+ if (!tcp.in) {
+ tcp.in = tcp.socket.input();
+ }
+ auto f = tcp.in->read_up_to(len);
+ if (!f.available()) {
+ dns_log.trace("Read {}: data unavailable", fd);
+ e.avail &= ~POLLIN;
+ use(fd);
+ f.then_wrapped([me = shared_from_this(), &e, fd](future<temporary_buffer<char>> f) {
+ try {
+ auto buf = f.get0();
+ dns_log.trace("Read {} -> {} bytes", fd, buf.size());
+ e.tcp.indata = std::move(buf);
+ } catch (...) {
+ dns_log.debug("Read {} failed: {}", fd, std::current_exception());
+ }
+ e.avail |= POLLIN; // always reset state
+ me->poll_sockets();
+ me->release(fd);
+ });
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+
+ try {
+ tcp.indata = f.get0();
+ continue; // loop will take care of data
+ } catch (std::system_error& e) {
+ errno = e.code().value();
+ return -1;
+ } catch (...) {
+ }
+ return -1;
+
+ }
+ case type::udp: {
+ auto & udp = e.udp;
+ if (udp.in) {
+ auto & p = udp.in->get_data();
+
+ dns_log.trace("Read {}. {} bytes available from {}", fd, p.len(), udp.in->get_src());
+
+ if (from != nullptr) {
+ *from = socket_address(udp.in->get_src()).as_posix_sockaddr();
+ if (from_len != nullptr) {
+ // TODO: ipvv6
+ *from_len = sizeof(sockaddr_in);
+ }
+ }
+
+ len = std::min(len, size_t(p.len()));
+ size_t rem = len;
+ auto * out = reinterpret_cast<char *>(dst);
+ for (auto & f : p.fragments()) {
+ auto n = std::min(rem, f.size);
+ out = std::copy_n(f.base, n, out);
+ rem = rem - n;
+ }
+ if (p.len() == len) {
+ udp.in = {};
+ } else {
+ p.trim_front(len);
+ }
+ return len;
+ }
+ auto f = udp.channel.receive();
+ if (!f.available()) {
+ e.avail &= ~POLLIN;
+ use(fd);
+ dns_log.trace("Read {}: data unavailable", fd);
+ f.then_wrapped([me = shared_from_this(), &e, fd](future<::net::udp_datagram> f) {
+ try {
+ auto d = f.get0();
+ dns_log.trace("Read {} -> {} bytes", fd, d.get_data().len());
+ e.udp.in = std::move(d);
+ e.avail |= POLLIN;
+ } catch (...) {
+ dns_log.debug("Read {} failed: {}", fd, std::current_exception());
+ }
+ me->poll_sockets();
+ me->release(fd);
+ });
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+
+ try {
+ udp.in = std::move(f.get0());
+ continue; // loop will take care of data
+ } catch (std::system_error& e) {
+ errno = e.code().value();
+ return -1;
+ } catch (...) {
+ }
+ return -1;
+ }
+ default:
+ return -1;
+ }
+ }
+ } catch (...) {
+ }
+ return -1;
+ }
+ ssize_t do_sendv(ares_socket_t fd, const iovec * vec, int len) {
+ if (_closed) {
+ return -1;
+ }
+ try {
+ auto& e = _sockets.at(fd);
+ dns_log.trace("Send {}({})", fd, int(e.typ));
+
+ // Assume we will be able to send data eventually very soon
+ // and just assume that unless we get immediate
+ // failures, we'll be ok. If we're not, the
+ // timeout logic will have to handle the problem.
+ //
+ // This saves us on two accounts:
+ // 1.) c-ares does not handle EWOULDBLOCK for
+ // udp sockets. Must pretend to finish
+ // immediately there anyway
+ // 2.) Doing so for tcp writes saves us having to
+ // match iovec->packet fragments. Downside is we
+ // have to copy the data, but we pretty much
+ // have to anyway, since we could otherwise
+ // get a query time out while we're sending
+ // with zero-copy and suddenly have freed
+ // memory in packets. Bad.
+
+
+ for (;;) {
+ // check if we're already writing.
+ if (e.typ == type::tcp && !(e.avail & POLLOUT)) {
+ dns_log.trace("Send already pending {}", fd);
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+
+ ::net::packet p;
+ p.reserve(len);
+ for (int i = 0; i < len; ++i) {
+ p = ::net::packet(std::move(p), ::net::fragment{reinterpret_cast<char *>(vec[i].iov_base), vec[i].iov_len});
+ }
+
+ auto bytes = p.len();
+ auto f = make_ready_future();
+
+ switch (e.typ) {
+ case type::tcp:
+ if (!e.tcp.out) {
+ e.tcp.out = e.tcp.socket.output();
+ }
+ f = e.tcp.out->write(std::move(p));
+ break;
+ case type::udp:
+ f = e.udp.channel.send(e.udp.dst, std::move(p));
+ break;
+ default:
+ return -1;
+ }
+
+ if (!f.available()) {
+ dns_log.trace("Send {} unavailable.", fd);
+ e.avail &= ~POLLOUT;
+ use(fd);
+ f.then_wrapped([me = shared_from_this(), &e, bytes, fd](future<> f) {
+ try {
+ f.get();
+ dns_log.trace("Send {}. {} bytes sent.", fd, bytes);
+ } catch (...) {
+ dns_log.debug("Send {} failed: {}", fd, std::current_exception());
+ }
+ e.avail |= POLLOUT;
+ me->poll_sockets();
+ me->release(fd);
+ });
+ // c-ares does _not_ use non-blocking retry for udp sockets. We just pretend
+ // all is fine even though we have no idea. Barring stack/adapter failure it
+ // is close to the same guarantee a "normal" message send would have anyway.
+ // For tcp we also pretend we're done, to make sure we don't have to deal with
+ // matching sent data
+ }
+ if (f.failed()) {
+ try {
+ f.get();
+ } catch (std::system_error& e) {
+ errno = e.code().value();
+ } catch (...) {
+ }
+ return -1;
+ }
+
+ return len;
+ }
+ } catch (...) {
+ }
+ return -1;
+ }
+ int next_fd() {
+ int fd = int(_sockets.size() + 1);
+ while (_sockets.count(fd)) {
+ ++fd;
+ }
+ return fd;
+ }
+ struct tcp_entry {
+ tcp_entry(connected_socket s)
+ : socket(std::move(s)) {
+ }
+ ;
+ connected_socket socket;
+ std::experimental::optional<input_stream<char>> in;
+ std::experimental::optional<output_stream<char>> out;
+ temporary_buffer<char> indata;
+ };
+ struct udp_entry {
+ udp_entry(::net::udp_channel c)
+ : channel(std::move(c)) {
+ }
+ ::net::udp_channel channel;
+ std::experimental::optional<::net::udp_datagram> in;;
+ socket_address dst;
+ };
+ struct sock_entry {
+ union {
+ tcp_entry tcp;
+ udp_entry udp;
+ };
+ type typ;
+ int avail = 0;
+ int pending = 0;
+ bool closed = false;
+
+ sock_entry(sock_entry&& e)
+ : typ(e.typ)
+ , avail(e.avail)
+ {
+ e.typ = type::none;
+ switch (typ) {
+ case type::tcp:
+ tcp = std::move(e.tcp);
+ break;
+ case type::udp:
+ udp = std::move(e.udp);
+ : tcp(tcp_entry{std::move(s)})
+ , typ(type::tcp)
+ {}
+ sock_entry(::net::udp_channel c)
+ : udp(udp_entry{std::move(c)})
+ , typ(type::udp)
+ {}
+ ~sock_entry() {
+ switch (typ) {
+ case type::tcp: tcp.~tcp_entry(); break;
+ case type::udp: udp.~udp_entry(); break;
+ default: break;
+ }
+ }
+ };
+
+ sock_entry& get_socket_entry(ares_socket_t fd) {
+ auto& e = _sockets.at(fd);
+ if (e.closed) {
+ throw std::runtime_error("Socket closed");
+ }
+ return e;
+ }
+
+
+ typedef std::unordered_map<ares_socket_t, sock_entry> socket_map;
+
+ friend struct dns_call;
+
+ socket_map _sockets;
+ ::network_stack & _stack;
+
+ ares_channel _channel = {};
+ uint64_t _ops = 0, _calls = 0;
+ std::chrono::milliseconds _timeout;
+ timer<> _timer;
+ gate _gate;
+ bool _closed = false;
+};
+
+seastar::net::dns_resolver::dns_resolver()
+ : dns_resolver(options())
+{}
+
+seastar::net::dns_resolver::dns_resolver(const options& opts)
+ : dns_resolver(engine().net(), opts)
+{}
+
+seastar::net::dns_resolver::dns_resolver(network_stack& stack, const options& opts)
+ : _impl(::make_shared<impl>(stack, opts))
+{}
+
+seastar::net::dns_resolver::~dns_resolver()
+{}
+
+seastar::net::dns_resolver::dns_resolver(dns_resolver&&) = default;
+seastar::net::dns_resolver& seastar::net::dns_resolver::operator=(dns_resolver&&) = default;
+
+future<seastar::net::hostent> seastar::net::dns_resolver::get_host_by_name(const sstring& name, opt_family family) {
+ return _impl->get_host_by_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<seastar::net::hostent> seastar::net::dns_resolver::get_host_by_addr(const inet_address& addr) {
+ return _impl->get_host_by_addr(addr);
+}
+
+future<seastar::net::inet_address> seastar::net::dns_resolver::resolve_name(const sstring& name, opt_family family) {
+ return _impl->resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns_resolver::resolve_addr(const inet_address& addr) {
+ return _impl->resolve_addr(addr);
+}
+
+future<> seastar::net::dns_resolver::close() {
+ return _impl->close();
+}
+
+static seastar::net::dns_resolver& resolver() {
+ static thread_local seastar::net::dns_resolver resolver;
+ return resolver;
+}
+
+
+future<seastar::net::hostent> seastar::net::dns::get_host_by_name(const sstring& name, opt_family family) {
+ return resolver().get_host_by_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<seastar::net::hostent> seastar::net::dns::get_host_by_addr(const inet_address& addr) {
+ return resolver().get_host_by_addr(addr);
+}
+
+future<seastar::net::inet_address> seastar::net::dns::resolve_name(const sstring& name, opt_family family) {
+ return resolver().resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns::resolve_addr(const inet_address& addr) {
+ return resolver().resolve_addr(addr);
+}
diff --git a/tests/dns_test.cc b/tests/dns_test.cc
new file mode 100644
index 0000000..2999d0e
--- /dev/null
+++ b/tests/dns_test.cc
@@ -0,0 +1,105 @@
+ * Copyright (C) 2016 ScyllaDB.
+ */
+#include <vector>
+#include <algorithm>
+
+#include "core/do_with.hh"
+#include "test-utils.hh"
+#include "core/sstring.hh"
+#include "core/reactor.hh"
+#include "core/do_with.hh"
+#include "core/future-util.hh"
+#include "net/dns.hh"
+#include "net/inet_address.hh"
+
+using namespace seastar;
+using namespace seastar::net;
+
+static const inet_address google_addr = inet_address("216.58.201.164");
+static const sstring google_name = "www.google.com";
+
+static future<> test_resolve(dns_resolver::options opts) {
+ auto d = ::make_lw_shared<dns_resolver>(std::move(opts));
+ return d->get_host_by_name(google_name, inet_address::family::INET).then([d](hostent e) {
+ //BOOST_REQUIRE(std::count(e.addr_list.begin(), e.addr_list.end(), google_addr));
+ return d->get_host_by_addr(e.addr_list.front()).then([d, a = e.addr_list.front()](hostent e) {
+ return d->get_host_by_name(e.names.front(), inet_address::family::INET).then([a](hostent e) {
+ BOOST_REQUIRE(std::count(e.addr_list.begin(), e.addr_list.end(), a));
+ });
+ });
+ }).finally([d]{
+ return d->close();
+ });
+}
+
+static future<> test_bad_name(dns_resolver::options opts) {
+ auto d = ::make_lw_shared<dns_resolver>(std::move(opts));
+ return d->get_host_by_name("apa.ninja.gnu", inet_address::family::INET).then_wrapped([d](future<hostent> f) {
+ try {
+ f.get();
+ BOOST_FAIL("should not succeed");
+ } catch (...) {
+ // ok.
+ }
+ }).finally([d]{
+ return d->close();
+ });
+}
+
+SEASTAR_TEST_CASE(test_resolve_udp) {
+ return test_resolve(dns_resolver::options());
+}
+
+SEASTAR_TEST_CASE(test_bad_name_udp) {
+ return test_bad_name(dns_resolver::options());
+}
+
+SEASTAR_TEST_CASE(test_timeout_udp) {
+ dns_resolver::options opts;
+ opts.servers = std::vector<inet_address>({ inet_address("1.2.3.4") }); // not a server
+ opts.timeout = std::chrono::milliseconds(500);
+
+ auto d = ::make_lw_shared<dns_resolver>(engine().net(), opts);
+ return d->get_host_by_name(google_name, inet_address::family::INET).then_wrapped([d](future<hostent> f) {
+ try {
+ f.get();
+ BOOST_FAIL("should not succeed");
+ } catch (...) {
+ // ok.
+ }
+ }).finally([d]{
+ return d->close();
+ });
+}
+
+SEASTAR_TEST_CASE(test_resolve_tcp) {
+ dns_resolver::options opts;
+ opts.use_tcp_query = true;
+ return test_resolve(opts);
+}
+
+SEASTAR_TEST_CASE(test_bad_name_tcp) {
+ dns_resolver::options opts;
+ opts.use_tcp_query = true;
+ return test_bad_name(opts);
+}
+
--
1.9.1
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 899 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1142 insertions(+), 1 deletion(-)
create mode 100644 net/dns.hh
create mode 100644 net/dns.cc
create mode 100644 tests/dns_test.cc
diff --git a/configure.py b/configure.py
index c091318..6742554 100755
--- a/configure.py
+++ b/configure.py
@@ -209,6 +209,7 @@ tests = [
'tests/scollectd_test',
'tests/perf/perf_fstream',
'tests/json_formatter_test',
+ 'tests/dns_test',
]
apps = [
@@ -269,6 +270,7 @@ libnet = [
'net/dhcp.cc',
'net/tls.cc',
'net/inet_address.cc',
+ 'net/dns.cc',
]
core = [
@@ -428,6 +430,7 @@ deps = {
'tests/scollectd_test': ['tests/scollectd_test.cc'] + core,
'tests/perf/perf_fstream': ['tests/perf/perf_fstream.cc'] + core,
'tests/json_formatter_test': ['tests/json_formatter_test.cc'] + core + http,
+ 'tests/dns_test': ['tests/dns_test.cc'] + core + libnet,
}
boost_tests = [
@@ -447,6 +450,7 @@ boost_tests = [
'tests/connect_test',
'tests/scollectd_test',
'tests/json_formatter_test',
+ 'tests/dns_test',
]
for bt in boost_tests:
@@ -673,6 +677,7 @@ with open(buildfile, 'w') as f:
build {dpdk_deps} : dpdkmake {dpdk_sources}
''').format(**globals()))
for mode in build_modes:
+ objdeps = {}
modeval = modes[mode]
if modeval['sanitize'] and not do_sanitize:
print('Note: --static disables debug mode sanitizers')
@@ -711,6 +716,7 @@ with open(buildfile, 'w') as f:
build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
+ objdeps['$builddir/' + mode + '/net/dns.o'] = ' $builddir/' + mode + '/' + cares_dir + '/ares_build.h'
compiles = {}
ragels = {}
swaggers = {}
@@ -773,7 +779,7 @@ with open(buildfile, 'w') as f:
ragels = {}
swaggers = {}
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
new file mode 100644
index 0000000..74b9020
--- /dev/null
+++ b/net/dns.hh
@@ -0,0 +1,131 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright 2016 Cloudius Systems
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ */
+
+#pragma once
+
+#include <vector>
+
+#pragma once
+
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/shared_ptr.hh"
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+
+namespace seastar {
+namespace net {
+
+/**
+namespace seastar {
+namespace net {
+
+ * A c++-esque version of a hostent
+ */
+struct hostent {
+ // Primary name is always first
+ std::vector<sstring> names;
+ // Primary address is also always first.
+ std::vector<inet_address> addr_list;
+};
+
+typedef std::experimental::optional<inet_address::family> opt_family;
+
+/**
+ * A DNS resolver object.
+ * Wraps the query logic & networking.
+ * Can be instantiated with options and your network
+ * stack of choice, though for "normal" non-test
+ * querying, you are probably better of with the
+ * global calls further down.
+ */
+class dns_resolver {
+public:
+ struct options {
+ std::experimental::optional<bool>
+ use_tcp_query;
+ std::experimental::optional<std::vector<inet_address>>
+ servers;
+ std::experimental::optional<std::chrono::milliseconds>
+ timeout;
+ std::experimental::optional<uint16_t>
+ tcp_port, udp_port;
+ std::experimental::optional<std::vector<sstring>>
+ domains;
+ };
+
+ dns_resolver();
+ dns_resolver(dns_resolver&&);
+ dns_resolver(const options&);
+ dns_resolver(::network_stack&, const options& = {});
+ ~dns_resolver();
+
+ dns_resolver& operator=(dns_resolver&&);
+
+ /**
+ * Resolves a hostname to one or more addresses and aliases
+ */
+ future<hostent> get_host_by_name(const sstring&, opt_family = {});
+ /**
+ * Resolves an address to one or more addresses and aliases
+ */
+ future<hostent> get_host_by_addr(const inet_address&);
+
+ /**
+ * Resolves a hostname to one (primary) address
+ */
+ future<inet_address> resolve_name(const sstring&, opt_family = {});
+ /**
+ * Resolves an address to one (primary) name
+ */
+ future<sstring> resolve_addr(const inet_address&);
+
+ /**
+ * Shuts the object down. Great for tests.
+ */
+ future<> close();
+private:
+ class impl;
+ ::shared_ptr<impl> _impl;
+};
+
+namespace dns {
+
+// See above. These functions simply queries using a shard-local
+// default-stack, default-opts resolver
+future<hostent> get_host_by_name(const sstring&, opt_family = {});
+future<hostent> get_host_by_addr(const inet_address&);
+
+future<inet_address> resolve_name(const sstring&, opt_family = {});
+future<sstring> resolve_addr(const inet_address&);
+
+}
+
+}
+}
diff --git a/net/dns.cc b/net/dns.cc
new file mode 100644
index 0000000..5c7bd10
--- /dev/null
+++ b/net/dns.cc
@@ -0,0 +1,899 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright 2016 Cloudius Systems
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ */
+
+#include <chrono>
+#include <experimental/string_view>
+
+#include <c-ares/ares.h>
+
+#include "ip.hh"
+#include "api.hh"
+#include "dns.hh"
+#include "core/sstring.hh"
+#include "core/timer.hh"
+#include "core/reactor.hh"
+#include "core/gate.hh"
+#include "util/log.hh"
+
+static seastar::logger dns_log("dns_resolver");
+
+class ares_error_category : public std::error_category {
+public:
+ constexpr ares_error_category() noexcept : std::error_category{} {}
+ const char * name() const noexcept {
+ return "C-Ares";
+ }
+ std::string message(int error) const {
+ switch (error) {
+ /* Server error codes (ARES_ENODATA indicates no relevant answer) */
+ case ARES_ENODATA: return "No data";
+ case ARES_EFORMERR: return "Form error";
+ case ARES_ESERVFAIL: return "Server failure";
+ case ARES_ENOTFOUND: return "Not found";
+ case ARES_ENOTIMP: return "Not implemented";
+ case ARES_EREFUSED: return "Refused";
+
+ /* Locally generated error codes */
+ case ARES_EBADQUERY: return "Bad query";
+ case ARES_EBADNAME: return "Bad name";
+ case ARES_EBADFAMILY: return "Bad family";
+ case ARES_EBADRESP: return "Bad response";
+ case ARES_ECONNREFUSED :return "Connection refused";
+ case ARES_ETIMEOUT: return "Timeout";
+ case ARES_EOF: return "EOF";
+ case ARES_EFILE: return "File error";
+ case ARES_ENOMEM: return "No memory";
+ case ARES_EDESTRUCTION: return "Destruction";
+ case ARES_EBADSTR: return "Bad string";
+
+ /* ares_getnameinfo error codes */
+ case ARES_EBADFLAGS: return "Invalid flags";
+
+ /* ares_getaddrinfo error codes */
+ case ARES_ENONAME: return "No name";
+ case ARES_EBADHINTS: return "Bad hints";
+
+ /* Uninitialized library error code */
+ case ARES_ENOTINITIALIZED: return "Not initialized";
+
+ /* ares_library_init error codes */
+ case ARES_ELOADIPHLPAPI: return "Load PHLPAPI";
+ case ARES_EADDRGETNETWORKPARAMS: return "Get network parameters";
+
+ /* More error codes */
+ case ARES_ECANCELLED: return "Cancelled";
+ default:
+ return "Unknown error";
+ }
+ }
+};
+
+static const ares_error_category ares_errorc;
+
+static void check_ares_error(int error) {
+ if (error != ARES_SUCCESS) {
+ throw std::system_error(error, ares_errorc);
+ }
+}
+
+struct ares_initializer {
+ ares_initializer() {
+ check_ares_error(ares_library_init(0));
+ }
+ ~ares_initializer() {
+ ares_library_cleanup();
+ }
+};
+
+class seastar::net::dns_resolver::impl
+ : public enable_shared_from_this<impl>
+{
+public:
+ impl(::network_stack& stack, const options& opts)
+ : _stack(stack)
+ , _timeout(opts.timeout ? *opts.timeout : std::chrono::milliseconds(5000) /* from ares private */)
+ , _timer(std::bind(&impl::poll_sockets, this))
+ {
+ static const ares_initializer a_init;
+
+ // this can "block" ever so slightly, because it will
+ // look in resolv.conf etc for query setup. We could
+ // do this ourselves, and instead set ares options
+ // here, but it seems more error prone (me parsing
+ // resolv.conf -> hah!)
+ ares_options a_opts = { 0, };
+ // TODO:
+ // We only allow querying dns server. We ignore hosts
+ // files. Either we need to add virtual file calls
+ // to c-ares (huger!)
+ // or we do our read+(partial)parse outselved for
+ // hosts files.
+ // Big downside here right now that we can essentially
+ // get different answers on a hsotfile based system
+ char buf[2] = "b";
+ a_opts.lookups = buf; // only net
+ // Alsways set the timeout
+ a_opts.timeout = _timeout.count();
+ int flags = ARES_OPT_LOOKUPS|ARES_OPT_TIMEOUTMS;
+
+ if (opts.use_tcp_query && *opts.use_tcp_query) {
+ a_opts.flags = ARES_FLAG_USEVC | ARES_FLAG_PRIMARY;
+ flags |= ARES_OPT_FLAGS;
+ }
+ std::vector<in_addr> addr_tmp;
+ if (opts.servers) {
+ std::transform(opts.servers->begin(), opts.servers->end(), std::back_inserter(addr_tmp), [](const inet_address& a) {
+ if (a.in_family != inet_address::family::INET) {
+ throw std::invalid_argument("Servers must be ipv4 addresses");
+ }
+ in_addr in = a;
+ return in;
+ });
+ a_opts.servers = addr_tmp.data();
+ a_opts.nservers = int(addr_tmp.size());
+ flags |= ARES_OPT_SERVERS;
+ }
+ std::vector<const char *> dom_tmp;
+ if (opts.domains) {
+ std::transform(opts.domains->begin(), opts.domains->end(), std::back_inserter(dom_tmp), [](const sstring& s) {
+ return s.data();
+ });
+ a_opts.domains = const_cast<char **>(dom_tmp.data());
+ a_opts.ndomains = int(dom_tmp.size());
+ flags |= ARES_OPT_DOMAINS;
+ }
+ if (opts.tcp_port) {
+ a_opts.tcp_port = *opts.tcp_port;
+ flags |= ARES_OPT_TCP_PORT;
+ }
+ if (opts.udp_port) {
+ a_opts.udp_port = *opts.udp_port;
+ flags |= ARES_OPT_UDP_PORT;
+ }
+
+ check_ares_error(ares_init_options(&_channel, &a_opts, flags));
+
+ static auto get_impl = [](void * p) { return reinterpret_cast<impl *>(p); };
+ static const ares_socket_functions callbacks = {
+ [](int af, int type, int protocol, void * p) { return get_impl(p)->do_socket(af, type, protocol); },
+ [](ares_socket_t s, void * p) { return get_impl(p)->do_close(s); },
+ [](ares_socket_t s, const struct sockaddr * addr, socklen_t len, void * p) { return get_impl(p)->do_connect(s, addr, len); },
+ [](ares_socket_t s, void * dst, size_t len, int flags, struct sockaddr * addr, socklen_t * alen, void * p) {
+ return get_impl(p)->do_recvfrom(s, dst, len, flags, addr, alen);
+ },
+ [](ares_socket_t s, const struct iovec * vec, int len, void * p) {
+ return get_impl(p)->do_sendv(s, vec, len);
+ },
+ };
+
+ ares_set_socket_functions(_channel, &callbacks, this);
+
+ // just in case you need printf-debug.
+ //dns_log.set_level(seastar::log_level::trace);
+ }
+ ~impl() {
+ _timer.cancel();
+ if (_channel) {
+ ares_destroy(_channel);
+ }
+ }
+
+ future<inet_address> resolve_name(sstring name, inet_address::family family) {
+ return get_host_by_name(std::move(name), family).then([](hostent h) {
+ return make_ready_future<inet_address>(h.addr_list.front());
+ });
+ }
+
+ future<hostent> get_host_by_name(sstring name, inet_address::family family) {
+ auto p = new promise<hostent>();
+ auto f = p->get_future();
+
+ dns_log.debug("Query name {} ({})", name, family);
+
+ dns_call call(*this);
+
+ ares_gethostbyname(_channel, name.c_str(), int(family), [](void* arg, int status, int timeouts, ::hostent* host) {
+ auto p = reinterpret_cast<promise<hostent> *>(arg);
+
+ switch (status) {
+ default:
+ dns_log.debug("Query failed: {}", status);
+ p->set_exception(std::system_error(status, ares_errorc));
+ break;
+ case ARES_SUCCESS:
+ p->set_value(make_hostent(*host));
+ break;
+ }
+
+ delete p;
+ }, reinterpret_cast<void *>(p));
+
+
+ poll_sockets();
+
+ return f.finally([this] {
+ end_call();
+ });
+ }
+
+ future<hostent> get_host_by_addr(inet_address addr) {
+ auto p = new promise<hostent>();
+ auto f = p->get_future();
+
+ dns_log.debug("Query addr {}", addr);
+
+ dns_call call(*this);
+
+ ares_gethostbyaddr(_channel, &addr.in, addr.size(), int(addr.in_family), [](void* arg, int status, int timeouts, ::hostent* host) {
+ auto p = reinterpret_cast<promise<hostent> *>(arg);
+
+ switch (status) {
+ default:
+ dns_log.debug("Query failed: {}", status);
+ p->set_exception(std::system_error(status, ares_errorc));
+ break;
+ case ARES_SUCCESS:
+ p->set_value(make_hostent(*host));
+ break;
+ }
+
+ delete p;
+ }, reinterpret_cast<void *>(p));
+
+
+ poll_sockets();
+
+ return f.finally([this] {
+ end_call();
+ });
+ }
+
+ future<sstring> resolve_addr(inet_address addr) {
+ return get_host_by_addr(addr).then([](hostent h) {
+ return make_ready_future<sstring>(h.names.front());
+ });
+ }
+
+ future<> close() {
+ _closed = true;
+ ares_cancel(_channel);
+ dns_log.trace("Shutting down {} sockets", _sockets.size());
+ for (auto & p : _sockets) {
+ do_close(p.first);
+ }
+ dns_log.trace("Closing gate");
+ return _gate.close();
+ }
+private:
+ enum class type {
+ none, tcp, udp
+ };
+ struct dns_call {
+ dns_call(impl & i)
+ : _i(i)
+ , _c(++i._calls)
+ {}
+ ~dns_call() {
+ // If a query does not immediately complete
+ // it might never do so, unless data actually
+ // comes back to us and a waiting recv promise
+ // is fulfilled.
+ // We need to add a timer to do polling at ~timeout
+ // ms later, so the ares logic can detect this and
+ // tell us we're over.
+ if (_c == 1 && _i._calls != 0) {
+ _i._timer.arm_periodic(_i._timeout);
+ }
+ }
+ impl& _i;
+ uint64_t _c;
+ };
+
+ void end_call() {
+ if (--_calls == 0) {
+ _timer.cancel();
+ }
+ }
+ void poll_sockets() {
+ fd_set readers, writers;
+ int n = 0;
+
+ dns_log.trace("Poll sockets");
+
+ do {
+ // Retrieve the set of file descriptors that the library wants us to monitor.
+ FD_ZERO(&readers);
+ FD_ZERO(&writers);
+
+ n = ares_fds(_channel, &readers, &writers);
+
+ dns_log.trace("ares_fds: {}", n);
+
+ if (n == 0) {
+ break;
+ }
+
+ n = 0;
+
+ for (auto & p : _sockets) {
+ auto & e = p.second;
+ auto fd = p.first;
+ auto r = FD_ISSET(p.first, &readers);
+ auto w = FD_ISSET(p.first, &writers);
+ auto ra = e.avail & POLLIN;
+ auto wa = e.avail & POLLOUT;
+
+ dns_log.trace("fd {} {}{}/{}{}", fd, (r ? "r" : ""),
+ (w ? "w" : ""), (ra ? "r" : ""),
+ (wa ? "w" : ""));
+
+ if (!wa) {
+ FD_CLR(fd, &writers);
+ }
+ if (!ra) {
+ FD_CLR(fd, &readers);
+ }
+ if (FD_ISSET(fd, &writers) || FD_ISSET(fd, &readers)) {
+ ++n;
+ }
+ }
+
+ ares_process(_channel, &readers, &writers);
+ } while (n != 0);
+ }
+
+ static hostent make_hostent(const ::hostent& host) {
+ hostent e;
+ e.names.emplace_back(host.h_name);
+ auto np = host.h_aliases;
+ while (*np != 0) {
+ e.names.emplace_back(*np++);
+ }
+ auto p = host.h_addr_list;
+ while (*p != nullptr) {
+ switch (host.h_addrtype) {
+ case AF_INET:
+ assert(size_t(host.h_length) >= sizeof(in_addr));
+ e.addr_list.emplace_back(*reinterpret_cast<const in_addr*>(*p));
+ break;
+ case AF_INET6:
+ assert(size_t(host.h_length) >= sizeof(in6_addr));
+ e.addr_list.emplace_back(*reinterpret_cast<const in6_addr*>(*p));
+ break;
+ default:
+ break;
+ }
+ ++p;
+ default:
+ break;
+ }
+ }
+
+ dns_log.debug("Query success: {}/{}", e.names.front(), e.addr_list.front());
+
+ return e;
+ }
+ // We need to partially ref-count our socket entries
+ // when we have pending reads/writes, so we don't erase the
+ // entry to early.
+ void use(ares_socket_t fd) {
+ _gate.enter();
+ auto& e = _sockets.at(fd);
+ ++e.pending;
+ }
+ void release(ares_socket_t fd) {
+ auto& e = _sockets.at(fd);
+ dns_log.trace("Release socket {} -> {}", fd, e.pending - 1);
+ if (--e.pending < 0) {
+ _sockets.erase(fd);
+ dns_log.trace("Released socket {}", fd);
+ }
+ _gate.leave();
+ }
+ ares_socket_t do_socket(int af, int type, int protocol) {
+ if (_closed) {
+ return -1;
+ }
+ int fd = next_fd();
+ switch (type) {
+ case SOCK_STREAM:
+ _sockets.emplace(fd, connected_socket());
+ dns_log.trace("Created tcp socket {}", fd);
+ break;
+ case SOCK_DGRAM:
+ _sockets.emplace(fd, _stack.make_udp_channel());
+ dns_log.trace("Created udp socket {}", fd);
+ break;
+ default: return -1;
+ }
+ return fd;
+ }
+ void do_close(ares_socket_t fd) {
+ dns_log.trace("Close socket {}", fd);
+ auto& e = _sockets.at(fd);
+
+ // Mark as closed.
+ if (std::exchange(e.closed, true)) {
+ return;
+ }
+
+ _gate.enter(); // "leave" is done in release(fd)
+
+ switch (e.typ) {
+ case type::tcp:
+ {
+ dns_log.trace("Close tcp socket {}, {} pending", fd, e.pending);
+ future<> f = make_ready_future();
+ if (e.tcp.in) {
+ f = e.tcp.socket.shutdown_input().then([fd] {
+ dns_log.trace("Closed tcp socket {} input", fd);
+ });
+ }
+ if (e.tcp.out) {
+ f = f.then([&e] {
+ return e.tcp.out->close();
+ }).then([fd] {
+ dns_log.trace("Closed tcp socket {} output", fd);
+ });
+ }
+ f = f.finally([me = shared_from_this(), fd] {
+ me->release(fd);
+ });
+ break;
+ }
+ case type::udp:
+ e.udp.channel.close();
+ release(fd);
+ break;
+ default:
+ // should not happen
+ _gate.leave();
+ break;
+ }
+ }
+ socket_address sock_addr(const sockaddr * addr, socklen_t len) {
+ if (addr->sa_family != AF_INET) {
+ throw std::invalid_argument("No ipv6 yet");
+ }
+ auto in = reinterpret_cast<const sockaddr_in *>(addr);
+ return *in;
+ }
+ int do_connect(ares_socket_t fd, const sockaddr * addr, socklen_t len) {
+ if (_closed) {
+ return -1;
+ }
+ try {
+ auto& e = get_socket_entry(fd);
+ auto sa = sock_addr(addr, len);
+
+ dns_log.trace("Connect {}({})->{}", fd, int(e.typ), sa);
+
+ assert(e.avail == 0);
+
+ e.avail = POLLOUT|POLLIN; // until we know otherwise
+
+ switch (e.typ) {
+ case type::tcp: {
+ auto f = _stack.connect(sa);
+ if (!f.available()) {
+ dns_log.trace("Connection pending: {}", fd);
+ e.avail = 0;
+ use(fd);
+ f.then_wrapped([me = shared_from_this(), &e, fd](future<connected_socket> f) {
+ try {
+ e.tcp.socket = f.get0();
+ dns_log.trace("Connection complete: {}", fd);
+ } catch (...) {
+ dns_log.debug("Connect {} failed: {}", fd, std::current_exception());
+ }
+ e.avail = POLLOUT|POLLIN;
+ me->poll_sockets();
+ me->release(fd);
+ });
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+ e.tcp.socket = f.get0();
+ break;
+ }
+ case type::udp:
+ // we do not have udp connect, so just keep
+ // track of the destination
+ e.udp.dst = sa;
+ break;
+ default:
+ return -1;
+ }
+ return 0;
+ } catch (...) {
+ return -1;
+ }
+ }
+ ssize_t do_recvfrom(ares_socket_t fd, void * dst, size_t len, int flags, struct sockaddr * from, socklen_t * from_len) {
+ if (_closed) {
+ return -1;
+ }
+ try {
+ auto& e = get_socket_entry(fd);
+ dns_log.trace("Read {}({})", fd, int(e.typ));
+ // check if we're already reading.
+ if (!(e.avail & POLLIN)) {
+ dns_log.trace("Read already pending {}", fd);
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+ for (;;) {
+ switch (e.typ) {
+ case type::tcp: {
+ auto & tcp = e.tcp;
+ if (!tcp.indata.empty()) {
+ dns_log.trace("Read {}. {} bytes available", fd, tcp.indata.size());
+ len = std::min(len, tcp.indata.size());
+ std::copy(tcp.indata.begin(), tcp.indata.begin() + len, reinterpret_cast<char *>(dst));
+ tcp.indata.trim_front(len);
+ return len;
+ }
+ if (!tcp.in) {
+ tcp.in = tcp.socket.input();
+ }
+ auto f = tcp.in->read_up_to(len);
+ if (!f.available()) {
+ dns_log.trace("Read {}: data unavailable", fd);
+ e.avail &= ~POLLIN;
+ use(fd);
+ f.then_wrapped([me = shared_from_this(), &e, fd](future<temporary_buffer<char>> f) {
+ try {
+ auto buf = f.get0();
+ dns_log.trace("Read {} -> {} bytes", fd, buf.size());
+ e.tcp.indata = std::move(buf);
+ } catch (...) {
+ dns_log.debug("Read {} failed: {}", fd, std::current_exception());
+ }
+ e.avail |= POLLIN; // always reset state
+ me->poll_sockets();
+ me->release(fd);
+ });
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+
+ try {
+ tcp.indata = f.get0();
+ continue; // loop will take care of data
+ } catch (std::system_error& e) {
+ errno = e.code().value();
+ return -1;
+ } catch (...) {
+ }
+ return -1;
+
+ }
+ case type::udp: {
+ auto & udp = e.udp;
+ if (udp.in) {
+ auto & p = udp.in->get_data();
+
+ dns_log.trace("Read {}. {} bytes available from {}", fd, p.len(), udp.in->get_src());
+
+ if (from != nullptr) {
+ *from = socket_address(udp.in->get_src()).as_posix_sockaddr();
+ if (from_len != nullptr) {
+ // TODO: ipvv6
+ *from_len = sizeof(sockaddr_in);
+ }
+ }
+
+ len = std::min(len, size_t(p.len()));
+ size_t rem = len;
+ auto * out = reinterpret_cast<char *>(dst);
+ for (auto & f : p.fragments()) {
+ auto n = std::min(rem, f.size);
+ out = std::copy_n(f.base, n, out);
+ rem = rem - n;
+ }
+ if (p.len() == len) {
+ udp.in = {};
+ } else {
+ p.trim_front(len);
+ }
+ return len;
+ }
+ auto f = udp.channel.receive();
+ if (!f.available()) {
+ e.avail &= ~POLLIN;
+ use(fd);
+ dns_log.trace("Read {}: data unavailable", fd);
+ f.then_wrapped([me = shared_from_this(), &e, fd](future<::net::udp_datagram> f) {
+ try {
+ auto d = f.get0();
+ dns_log.trace("Read {} -> {} bytes", fd, d.get_data().len());
+ e.udp.in = std::move(d);
+ e.avail |= POLLIN;
+ } catch (...) {
+ dns_log.debug("Read {} failed: {}", fd, std::current_exception());
+ }
+ me->poll_sockets();
+ me->release(fd);
+ });
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+
+ try {
+ udp.in = std::move(f.get0());
+ continue; // loop will take care of data
+ } catch (std::system_error& e) {
+ errno = e.code().value();
+ return -1;
+ } catch (...) {
+ }
+ return -1;
+ }
+ default:
+ return -1;
+ }
+ }
+ } catch (...) {
+ }
+ return -1;
+ }
+ ssize_t do_sendv(ares_socket_t fd, const iovec * vec, int len) {
+ if (_closed) {
+ return -1;
+ }
+ try {
+ auto& e = _sockets.at(fd);
+ dns_log.trace("Send {}({})", fd, int(e.typ));
+
+ // Assume we will be able to send data eventually very soon
+ // and just assume that unless we get immediate
+ // failures, we'll be ok. If we're not, the
+ // timeout logic will have to handle the problem.
+ //
+ // This saves us on two accounts:
+ // 1.) c-ares does not handle EWOULDBLOCK for
+ // udp sockets. Must pretend to finish
+ // immediately there anyway
+ // 2.) Doing so for tcp writes saves us having to
+ // match iovec->packet fragments. Downside is we
+ // have to copy the data, but we pretty much
+ // have to anyway, since we could otherwise
+ // get a query time out while we're sending
+ // with zero-copy and suddenly have freed
+ // memory in packets. Bad.
+
+
+ for (;;) {
+ // check if we're already writing.
+ if (e.typ == type::tcp && !(e.avail & POLLOUT)) {
+ dns_log.trace("Send already pending {}", fd);
+ errno = EWOULDBLOCK;
+ return -1;
+ }
+
+ ::net::packet p;
+ p.reserve(len);
+ for (int i = 0; i < len; ++i) {
+ p = ::net::packet(std::move(p), ::net::fragment{reinterpret_cast<char *>(vec[i].iov_base), vec[i].iov_len});
+ }
+
+ auto bytes = p.len();
+ auto f = make_ready_future();
+
+ switch (e.typ) {
+ case type::tcp:
+ if (!e.tcp.out) {
+ e.tcp.out = e.tcp.socket.output();
+ }
+ f = e.tcp.out->write(std::move(p));
+ break;
+ case type::udp:
+ f = e.udp.channel.send(e.udp.dst, std::move(p));
+ break;
+ default:
+ return -1;
+ }
+
+ if (!f.available()) {
+ dns_log.trace("Send {} unavailable.", fd);
+ e.avail &= ~POLLOUT;
+ use(fd);
+ f.then_wrapped([me = shared_from_this(), &e, bytes, fd](future<> f) {
+ try {
+ f.get();
+ dns_log.trace("Send {}. {} bytes sent.", fd, bytes);
+ } catch (...) {
+ dns_log.debug("Send {} failed: {}", fd, std::current_exception());
+ }
+ e.avail |= POLLOUT;
+ me->poll_sockets();
+ me->release(fd);
+ });
+ // c-ares does _not_ use non-blocking retry for udp sockets. We just pretend
+ // all is fine even though we have no idea. Barring stack/adapter failure it
+ // is close to the same guarantee a "normal" message send would have anyway.
+ // For tcp we also pretend we're done, to make sure we don't have to deal with
+ // matching sent data
+ }
+ if (f.failed()) {
+ try {
+ f.get();
+ } catch (std::system_error& e) {
+ errno = e.code().value();
+ } catch (...) {
+ }
+ return -1;
+ }
+
+ return len;
+ }
+ } catch (...) {
+ }
+ return -1;
+ }
+ int next_fd() {
+ int fd = int(_sockets.size() + 1);
+ while (_sockets.count(fd)) {
+ ++fd;
+ }
+ return fd;
+ }
+ struct tcp_entry {
+ tcp_entry(connected_socket s)
+ : socket(std::move(s)) {
+ }
+ ;
+ connected_socket socket;
+ std::experimental::optional<input_stream<char>> in;
+ std::experimental::optional<output_stream<char>> out;
+ temporary_buffer<char> indata;
+ };
+ struct udp_entry {
+ udp_entry(::net::udp_channel c)
+ : channel(std::move(c)) {
+ }
+ ::net::udp_channel channel;
+ std::experimental::optional<::net::udp_datagram> in;;
+ socket_address dst;
+ };
+ struct sock_entry {
+ union {
+ tcp_entry tcp;
+ udp_entry udp;
+ };
+ type typ;
+ int avail = 0;
+ int pending = 0;
+ bool closed = false;
+
+ sock_entry(sock_entry&& e)
+ : typ(e.typ)
+ , avail(e.avail)
+ {
+ e.typ = type::none;
+ switch (typ) {
+ case type::tcp:
+ tcp = std::move(e.tcp);
+ break;
+ case type::udp:
+ udp = std::move(e.udp);
+ break;
+ default:
+ break;
+ }
+ }
+ sock_entry(connected_socket s)
+ default:
+ break;
+ }
+ }
+ : tcp(tcp_entry{std::move(s)})
+ , typ(type::tcp)
+ {}
+ sock_entry(::net::udp_channel c)
+ : udp(udp_entry{std::move(c)})
+ , typ(type::udp)
+ {}
+ ~sock_entry() {
+ switch (typ) {
+ case type::tcp: tcp.~tcp_entry(); break;
+ case type::udp: udp.~udp_entry(); break;
+ default: break;
+ }
+ }
+ };
+
+ sock_entry& get_socket_entry(ares_socket_t fd) {
+ auto& e = _sockets.at(fd);
+ if (e.closed) {
+ throw std::runtime_error("Socket closed");
+ }
+ return e;
+ }
+
+
+ typedef std::unordered_map<ares_socket_t, sock_entry> socket_map;
+
+ friend struct dns_call;
+
+ socket_map _sockets;
+ ::network_stack & _stack;
+
+ ares_channel _channel = {};
+ uint64_t _ops = 0, _calls = 0;
+ std::chrono::milliseconds _timeout;
+ timer<> _timer;
+ gate _gate;
+ bool _closed = false;
+};
+
+seastar::net::dns_resolver::dns_resolver()
+ : dns_resolver(options())
+{}
+
+seastar::net::dns_resolver::dns_resolver(const options& opts)
+ : dns_resolver(engine().net(), opts)
+{}
+
+seastar::net::dns_resolver::dns_resolver(network_stack& stack, const options& opts)
+ : _impl(::make_shared<impl>(stack, opts))
+{}
+
+seastar::net::dns_resolver::~dns_resolver()
+{}
+
+seastar::net::dns_resolver::dns_resolver(dns_resolver&&) = default;
+seastar::net::dns_resolver& seastar::net::dns_resolver::operator=(dns_resolver&&) = default;
+
+future<seastar::net::hostent> seastar::net::dns_resolver::get_host_by_name(const sstring& name, opt_family family) {
+ return _impl->get_host_by_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<seastar::net::hostent> seastar::net::dns_resolver::get_host_by_addr(const inet_address& addr) {
+ return _impl->get_host_by_addr(addr);
+}
+
+future<seastar::net::inet_address> seastar::net::dns_resolver::resolve_name(const sstring& name, opt_family family) {
+ return _impl->resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns_resolver::resolve_addr(const inet_address& addr) {
+ return _impl->resolve_addr(addr);
+}
+
+future<> seastar::net::dns_resolver::close() {
+ return _impl->close();
+}
+
+static seastar::net::dns_resolver& resolver() {
+ static thread_local seastar::net::dns_resolver resolver;
+ return resolver;
+}
+
+
+future<seastar::net::hostent> seastar::net::dns::get_host_by_name(const sstring& name, opt_family family) {
+ return resolver().get_host_by_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<seastar::net::hostent> seastar::net::dns::get_host_by_addr(const inet_address& addr) {
+ return resolver().get_host_by_addr(addr);
+}
+
+future<seastar::net::inet_address> seastar::net::dns::resolve_name(const sstring& name, opt_family family) {
+ return resolver().resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns::resolve_addr(const inet_address& addr) {
+ return resolver().resolve_addr(addr);
+}
diff --git a/tests/dns_test.cc b/tests/dns_test.cc
new file mode 100644
index 0000000..2999d0e
--- /dev/null
+++ b/tests/dns_test.cc
@@ -0,0 +1,105 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+ * Copyright (C) 2016 ScyllaDB.
+ */
+#include <vector>
+#include <algorithm>
+
+#include "core/do_with.hh"
+#include "test-utils.hh"
+#include "core/sstring.hh"
+#include "core/reactor.hh"
+#include "core/do_with.hh"
+#include "core/future-util.hh"
+#include "net/dns.hh"
+#include "net/inet_address.hh"
+
+using namespace seastar;
+using namespace seastar::net;
+
+static const inet_address google_addr = inet_address("216.58.201.164");
+static const sstring google_name = "www.google.com";
+
+static future<> test_resolve(dns_resolver::options opts) {
+ auto d = ::make_lw_shared<dns_resolver>(std::move(opts));
+ return d->get_host_by_name(google_name, inet_address::family::INET).then([d](hostent e) {
+ //BOOST_REQUIRE(std::count(e.addr_list.begin(), e.addr_list.end(), google_addr));
+ return d->get_host_by_addr(e.addr_list.front()).then([d, a = e.addr_list.front()](hostent e) {
+ return d->get_host_by_name(e.names.front(), inet_address::family::INET).then([a](hostent e) {
+ BOOST_REQUIRE(std::count(e.addr_list.begin(), e.addr_list.end(), a));
+ });
+ });
+ }).finally([d]{
+ return d->close();
+ });
+}
+
+static future<> test_bad_name(dns_resolver::options opts) {
+ auto d = ::make_lw_shared<dns_resolver>(std::move(opts));
+ return d->get_host_by_name("apa.ninja.gnu", inet_address::family::INET).then_wrapped([d](future<hostent> f) {
+ try {
+ f.get();
+ BOOST_FAIL("should not succeed");
+ } catch (...) {
+ // ok.
+ }
+ }).finally([d]{
+ return d->close();
+ });
+}
+
+SEASTAR_TEST_CASE(test_resolve_udp) {
+ return test_resolve(dns_resolver::options());
+}
+
+SEASTAR_TEST_CASE(test_bad_name_udp) {
+ return test_bad_name(dns_resolver::options());
+}
+
+SEASTAR_TEST_CASE(test_timeout_udp) {
+ dns_resolver::options opts;
+ opts.servers = std::vector<inet_address>({ inet_address("1.2.3.4") }); // not a server
+ opts.timeout = std::chrono::milliseconds(500);
+
+ auto d = ::make_lw_shared<dns_resolver>(engine().net(), opts);
+ return d->get_host_by_name(google_name, inet_address::family::INET).then_wrapped([d](future<hostent> f) {
+ try {
+ f.get();
+ BOOST_FAIL("should not succeed");
+ } catch (...) {
+ // ok.
+ }
+ }).finally([d]{
+ return d->close();
+ });
+}
+
+SEASTAR_TEST_CASE(test_resolve_tcp) {
+ dns_resolver::options opts;
+ opts.use_tcp_query = true;
+ return test_resolve(opts);
+}
+
+SEASTAR_TEST_CASE(test_bad_name_tcp) {
+ dns_resolver::options opts;
+ opts.use_tcp_query = true;
+ return test_bad_name(opts);
+}
+
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 3:55:22 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index d19ad50..d8ba3f6 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
@@ -67,6 +67,14 @@ struct inet_address {
operator const ::in_addr&() const;
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
std::ostream& operator<<(std::ostream&, const inet_address&);
diff --git a/net/inet_address.cc b/net/inet_address.cc
index f0cec76..53ff0c1 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -24,6 +24,7 @@
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
seastar::net::inet_address::inet_address()
: in_family(family::INET) {
@@ -87,6 +88,40 @@ size_t seastar::net::inet_address::size() const {
}
}
+future<sstring> seastar::net::inet_address::hostname() const {
+ return dns::resolve_addr(*this);
+}
+
+future<std::vector<sstring>> seastar::net::inet_address::aliases() const {
+ return dns::get_host_by_addr(*this).then([](hostent e) {
+ return make_ready_future<std::vector<sstring>>(std::move(e.names));
+ });
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name) {
+ return dns::resolve_name(name);
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name, family f) {
+ return dns::resolve_name(name, f);
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name) {
+ return dns::get_host_by_name(name).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name, family f) {
+ return dns::get_host_by_name(name, f).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
char buffer[64];
1.9.1
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index d19ad50..d8ba3f6 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
@@ -67,6 +67,14 @@ struct inet_address {
operator const ::in_addr&() const;
operator const ::in6_addr&() const;
+
+ future<sstring> hostname() const;
+
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
std::ostream& operator<<(std::ostream&, const inet_address&);
diff --git a/net/inet_address.cc b/net/inet_address.cc
index f0cec76..53ff0c1 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -24,6 +24,7 @@
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
seastar::net::inet_address::inet_address()
: in_family(family::INET) {
}
}
+future<sstring> seastar::net::inet_address::hostname() const {
+ return dns::resolve_addr(*this);
+}
+
+future<std::vector<sstring>> seastar::net::inet_address::aliases() const {
+ return dns::get_host_by_addr(*this).then([](hostent e) {
+ return make_ready_future<std::vector<sstring>>(std::move(e.names));
+ });
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name) {
+ return dns::resolve_name(name);
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name, family f) {
+ return dns::resolve_name(name, f);
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name) {
+ return dns::get_host_by_name(name).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name, family f) {
+ return dns::get_host_by_name(name, f).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
char buffer[64];
return os << inet_ntop(int(addr.in_family), &addr.in, buffer, sizeof(buffer));
--
1.9.1
Pekka Enberg
<penberg@scylladb.com>Nov 29, 2016, 4:03:58 AM11/29/16
to Calle Wilund, seastar-dev, Avi Kivity
On Tue, Nov 29, 2016 at 10:54 AM, Calle Wilund <ca...@scylladb.com> wrote:
> Note: this adds my fork of the library, containing changes for
> virtual IO functions, required for the lib to be really useful to
> scylla.
>
> Should be changed to either fork in the scylladb account, or
> to main trunk of library once the above changes are accepted (if).
The latter is our preferred choice. Did you submit your changes
> Note: this adds my fork of the library, containing changes for
> virtual IO functions, required for the lib to be really useful to
> scylla.
>
> Should be changed to either fork in the scylladb account, or
> to main trunk of library once the above changes are accepted (if).
upstream already?
- Pekka
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 4:06:03 AM11/29/16
to Pekka Enberg, seastar-dev, Avi Kivity
dealt with first,
so I can be sure the API devised works for us. But the maintainer is
amendable to the
changes, so it should be fairly doable.
> - Pekka
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:09:31 AM11/29/16
to calle@scylladb.com, Pekka Enberg, seastar-dev
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:11:13 AM11/29/16
to Pekka Enberg, Calle Wilund, seastar-dev
guard against the main repo moving. But you are of course correct that
we really want to upstream these changes.
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 4:13:34 AM11/29/16
to Avi Kivity, Pekka Enberg, seastar-dev
https://github.com/elcallio/c-ares/commits/master
also. In colour!
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:16:14 AM11/29/16
to Calle Wilund, seastar-dev@googlegroups.com
do it for you.
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:29:02 AM11/29/16
to Calle Wilund, seastar-dev@googlegroups.com
On 11/29/2016 10:55 AM, Calle Wilund wrote:
> + dns_resolver(const options&);
explicit
> + dns_resolver(::network_stack&, const options& = {});
resolvers left and right.
> + // TODO:
> + // We only allow querying dns server. We ignore hosts
> + // files. Either we need to add virtual file calls
> + // to c-ares (huger!)
> + // or we do our read+(partial)parse outselved for
> + // hosts files.
> + // Big downside here right now that we can essentially
> + // get different answers on a hsotfile based system
doesn't do so on each query (even if it does, we can expect the read
never to block).
pointer to errno, or return errno and accept the main return value via a
pointer (like posix_foo).
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:34:18 AM11/29/16
to Calle Wilund, seastar-dev@googlegroups.com
On 11/29/2016 10:55 AM, Calle Wilund wrote:
> Adds seastar::net::dns_resolver with methods to query names/addresses.
> Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
> and uses our patched virtual IO provider struct for c-ares.
>
> Like seastar, only handles ipv4 atm.
> Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
> and uses our patched virtual IO provider struct for c-ares.
>
> Like seastar, only handles ipv4 atm.
> + int next_fd() {
> + int fd = int(_sockets.size() + 1);
> + while (_sockets.count(fd)) {
> + ++fd;
> + }
> + return fd;
> + }
>
Let's start the first fd returned at 1'000'000'000, so that if we have
> + int fd = int(_sockets.size() + 1);
> + while (_sockets.count(fd)) {
> + ++fd;
> + }
> + return fd;
> + }
>
c-ares accidentally calling a system call, we fail fast, and not in a
weird data-corrupting way.
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 4:34:53 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
>> + dns_resolver(const options&);
>
> explicit
>
>> + dns_resolver(::network_stack&, const options& = {});
>
> explicit
>
Why? Afaict these constructors have no chance of aliasing? (Not that I
>
> explicit
>
>> + dns_resolver(::network_stack&, const options& = {});
>
> explicit
>
am against, just curious)
blockingness of the read is low.
there is no errno for closed fd.
(Also, the library is actually pretty strict, and will not hit the above
check really, its there just because I am paranoid.
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:35:16 AM11/29/16
to calle@scylladb.com, Pekka Enberg, seastar-dev
open a pull request and we'll see what happens next.
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 4:43:49 AM11/29/16
to calle@scylladb.com, seastar-dev@googlegroups.com
and is passed a network stack instead, I think the compiler should complain.
just virtualize it as blocking calls, and run it in a seastar::thread.
But we can start without it, we won't have many dns calls and the odds
of any single one blocking are very low.
> (Also, the library is actually pretty strict, and will not hit the
> above check really, its there just because I am paranoid.
>
>>> + int fd = next_fd();
>>> + switch (type) {
>>> + case SOCK_STREAM:
>>> + _sockets.emplace(fd, connected_socket());
>>> + dns_log.trace("Created tcp socket {}", fd);
>>> + break;
>>> + case SOCK_DGRAM:
>>> + _sockets.emplace(fd, _stack.make_udp_channel());
>>> + dns_log.trace("Created udp socket {}", fd);
>>> + break;
>>> + default: return -1;
ones are being closed, but let's assume c-ares is well-behaved and won't
do that.
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 4:50:06 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
(some already are) the c-ares parse routines.
Luckily hosts files are not overly complex.
I'm pretty sure c-ares does not cache host file info because a.) its
less unix (unix always reads the file), and b.) weird pitfals with
timestamps on some systems.
We could ignore this and just read the file + timestamp check if it
becomes an issue. But not before. For scylla, it should not be much of
an issue (at least not config). But for a http server etc, it might be.
(reverse resolve on each call).
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 5:21:55 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
std::experimental::variant exists on all
gcc versions we support. For example, my v5.3.0 lacks it sorely.
what he does"
attitude that the corresponding struct usage normally has. But I'm
willing to be strict.
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 5:26:31 AM11/29/16
to calle@scylladb.com, seastar-dev@googlegroups.com
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 5:27:51 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 5:55:30 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
instead makes all interaction with the posix/BSD functions (like
inet_pton, or even c-ares) a bit of a pain... And it also adds redundant
data, since you would suddenly
have both the in_family field, and the invisible "which" of the variant.
And ints matter! :-P
So, either synthesize family from variant info (gah - cross bridge for
water), or have redundant fields, or, with a variant case, having to add
essentially a "union-maker", i.e. an "void * addr() const" function that
returns one of the two types (i.e. boths) address, so we can be friends
with posix calls.
Seems like a lot of hoobla for such a simplistic type... :-(
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 5:57:43 AM11/29/16
to calle@scylladb.com, seastar-dev@googlegroups.com
method instead of state).
>
> So, either synthesize family from variant info (gah - cross bridge for
> water), or have redundant fields, or, with a variant case, having to
> add essentially a "union-maker", i.e. an "void * addr() const"
> function that returns one of the two types (i.e. boths) address, so we
> can be friends with posix calls.
>
> Seems like a lot of hoobla for such a simplistic type... :-(
conversion operators.
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 8:14:18 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
That is unfortunately not possible, because c-ares uses fd_set:s to
communicate readiness of fd:s from client (poller)
and library. And those are bitsets that only handle a limited set of fd
values.
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 8:16:51 AM11/29/16
to calle@scylladb.com, seastar-dev@googlegroups.com
Carl Wilund
<calle@scylladb.com>Nov 29, 2016, 8:22:27 AM11/29/16
to Avi Kivity, seastar-dev@googlegroups.com
distinguish the fd. fd_set capcity is typically 1024.
Avi Kivity
<avi@scylladb.com>Nov 29, 2016, 8:23:58 AM11/29/16
to calle@scylladb.com, seastar-dev@googlegroups.com
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:12 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Note: this adds my fork of the library, containing changes for
virtual IO functions, required for the lib to be really useful to
scylla.
Should be changed to either fork in the scylladb account, or
to main trunk of library once the above changes are accepted (if).
---
virtual IO functions, required for the lib to be really useful to
scylla.
Should be changed to either fork in the scylladb account, or
to main trunk of library once the above changes are accepted (if).
.gitmodules | 3 +++
c-ares | 1 +
2 files changed, 4 insertions(+)
create mode 160000 c-ares
diff --git a/.gitmodules b/.gitmodules
index ac15ad4..fd96ea7 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -4,3 +4,6 @@
[submodule "fmt"]
path = fmt
url = ../fmt
+[submodule "c-ares"]
+ path = c-ares
+ url = https://github.com/elcallio/c-ares.git
diff --git a/c-ares b/c-ares
new file mode 160000
index 0000000..9668918
--- /dev/null
+++ b/c-ares
@@ -0,0 +1 @@
+Subproject commit 966891821b4d5ae622d9ab31f38b740f39650106
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:13 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
diff --git a/configure.py b/configure.py
index 9890cf1..a1be833 100755
--- a/configure.py
+++ b/configure.py
@@ -155,12 +155,14 @@ modes = {
'sanitize_libs': '-lasan -lubsan',
'opt': '-O0 -DDEBUG -DDEBUG_SHARED_PTR -DDEFAULT_ALLOCATOR -DSEASTAR_THREAD_STACK_GUARDS',
'libs': '',
+ 'cares_opts': '--enable-debug',
},
'release': {
'sanitize': '',
'sanitize_libs': '',
'opt': '-O2',
'libs': '',
+ 'cares_opts': '',
},
}
@@ -608,6 +610,25 @@ if args.dpdk:
if file.endswith('.h') or file.endswith('.c')]
dpdk_sources = ' '.join(dpdk_sources)
+# both source and builddir location
+cares_dir = 'c-ares'
+cares_lib = 'cares-seastar'
+cares_src_lib = cares_dir + '/.libs/libcares.a'
+
+if not os.path.exists(cares_dir) or not os.listdir(cares_dir):
+ raise Exception(cares_dir + ' is empty. Run "git submodule update --init".')
+
+if not os.path.exists(cares_dir + '/configure'):
+ subprocess.check_call('sh -x buildconf', cwd=cares_dir, shell=True)
+
+cares_sources = []
+for root, dirs, files in os.walk('c-ares'):
+ cares_sources += [os.path.join(root, file)
+ for file in files
+ if file.endswith('.h') or file.endswith('.c')]
+cares_sources = ' '.join(cares_sources)
+libs += ' -l' + cares_lib
+
outdir = 'build'
buildfile = 'build.ninja'
os.makedirs(outdir, exist_ok = True)
@@ -641,6 +662,8 @@ with open(buildfile, 'w') as f:
rule protobuf
command = protoc --cpp_out=$outdir $in
description = PROTOC $out
+ rule copy_file
+ command = cp $in $out
''').format(**globals()))
if args.dpdk:
f.write(textwrap.dedent('''\
@@ -657,26 +680,36 @@ with open(buildfile, 'w') as f:
elif modeval['sanitize']:
modeval['sanitize'] += ' -DASAN_ENABLED'
f.write(textwrap.dedent('''\
- cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen
+ cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen -I $builddir/{mode}/c-ares
libs_{mode} = {sanitize_libs} {libs}
rule cxx.{mode}
command = $cxx -MMD -MT $out -MF $out.d $cxxflags_{mode} $cxxflags -c -o $out $in
description = CXX $out
depfile = $out.d
rule link.{mode}
- command = $cxx $cxxflags_{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK $out
pool = link_pool
rule link_stripped.{mode}
- command = $cxx $cxxflags_{mode} -s $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -s -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK (stripped) $out
pool = link_pool
rule ar.{mode}
command = rm -f $out; ar cr $out $in; ranlib $out
description = AR $out
''').format(mode = mode, **modeval))
- f.write('build {mode}: phony {artifacts}\n'.format(mode = mode,
+ f.write('build {mode}: phony $builddir/{mode}/lib{cares_lib}.a {artifacts}\n'.format(mode = mode, cares_lib=cares_lib,
artifacts = str.join(' ', ('$builddir/' + mode + '/' + x for x in build_artifacts))))
+ f.write(textwrap.dedent('''\
+ rule caresmake_{mode}
+ command = make -C build/{mode}/{cares_dir}
+ rule caresconfigure_{mode}
+ command = mkdir -p $builddir/{mode}/{cares_dir} && cd $builddir/{mode}/{cares_dir} && {srcdir}/$in {cares_opts}
+ build $builddir/{mode}/{cares_dir}/Makefile : caresconfigure_{mode} {cares_dir}/configure
+ build $builddir/{mode}/{cares_dir}/ares_build.h : phony $builddir/{mode}/{cares_dir}/Makefile
+ build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
+ build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
+ ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
f.write('build $builddir/{}/{}_g: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
f.write(' extralibs = {}\n'.format(' '.join(extralibs)))
else:
- f.write('build $builddir/{}/{}: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
+ f.write('build $builddir/{}/{}: link.{} {} | {} $builddir/{}/lib{}.a\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps, mode, cares_lib))
for src in srcs:
if src.endswith('.cc'):
@@ -756,7 +789,7 @@ with open(buildfile, 'w') as f:
rule configure
command = python3 configure.py $configure_args
generator = 1
- build build.ninja: configure | configure.py
+ build build.ninja: configure | configure.py {cares_dir}/configure
rule cscope
command = find -name '*.[chS]' -o -name "*.cc" -o -name "*.hh" | cscope -bq -i-
description = CSCOPE
--
1.9.1
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
diff --git a/configure.py b/configure.py
index 9890cf1..a1be833 100755
--- a/configure.py
+++ b/configure.py
@@ -155,12 +155,14 @@ modes = {
'sanitize_libs': '-lasan -lubsan',
'opt': '-O0 -DDEBUG -DDEBUG_SHARED_PTR -DDEFAULT_ALLOCATOR -DSEASTAR_THREAD_STACK_GUARDS',
'libs': '',
+ 'cares_opts': '--enable-debug',
},
'release': {
'sanitize': '',
'sanitize_libs': '',
'opt': '-O2',
'libs': '',
+ 'cares_opts': '',
},
}
@@ -608,6 +610,25 @@ if args.dpdk:
if file.endswith('.h') or file.endswith('.c')]
dpdk_sources = ' '.join(dpdk_sources)
+# both source and builddir location
+cares_dir = 'c-ares'
+cares_lib = 'cares-seastar'
+cares_src_lib = cares_dir + '/.libs/libcares.a'
+
+if not os.path.exists(cares_dir) or not os.listdir(cares_dir):
+ raise Exception(cares_dir + ' is empty. Run "git submodule update --init".')
+
+if not os.path.exists(cares_dir + '/configure'):
+ subprocess.check_call('sh -x buildconf', cwd=cares_dir, shell=True)
+
+cares_sources = []
+for root, dirs, files in os.walk('c-ares'):
+ cares_sources += [os.path.join(root, file)
+ for file in files
+ if file.endswith('.h') or file.endswith('.c')]
+cares_sources = ' '.join(cares_sources)
+libs += ' -l' + cares_lib
+
outdir = 'build'
buildfile = 'build.ninja'
os.makedirs(outdir, exist_ok = True)
@@ -641,6 +662,8 @@ with open(buildfile, 'w') as f:
rule protobuf
command = protoc --cpp_out=$outdir $in
description = PROTOC $out
+ rule copy_file
+ command = cp $in $out
''').format(**globals()))
if args.dpdk:
f.write(textwrap.dedent('''\
@@ -657,26 +680,36 @@ with open(buildfile, 'w') as f:
elif modeval['sanitize']:
modeval['sanitize'] += ' -DASAN_ENABLED'
f.write(textwrap.dedent('''\
- cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen
+ cxxflags_{mode} = {sanitize} {opt} -I $builddir/{mode}/gen -I $builddir/{mode}/c-ares
libs_{mode} = {sanitize_libs} {libs}
rule cxx.{mode}
command = $cxx -MMD -MT $out -MF $out.d $cxxflags_{mode} $cxxflags -c -o $out $in
description = CXX $out
depfile = $out.d
rule link.{mode}
- command = $cxx $cxxflags_{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK $out
pool = link_pool
rule link_stripped.{mode}
- command = $cxx $cxxflags_{mode} -s $ldflags -o $out $in $libs $libs_{mode} $extralibs
+ command = $cxx $cxxflags_{mode} -s -L$builddir/{mode} $ldflags -o $out $in $libs $libs_{mode} $extralibs
description = LINK (stripped) $out
pool = link_pool
rule ar.{mode}
command = rm -f $out; ar cr $out $in; ranlib $out
description = AR $out
''').format(mode = mode, **modeval))
- f.write('build {mode}: phony {artifacts}\n'.format(mode = mode,
+ f.write('build {mode}: phony $builddir/{mode}/lib{cares_lib}.a {artifacts}\n'.format(mode = mode, cares_lib=cares_lib,
artifacts = str.join(' ', ('$builddir/' + mode + '/' + x for x in build_artifacts))))
+ f.write(textwrap.dedent('''\
+ rule caresmake_{mode}
+ command = make -C build/{mode}/{cares_dir}
+ rule caresconfigure_{mode}
+ command = mkdir -p $builddir/{mode}/{cares_dir} && cd $builddir/{mode}/{cares_dir} && {srcdir}/$in {cares_opts}
+ build $builddir/{mode}/{cares_dir}/Makefile : caresconfigure_{mode} {cares_dir}/configure
+ build $builddir/{mode}/{cares_dir}/ares_build.h : phony $builddir/{mode}/{cares_dir}/Makefile
+ build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
+ build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
+ ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
compiles = {}
ragels = {}
swaggers = {}
@@ -718,7 +751,7 @@ with open(buildfile, 'w') as f:
ragels = {}
swaggers = {}
f.write('build $builddir/{}/{}_g: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
f.write(' extralibs = {}\n'.format(' '.join(extralibs)))
else:
- f.write('build $builddir/{}/{}: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
+ f.write('build $builddir/{}/{}: link.{} {} | {} $builddir/{}/lib{}.a\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps, mode, cares_lib))
for src in srcs:
if src.endswith('.cc'):
@@ -756,7 +789,7 @@ with open(buildfile, 'w') as f:
rule configure
command = python3 configure.py $configure_args
generator = 1
- build build.ninja: configure | configure.py
+ build build.ninja: configure | configure.py {cares_dir}/configure
rule cscope
command = find -name '*.[chS]' -o -name "*.cc" -o -name "*.hh" | cscope -bq -i-
description = CSCOPE
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:14 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
C-ares based DNS query module. Based on fork of the library using virtual
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
v2:
* Allow file (/etc/hosts) resolution, even though file io is
not virtualized. Until proven to be an issue
* Added "noexcept" and "explicit" where appropriate to dns_resolver
contructor/assign-op
* inet_address is now more encapsulated + cast operators throw if
types mismatch
Calle Wilund (9):
Add c-ares (dns query library) as submodule
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
configure.py | 54 +++-
net/dns.hh | 131 ++++++++
net/inet_address.hh | 90 ++++++
net/packet.hh | 2 +-
net/socket_defs.hh | 6 +
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 157 +++++++++
net/posix-stack.cc | 6 +
net/stack.cc | 9 +
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
13 files changed, 1463 insertions(+), 8 deletions(-)
create mode 100644 net/dns.hh
create mode 100644 net/inet_address.hh
create mode 100644 net/dns.cc
create mode 100644 net/inet_address.cc
create mode 100644 tests/dns_test.cc
create mode 160000 c-ares
--
1.9.1
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
v2:
* Allow file (/etc/hosts) resolution, even though file io is
not virtualized. Until proven to be an issue
* Added "noexcept" and "explicit" where appropriate to dns_resolver
contructor/assign-op
* inet_address is now more encapsulated + cast operators throw if
types mismatch
Calle Wilund (9):
Add c-ares (dns query library) as submodule
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
configure.py | 54 +++-
net/dns.hh | 131 ++++++++
net/inet_address.hh | 90 ++++++
net/packet.hh | 2 +-
net/socket_defs.hh | 6 +
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 157 +++++++++
net/posix-stack.cc | 6 +
net/stack.cc | 9 +
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
13 files changed, 1463 insertions(+), 8 deletions(-)
create mode 100644 net/dns.hh
create mode 100644 net/inet_address.hh
create mode 100644 net/dns.cc
create mode 100644 net/inet_address.cc
create mode 100644 tests/dns_test.cc
create mode 160000 c-ares
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:14 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/packet.hh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/packet.hh b/net/packet.hh
index bd0b32e..f6f4db0 100644
--- a/net/packet.hh
+++ b/net/packet.hh
@@ -381,7 +381,7 @@ class packet final {
packet::packet(Iterator begin, Iterator end, deleter del) {
unsigned nr_frags = 0, len = 0;
nr_frags = std::distance(begin, end);
- std::for_each(begin, end, [&] (fragment& frag) { len += frag.size; });
+ std::for_each(begin, end, [&] (const fragment& frag) { len += frag.size; });
_impl = impl::allocate(nr_frags);
_impl->_deleter = std::move(del);
_impl->_len = len;
--
1.9.1
net/packet.hh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/packet.hh b/net/packet.hh
index bd0b32e..f6f4db0 100644
--- a/net/packet.hh
+++ b/net/packet.hh
@@ -381,7 +381,7 @@ class packet final {
packet::packet(Iterator begin, Iterator end, deleter del) {
unsigned nr_frags = 0, len = 0;
nr_frags = std::distance(begin, end);
- std::for_each(begin, end, [&] (fragment& frag) { len += frag.size; });
+ std::for_each(begin, end, [&] (const fragment& frag) { len += frag.size; });
_impl = impl::allocate(nr_frags);
_impl->_deleter = std::move(del);
_impl->_len = len;
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:15 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 1 +
net/inet_address.hh | 82 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+)
create mode 100644 net/inet_address.hh
create mode 100644 net/inet_address.cc
diff --git a/configure.py b/configure.py
index a1be833..c091318 100755
--- a/configure.py
+++ b/configure.py
--- /dev/null
+++ b/net/inet_address.hh
@@ -0,0 +1,82 @@
+class inet_address {
+public:
+ family _in_family;
+
+ union {
+ ::in_addr _in;
+ ::in6_addr _in6;
+ };
+public:
+ return _in_family;
+ }
index 0000000..ed6c0ce
--- /dev/null
+++ b/net/inet_address.cc
@@ -0,0 +1,116 @@
+ in._in_family = family::INET;
+ return in;
+ }
+ if (::inet_pton(AF_INET6, addr.c_str(), &in._in6)) {
+ in._in_family = family::INET6;
+ return false;
+ }
+ switch (_in_family) {
+ case family::INET:
+ return _in.s_addr == o._in.s_addr;
+ case family::INET6:
+ return std::equal(std::begin(_in6.s6_addr), std::end(_in6.s6_addr),
+ std::begin(o._in6.s6_addr));
+ throw std::invalid_argument("Not an ipv4 address");
+ }
+ return _in;
+ throw std::invalid_argument("Not an ipv6 address");
+ }
+ return _in6;
+}
+ case family::INET:
+ return sizeof(::in_addr);
+ case family::INET6:
+ return sizeof(::in6_addr);
+ return &_in;
1.9.1
configure.py | 1 +
net/inet_address.hh | 82 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+)
create mode 100644 net/inet_address.hh
create mode 100644 net/inet_address.cc
diff --git a/configure.py b/configure.py
index a1be833..c091318 100755
--- a/configure.py
+++ b/configure.py
@@ -268,6 +268,7 @@ libnet = [
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
new file mode 100644
index 0000000..de97837
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
new file mode 100644
--- /dev/null
+++ b/net/inet_address.hh
@@ -0,0 +1,82 @@
+public:
+ enum class family {
+ INET = AF_INET, INET6 = AF_INET6
+ };
+private:
+ INET = AF_INET, INET6 = AF_INET6
+ };
+ family _in_family;
+
+ union {
+ ::in_addr _in;
+ ::in6_addr _in6;
+ };
+public:
+
+ inet_address();
+ inet_address(::in_addr i);
+ inet_address(::in6_addr i);
+ // NOTE: does _not_ resolve the address. Only parses
+ // ipv4/ipv6 numerical address
+ inet_address(const sstring&);
+ inet_address(inet_address&&) = default;
+ inet_address(const inet_address&) = default;
+
+ inet_address& operator=(const inet_address&) = default;
+ bool operator==(const inet_address&) const;
+
+ family in_family() const {
+ inet_address();
+ inet_address(::in_addr i);
+ inet_address(::in6_addr i);
+ // NOTE: does _not_ resolve the address. Only parses
+ // ipv4/ipv6 numerical address
+ inet_address(const sstring&);
+ inet_address(inet_address&&) = default;
+ inet_address(const inet_address&) = default;
+
+ inet_address& operator=(const inet_address&) = default;
+ bool operator==(const inet_address&) const;
+
+ return _in_family;
+ }
+
+ size_t size() const;
+ const void * data() const;
+ size_t size() const;
+
+ operator const ::in_addr&() const;
+ operator const ::in6_addr&() const;
+};
+
+std::ostream& operator<<(std::ostream&, const inet_address&);
+std::ostream& operator<<(std::ostream&, const inet_address::family&);
+
+}
+}
diff --git a/net/inet_address.cc b/net/inet_address.cc
new file mode 100644
+ operator const ::in_addr&() const;
+ operator const ::in6_addr&() const;
+};
+
+std::ostream& operator<<(std::ostream&, const inet_address&);
+std::ostream& operator<<(std::ostream&, const inet_address::family&);
+
+}
+}
diff --git a/net/inet_address.cc b/net/inet_address.cc
index 0000000..ed6c0ce
--- /dev/null
+++ b/net/inet_address.cc
@@ -0,0 +1,116 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+#include <ostream>
+#include <arpa/inet.h>
+
+#include "inet_address.hh"
+
+seastar::net::inet_address::inet_address()
+ : inet_address(::in_addr{ 0, })
+#include <arpa/inet.h>
+
+#include "inet_address.hh"
+
+seastar::net::inet_address::inet_address()
+{}
+
+seastar::net::inet_address::inet_address(::in_addr i)
+ : _in_family(family::INET), _in(i) {
+
+seastar::net::inet_address::inet_address(::in_addr i)
+}
+
+seastar::net::inet_address::inet_address(::in6_addr i)
+ : _in_family(family::INET6), _in6(i) {
+
+seastar::net::inet_address::inet_address(::in6_addr i)
+}
+
+seastar::net::inet_address::inet_address(const sstring& addr)
+ : inet_address([&addr] {
+ inet_address in;
+ if (::inet_pton(AF_INET, addr.c_str(), &in._in)) {
+
+seastar::net::inet_address::inet_address(const sstring& addr)
+ : inet_address([&addr] {
+ inet_address in;
+ in._in_family = family::INET;
+ return in;
+ }
+ if (::inet_pton(AF_INET6, addr.c_str(), &in._in6)) {
+ in._in_family = family::INET6;
+ return in;
+ }
+ throw std::invalid_argument(addr);
+}())
+{}
+
+bool seastar::net::inet_address::operator==(const inet_address& o) const {
+ if (o._in_family != _in_family) {
+ }
+ throw std::invalid_argument(addr);
+}())
+{}
+
+bool seastar::net::inet_address::operator==(const inet_address& o) const {
+ return false;
+ }
+ switch (_in_family) {
+ case family::INET:
+ return _in.s_addr == o._in.s_addr;
+ case family::INET6:
+ return std::equal(std::begin(_in6.s6_addr), std::end(_in6.s6_addr),
+ std::begin(o._in6.s6_addr));
+ default:
+ return false;
+ }
+}
+
+seastar::net::inet_address::operator const ::in_addr&() const {
+ if (_in_family != family::INET) {
+ return false;
+ }
+}
+
+seastar::net::inet_address::operator const ::in_addr&() const {
+ throw std::invalid_argument("Not an ipv4 address");
+ }
+ return _in;
+}
+
+seastar::net::inet_address::operator const ::in6_addr&() const {
+ if (_in_family != family::INET6) {
+
+seastar::net::inet_address::operator const ::in6_addr&() const {
+ throw std::invalid_argument("Not an ipv6 address");
+ }
+ return _in6;
+}
+
+size_t seastar::net::inet_address::size() const {
+ switch (_in_family) {
+size_t seastar::net::inet_address::size() const {
+ case family::INET:
+ return sizeof(::in_addr);
+ case family::INET6:
+ return sizeof(::in6_addr);
+ default:
+ return 0;
+ }
+}
+
+const void * seastar::net::inet_address::data() const {
+ return 0;
+ }
+}
+
+ return &_in;
+}
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
+ char buffer[64];
+ return os << inet_ntop(int(addr.in_family()), addr.data(), buffer, sizeof(buffer));
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
+ char buffer[64];
+}
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address::family& f) {
+ switch (f) {
+ case inet_address::family::INET:
+ os << "INET";
+ break;
+ case inet_address::family::INET6:
+ os << "INET6";
+ break;
+ default:
+ break;
+ }
+ return os;
+}
+
--
+
+std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address::family& f) {
+ switch (f) {
+ case inet_address::family::INET:
+ os << "INET";
+ break;
+ case inet_address::family::INET6:
+ os << "INET6";
+ break;
+ default:
+ break;
+ }
+ return os;
+}
+
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:15 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
diff --git a/net/inet_address.cc b/net/inet_address.cc
index ed6c0ce..f3a31d9 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -23,6 +23,7 @@
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
@@ -114,3 +115,8 @@ const void * seastar::net::inet_address::data() const {
return os;
}
+std::ostream& operator<<(std::ostream& os, const socket_address& a) {
+ return os << seastar::net::inet_address(a.as_posix_sockaddr_in().sin_addr)
+ << ":" << a.u.in.sin_port
+ ;
+}
diff --git a/net/stack.cc b/net/stack.cc
index 5ef67e8..d7c0a6b 100644
--- a/net/stack.cc
+++ b/net/stack.cc
@@ -148,3 +148,12 @@ void server_socket::abort_accept() {
: socket_address(make_ipv4_address(addr))
{}
+
+bool socket_address::operator==(const socket_address& a) const {
+ // TODO: handle ipv6
+ return std::tie(u.in.sin_family, u.in.sin_port, u.in.sin_addr.s_addr)
+ == std::tie(a.u.in.sin_family, a.u.in.sin_port,
+ a.u.in.sin_addr.s_addr);
+}
+
+
--
1.9.1
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
* Copyright (C) 2016 ScyllaDB.
*/
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
diff --git a/net/inet_address.cc b/net/inet_address.cc
index ed6c0ce..f3a31d9 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -23,6 +23,7 @@
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
@@ -114,3 +115,8 @@ const void * seastar::net::inet_address::data() const {
return os;
}
+std::ostream& operator<<(std::ostream& os, const socket_address& a) {
+ return os << seastar::net::inet_address(a.as_posix_sockaddr_in().sin_addr)
+ << ":" << a.u.in.sin_port
+ ;
+}
diff --git a/net/stack.cc b/net/stack.cc
index 5ef67e8..d7c0a6b 100644
--- a/net/stack.cc
+++ b/net/stack.cc
@@ -148,3 +148,12 @@ void server_socket::abort_accept() {
: socket_address(make_ipv4_address(addr))
{}
+
+bool socket_address::operator==(const socket_address& a) const {
+ // TODO: handle ipv6
+ return std::tie(u.in.sin_family, u.in.sin_port, u.in.sin_addr.s_addr)
+ == std::tie(a.u.in.sin_family, a.u.in.sin_port,
+ a.u.in.sin_addr.s_addr);
+}
+
+
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:16 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Otherwise there is no way to get them to wake up.
---
net/udp.cc | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/udp.cc b/net/udp.cc
index 598ba71..8fedf31 100644
--- a/net/udp.cc
+++ b/net/udp.cc
@@ -108,6 +108,7 @@ class native_channel : public udp_channel_impl {
}
virtual void close() override {
+ _state->_queue.abort(std::make_exception_ptr(std::system_error(EBADF, std::system_category())));
_reg.unregister();
_closed = true;
}
--
1.9.1
---
net/udp.cc | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/udp.cc b/net/udp.cc
index 598ba71..8fedf31 100644
--- a/net/udp.cc
+++ b/net/udp.cc
@@ -108,6 +108,7 @@ class native_channel : public udp_channel_impl {
}
virtual void close() override {
+ _state->_queue.abort(std::make_exception_ptr(std::system_error(EBADF, std::system_category())));
_reg.unregister();
_closed = true;
}
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:17 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Or they shall sleep forever more
---
net/posix-stack.cc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/net/posix-stack.cc b/net/posix-stack.cc
index cd02c39..64879a3 100644
--- a/net/posix-stack.cc
+++ b/net/posix-stack.cc
@@ -433,6 +433,9 @@ class posix_udp_channel : public udp_channel_impl {
virtual future<> send(ipv4_addr dst, packet p);
virtual void close() override {
_closed = true;
+ auto p = std::make_exception_ptr(std::system_error(EBADF, std::system_category()));
+ _fd->abort_reader(p);
+ _fd->abort_writer(p);
_fd.reset();
}
virtual bool is_closed() const override { return _closed; }
@@ -476,6 +479,9 @@ class posix_datagram : public udp_datagram_impl {
auto dst = ipv4_addr(_recv._cmsg.pktinfo.ipi_addr.s_addr, _address.port);
return make_ready_future<udp_datagram>(udp_datagram(std::make_unique<posix_datagram>(
_recv._src_addr, dst, packet(fragment{_recv._buffer, size}, make_deleter([buf = _recv._buffer] { delete[] buf; })))));
+ }).handle_exception([p = _recv._buffer](auto ep) {
+ delete[] p;
+ return make_exception_future<udp_datagram>(std::move(ep));
});
}
--
1.9.1
---
net/posix-stack.cc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/net/posix-stack.cc b/net/posix-stack.cc
index cd02c39..64879a3 100644
--- a/net/posix-stack.cc
+++ b/net/posix-stack.cc
@@ -433,6 +433,9 @@ class posix_udp_channel : public udp_channel_impl {
virtual future<> send(ipv4_addr dst, packet p);
virtual void close() override {
_closed = true;
+ auto p = std::make_exception_ptr(std::system_error(EBADF, std::system_category()));
+ _fd->abort_reader(p);
+ _fd->abort_writer(p);
_fd.reset();
}
virtual bool is_closed() const override { return _closed; }
@@ -476,6 +479,9 @@ class posix_datagram : public udp_datagram_impl {
auto dst = ipv4_addr(_recv._cmsg.pktinfo.ipi_addr.s_addr, _address.port);
return make_ready_future<udp_datagram>(udp_datagram(std::make_unique<posix_datagram>(
_recv._src_addr, dst, packet(fragment{_recv._buffer, size}, make_deleter([buf = _recv._buffer] { delete[] buf; })))));
+ }).handle_exception([p = _recv._buffer](auto ep) {
+ delete[] p;
+ return make_exception_future<udp_datagram>(std::move(ep));
});
}
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:18 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
Adds seastar::net::dns_resolver with methods to query names/addresses.
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
configure.py | 8 +-
net/dns.hh | 131 ++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1149 insertions(+), 1 deletion(-)
create mode 100644 net/dns.hh
create mode 100644 net/dns.cc
create mode 100644 tests/dns_test.cc
diff --git a/configure.py b/configure.py
create mode 100644 tests/dns_test.cc
index c091318..6742554 100755
--- a/configure.py
+++ b/configure.py
build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
+ objdeps['$builddir/' + mode + '/net/dns.o'] = ' $builddir/' + mode + '/' + cares_dir + '/ares_build.h'
compiles = {}
ragels = {}
swaggers = {}
ragels = {}
swaggers = {}
@@ -773,7 +779,7 @@ with open(buildfile, 'w') as f:
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
new file mode 100644
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
index 0000000..fdf4027
--- /dev/null
+++ b/net/dns.hh
@@ -0,0 +1,131 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright 2016 Cloudius Systems
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ */
+
+#pragma once
+
+
+#pragma once
+
+#include <vector>
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/shared_ptr.hh"
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+
+namespace seastar {
+namespace net {
+
+namespace seastar {
+namespace net {
+
+ explicit dns_resolver(const options&);
+ explicit dns_resolver(::network_stack&, const options& = {});
+ ~dns_resolver();
+
+ dns_resolver& operator=(dns_resolver&&) noexcept;
+}
+}
diff --git a/net/dns.cc b/net/dns.cc
new file mode 100644
index 0000000..38b5993
--- /dev/null
+++ b/net/dns.cc
@@ -0,0 +1,906 @@
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ // For now, use the default "fb" query order
+ // (set explicitly lest we forget).
+ // We only do querying dns server really async.
+ // Reading hosts files is doen by c-ares internally
+ // and with normal fread calls. Thus they theorectically
+ // block. This can potentially be an issue for some application
+ // and if so, we need to revisit this. For now, assume
+ // it won't block us in any measurable way.
+ char buf[3] = "fb";
+ a_opts.lookups = buf; // only net
+ // Always set the timeout
+ a_opts.timeout = _timeout.count();
+ int flags = ARES_OPT_LOOKUPS|ARES_OPT_TIMEOUTMS;
+
+ if (opts.use_tcp_query && *opts.use_tcp_query) {
+ a_opts.flags = ARES_FLAG_USEVC | ARES_FLAG_PRIMARY;
+ flags |= ARES_OPT_FLAGS;
+ }
+ std::vector<in_addr> addr_tmp;
+ if (opts.servers) {
+ std::transform(opts.servers->begin(), opts.servers->end(), std::back_inserter(addr_tmp), [](const inet_address& a) {
+ if (a.in_family() != inet_address::family::INET) {
+ int flags = ARES_OPT_LOOKUPS|ARES_OPT_TIMEOUTMS;
+
+ if (opts.use_tcp_query && *opts.use_tcp_query) {
+ a_opts.flags = ARES_FLAG_USEVC | ARES_FLAG_PRIMARY;
+ flags |= ARES_OPT_FLAGS;
+ }
+ std::vector<in_addr> addr_tmp;
+ if (opts.servers) {
+ std::transform(opts.servers->begin(), opts.servers->end(), std::back_inserter(addr_tmp), [](const inet_address& a) {
+ break;
+ default:
+ break;
+ }
+ default:
+ break;
+ }
+ int fd = next_fd();
+ switch (type) {
+ case SOCK_STREAM:
+ _sockets.emplace(fd, connected_socket());
+ dns_log.trace("Created tcp socket {}", fd);
+ break;
+ case SOCK_DGRAM:
+ _sockets.emplace(fd, _stack.make_udp_channel());
+ dns_log.trace("Created udp socket {}", fd);
+ break;
+ default: return -1;
+ switch (type) {
+ case SOCK_STREAM:
+ _sockets.emplace(fd, connected_socket());
+ dns_log.trace("Created tcp socket {}", fd);
+ break;
+ case SOCK_DGRAM:
+ _sockets.emplace(fd, _stack.make_udp_channel());
+ dns_log.trace("Created udp socket {}", fd);
+ break;
+ default: return -1;
+ }
+ return fd;
+ }
+ return fd;
+ }
+ break;
+ }
+ case type::udp:
+ e.udp.channel.close();
+ release(fd);
+ break;
+ }
+ case type::udp:
+ e.udp.channel.close();
+ release(fd);
+ default:
+ default:
+ return -1;
+ }
+ return 0;
+ } catch (...) {
+ }
+ return 0;
+ } catch (...) {
+ default:
+ // ~1024 fd:s. Set to something below that in case you need to
+ // debug (maybe)
+ static constexpr ares_socket_t socket_offset = 1;
+
+ ares_socket_t next_fd() {
+ ares_socket_t fd = ares_socket_t(_sockets.size() + socket_offset);
+ while (_sockets.count(fd)) {
+ ++fd;
+ }
+ return fd;
+ }
+ ++fd;
+ }
+ return fd;
+ }
+ break;
+ default:
+ break;
+ }
+ }
+ default:
+ break;
+ }
+ }
+seastar::net::dns_resolver& seastar::net::dns_resolver::operator=(dns_resolver&&) noexcept = default;
index 0000000..2999d0e
--- /dev/null
+++ b/tests/dns_test.cc
@@ -0,0 +1,105 @@
--- /dev/null
+++ b/tests/dns_test.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 29, 2016, 8:29:19 AM11/29/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index de97837..a2904a5 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
@@ -73,6 +73,14 @@ class inet_address {
operator const ::in_addr&() const;
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
std::ostream& operator<<(std::ostream&, const inet_address&);
diff --git a/net/inet_address.cc b/net/inet_address.cc
index f3a31d9..709434e 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -24,6 +24,7 @@
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
@@ -96,6 +97,40 @@ const void * seastar::net::inet_address::data() const {
return &_in;
}
+future<sstring> seastar::net::inet_address::hostname() const {
+ return dns::resolve_addr(*this);
+}
+
+future<std::vector<sstring>> seastar::net::inet_address::aliases() const {
+ return dns::get_host_by_addr(*this).then([](hostent e) {
+ return make_ready_future<std::vector<sstring>>(std::move(e.names));
+ });
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name) {
+ return dns::resolve_name(name);
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name, family f) {
+ return dns::resolve_name(name, f);
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name) {
+ return dns::get_host_by_name(name).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name, family f) {
+ return dns::get_host_by_name(name, f).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
char buffer[64];
1.9.1
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index de97837..a2904a5 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
@@ -73,6 +73,14 @@ class inet_address {
operator const ::in_addr&() const;
operator const ::in6_addr&() const;
+
+ future<sstring> hostname() const;
+
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
std::ostream& operator<<(std::ostream&, const inet_address&);
diff --git a/net/inet_address.cc b/net/inet_address.cc
index f3a31d9..709434e 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -24,6 +24,7 @@
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
return &_in;
}
+future<sstring> seastar::net::inet_address::hostname() const {
+ return dns::resolve_addr(*this);
+}
+
+future<std::vector<sstring>> seastar::net::inet_address::aliases() const {
+ return dns::get_host_by_addr(*this).then([](hostent e) {
+ return make_ready_future<std::vector<sstring>>(std::move(e.names));
+ });
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name) {
+ return dns::resolve_name(name);
+}
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name, family f) {
+ return dns::resolve_name(name, f);
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name) {
+ return dns::get_host_by_name(name).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name, family f) {
+ return dns::get_host_by_name(name, f).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
char buffer[64];
return os << inet_ntop(int(addr.in_family()), addr.data(), buffer, sizeof(buffer));
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:50 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Note: this adds my fork of the library, containing changes for
virtual IO functions, required for the lib to be really useful to
scylla.
Should be changed to either fork in the scylladb account, or
to main trunk of library once the above changes are accepted (if).
---
.gitmodules | 3 +++
c-ares | 1 +
2 files changed, 4 insertions(+)
create mode 160000 c-ares
diff --git a/.gitmodules b/.gitmodules
index ac15ad4..fd96ea7 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -4,3 +4,6 @@
[submodule "fmt"]
path = fmt
url = ../fmt
+[submodule "c-ares"]
+ path = c-ares
+ url = https://github.com/elcallio/c-ares.git
diff --git a/c-ares b/c-ares
new file mode 160000
index 0000000..7a1092b
virtual IO functions, required for the lib to be really useful to
scylla.
Should be changed to either fork in the scylladb account, or
to main trunk of library once the above changes are accepted (if).
---
.gitmodules | 3 +++
c-ares | 1 +
2 files changed, 4 insertions(+)
create mode 160000 c-ares
diff --git a/.gitmodules b/.gitmodules
index ac15ad4..fd96ea7 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -4,3 +4,6 @@
[submodule "fmt"]
path = fmt
url = ../fmt
+[submodule "c-ares"]
+ path = c-ares
+ url = https://github.com/elcallio/c-ares.git
diff --git a/c-ares b/c-ares
new file mode 160000
--- /dev/null
+++ b/c-ares
@@ -0,0 +1 @@
+Subproject commit 7a1092b684301a98c4e30ed2126a04fb2d9672b3
+++ b/c-ares
@@ -0,0 +1 @@
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:50 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
C-ares based DNS query module. Based on fork of the library using virtual
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
v2:
* Allow file (/etc/hosts) resolution, even though file io is
not virtualized. Until proven to be an issue
* Added "noexcept" and "explicit" where appropriate to dns_resolver
contructor/assign-op
* inet_address is now more encapsulated + cast operators throw if
types mismatch
v3:
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
v2:
* Allow file (/etc/hosts) resolution, even though file io is
not virtualized. Until proven to be an issue
* Added "noexcept" and "explicit" where appropriate to dns_resolver
contructor/assign-op
* inet_address is now more encapsulated + cast operators throw if
types mismatch
* Updated submodule reference
* Added namespace qualification in net/api.hh
Calle Wilund (10):
Add c-ares (dns query library) as submodule
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
net/api.hh: qualify net::socket_impl better -> ::net::socket_impl etc
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
configure.py | 54 +++-
net/api.hh | 4 +-
net/dns.hh | 131 ++++++++
net/inet_address.hh | 90 ++++++
net/packet.hh | 2 +-
net/socket_defs.hh | 6 +
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 157 +++++++++
net/posix-stack.cc | 6 +
net/stack.cc | 9 +
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
14 files changed, 1465 insertions(+), 10 deletions(-)
net/inet_address.hh | 90 ++++++
net/packet.hh | 2 +-
net/socket_defs.hh | 6 +
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 157 +++++++++
net/posix-stack.cc | 6 +
net/stack.cc | 9 +
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
create mode 100644 net/dns.hh
create mode 100644 net/inet_address.hh
create mode 100644 net/dns.cc
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:51 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
diff --git a/configure.py b/configure.py
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
index 9890cf1..a1be833 100755
--- a/configure.py
+++ b/configure.py
+ build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
+ build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
+ ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
compiles = {}
ragels = {}
swaggers = {}
ragels = {}
swaggers = {}
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:51 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:52 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 1 +
net/inet_address.hh | 82 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+)
create mode 100644 net/inet_address.hh
configure.py | 1 +
net/inet_address.hh | 82 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+)
create mode 100644 net/inet_address.cc
diff --git a/configure.py b/configure.py
index a1be833..c091318 100755
--- a/configure.py
+++ b/configure.py
@@ -268,6 +268,7 @@ libnet = [
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
new file mode 100644
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
index 0000000..de97837
--- /dev/null
+++ b/net/inet_address.hh
@@ -0,0 +1,82 @@
--- /dev/null
+++ b/net/inet_address.hh
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+ */
+#pragma once
+
+#include <iosfwd>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <stdexcept>
+#include <vector>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <stdexcept>
+#include <vector>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+
+namespace seastar {
+namespace net {
+
+namespace seastar {
+namespace net {
+
+};
+
+std::ostream& operator<<(std::ostream&, const inet_address&);
+std::ostream& operator<<(std::ostream&, const inet_address::family&);
+
+}
+}
diff --git a/net/inet_address.cc b/net/inet_address.cc
new file mode 100644
index 0000000..ed6c0ce
--- /dev/null
+++ b/net/inet_address.cc
@@ -0,0 +1,116 @@
--- /dev/null
+++ b/net/inet_address.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+ */
+ break;
+ default:
+ break;
+ }
+ default:
+ break;
+ }
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:53 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
* Copyright (C) 2016 ScyllaDB.
*/
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
diff --git a/net/inet_address.cc b/net/inet_address.cc
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
index ed6c0ce..f3a31d9 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -23,6 +23,7 @@
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
: inet_address(::in_addr{ 0, })
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:54 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:54 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:55 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index de97837..a2904a5 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
@@ -73,6 +73,14 @@ class inet_address {
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index de97837..a2904a5 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
operator const ::in_addr&() const;
operator const ::in6_addr&() const;
+
+
+ future<sstring> hostname() const;
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
std::ostream& operator<<(std::ostream&, const inet_address&);
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
diff --git a/net/inet_address.cc b/net/inet_address.cc
index f3a31d9..709434e 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -24,6 +24,7 @@
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
: inet_address(::in_addr{ 0, })
return os << inet_ntop(int(addr.in_family()), addr.data(), buffer, sizeof(buffer));
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:55 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Adds seastar::net::dns_resolver with methods to query names/addresses.
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1149 insertions(+), 1 deletion(-)
create mode 100644 net/dns.hh
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1149 insertions(+), 1 deletion(-)
create mode 100644 net/dns.cc
create mode 100644 tests/dns_test.cc
diff --git a/configure.py b/configure.py
create mode 100644 tests/dns_test.cc
index c091318..6742554 100755
--- a/configure.py
+++ b/configure.py
build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
+ objdeps['$builddir/' + mode + '/net/dns.o'] = ' $builddir/' + mode + '/' + cares_dir + '/ares_build.h'
compiles = {}
ragels = {}
swaggers = {}
ragels = {}
swaggers = {}
@@ -773,7 +779,7 @@ with open(buildfile, 'w') as f:
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
new file mode 100644
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
index 0000000..fdf4027
--- /dev/null
+++ b/net/dns.hh
@@ -0,0 +1,131 @@
--- /dev/null
+++ b/net/dns.hh
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright 2016 Cloudius Systems
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ */
+
+#pragma once
+
+
+#pragma once
+
+#include <vector>
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/shared_ptr.hh"
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+
+namespace seastar {
+namespace net {
+
+namespace seastar {
+namespace net {
+
+}
+}
diff --git a/net/dns.cc b/net/dns.cc
new file mode 100644
index 0000000..38b5993
--- /dev/null
+++ b/net/dns.cc
@@ -0,0 +1,906 @@
--- /dev/null
+++ b/net/dns.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ break;
+ default:
+ break;
+ }
+ default:
+ break;
+ }
+ default:
+ default:
+ default:
+ break;
+ default:
+ break;
+ }
+ }
+ default:
+ break;
+ }
+ }
+
+future<seastar::net::hostent> seastar::net::dns_resolver::get_host_by_addr(const inet_address& addr) {
+ return _impl->get_host_by_addr(addr);
+}
+ return _impl->get_host_by_addr(addr);
+
+future<seastar::net::inet_address> seastar::net::dns_resolver::resolve_name(const sstring& name, opt_family family) {
+ return _impl->resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns_resolver::resolve_addr(const inet_address& addr) {
+ return _impl->resolve_addr(addr);
+}
+ return _impl->resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns_resolver::resolve_addr(const inet_address& addr) {
+ return _impl->resolve_addr(addr);
+
+future<> seastar::net::dns_resolver::close() {
+ return _impl->close();
+}
+
+static seastar::net::dns_resolver& resolver() {
+ static thread_local seastar::net::dns_resolver resolver;
+ return resolver;
+}
+
+ return _impl->close();
+}
+
+static seastar::net::dns_resolver& resolver() {
+ static thread_local seastar::net::dns_resolver resolver;
+ return resolver;
+}
+
+future<seastar::net::hostent> seastar::net::dns::get_host_by_name(const sstring& name, opt_family family) {
+ return resolver().get_host_by_name(name, family.value_or(inet_address::family::INET));
+}
+ return resolver().get_host_by_name(name, family.value_or(inet_address::family::INET));
+
+future<seastar::net::hostent> seastar::net::dns::get_host_by_addr(const inet_address& addr) {
+ return resolver().get_host_by_addr(addr);
+}
+ return resolver().get_host_by_addr(addr);
+
+future<seastar::net::inet_address> seastar::net::dns::resolve_name(const sstring& name, opt_family family) {
+ return resolver().resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns::resolve_addr(const inet_address& addr) {
+ return resolver().resolve_addr(addr);
+}
diff --git a/tests/dns_test.cc b/tests/dns_test.cc
new file mode 100644
+ return resolver().resolve_name(name, family.value_or(inet_address::family::INET));
+}
+
+future<sstring> seastar::net::dns::resolve_addr(const inet_address& addr) {
+ return resolver().resolve_addr(addr);
+}
diff --git a/tests/dns_test.cc b/tests/dns_test.cc
index 0000000..2999d0e
--- /dev/null
+++ b/tests/dns_test.cc
@@ -0,0 +1,105 @@
--- /dev/null
+++ b/tests/dns_test.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 7:27:56 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
We have both seastar::net and ::net namespaces. We should not.
---
net/api.hh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/api.hh b/net/api.hh
index 55c5fb0..9613968 100644
--- a/net/api.hh
+++ b/net/api.hh
@@ -216,12 +216,12 @@ class connected_socket {
/// A \c socket that allows a connection to be established between
/// two endpoints.
class socket {
- std::unique_ptr<net::socket_impl> _si;
+ std::unique_ptr<::net::socket_impl> _si;
public:
~socket();
/// \cond internal
- explicit socket(std::unique_ptr<net::socket_impl> si);
+ explicit socket(std::unique_ptr<::net::socket_impl> si);
/// \endcond
/// Moves a \c seastar::socket object.
socket(socket&&) noexcept;
--
1.9.1
---
net/api.hh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/api.hh b/net/api.hh
index 55c5fb0..9613968 100644
--- a/net/api.hh
+++ b/net/api.hh
@@ -216,12 +216,12 @@ class connected_socket {
/// A \c socket that allows a connection to be established between
/// two endpoints.
class socket {
- std::unique_ptr<net::socket_impl> _si;
+ std::unique_ptr<::net::socket_impl> _si;
public:
~socket();
/// \cond internal
- explicit socket(std::unique_ptr<net::socket_impl> si);
+ explicit socket(std::unique_ptr<::net::socket_impl> si);
/// \endcond
/// Moves a \c seastar::socket object.
socket(socket&&) noexcept;
--
1.9.1
Avi Kivity
<avi@scylladb.com>Nov 30, 2016, 7:58:58 AM11/30/16
to Calle Wilund, seastar-dev@googlegroups.com
On 11/30/2016 02:27 PM, Calle Wilund wrote:
> Adds seastar::net::dns_resolver with methods to query names/addresses.
> Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
> and uses our patched virtual IO provider struct for c-ares.
>
>
> + static auto get_impl = [](void * p) { return reinterpret_cast<impl *>(p); };
> + static const ares_socket_functions callbacks = {
> + [](int af, int type, int protocol, void * p) { return get_impl(p)->do_socket(af, type, protocol); },
> + [](ares_socket_t s, void * p) { return get_impl(p)->do_close(s); },
> + [](ares_socket_t s, const struct sockaddr * addr, socklen_t len, void * p) { return get_impl(p)->do_connect(s, addr, len); },
> + [](ares_socket_t s, void * dst, size_t len, int flags, struct sockaddr * addr, socklen_t * alen, void * p) {
> + return get_impl(p)->do_recvfrom(s, dst, len, flags, addr, alen);
> + },
> + [](ares_socket_t s, const struct iovec * vec, int len, void * p) {
> + return get_impl(p)->do_sendv(s, vec, len);
> + },
> + };
> +
> + ares_set_socket_functions(_channel, &callbacks, this);
>
How is backward compatibility planned for this? Suppose ares needs more
> + static const ares_socket_functions callbacks = {
> + [](int af, int type, int protocol, void * p) { return get_impl(p)->do_socket(af, type, protocol); },
> + [](ares_socket_t s, void * p) { return get_impl(p)->do_close(s); },
> + [](ares_socket_t s, const struct sockaddr * addr, socklen_t len, void * p) { return get_impl(p)->do_connect(s, addr, len); },
> + [](ares_socket_t s, void * dst, size_t len, int flags, struct sockaddr * addr, socklen_t * alen, void * p) {
> + return get_impl(p)->do_recvfrom(s, dst, len, flags, addr, alen);
> + },
> + [](ares_socket_t s, const struct iovec * vec, int len, void * p) {
> + return get_impl(p)->do_sendv(s, vec, len);
> + },
> + };
> +
> + ares_set_socket_functions(_channel, &callbacks, this);
>
callbacks implemented?
One way to solve this is to pass sizeof(callbacks) to
ares_set_socket_functions(), which memcpy() it into a zero-initialized
struct ares_socket_functions. Then ares_set_socket_functions() knows
some of the functions were missing, and can work around them (by calling
basic-er functions that do exist, not calling optional functions, or
returning an error when the functionality is critical).
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 8:07:02 AM11/30/16
to Avi Kivity, seastar-dev@googlegroups.com
struct ares_more_functions {
struct ares_socket_functions callbacks base;
.... more calls
};
and add a new setter, eliminating any aliasing at all. Which imho seems
more in line with how the lib maintainer want
options/settables to move forward.
Avi Kivity
<avi@scylladb.com>Nov 30, 2016, 8:14:51 AM11/30/16
to Calle Wilund, seastar-dev@googlegroups.com
Let's just make sure it's addressed somehow.
Does it look on track for merging?
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 8:18:20 AM11/30/16
to Avi Kivity, seastar-dev@googlegroups.com
the whole curl thing, it might be a while.
We could just set up a scylladb fork based on my fork/patches and assume
that any final result of the merge debate will
still end up somewhere in the same neighbourhood of functionality, i.e.
within the limits of code adaption on our part.
Avi Kivity
<avi@scylladb.com>Nov 30, 2016, 8:24:45 AM11/30/16
to Calle Wilund, seastar-dev@googlegroups.com
guidance is, and take action according to that. If he doesn't respond
we'll fork, but I'm optimistic he'll respond soon. So can also try on
the mailing list, I think he mentioned it when I brought up the subject
(maybe a year ago) and evaporated.
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 8:26:51 AM11/30/16
to Avi Kivity, seastar-dev@googlegroups.com
he'll probably get back to me within a day or two.
To be fair, I don't think the travis build has even finished yet. The
wait queue for OSX builds seems to be absurd.
Avi Kivity
<avi@scylladb.com>Nov 30, 2016, 8:29:04 AM11/30/16
to Calle Wilund, seastar-dev@googlegroups.com
> But I sent him an email also, he'll probably get back to me within a
> day or two.
> To be fair, I don't think the travis build has even finished yet. The
> wait queue for OSX builds seems to be absurd.
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:08 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
C-ares based DNS query module. Based on fork of the library using virtual
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
v2:
* Allow file (/etc/hosts) resolution, even though file io is
not virtualized. Until proven to be an issue
* Added "noexcept" and "explicit" where appropriate to dns_resolver
contructor/assign-op
* inet_address is now more encapsulated + cast operators throw if
types mismatch
v3:
* Updated submodule reference
* Added namespace qualification in net/api.hh
v4:
socket io functions. With some luck, once we're happy about this code, it
can be put into the mainline library.
* Adds (fork of) c-ares as submodule + building and linking with the same.
* Adds seastar::net::dns_resolver object to handle lookups and deal with
IO handling for c-ares.
* Adds inet_address type for, well, you can guess...
* Adds convinience calls for inet_address lookup.
"Normal" address resolution is done via thread_local global resolvers
attached to the active network stack. But separate objects can be used
with different settings and stacks if need be.
The resolver object does _not_ cache lookups. Such as cache is probably
better suited to build on top of it, though query of SOA records needs
to be wrapped in that case.
v2:
* Allow file (/etc/hosts) resolution, even though file io is
not virtualized. Until proven to be an issue
* Added "noexcept" and "explicit" where appropriate to dns_resolver
contructor/assign-op
* inet_address is now more encapsulated + cast operators throw if
types mismatch
v3:
* Updated submodule reference
* Added namespace qualification in net/api.hh
* More namespace qualifications in net/*
* Added ipv4_addr constructor from inet_address
Calle Wilund (11):
Add c-ares (dns query library) as submodule
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
net/*: qualify net::* refs better -> ::net::* etc
configure.py: Add building and linking with c-ares library
packet: Fix constness in templated size summary loop
Add seastar::net::inet_address type
socket_address: Add ostream<< operator and comparison
native_stack:udp: Make udp_channel_impl::close wake pending r/w ops
posix_stack:udp_channel_impl: Make close wake up pending r/w ops
Add c-ares based dns query support
seastar::net::inet_address: Add dns query wrapper functions for "easy"
lookup
net/ipv4_addr: Add constructor from seastar::net::inet_address
configure.py | 54 +++-
net/api.hh | 6 +-
net/dns.hh | 131 ++++++++
net/inet_address.hh | 90 ++++++
net/ip.hh | 6 +-
net/inet_address.hh | 90 ++++++
net/net.hh | 8 +-
net/packet.hh | 2 +-
net/posix-stack.hh | 2 +-
net/socket_defs.hh | 12 +
net/tcp.hh | 10 +-
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 157 +++++++++
net/native-stack.cc | 4 +-
net/inet_address.cc | 157 +++++++++
net/net.cc | 8 +
net/posix-stack.cc | 6 +
net/stack.cc | 11 +-
net/tls.cc | 22 +-
net/udp.cc | 1 +
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
21 files changed, 1507 insertions(+), 38 deletions(-)
tests/dns_test.cc | 105 ++++++
.gitmodules | 3 +
c-ares | 1 +
create mode 100644 net/dns.hh
create mode 100644 net/inet_address.hh
create mode 100644 net/dns.cc
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:09 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:09 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
diff --git a/configure.py b/configure.py
configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
1 file changed, 39 insertions(+), 6 deletions(-)
index 9890cf1..a1be833 100755
--- a/configure.py
+++ b/configure.py
+ build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
+ build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
+ ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
compiles = {}
ragels = {}
swaggers = {}
ragels = {}
swaggers = {}
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:10 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:11 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
configure.py | 1 +
net/inet_address.hh | 82 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+)
create mode 100644 net/inet_address.hh
configure.py | 1 +
net/inet_address.hh | 82 +++++++++++++++++++++++++++++++++++++
net/inet_address.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+)
create mode 100644 net/inet_address.cc
diff --git a/configure.py b/configure.py
index a1be833..c091318 100755
--- a/configure.py
+++ b/configure.py
@@ -268,6 +268,7 @@ libnet = [
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
new file mode 100644
'net/tcp.cc',
'net/dhcp.cc',
'net/tls.cc',
+ 'net/inet_address.cc',
]
core = [
diff --git a/net/inet_address.hh b/net/inet_address.hh
index 0000000..de97837
--- /dev/null
+++ b/net/inet_address.hh
@@ -0,0 +1,82 @@
--- /dev/null
+++ b/net/inet_address.hh
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+
+ */
+#pragma once
+
+#include <iosfwd>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <stdexcept>
+#include <vector>
+#include <sys/types.h>
+#include <netinet/in.h>
+#include <stdexcept>
+#include <vector>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+
+namespace seastar {
+namespace net {
+
+namespace seastar {
+namespace net {
+
+};
+
+std::ostream& operator<<(std::ostream&, const inet_address&);
+std::ostream& operator<<(std::ostream&, const inet_address::family&);
+
+}
+}
diff --git a/net/inet_address.cc b/net/inet_address.cc
new file mode 100644
index 0000000..ed6c0ce
--- /dev/null
+++ b/net/inet_address.cc
@@ -0,0 +1,116 @@
--- /dev/null
+++ b/net/inet_address.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) 2016 ScyllaDB.
+ */
+ */
+ break;
+ default:
+ break;
+ }
+ default:
+ break;
+ }
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:11 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
net/socket_defs.hh | 6 ++++++
net/inet_address.cc | 6 ++++++
net/stack.cc | 9 +++++++++
3 files changed, 21 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index 5638673..f23a261 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -19,6 +19,8 @@
* Copyright (C) 2016 ScyllaDB.
*/
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
diff --git a/net/inet_address.cc b/net/inet_address.cc
#pragma once
+
+#include <iosfwd>
#include <sys/socket.h>
#include <netinet/ip.h>
#include "net/byteorder.hh"
@@ -41,8 +43,12 @@ class socket_address {
::sockaddr_in& as_posix_sockaddr_in() { return u.in; }
const ::sockaddr& as_posix_sockaddr() const { return u.sa; }
const ::sockaddr_in& as_posix_sockaddr_in() const { return u.in; }
+
+ bool operator==(const socket_address&) const;
};
+std::ostream& operator<<(std::ostream&, const socket_address&);
+
namespace seastar {
enum class transport {
index ed6c0ce..f3a31d9 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -23,6 +23,7 @@
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
#include <arpa/inet.h>
#include "inet_address.hh"
+#include "socket_defs.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
: inet_address(::in_addr{ 0, })
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:12 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:13 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:14 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
Adds seastar::net::dns_resolver with methods to query names/addresses.
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Wraps nasty state machine to emulate bsd-nonblocking IO on seastar
and uses our patched virtual IO provider struct for c-ares.
Like seastar, only handles ipv4 atm.
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1149 insertions(+), 1 deletion(-)
create mode 100644 net/dns.hh
---
configure.py | 8 +-
net/dns.hh | 131 ++++++++
net/dns.cc | 906 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dns_test.cc | 105 +++++++
4 files changed, 1149 insertions(+), 1 deletion(-)
create mode 100644 net/dns.cc
create mode 100644 tests/dns_test.cc
diff --git a/configure.py b/configure.py
create mode 100644 tests/dns_test.cc
index c091318..6742554 100755
--- a/configure.py
+++ b/configure.py
build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
+ objdeps['$builddir/' + mode + '/net/dns.o'] = ' $builddir/' + mode + '/' + cares_dir + '/ares_build.h'
compiles = {}
ragels = {}
swaggers = {}
ragels = {}
swaggers = {}
@@ -773,7 +779,7 @@ with open(buildfile, 'w') as f:
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
new file mode 100644
for obj in compiles:
src = compiles[obj]
gen_headers = list(ragels.keys()) + list(swaggers.keys()) + list(protobufs.keys())
- f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps))
+ f.write('build {}: cxx.{} {} || {} \n'.format(obj, mode, src, ' '.join(gen_headers) + dpdk_deps + objdeps.get(obj, '')))
for hh in ragels:
src = ragels[hh]
f.write('build {}: ragel {}\n'.format(hh, src))
diff --git a/net/dns.hh b/net/dns.hh
index 0000000..fdf4027
--- /dev/null
+++ b/net/dns.hh
@@ -0,0 +1,131 @@
--- /dev/null
+++ b/net/dns.hh
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright 2016 Cloudius Systems
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ */
+
+#pragma once
+
+
+#pragma once
+
+#include <vector>
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+#include <unordered_map>
+#include <memory>
+#include <experimental/optional>
+
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/future.hh"
+#include "../core/sstring.hh"
+#include "../core/shared_ptr.hh"
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+#include "inet_address.hh"
+
+struct ipv4_addr;
+
+class socket_address;
+class network_stack;
+
+/**
+ * C-ares based dns query support.
+ * Handles name- and ip-based resolution.
+ *
+ */
+
+namespace seastar {
+namespace net {
+
+namespace seastar {
+namespace net {
+
+}
+}
diff --git a/net/dns.cc b/net/dns.cc
new file mode 100644
index 0000000..38b5993
--- /dev/null
+++ b/net/dns.cc
@@ -0,0 +1,906 @@
--- /dev/null
+++ b/net/dns.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ static auto get_impl = [](void * p) { return reinterpret_cast<impl *>(p); };
+ static const ares_socket_functions callbacks = {
+ [](int af, int type, int protocol, void * p) { return get_impl(p)->do_socket(af, type, protocol); },
+ [](ares_socket_t s, void * p) { return get_impl(p)->do_close(s); },
+ [](ares_socket_t s, const struct sockaddr * addr, socklen_t len, void * p) { return get_impl(p)->do_connect(s, addr, len); },
+ [](ares_socket_t s, void * dst, size_t len, int flags, struct sockaddr * addr, socklen_t * alen, void * p) {
+ return get_impl(p)->do_recvfrom(s, dst, len, flags, addr, alen);
+ },
+ [](ares_socket_t s, const struct iovec * vec, int len, void * p) {
+ return get_impl(p)->do_sendv(s, vec, len);
+ },
+ };
+
+ ares_set_socket_functions(_channel, &callbacks, this);
+ static const ares_socket_functions callbacks = {
+ [](int af, int type, int protocol, void * p) { return get_impl(p)->do_socket(af, type, protocol); },
+ [](ares_socket_t s, void * p) { return get_impl(p)->do_close(s); },
+ [](ares_socket_t s, const struct sockaddr * addr, socklen_t len, void * p) { return get_impl(p)->do_connect(s, addr, len); },
+ [](ares_socket_t s, void * dst, size_t len, int flags, struct sockaddr * addr, socklen_t * alen, void * p) {
+ return get_impl(p)->do_recvfrom(s, dst, len, flags, addr, alen);
+ },
+ [](ares_socket_t s, const struct iovec * vec, int len, void * p) {
+ return get_impl(p)->do_sendv(s, vec, len);
+ },
+ };
+
+ ares_set_socket_functions(_channel, &callbacks, this);
+ break;
+ default:
+ break;
+ }
+ default:
+ break;
+ }
+ default:
+ default:
+ default:
+ break;
+ default:
+ break;
+ }
+ }
+ default:
+ break;
+ }
+ }
index 0000000..2999d0e
--- /dev/null
+++ b/tests/dns_test.cc
@@ -0,0 +1,105 @@
--- /dev/null
+++ b/tests/dns_test.cc
+/*
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+ * This file is open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * distributed with this work for additional information regarding copyright
+ * ownership. You may not use this file except in compliance with the License.
+ *
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:14 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index de97837..a2904a5 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
@@ -73,6 +73,14 @@ class inet_address {
net/inet_address.hh | 8 ++++++++
net/inet_address.cc | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/net/inet_address.hh b/net/inet_address.hh
index de97837..a2904a5 100644
--- a/net/inet_address.hh
+++ b/net/inet_address.hh
operator const ::in_addr&() const;
operator const ::in6_addr&() const;
+
+
+ future<sstring> hostname() const;
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
std::ostream& operator<<(std::ostream&, const inet_address&);
+ future<std::vector<sstring>> aliases() const;
+
+ static future<inet_address> find(const sstring&);
+ static future<inet_address> find(const sstring&, family);
+ static future<std::vector<inet_address>> find_all(const sstring&);
+ static future<std::vector<inet_address>> find_all(const sstring&, family);
};
diff --git a/net/inet_address.cc b/net/inet_address.cc
index f3a31d9..709434e 100644
--- a/net/inet_address.cc
+++ b/net/inet_address.cc
@@ -24,6 +24,7 @@
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
#include "inet_address.hh"
#include "socket_defs.hh"
+#include "dns.hh"
seastar::net::inet_address::inet_address()
: inet_address(::in_addr{ 0, })
: inet_address(::in_addr{ 0, })
@@ -96,6 +97,40 @@ const void * seastar::net::inet_address::data() const {
return &_in;
}
+future<sstring> seastar::net::inet_address::hostname() const {
+ return dns::resolve_addr(*this);
+}
+
+future<std::vector<sstring>> seastar::net::inet_address::aliases() const {
+ return dns::get_host_by_addr(*this).then([](hostent e) {
+ return make_ready_future<std::vector<sstring>>(std::move(e.names));
+ });
+}
return &_in;
}
+future<sstring> seastar::net::inet_address::hostname() const {
+ return dns::resolve_addr(*this);
+}
+
+future<std::vector<sstring>> seastar::net::inet_address::aliases() const {
+ return dns::get_host_by_addr(*this).then([](hostent e) {
+ return make_ready_future<std::vector<sstring>>(std::move(e.names));
+ });
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name) {
+ return dns::resolve_name(name);
+}
+ const sstring& name) {
+ return dns::resolve_name(name);
+
+future<seastar::net::inet_address> seastar::net::inet_address::find(
+ const sstring& name, family f) {
+ return dns::resolve_name(name, f);
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name) {
+ return dns::get_host_by_name(name).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name, family f) {
+ return dns::get_host_by_name(name, f).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
char buffer[64];
+ const sstring& name, family f) {
+ return dns::resolve_name(name, f);
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name) {
+ return dns::get_host_by_name(name).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
+future<std::vector<seastar::net::inet_address>> seastar::net::inet_address::find_all(
+ const sstring& name, family f) {
+ return dns::get_host_by_name(name, f).then([](hostent e) {
+ return make_ready_future<std::vector<seastar::net::inet_address>>(std::move(e.addr_list));
+ });
+}
+
std::ostream& seastar::net::operator<<(std::ostream& os, const inet_address& addr) {
char buffer[64];
return os << inet_ntop(int(addr.in_family()), addr.data(), buffer, sizeof(buffer));
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:15 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
We have both seastar::net and ::net namespaces. We should not.
But there it is...
---
net/api.hh | 6 +++---
net/ip.hh | 6 +++---
net/net.hh | 8 ++++----
net/posix-stack.hh | 2 +-
net/tcp.hh | 10 +++++-----
net/native-stack.cc | 4 ++--
net/stack.cc | 2 +-
net/tls.cc | 22 +++++++++++-----------
8 files changed, 30 insertions(+), 30 deletions(-)
diff --git a/net/api.hh b/net/api.hh
index 55c5fb0..3797014 100644
--- a/net/api.hh
+++ b/net/api.hh
@@ -216,12 +216,12 @@ class connected_socket {
/// A \c socket that allows a connection to be established between
/// two endpoints.
class socket {
- std::unique_ptr<net::socket_impl> _si;
+ std::unique_ptr<::net::socket_impl> _si;
public:
~socket();
/// \cond internal
- explicit socket(std::unique_ptr<net::socket_impl> si);
+ explicit socket(std::unique_ptr<::net::socket_impl> si);
/// \endcond
/// Moves a \c seastar::socket object.
socket(socket&&) noexcept;
@@ -287,7 +287,7 @@ class network_stack {
+++ b/net/api.hh
@@ -216,12 +216,12 @@ class connected_socket {
/// A \c socket that allows a connection to be established between
/// two endpoints.
class socket {
- std::unique_ptr<net::socket_impl> _si;
+ std::unique_ptr<::net::socket_impl> _si;
public:
~socket();
/// \cond internal
- explicit socket(std::unique_ptr<net::socket_impl> si);
+ explicit socket(std::unique_ptr<::net::socket_impl> si);
/// \endcond
/// Moves a \c seastar::socket object.
socket(socket&&) noexcept;
return socket().connect(sa, local, proto);
}
virtual seastar::socket socket() = 0;
- virtual net::udp_channel make_udp_channel(ipv4_addr addr = {}) = 0;
+ virtual ::net::udp_channel make_udp_channel(ipv4_addr addr = {}) = 0;
virtual future<> initialize() {
return make_ready_future();
}
diff --git a/net/ip.hh b/net/ip.hh
index a33a050..5473555 100644
--- a/net/ip.hh
+++ b/net/ip.hh
@@ -134,7 +134,7 @@ struct ipv4_traits {
static void udp_pseudo_header_checksum(checksummer& csum, ipv4_address src, ipv4_address dst, uint16_t len) {
csum.sum_many(src.ip.raw, dst.ip.raw, uint8_t(0), uint8_t(ip_protocol_num::udp), len);
}
- static constexpr uint8_t ip_hdr_len_min = net::ipv4_hdr_len_min;
+ static constexpr uint8_t ip_hdr_len_min = ipv4_hdr_len_min;
};
template <ip_protocol_num ProtoNum>
@@ -399,8 +399,8 @@ class ipv4 {
tcp<ipv4_traits>& get_tcp() { return *_tcp._tcp; }
ipv4_udp& get_udp() { return _udp; }
void register_l4(proto_type id, ip_protocol* handler);
- const net::hw_features& hw_features() const { return _netif->hw_features(); }
- static bool needs_frag(packet& p, ip_protocol_num proto_num, net::hw_features hw_features);
+ const ::net::hw_features& hw_features() const { return _netif->hw_features(); }
+ static bool needs_frag(packet& p, ip_protocol_num proto_num, ::net::hw_features hw_features);
void learn(ethernet_address l2, ipv4_address l3) {
_arp.learn(l2, l3);
}
diff --git a/net/net.hh b/net/net.hh
index a822892..1283ffe 100644
--- a/net/net.hh
+++ b/net/net.hh
@@ -81,7 +81,7 @@ struct hw_features {
// Maximum Transmission Unit
uint16_t mtu = 1500;
// Maximun packet len when TCP/UDP offload is enabled
- uint16_t max_packet_len = net::ip_packet_len_max - net::eth_hdr_len;
+ uint16_t max_packet_len = ip_packet_len_max - eth_hdr_len;
};
class l3_protocol {
@@ -115,14 +115,14 @@ class interface {
std::shared_ptr<device> _dev;
subscription<packet> _rx;
ethernet_address _hw_address;
- net::hw_features _hw_features;
+ ::net::hw_features _hw_features;
std::vector<l3_protocol::packet_provider_type> _pkt_providers;
private:
future<> dispatch_packet(packet p);
public:
explicit interface(std::shared_ptr<device> dev);
ethernet_address hw_address() { return _hw_address; }
- const net::hw_features& hw_features() const { return _hw_features; }
+ const ::net::hw_features& hw_features() const { return _hw_features; }
subscription<packet, ethernet_address> register_l3(eth_protocol_num proto_num,
std::function<future<> (packet p, ethernet_address from)> next,
std::function<bool (forward_hash&, packet&, size_t)> forward);
@@ -267,7 +267,7 @@ class device {
void l2receive(packet p) { _queues[engine().cpu_id()]->_rx_stream.produce(std::move(p)); }
subscription<packet> receive(std::function<future<> (packet)> next_packet);
virtual ethernet_address hw_address() = 0;
- virtual net::hw_features hw_features() = 0;
+ virtual ::net::hw_features hw_features() = 0;
virtual const rss_key_type& rss_key() const { return default_rsskey_40bytes; }
virtual uint16_t hw_queues_count() { return 1; }
virtual future<> link_ready() { return make_ready_future<>(); }
diff --git a/net/posix-stack.hh b/net/posix-stack.hh
index 7ecd56c..970a8a0 100644
--- a/net/posix-stack.hh
+++ b/net/posix-stack.hh
@@ -106,7 +106,7 @@ class posix_network_stack : public network_stack {
explicit posix_network_stack(boost::program_options::variables_map opts) : _reuseport(engine().posix_reuseport_available()) {}
virtual server_socket listen(socket_address sa, listen_options opts) override;
virtual ::seastar::socket socket() override;
- virtual net::udp_channel make_udp_channel(ipv4_addr addr) override;
+ virtual ::net::udp_channel make_udp_channel(ipv4_addr addr) override;
static future<std::unique_ptr<network_stack>> create(boost::program_options::variables_map opts) {
return make_ready_future<std::unique_ptr<network_stack>>(std::unique_ptr<network_stack>(new posix_network_stack(opts)));
}
diff --git a/net/tcp.hh b/net/tcp.hh
index bbccb14..faa304e 100644
--- a/net/tcp.hh
+++ b/net/tcp.hh
@@ -519,7 +519,7 @@ class tcp {
return size;
}
uint16_t local_mss() {
- return _tcp.hw_features().mtu - net::tcp_hdr_len_min - InetTraits::ip_hdr_len_min;
+ return _tcp.hw_features().mtu - ::net::tcp_hdr_len_min - InetTraits::ip_hdr_len_min;
}
void queue_packet(packet p) {
_packetq.emplace_back(typename InetTraits::l4packet{_foreign_ip, std::move(p)});
@@ -711,7 +711,7 @@ class tcp {
bool forward(forward_hash& out_hash_data, packet& p, size_t off);
listener listen(uint16_t port, size_t queue_length = 100);
connection connect(socket_address sa);
- const net::hw_features& hw_features() const { return _inet._inet.hw_features(); }
+ const ::net::hw_features& hw_features() const { return _inet._inet.hw_features(); }
future<> poll_tcb(ipaddr to, lw_shared_ptr<tcb> tcb);
void add_connected_tcb(lw_shared_ptr<tcb> tcbp, uint16_t local_port) {
auto it = _listening.find(local_port);
@@ -785,7 +785,7 @@ class tcp {
connid id;
auto src_ip = _inet._inet.host_address();
auto dst_ip = ipv4_address(sa);
- auto dst_port = net::ntoh(sa.u.in.sin_port);
+ auto dst_port = ::net::ntoh(sa.u.in.sin_port);
do {
src_port = _port_dist(_e);
@@ -1527,9 +1527,9 @@ class tcp {
uint32_t len;
if (_tcp.hw_features().tx_tso) {
// FIXME: Info tap device the size of the splitted packet
- len = _tcp.hw_features().max_packet_len - net::tcp_hdr_len_min - InetTraits::ip_hdr_len_min;
+ len = _tcp.hw_features().max_packet_len - ::net::tcp_hdr_len_min - InetTraits::ip_hdr_len_min;
} else {
- len = std::min(uint16_t(_tcp.hw_features().mtu - net::tcp_hdr_len_min - InetTraits::ip_hdr_len_min), _snd.mss);
+ len = std::min(uint16_t(_tcp.hw_features().mtu - ::net::tcp_hdr_len_min - InetTraits::ip_hdr_len_min), _snd.mss);
}
can_send = std::min(can_send, len);
// easy case: one small packet
diff --git a/net/native-stack.cc b/net/native-stack.cc
index 168a077..756a4c0 100644
--- a/net/native-stack.cc
+++ b/net/native-stack.cc
@@ -221,12 +221,12 @@ class native_network_stack : public network_stack {
auto & ns = static_cast<native_network_stack&>(engine().net());
ns.set_ipv4_packet_filter(f);
}).then([this, d = std::move(d), is_renew, res]() mutable {
- net::dhcp::result_type fut = is_renew ? d.renew(res) : d.discover();
+ ::net::dhcp::result_type fut = is_renew ? d.renew(res) : d.discover();
return fut.then([this, is_renew](bool success, const dhcp::lease & res) {
return smp::invoke_on_all([] {
auto & ns = static_cast<native_network_stack&>(engine().net());
ns.set_ipv4_packet_filter(nullptr);
- }).then(std::bind(&net::native_network_stack::on_dhcp, this, success, res, is_renew));
+ }).then(std::bind(&::net::native_network_stack::on_dhcp, this, success, res, is_renew));
}).finally([d = std::move(d)] {});
});
}
diff --git a/net/stack.cc b/net/stack.cc
index d7c0a6b..c6787af 100644
--- a/net/stack.cc
+++ b/net/stack.cc
@@ -109,7 +109,7 @@ void connected_socket::set_keepalive_parameters(const net::keepalive_params& p)
{}
seastar::socket::socket(
- std::unique_ptr<net::socket_impl> si)
+ std::unique_ptr<::net::socket_impl> si)
: _si(std::move(si)) {
}
diff --git a/net/tls.cc b/net/tls.cc
index 2705a32..844f771 100644
--- a/net/tls.cc
+++ b/net/tls.cc
@@ -503,7 +503,7 @@ void seastar::tls::credentials_builder::apply_to(certificate_credentials& creds)
* of these, since we handle handshake etc.
*
*/
-class session: public net::connected_socket_impl {
+class session: public ::net::connected_socket_impl {
public:
enum class type
: uint32_t {
@@ -511,7 +511,7 @@ class session: public net::connected_socket_impl {
};
session(type t, ::shared_ptr<certificate_credentials> creds,
- std::unique_ptr<net::connected_socket_impl> sock, sstring name = { })
+ std::unique_ptr<::net::connected_socket_impl> sock, sstring name = { })
: _type(t), _sock(std::move(sock)), _creds(std::move(creds)), _hostname(
std::move(name)), _in(_sock->source()), _out(_sock->sink()), _output_pending(
make_ready_future<>()), _session([t] {
@@ -538,7 +538,7 @@ class session: public net::connected_socket_impl {
}
session(type t, ::shared_ptr<certificate_credentials> creds,
::connected_socket sock, sstring name = { })
- : session(t, std::move(creds), net::get_impl::get(std::move(sock)),
+ : session(t, std::move(creds), ::net::get_impl::get(std::move(sock)),
std::move(name)) {
}
@@ -811,10 +811,10 @@ class session: public net::connected_socket_impl {
bool get_keepalive() const override {
return _sock->get_keepalive();
}
- void set_keepalive_parameters(const net::keepalive_params& p) override {
+ void set_keepalive_parameters(const ::net::keepalive_params& p) override {
_sock->set_keepalive_parameters(p);
}
- net::keepalive_params get_keepalive_parameters() const override {
+ ::net::keepalive_params get_keepalive_parameters() const override {
return _sock->get_keepalive_parameters();
}
@@ -828,7 +828,7 @@ class session: public net::connected_socket_impl {
type _type;
- std::unique_ptr<net::connected_socket_impl> _sock;
+ std::unique_ptr<::net::connected_socket_impl> _sock;
::shared_ptr<certificate_credentials> _creds;
const sstring _hostname;
data_source _in;
@@ -909,9 +909,9 @@ class session::sink_impl: public ::data_sink_impl {
: _session(s) {
}
private:
- typedef net::fragment* frag_iter;
+ typedef ::net::fragment* frag_iter;
- future<> put(net::packet p, frag_iter i, frag_iter e, size_t off = 0) {
+ future<> put(::net::packet p, frag_iter i, frag_iter e, size_t off = 0) {
while (i != e) {
auto ptr = i->base;
auto size = i->size;
@@ -946,7 +946,7 @@ class session::sink_impl: public ::data_sink_impl {
future<> flush() override {
return _session.flush();
}
- future<> put(net::packet p) override {
+ future<> put(::net::packet p) override {
auto i = p.fragments().begin();
auto e = p.fragments().end();
return put(std::move(p), i, e);
@@ -961,7 +961,7 @@ class session::sink_impl: public ::data_sink_impl {
session& _session;
};
-class server_session : public net::server_socket_impl {
+class server_session : public ::net::server_socket_impl {
public:
server_session(::shared_ptr<server_credentials> creds, ::server_socket sock)
: _creds(std::move(creds)), _sock(std::move(sock)) {
@@ -984,7 +984,7 @@ class server_session : public net::server_socket_impl {
::server_socket _sock;
};
-class tls_socket_impl : public net::socket_impl {
+class tls_socket_impl : public ::net::socket_impl {
::shared_ptr<certificate_credentials> _cred;
sstring _name;
seastar::socket _socket;
--
1.9.1
Calle Wilund
<calle@scylladb.com>Nov 30, 2016, 10:26:15 AM11/30/16
to seastar-dev@googlegroups.com, Calle Wilund
---
net/socket_defs.hh | 6 ++++++
net/net.cc | 8 ++++++++
2 files changed, 14 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index f23a261..a274a24 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -56,6 +56,11 @@ enum class transport {
SCTP = IPPROTO_SCTP
};
+
+namespace net {
+class inet_address;
+}
+
}
struct listen_options {
@@ -75,6 +80,7 @@ struct ipv4_addr {
ipv4_addr(uint16_t port) : ip(0), port(port) {}
ipv4_addr(const std::string &addr);
ipv4_addr(const std::string &addr, uint16_t port);
+ ipv4_addr(const seastar::net::inet_address&, uint16_t);
ipv4_addr(const socket_address &sa) {
ip = net::ntoh(sa.u.in.sin_addr.s_addr);
diff --git a/net/net.cc b/net/net.cc
index ad0165b..e667f65 100644
--- a/net/net.cc
+++ b/net/net.cc
@@ -25,6 +25,7 @@
#include "net.hh"
#include <utility>
#include "toeplitz.hh"
+#include "inet_address.hh"
using std::move;
@@ -44,6 +45,13 @@
ipv4_addr::ipv4_addr(const std::string &addr, uint16_t port_) : ip(boost::asio::ip::address_v4::from_string(addr).to_ulong()), port(port_) {}
+ipv4_addr::ipv4_addr(const seastar::net::inet_address& a, uint16_t port)
+ : ipv4_addr([&a] {
+}(), port)
+{}
+
namespace net {
inline
--
1.9.1
net/socket_defs.hh | 6 ++++++
net/net.cc | 8 ++++++++
2 files changed, 14 insertions(+)
diff --git a/net/socket_defs.hh b/net/socket_defs.hh
index f23a261..a274a24 100644
--- a/net/socket_defs.hh
+++ b/net/socket_defs.hh
@@ -56,6 +56,11 @@ enum class transport {
SCTP = IPPROTO_SCTP
};
+
+namespace net {
+class inet_address;
+}
+
}
struct listen_options {
@@ -75,6 +80,7 @@ struct ipv4_addr {
ipv4_addr(uint16_t port) : ip(0), port(port) {}
ipv4_addr(const std::string &addr);
ipv4_addr(const std::string &addr, uint16_t port);
+ ipv4_addr(const seastar::net::inet_address&, uint16_t);
ipv4_addr(const socket_address &sa) {
ip = net::ntoh(sa.u.in.sin_addr.s_addr);
diff --git a/net/net.cc b/net/net.cc
index ad0165b..e667f65 100644
--- a/net/net.cc
+++ b/net/net.cc
@@ -25,6 +25,7 @@
#include "net.hh"
#include <utility>
#include "toeplitz.hh"
+#include "inet_address.hh"
using std::move;
@@ -44,6 +45,13 @@
ipv4_addr::ipv4_addr(const std::string &addr, uint16_t port_) : ip(boost::asio::ip::address_v4::from_string(addr).to_ulong()), port(port_) {}
+ipv4_addr::ipv4_addr(const seastar::net::inet_address& a, uint16_t port)
+ : ipv4_addr([&a] {
+ ::in_addr in = a;
+ return ::net::ntoh(in.s_addr);
+}(), port)
+{}
+
namespace net {
inline
--
1.9.1
Carl Wilund
<calle@scylladb.com>Dec 6, 2016, 7:48:35 AM12/6/16
to Avi Kivity, seastar-dev@googlegroups.com
for c-ares, and they currently have a bit of a backlog, so I'm not sure
how fast you should really expect feedback (esp. given that all
maintainers are working other projects also).
If push comes to shove, I'm seeing the maintainer next week again, but...
Avi Kivity
<avi@scylladb.com>Dec 6, 2016, 10:17:27 AM12/6/16
to calle@scylladb.com, seastar-dev@googlegroups.com
But do send an email to the project mailing list, it also gives other
contributors a chance to weigh in. I see the maintainer's last message
is from Oct 18.
Calle Wilund
<calle@scylladb.com>Dec 12, 2016, 11:05:11 AM12/12/16
to Avi Kivity, seastar-dev@googlegroups.com
fix a few nitpicks, which are on the level of white space error +
improve some comments/docs, and add a few tests to the c-ares suite to
ensure the changes stay healthy, but nothing major.
Avi Kivity
<avi@scylladb.com>Dec 12, 2016, 11:10:02 AM12/12/16
to Calle Wilund, seastar-dev@googlegroups.com
On 12/12/2016 06:05 PM, Calle Wilund wrote:
>
>>
>> Let's wait until early next week, then we'll move if c-ares doesn't.
>> But do send an email to the project mailing list, it also gives other
>> contributors a chance to weigh in. I see the maintainer's last
>> message is from Oct 18.
>>
> Fyi/fwiv: Maintainer commented on the pull, in general positive. I
> will fix a few nitpicks, which are on the level of white space error +
> improve some comments/docs, and add a few tests to the c-ares suite to
> ensure the changes stay healthy, but nothing major.
>
without local patches.
Avi Kivity
<avi@scylladb.com>Dec 12, 2016, 11:18:49 AM12/12/16
to Calle Wilund, seastar-dev@googlegroups.com
On 11/30/2016 02:27 PM, Calle Wilund wrote:
> ---
> configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
> 1 file changed, 39 insertions(+), 6 deletions(-)
>
> diff --git a/configure.py b/configure.py
> configure.py | 45 +++++++++++++++++++++++++++++++++++++++------
> 1 file changed, 39 insertions(+), 6 deletions(-)
>
> index 9890cf1..a1be833 100755
> --- a/configure.py
> +++ b/configure.py
> @@ -155,12 +155,14 @@ modes = {
> 'sanitize_libs': '-lasan -lubsan',
> 'opt': '-O0 -DDEBUG -DDEBUG_SHARED_PTR -DDEFAULT_ALLOCATOR -DSEASTAR_THREAD_STACK_GUARDS',
> 'libs': '',
> + 'cares_opts': '--enable-debug',
> },
> 'release': {
> 'sanitize': '',
> 'sanitize_libs': '',
> 'opt': '-O2',
> 'libs': '',
> + 'cares_opts': '',
> },
> }
>
> @@ -608,6 +610,25 @@ if args.dpdk:
> if file.endswith('.h') or file.endswith('.c')]
> dpdk_sources = ' '.join(dpdk_sources)
>
> +# both source and builddir location
> +cares_dir = 'c-ares'
> +cares_lib = 'cares-seastar'
> +cares_src_lib = cares_dir + '/.libs/libcares.a'
Don't we build under build/{mode}/c-ares?
> 'sanitize_libs': '-lasan -lubsan',
> 'opt': '-O0 -DDEBUG -DDEBUG_SHARED_PTR -DDEFAULT_ALLOCATOR -DSEASTAR_THREAD_STACK_GUARDS',
> 'libs': '',
> + 'cares_opts': '--enable-debug',
> },
> 'release': {
> 'sanitize': '',
> 'sanitize_libs': '',
> 'opt': '-O2',
> 'libs': '',
> + 'cares_opts': '',
> },
> }
>
> @@ -608,6 +610,25 @@ if args.dpdk:
> if file.endswith('.h') or file.endswith('.c')]
> dpdk_sources = ' '.join(dpdk_sources)
>
> +# both source and builddir location
> +cares_dir = 'c-ares'
> +cares_lib = 'cares-seastar'
> +cares_src_lib = cares_dir + '/.libs/libcares.a'
that you can skip based on timestamps &c, since it reads in info from
the system.
> + build $builddir/{mode}/{cares_dir}/Makefile : caresconfigure_{mode} {cares_dir}/configure
> + build $builddir/{mode}/{cares_src_lib} : caresmake_{mode} $builddir/{mode}/{cares_dir}/Makefile | {cares_sources}
> + build $builddir/{mode}/lib{cares_lib}.a : copy_file $builddir/{mode}/{cares_src_lib}
> + ''').format(srcdir = os.getcwd(), cares_opts=modeval['cares_opts'], **globals()))
> compiles = {}
> ragels = {}
> swaggers = {}
> ragels = {}
> swaggers = {}
> @@ -718,7 +751,7 @@ with open(buildfile, 'w') as f:
> f.write('build $builddir/{}/{}_g: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
> f.write(' extralibs = {}\n'.format(' '.join(extralibs)))
> else:
> - f.write('build $builddir/{}/{}: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
> + f.write('build $builddir/{}/{}: link.{} {} | {} $builddir/{}/lib{}.a\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps, mode, cares_lib))
>
> for src in srcs:
> if src.endswith('.cc'):
> @@ -756,7 +789,7 @@ with open(buildfile, 'w') as f:
> rule configure
> command = python3 configure.py $configure_args
> generator = 1
> - build build.ninja: configure | configure.py
> + build build.ninja: configure | configure.py {cares_dir}/configure
> rule cscope
> command = find -name '*.[chS]' -o -name "*.cc" -o -name "*.hh" | cscope -bq -i-
> description = CSCOPE
I also expected changes to seastar.pc, aren't they needed?
> f.write('build $builddir/{}/{}_g: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
> f.write(' extralibs = {}\n'.format(' '.join(extralibs)))
> else:
> - f.write('build $builddir/{}/{}: link.{} {} | {}\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps))
> + f.write('build $builddir/{}/{}: link.{} {} | {} $builddir/{}/lib{}.a\n'.format(mode, binary, mode, str.join(' ', objs), dpdk_deps, mode, cares_lib))
>
> for src in srcs:
> if src.endswith('.cc'):
> @@ -756,7 +789,7 @@ with open(buildfile, 'w') as f:
> rule configure
> command = python3 configure.py $configure_args
> generator = 1
> - build build.ninja: configure | configure.py
> + build build.ninja: configure | configure.py {cares_dir}/configure
> rule cscope
> command = find -name '*.[chS]' -o -name "*.cc" -o -name "*.hh" | cscope -bq -i-
> description = CSCOPE
Calle Wilund
<calle@scylladb.com>Dec 12, 2016, 11:25:08 AM12/12/16
to Avi Kivity, seastar-dev@googlegroups.com
is not known yet, but only in the horrible python-expansion-blob further
down.
configure, and then run the configure at build time to generate the
Makefiles for debug resp. release. This is again mainly because the
concepts of debug/release are neither known nor invocation specific,
i.e. we need to generate separate rules for each.
So, configure.py makes sure c-ares/configure is generated from
configure.in/autotools, then we set up rules to build
build/{mode}/c-ares/Makefile (and thus out-of-tree compilation) for each
of the modes.
included. (Verified be the fact that I could use it from scylla).
Avi Kivity
<avi@scylladb.com>Dec 13, 2016, 3:18:39 AM12/13/16
to Calle Wilund, seastar-dev@googlegroups.com
second ninja take?
Calle Wilund
<calle@scylladb.com>Dec 13, 2016, 3:27:23 AM12/13/16
to Avi Kivity, seastar-dev@googlegroups.com
time ninja
ninja: no work to do.
real 0m0.015s
user 0m0.004s
sys 0m0.008s
Shlomi Livne
<shlomi@scylladb.com>Dec 23, 2016, 12:53:27 PM12/23/16
to Avi Kivity, Calle Wilund, seastar-dev
Ping ...
Does that mean we are waiting for upstream to absorb before merging or merging what Calle sent ?
--
You received this message because you are subscribed to the Google Groups "seastar-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to seastar-dev+unsubscribe@googlegroups.com.
To post to this group, send email to seast...@googlegroups.com.
Visit this group at https://groups.google.com/group/seastar-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/seastar-dev/82bb2694-ea21-20f0-530f-75944e870044%40scylladb.com.
Avi Kivity
<avi@scylladb.com>Dec 25, 2016, 3:31:32 AM12/25/16
to Shlomi Livne, Calle Wilund, seastar-dev
On 12/23/2016 07:53 PM, Shlomi Livne
wrote:
Ping ...Does that mean we are waiting for upstream to absorb before merging or merging what Calle sent ?
Yes.
Shlomi Livne
<shlomi@scylladb.com>Dec 25, 2016, 3:34:07 AM12/25/16
to Avi Kivity, Shlomi Livne, Calle Wilund, seastar-dev
Yes what ?
1. Merging Calle'sAvi Kivity
<avi@scylladb.com>Dec 25, 2016, 4:02:26 AM12/25/16
to Shlomi Livne, Calle Wilund, seastar-dev
Sorry. Waiting for upstream.
Calle Wilund
<calle@scylladb.com>Jan 31, 2017, 9:10:20 AM1/31/17
to Avi Kivity, Shlomi Livne, seastar-dev
The virtual IO patch set was merged to c-ares today.
Shlomi Livne
<shlomi@scylladb.com>Jan 31, 2017, 9:16:55 AM1/31/17
to Calle Wilund, Avi Kivity, seastar-dev
SO lets move and use that .... and retest our code
Nice :)Avi Kivity
<avi@scylladb.com>Jan 31, 2017, 9:26:17 AM1/31/17
to Calle Wilund, seastar-dev@googlegroups.com
On 11/30/2016 02:27 PM, Calle Wilund wrote:
> Note: this adds my fork of the library, containing changes for
> virtual IO functions, required for the lib to be really useful to
> scylla.
>
> Should be changed to either fork in the scylladb account, or
> to main trunk of library once the above changes are accepted (if).
> ---
> .gitmodules | 3 +++
> c-ares | 1 +
> 2 files changed, 4 insertions(+)
> create mode 160000 c-ares
>
> diff --git a/.gitmodules b/.gitmodules
> index ac15ad4..fd96ea7 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -4,3 +4,6 @@
> [submodule "fmt"]
> path = fmt
> url = ../fmt
> +[submodule "c-ares"]
> + path = c-ares
> + url = https://github.com/elcallio/c-ares.git
I cloned it to the scylladb org, so this should read ../c-ares.
> virtual IO functions, required for the lib to be really useful to
> scylla.
>
> Should be changed to either fork in the scylladb account, or
> to main trunk of library once the above changes are accepted (if).
> ---
> .gitmodules | 3 +++
> c-ares | 1 +
> 2 files changed, 4 insertions(+)
> create mode 160000 c-ares
>
> diff --git a/.gitmodules b/.gitmodules
> index ac15ad4..fd96ea7 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -4,3 +4,6 @@
> [submodule "fmt"]
> path = fmt
> url = ../fmt
> +[submodule "c-ares"]
> + path = c-ares
> + url = https://github.com/elcallio/c-ares.git
> diff --git a/c-ares b/c-ares
> new file mode 160000
> index 0000000..7a1092b
> --- /dev/null
> +++ b/c-ares
> @@ -0,0 +1 @@
> +Subproject commit 7a1092b684301a98c4e30ed2126a04fb2d9672b3
Reply all
Reply to author
Forward
0 new messages