brat (simstring) dependency build failure on WSL/Ubuntu18.04

[Mukund Parthasarathy]

Hello-

 I'm trying to create a norm db using tools/norm_db_init.py that requires simstring to be installed.

  -using g++ (Ubuntu 7.5.0)

  -running make (simstring-1.0) results in following errors:

$ make

make  all-recursive

make[1]: Entering directory '/home/mukund/tools/simstring-1.0'

Making all in include

make[2]: Entering directory '/home/mukund/tools/simstring-1.0/include'

make[2]: Nothing to be done for 'all'.

make[2]: Leaving directory '/home/mukund/tools/simstring-1.0/include'

Making all in frontend

make[2]: Entering directory '/home/mukund/tools/simstring-1.0/frontend'

g++ -DHAVE_CONFIG_H -I. -I.. -I.. -I../include   -O3 -ffast-math  -O3 -ffast-math  -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp

In file included from ../include/simstring/memory_mapped_file.h:62:0,

                 from ../include/simstring/simstring.h:52,

                 from main.cpp:43:

../include/simstring/memory_mapped_file_posix.h: In member function ‘void memory_mapped_file_posix::open(const string&, std::ios_base::openmode)’:

../include/simstring/memory_mapped_file_posix.h:91:19: error: ‘::close’ has not been declared

                 ::close(m_fd);

                   ^~~~~

../include/simstring/memory_mapped_file_posix.h:91:19: note: suggested alternative: ‘pclose’

                 ::close(m_fd);

                   ^~~~~

                   pclose

../include/simstring/memory_mapped_file_posix.h: In member function ‘void memory_mapped_file_posix::close()’:

../include/simstring/memory_mapped_file_posix.h:106:15: error: ‘::close’ has not been declared

             ::close(m_fd);

               ^~~~~

../include/simstring/memory_mapped_file_posix.h:106:15: note: suggested alternative: ‘pclose’

             ::close(m_fd);

               ^~~~~

               pclose

../include/simstring/memory_mapped_file_posix.h: In member function ‘bool memory_mapped_file_posix::resize(memory_mapped_file_posix::size_type)’:

../include/simstring/memory_mapped_file_posix.h:126:19: error: ‘::lseek’ has not been declared

             if (::lseek(m_size, size, SEEK_SET) >= 0) {

                   ^~~~~

../include/simstring/memory_mapped_file_posix.h:126:19: note: suggested alternative: ‘fseek’

             if (::lseek(m_size, size, SEEK_SET) >= 0) {

                   ^~~~~

                   fseek

../include/simstring/memory_mapped_file_posix.h:128:21: error: ‘read’ was not declared in this scope

                 if (read(m_fd, &c, sizeof(char)) == -1) {

                     ^~~~

../include/simstring/memory_mapped_file_posix.h:128:21: note: suggested alternative: ‘fread’

                 if (read(m_fd, &c, sizeof(char)) == -1) {

                     ^~~~

                     fread

../include/simstring/memory_mapped_file_posix.h:131:21: error: ‘write’ was not declared in this scope

                 if (write(m_fd, &c, sizeof(char)) == -1) {

                     ^~~~~

../include/simstring/memory_mapped_file_posix.h:131:21: note: suggested alternative: ‘fwrite’

                 if (write(m_fd, &c, sizeof(char)) == -1) {

                     ^~~~~

                     fwrite

Makefile:268: recipe for target 'main.o' failed

make[2]: *** [main.o] Error 1

make[2]: Leaving directory '/home/mukund/tools/simstring-1.0/frontend'

Makefile:327: recipe for target 'all-recursive' failed

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory '/home/mukund/tools/simstring-1.0'

Makefile:241: recipe for target 'all' failed

make: *** [all] Error 2

  -Is there an abstraction around or substitute for simstring? I know simstring is an ext dependency, any pointers appreciated!

 Creating the norm db had other errors with Python 3.8.4 (utf-8 codec issues), for now skipping errors (i.e override errors='ignore').

Thanks,

--Mukund

[Goran Topic]

As you say, simstring is not maintained by us.

I have encountered your error before though, and I believe I know the solution:

Edit `include/simstring/memory_mapped_file_posix.h` and add `#include <unistd.h>` where the other includes are.

If that does not work, you will have to contact simstring author.

Also, I reworked simstring handling, so that Python bindings are not needed any more, as it uses the executable simstring; if you use that, UTF8 codec issues should go away, since command-line simstring, unlike python bindings, supports UTF8 mode.

Just make sure to set `SIMSTRING_EXECUTABLE` in config.

Goran

--

---
You received this message because you are subscribed to the Google Groups "brat-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brat-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/brat-users/bf9c0a40-adfa-4608-8e28-41ee90269110o%40googlegroups.com.

[Mukund Parthasarathy]

Thanks, that helped with successful compilation!

Next step trying to build the swig/python binding has few errors and a bunch of warnings. Have encountered any of these errors below? Looks like binding code generation is out of date...appreciate your thoughts (any plans to switch to simstring-pure py package? May be we can avoid this pain):

 python3 setup.py build_ext

running build_ext

building '_simstring' extension

creating build

creating build/temp.linux-x86_64-3.8

g++ -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/mukund/tools/simstring-1.0/include -I/usr/include/python3.8 -c export.cpp -o build/temp.linux-x86_64-3.8/export.o

In file included from export.cpp:8:0:

/home/mukund/tools/simstring-1.0/include/simstring/simstring.h: In member function ‘bool simstring::reader::open(const string&)’:

/home/mukund/tools/simstring-1.0/include/simstring/simstring.h:830:18: warning: variable ‘num_entries’ set but not used [-Wunused-but-set-variable]

         uint32_t num_entries, max_size;

                  ^~~~~~~~~~~

export.cpp: In destructor ‘virtual writer::~writer()’:

export.cpp:95:38: warning: throw will always call terminate() [-Wterminate]

      throw std::runtime_error(message);

                                      ^

export.cpp:95:38: note: in C++11 destructors default to noexcept

export.cpp:109:38: warning: throw will always call terminate() [-Wterminate]

      throw std::runtime_error(message);

                                      ^

export.cpp:109:38: note: in C++11 destructors default to noexcept

g++ -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/mukund/tools/simstring-1.0/include -I/usr/include/python3.8 -c export_wrap.cpp -o build/temp.linux-x86_64-3.8/export_wrap.o

export_wrap.cpp: In function ‘void SWIG_Python_AddErrorMsg(const char*)’:

export_wrap.cpp:883:33: error: ‘PyString_AsString’ was not declared in this scope

     PyErr_Format(type, "%s %s", PyString_AsString(old_str), mesg);

                                 ^~~~~~~~~~~~~~~~~

export_wrap.cpp:883:33: note: suggested alternative: ‘PyBytes_AsString’

     PyErr_Format(type, "%s %s", PyString_AsString(old_str), mesg);

                                 ^~~~~~~~~~~~~~~~~

                                 PyBytes_AsString

export_wrap.cpp: In function ‘PySwigClientData* PySwigClientData_New(PyObject*)’:

export_wrap.cpp:1274:9: error: ‘PyClass_Check’ was not declared in this scope

     if (PyClass_Check(obj)) {

         ^~~~~~~~~~~~~

export_wrap.cpp:1274:9: note: suggested alternative: ‘PyCell_Check’

     if (PyClass_Check(obj)) {

         ^~~~~~~~~~~~~

         PyCell_Check

export_wrap.cpp: In function ‘PyObject* PySwigObject_format(const char*, PySwigObject*)’:

export_wrap.cpp:1347:24: error: ‘PyString_FromString’ was not declared in this scope

       PyObject *ofmt = PyString_FromString(fmt);

                        ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1347:24: note: suggested alternative: ‘PyLong_FromString’

       PyObject *ofmt = PyString_FromString(fmt);

                        ^~~~~~~~~~~~~~~~~~~

                        PyLong_FromString

export_wrap.cpp:1349:8: error: ‘PyString_Format’ was not declared in this scope

  res = PyString_Format(ofmt,args);

        ^~~~~~~~~~~~~~~

export_wrap.cpp:1349:8: note: suggested alternative: ‘PyErr_Format’

  res = PyString_Format(ofmt,args);

        ^~~~~~~~~~~~~~~

        PyErr_Format

export_wrap.cpp: In function ‘PyObject* PySwigObject_repr(PySwigObject*)’:

export_wrap.cpp:1379:84: error: ‘PyString_AsString’ was not declared in this scope

   PyObject *repr = PyString_FromFormat("<Swig Object of type '%s' at 0x%s>", name, PyString_AsString(hex));

                                                                                    ^~~~~~~~~~~~~~~~~

export_wrap.cpp:1379:84: note: suggested alternative: ‘PyBytes_AsString’

   PyObject *repr = PyString_FromFormat("<Swig Object of type '%s' at 0x%s>", name, PyString_AsString(hex));

                                                                                    ^~~~~~~~~~~~~~~~~

                                                                                    PyBytes_AsString

export_wrap.cpp:1379:20: error: ‘PyString_FromFormat’ was not declared in this scope

   PyObject *repr = PyString_FromFormat("<Swig Object of type '%s' at 0x%s>", name, PyString_AsString(hex));

                    ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1379:20: note: suggested alternative: ‘PyBytes_FromFormat’

   PyObject *repr = PyString_FromFormat("<Swig Object of type '%s' at 0x%s>", name, PyString_AsString(hex));

                    ^~~~~~~~~~~~~~~~~~~

                    PyBytes_FromFormat

export_wrap.cpp:1387:5: error: ‘PyString_ConcatAndDel’ was not declared in this scope

     PyString_ConcatAndDel(&repr,nrep);

     ^~~~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1387:5: note: suggested alternative: ‘PyBytes_ConcatAndDel’

     PyString_ConcatAndDel(&repr,nrep);

     ^~~~~~~~~~~~~~~~~~~~~

     PyBytes_ConcatAndDel

export_wrap.cpp: In function ‘int PySwigObject_print(PySwigObject*, FILE*, int)’:

export_wrap.cpp:1401:11: error: ‘PyString_AsString’ was not declared in this scope

     fputs(PyString_AsString(repr), fp);

           ^~~~~~~~~~~~~~~~~

export_wrap.cpp:1401:11: note: suggested alternative: ‘PyBytes_AsString’

     fputs(PyString_AsString(repr), fp);

           ^~~~~~~~~~~~~~~~~

           PyBytes_AsString

export_wrap.cpp: In function ‘PyObject* PySwigObject_str(PySwigObject*)’:

export_wrap.cpp:1414:5: error: ‘PyString_FromString’ was not declared in this scope

     PyString_FromString(result) : 0;

     ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1414:5: note: suggested alternative: ‘PyLong_FromString’

     PyString_FromString(result) : 0;

     ^~~~~~~~~~~~~~~~~~~

     PyLong_FromString

export_wrap.cpp: In function ‘PyTypeObject* _PySwigObject_type()’:

export_wrap.cpp:1623:6: error: ‘coercion’ was not declared in this scope

     (coercion)0,   /*nb_coerce*/

      ^~~~~~~~

export_wrap.cpp:1623:6: note: suggested alternative: ‘ctermid’

     (coercion)0,   /*nb_coerce*/

      ^~~~~~~~

      ctermid

export_wrap.cpp:1623:15: error: expected ‘}’ before numeric constant

     (coercion)0,   /*nb_coerce*/

               ^

export_wrap.cpp:1623:15: error: expected ‘,’ or ‘;’ before numeric constant

export_wrap.cpp:1603:15: warning: unused variable ‘swigobject_doc’ [-Wunused-variable]

   static char swigobject_doc[] = "Swig object carries a C/C++ instance pointer";

               ^~~~~~~~~~~~~~

export_wrap.cpp:1605:26: warning: unused variable ‘PySwigObject_as_number’ [-Wunused-variable]

   static PyNumberMethods PySwigObject_as_number = {

                          ^~~~~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1636:3: warning: no return statement in function returning non-void [-Wreturn-type]

   };

   ^

export_wrap.cpp: At global scope:

export_wrap.cpp:1640:3: error: expected unqualified-id before ‘if’

   if (!type_init) {

   ^~

export_wrap.cpp:1706:3: error: expected unqualified-id before ‘return’

   return &pyswigobject_type;

   ^~~~~~

export_wrap.cpp: In function ‘PyObject* PySwigPacked_repr(PySwigPacked*)’:

export_wrap.cpp:1752:12: error: ‘PyString_FromFormat’ was not declared in this scope

     return PyString_FromFormat("<Swig Packed at %s%s>", result, v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1752:12: note: suggested alternative: ‘PyBytes_FromFormat’

     return PyString_FromFormat("<Swig Packed at %s%s>", result, v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

            PyBytes_FromFormat

export_wrap.cpp:1754:12: error: ‘PyString_FromFormat’ was not declared in this scope

     return PyString_FromFormat("<Swig Packed %s>", v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1754:12: note: suggested alternative: ‘PyBytes_FromFormat’

     return PyString_FromFormat("<Swig Packed %s>", v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

            PyBytes_FromFormat

export_wrap.cpp: In function ‘PyObject* PySwigPacked_str(PySwigPacked*)’:

export_wrap.cpp:1763:12: error: ‘PyString_FromFormat’ was not declared in this scope

     return PyString_FromFormat("%s%s", result, v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1763:12: note: suggested alternative: ‘PyBytes_FromFormat’

     return PyString_FromFormat("%s%s", result, v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

            PyBytes_FromFormat

export_wrap.cpp:1765:12: error: ‘PyString_FromString’ was not declared in this scope

     return PyString_FromString(v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1765:12: note: suggested alternative: ‘PyLong_FromString’

     return PyString_FromString(v->ty->name);

            ^~~~~~~~~~~~~~~~~~~

            PyLong_FromString

export_wrap.cpp: In function ‘PyTypeObject* _PySwigPacked_type()’:

export_wrap.cpp:1819:3: error: ‘cmpfunc’ was not declared in this scope

  (cmpfunc)PySwigPacked_compare,     /* tp_compare */

   ^~~~~~~

export_wrap.cpp:1819:3: note: suggested alternative: ‘newfunc’

  (cmpfunc)PySwigPacked_compare,     /* tp_compare */

   ^~~~~~~

   newfunc

export_wrap.cpp:1819:11: error: expected ‘}’ before ‘PySwigPacked_compare’

  (cmpfunc)PySwigPacked_compare,     /* tp_compare */

           ^~~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1819:11: error: expected ‘,’ or ‘;’ before ‘PySwigPacked_compare’

export_wrap.cpp:1808:24: warning: unused variable ‘tmp’ [-Wunused-variable]

     const PyTypeObject tmp

                        ^~~

export_wrap.cpp:1865:25: error: ‘tmp’ was not declared in this scope

     pyswigpacked_type = tmp;

                         ^~~

export_wrap.cpp:1865:25: note: suggested alternative: ‘tm’

     pyswigpacked_type = tmp;

                         ^~~

                         tm

export_wrap.cpp:1866:23: error: ‘PyTypeObject {aka struct _typeobject}’ has no member named ‘ob_type’; did you mean ‘ob_base’?

     pyswigpacked_type.ob_type = &PyType_Type;

                       ^~~~~~~

                       ob_base

export_wrap.cpp:1804:15: warning: unused variable ‘swigpacked_doc’ [-Wunused-variable]

   static char swigpacked_doc[] = "Swig object carries a C/C++ instance pointer";

               ^~~~~~~~~~~~~~

export_wrap.cpp:1868:3: warning: no return statement in function returning non-void [-Wreturn-type]

   }

   ^

export_wrap.cpp: At global scope:

export_wrap.cpp:1869:3: error: expected unqualified-id before ‘return’

   return &pyswigpacked_type;

   ^~~~~~

export_wrap.cpp:1870:1: error: expected declaration before ‘}’ token

 }

 ^

export_wrap.cpp:1804:15: warning: ‘swigpacked_doc’ defined but not used [-Wunused-variable]

   static char swigpacked_doc[] = "Swig object carries a C/C++ instance pointer";

               ^~~~~~~~~~~~~~

export_wrap.cpp:1639:14: warning: ‘type_init’ defined but not used [-Wunused-variable]

   static int type_init = 0;

              ^~~~~~~~~

export_wrap.cpp:1638:23: warning: ‘pyswigobject_type’ defined but not used [-Wunused-variable]

   static PyTypeObject pyswigobject_type;

                       ^~~~~~~~~~~~~~~~~

export_wrap.cpp:1605:26: warning: ‘PySwigObject_as_number’ defined but not used [-Wunused-variable]

   static PyNumberMethods PySwigObject_as_number = {

                          ^~~~~~~~~~~~~~~~~~~~~~

export_wrap.cpp:1603:15: warning: ‘swigobject_doc’ defined but not used [-Wunused-variable]

   static char swigobject_doc[] = "Swig object carries a C/C++ instance pointer";

               ^~~~~~~~~~~~~~

export_wrap.cpp:1571:1: warning: ‘swigobject_methods’ defined but not used [-Wunused-variable]

 swigobject_methods[] = {

 ^~~~~~~~~~~~~~~~~~

error: command 'g++' failed with exit status 1

To unsubscribe from this group and stop receiving emails from it, send an email to brat-...@googlegroups.com.

[Mukund Parthasarathy]

Thanks for this, Goran. The exec string config works, I'm able to proceed with the setup of conf files for norm. Pls ignore the other swig/python errors, I was able to resolve those (py runtime config issue).

--Mukund

You received this message because you are subscribed to a topic in the Google Groups "brat-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/brat-users/gqZUwkaRjJU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to brat-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/brat-users/CAGMASUNXVJWq9Pmc_ni%3DOw2%2BoUTpihNO10ZmfDkMQzURcf4Frw%40mail.gmail.com.

[Goran Topic]

I have not been able to compile the swig bindings for mac though, which is why I reworked brat so it can use the simstring executable.

And as I said, the executable is actually more powerful, as it supports utf8.

I tried simstring-pure, it is even in the codebase as of a couple of months ago, but it proved to be glacially slow and unusable when actually applied, so i ripped it out and wrapped the executable because I could reliably compile that.

Goran

To view this discussion on the web, visit https://groups.google.com/d/msgid/brat-users/CANtfj_AAQbTDVA-NSL-C11baim%2BsG2ZUXxCzt07dNLnGDWN-JQ%40mail.gmail.com.