Perl API

[David Lukeš]

Hi all,

is the SWIG-generated Perl interface still supposed to work as of the latest release of Manatee, i.e. 2.214.1? If I configure Manatee with --with-pcre --enable-perl, building fails with the following error:

/usr/bin/swig -perl -c++ -shadow -o manatee_pe_wrap.cc manatee.i manatee.i:28: Error: Unable to find 'std_unordered_set.i'

Looking at the interface files that come bundled with SWIG, std_unordered_set.i is available for Java, Python and Ruby, but not Perl. If I just naively disable portions of manatee.i which look related to this…

diff --git a/api/manatee.i b/api/manatee.i index 01c2f7c..a2de2cd 100644 --- a/api/manatee.i +++ b/api/manatee.i @@ -25,8 +25,10 @@ %include common.i %include typemaps.i +#ifndef SWIGPERL %include std_unordered_set.i %template() std::unordered_set<std::string>; +#endif void languages (std::vector<std::string> &out); @@ -946,6 +948,7 @@ struct kwitem { %apply std::vector<std::string>& INPUT {std::vector<std::string> &filters } %apply std::vector<string>& INPUT {std::vector<string> &addfreqs } +#ifndef SWIGPERL %apply std::unordered_set<std::string>& INPUT {std::unordered_set<std::string> &whitelist, std::unordered_set<std::string> &blacklist } class Keyword { @@ -966,6 +969,7 @@ public: } } }; +#endif class TokenLevel;

… I get a slew of warnings from SWIG and ultimately a few compilation errors, one of which says 'PyObject' does not name a type. I get something similar when trying to compile with --enable-ruby instead, even though Ruby has std_unordered_set.i.

This sort of looks like manatee.i has become Python-specific, in which case it might be helpful to remove the other --enable-... options to signal this to users? Or am I missing something?

Best,

David

​

[David Lukeš]

Sorry, somewhere along the way, the message got converted to plain text in a way that made it less readable.

Here’s the SWIG error:

/usr/bin/swig -perl -c++ -shadow -o manatee_pe_wrap.cc manatee.i
manatee.i:28: Error: Unable to find 'std_unordered_set.i'

And here’s the patch I used to conditionally disable parts of manatee.i:

diff --git a/api/manatee.i b/api/manatee.i
index 01c2f7c..a2de2cd 100644
--- a/api/manatee.i
+++ b/api/manatee.i
@@ -25,8 +25,10 @@

 %include common.i
 %include typemaps.i
+#ifndef SWIGPERL
 %include std_unordered_set.i
 %template() std::unordered_set<std::string>;
+#endif

 void languages (std::vector<std::string> &out);

@@ -946,6 +948,7 @@ struct kwitem {

 %apply std::vector<std::string>& INPUT {std::vector<std::string> &filters }
 %apply std::vector<string>& INPUT {std::vector<string> &addfreqs }
+#ifndef SWIGPERL
 %apply std::unordered_set<std::string>& INPUT {std::unordered_set<std::string> &whitelist, std::unordered_set<std::string> &blacklist }

 class Keyword {
@@ -966,6 +969,7 @@ public:
         }
     }
 };
+#endif


 class TokenLevel;

D.

[Miloš Jakubíček]

Hi David,

We have only supported the Python API for the last couple of years -- the rest might or might not work depending on circumstances, and might or might not be easy to implement depending on the memory issues in the interface that are language-dependent.

So:

This sort of looks like manatee.i has become Python-specific, in which case it might be helpful to remove the other --enable-... options to signal this to users? Or am I missing something?

In principle yes, the only question is whether some users (like you?) were only interested in selected parts of the API that could work with other languages.

Do I understand it correctly that after applying your patch you were able to successfully build the rest of the API for Perl?

Best

Milos

Milos Jakubicek

CEO, Lexical Computing

Brno, CZ | Brighton, UK

http://www.lexicalcomputing.com

http://www.sketchengine.eu

--
You received this message because you are subscribed to the Google Groups "NoSketch Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to noske+un...@sketchengine.co.uk.
To view this discussion on the web visit https://groups.google.com/a/sketchengine.co.uk/d/msgid/noske/CAEPTPExPQ-MODMSQz6qLkAPSVHd7Egk%2BftVHbB1SHBUMg9wuoA%40mail.gmail.com.

[David Lukeš]

Dear Miloš,

Thank you for the quick reply!

We have only supported the Python API for the last couple of years

Understood!

Do I understand it correctly that after applying your patch you were able to successfully build the rest of the API for Perl?

Not exactly — that patch just removed stuff that (I assumed) requires std_unordered_set.i, which is not necessarily Python-specific, it’s just not available for Perl (yet?) in SWIG. For the Perl API to compile, I had to remove one additional construct referencing PyObject. Here’s the full patch:

diff --git a/api/manatee.i b/api/manatee.i
index 01c2f7c..f86f6af 100644

--- a/api/manatee.i
+++ b/api/manatee.i
@@ -25,8 +25,10 @@

 %include common.i
 %include typemaps.i
+#ifndef SWIGPERL
 %include std_unordered_set.i
 %template() std::unordered_set<std::string>;
+#endif

 void languages (std::vector<std::string> &out);

@@ -925,6 +927,7 @@ CorpInfo *loadCorpInfoDefaults (const char *corp_name_or_path);
 /** Creates CorpInfo object by path/name of corpus only using the registry contents */
 CorpInfo *loadCorpInfoNoDefaults (const char *corp_name_or_path);

+#ifdef SWIGPYTHON
 %nodefaultctor kwitem;
 struct kwitem {
     int id1, id2;
@@ -943,11 +946,15 @@ struct kwitem {
         }
     }
 };
+#endif

 %apply std::vector<std::string>& INPUT {std::vector<std::string> &filters }
 %apply std::vector<string>& INPUT {std::vector<string> &addfreqs }
+#ifndef SWIGPERL
 %apply std::unordered_set<std::string>& INPUT {std::unordered_set<std::string> &whitelist, std::unordered_set<std::string> &blacklist }

+#endif

+#ifdef SWIGPYTHON
 class Keyword {
 public:
     Keyword (Corpus *c1, Corpus *c2, WordList *wl1, WordList *wl2, float N,
@@ -966,6 +973,7 @@ public:
         }
     }
 };
+#endif


 class TokenLevel;

With this patch applied, compilation exited normally. But I have no idea whether the result works at least as well as 2.167.10, which is the last version which didn’t need any patch in this respect.

Best,

David

[Miloš Jakubíček]

Hi David,

I've merged a slightly modified version (https://paste.centos.org/view/6135290a).

I assume that keyword extraction would not work but did not test this in Perl.

Best

Milos

Milos Jakubicek

CEO, Lexical Computing

Brno, CZ | Brighton, UK

http://www.lexicalcomputing.com

http://www.sketchengine.eu

[David Lukeš]

Hi Miloš,

That’s great, thank you very much!

David