Fwd: About textgrounder

[Jason Baldridge]

Damien -- I'm sending this to the textgrounder help/open mailing list in case Ben, Mike, or someone else can help out. It looks like basic installation issues, so the first thing is to make sure you are using the latest version, or the correct version, for what you want to do.

Can the rest of you all help Damien out? Thanks!

---------- Forwarded message ----------
From: Damien Palacio <damien....@gmail.com>
Date: Tue, Feb 21, 2012 at 3:50 AM
Subject: About textgrounder
To: jbal...@mail.utexas.edu

Dear Jason Baldridge,

I'm working with Ross Purves in Zurich University, and we want to test it and compare it with approaches we are working on currently.

So I tried to use Textgrounder but I had some problems.

First wich documentation should be used? It is written that wiki is more up-to-date but I notice some steps are not possible anymore (like ImportGazetteer seems to have been put in old folder),

I tried first to use it on Windows 7 but as there are some space in java path and libs (QTjava) (because there are by default in "Program Files") it doesn't work. For Java it is possible to avoid the problem by using "java ..." and not "$JAVA_HOME ..." (as java bin is in the path) but still problems with libs,

I tried then on Ubuntu. No problems with space in paths. But during compiling I have some problems (see below).

But I saw that I can ran tg-geolocate. I wanted to try if it works with Twitter (tg-geolocate twitter). I dowloaded Twitter data you provide. After changed names of folder (the name is wikigrounder-twitter but by default he looks for twitter-geotext and inside it is output-5-docthresh and it looks for doctresh-5) I have an error. Seems it is not the right resources but I'm not sure or if it because of previous compiling error?

if you have an idea about what the problem is?

Thanks in advance

Best regards

Damien Palacio

# tg-geolocate twitter error:

Exception in thread "main" opennlp.textgrounder.util.ioutil.package$FileFormatException: Found no schema files (matching -training-unigram-counts-schema\.txt(\.[a-zA-Z0-9]+)?$) in directory /home/damien/textgrounder/corpora/twitter-geotext/docthresh-5
   at opennlp.textgrounder.util.ioutil.package$CorpusFileProcessor.find_schema_file(ioutil.scala:1154)
   at opennlp.textgrounder.util.ioutil.package$CorpusFileProcessor.read_schema_from_corpus(ioutil.scala:1170)
   at opennlp.textgrounder.gridlocate.DistDocumentTable$$anonfun$read_training_documents$1.apply$mcZI$sp(DistDocument.scala:332)
   at opennlp.textgrounder.gridlocate.DistDocumentTable$$anonfun$read_training_documents$1.apply(DistDocument.scala:328)
   at opennlp.textgrounder.gridlocate.DistDocumentTable$$anonfun$read_training_documents$1.apply(DistDocument.scala:328)
   at scala.collection.immutable.Range.foreach(Range.scala:78)
   at opennlp.textgrounder.gridlocate.DistDocumentTable.read_training_documents(DistDocument.scala:328)
   at opennlp.textgrounder.gridlocate.GridLocateDriver$$anonfun$read_documents$1.apply(GridLocate.scala:798)
   at opennlp.textgrounder.gridlocate.GridLocateDriver$$anonfun$read_documents$1.apply(GridLocate.scala:797)
   at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
   at scala.collection.immutable.List.foreach(List.scala:45)
   at opennlp.textgrounder.gridlocate.GridLocateDriver.read_documents(GridLocate.scala:797)
   at opennlp.textgrounder.gridlocate.GridLocateDriver.setup_for_run(GridLocate.scala:810)
   at opennlp.textgrounder.geolocate.GeolocateDocumentTypeDriver.setup_for_run(Geolocate.scala:477)
   at opennlp.textgrounder.util.experiment.package$ExperimentDriverApp.run_program(experiment.scala:680)
   at opennlp.textgrounder.gridlocate.GridLocateApp.run_program(GridLocate.scala:873)
   at opennlp.textgrounder.util.experiment.package$ExperimentApp.implement_main(experiment.scala:569)
   at opennlp.textgrounder.util.experiment.package$ExperimentApp.main(experiment.scala:583)
   at opennlp.textgrounder.geolocate.GeolocateDocumentApp.main(Geolocate.scala)

## textgrounder build-all :

[info] Loading project definition from /home/damien/textgrounder/project
[info] Set current project to TextGrounder (in build file:/home/damien/textgrounder/)
[success] Total time: 0 s, completed Feb 20, 2012 11:05:58 AM
[info] Compiling 56 Scala sources and 4 Java sources to /home/damien/textgrounder/target/classes...
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/KDTreeCellGrid.scala:97: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error]    val task = new ExperimentMeteredTask(table.driver, "K-d tree structure",
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/MultiRegularCell.scala:279: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error]    SphereCoord(coord.lat - subval, coord.long - subval))
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:247: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error]    SphereCoord((sw.lat + ne.lat) / 2.0, (sw.long + ne.long) / 2.0)
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:250: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error]    SphereCoord(centroid(0) / num_docs, centroid(1) / num_docs);
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:271: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val nw = SphereCoord(ne.lat, sw.long)
[error]                      ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:272: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val se = SphereCoord(sw.lat, ne.long)
[error]                      ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:146: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error]    yield SphereCoord((center.lat + coord.lat) / 2.0,
[error]                     ^
[warn] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:166: dead code following this construct
[warn]      coord.long, coord.lat, fracprob * params.kml_max_height)
[warn]      ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/SphereCell.scala:166: value long is not a member of Nothing
[error]      coord.long, coord.lat, fracprob * params.kml_max_height)
[error]            ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/Hadoop.scala:228: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error]   val task = new ExperimentMeteredTask(driver, "document", "evaluating")
[error]           ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/KDTreeCellGrid.scala:174: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error]    val task = new ExperimentMeteredTask(table.driver, "K-d tree cell",
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/MultiRegularCell.scala:227: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] coord_to_tiling_cell_index(SphereCoord(minimum_latitude, minimum_longitude))
[error]                                        ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/MultiRegularCell.scala:222: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] coord_to_tiling_cell_index(SphereCoord(maximum_latitude - 1e-10,
[error]                                        ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/MultiRegularCell.scala:468: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error] val task = new ExperimentMeteredTask(table.driver, "Earth-tiling cell",
[error]             ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/toponym/Toponym.scala:269: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val topleft = SphereCoord((for (x <- goodlocs) yield x.coord.lat) min,
[error]                           ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/toponym/Toponym.scala:271: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val botright = SphereCoord((for (x <- goodlocs) yield x.coord.lat) max,
[error]                            ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/toponym/Toponym.scala:1150: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error]          wordstruct.coord = SphereCoord(lat.toDouble, long.toDouble)
[error]                                        ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/toponym/Toponym.scala:1328: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val loc = new Locality(name, SphereCoord(lat.toInt / 100., long.toInt / 100.),
[error]                                          ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/geolocate/toponym/Toponym.scala:1386: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error] val task = new ExperimentMeteredTask(cell_grid.table.driver,
[error]             ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/gridlocate/Cell.scala:589: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error]    val task = new ExperimentMeteredTask(table.driver, "non-empty cell",
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/gridlocate/DistDocument.scala:280: not enough arguments for constructor ExperimentMeteredTask: (driver: opennlp.textgrounder.util.experiment.package.ExperimentDriver, item_name: String, verb: String, secs_between_output: Double, maxtime: Double)opennlp.textgrounder.util.experiment.package.ExperimentMeteredTask
[error]      new ExperimentMeteredTask(driver, "document", "reading pass " + pass,
[error]      ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/FrobCorpus.scala:177: object opennlp.textgrounder.util.ioutil.package.Schema does not take parameters
[error]      Schema(fieldnames, modify_fixed_values(schema.fixed_values))
[error]            ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/FrobCorpus.scala:212: object opennlp.textgrounder.util.ioutil.package.Schema does not take parameters
[error]        Schema(new_fieldnames, modify_fixed_values(new_fixed_values))
[error]              ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/TwitterPullLocationVariance.scala:105: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val avgpoint = SphereCoord(lats.sum / lats.length, lngs.sum / lngs.length)
[error]                            ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/TwitterPullLocationVariance.scala:106: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error] val allpoints = latlngs.map(ll => SphereCoord(ll._1, ll._2))
[error]                                               ^
[warn] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/TwitterPullLocationVariance.scala:107: dead code following this construct
[warn] val distances = allpoints.map(spheredist(_, avgpoint))
[warn]                                           ^
[warn] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/TwitterPullLocationVariance.scala:112: dead code following this construct
[warn]                      .map{case (a, b) => spheredist(a, b)}.max
[warn]                                                     ^
[warn] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/preprocess/TwitterPullLocationVariance.scala:112: dead code following this construct
[warn]                      .map{case (a, b) => spheredist(a, b)}.max
[warn]                                                        ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/util/collectionutil.scala:478: not enough arguments for constructor LRUCache: (maxsize: Int)opennlp.textgrounder.util.collectionutil.package.LRUCache[T,U]
[error] override def empty = new LRUCache[T,U]()
[error]                       ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/util/collectionutil.scala:483: not enough arguments for constructor LRUCache: (maxsize: Int)opennlp.textgrounder.util.collectionutil.package.LRUCache[T,U]
[error] def empty[T,U] = new LRUCache[T,U]()
[error]                   ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/util/distances.scala:140: not enough arguments for method apply: (lat: Double, long: Double, method: String)opennlp.textgrounder.util.distances.package.SphereCoord in object SphereCoord.
[error] Unspecified value parameter method.
[error]    SphereCoord(lat.toDouble, long.toDouble)
[error]               ^
[error] /home/damien/textgrounder/src/main/scala/opennlp/textgrounder/util/distances.scala:157: type mismatch;
[error]  found   : opennlp.textgrounder.util.distances.package.opennlp.textgrounder.util.distances.Year
[error]  required: opennlp.textgrounder.util.distances.package.(some other)opennlp.textgrounder.util.distances.Year
[error] def deserialize(foo: String) = Year(foo.toDouble)
[error]                                     ^
[warn] four warnings found
[error] 28 errors found
[error] {file:/home/damien/textgrounder/}default-100c57/compile:compile: Compilation failed
[error] Total time: 32 s, completed Feb 20, 2012 11:06:30 AM

--

##############################################################

# Dr. Damien Palacio

# Postdoc in GIS-Geocomputation unit, 
# Department of Geography, University of Zurich

#

# Mob.: +41-7-74-50-79-44
# Skype: damienpalacio
# Email: damien....@gmail.com

# Web: http://damienpalacio.eu

##############################################################

--
Jason Baldridge
Associate Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge

[Stephen Roller]

Hi Damien,

I believe you're using an old version of the GeoText data. Our data format underwent many significant changes recently due to a paper submission and some efforts to generalize the system.

Below is a copy of the geotext data in the latest format. If this doesn't work, let us know and we'll fix you up.

http://www.cs.utexas.edu/~roller/geotext.tar.gz

(On a side note, there's no way Textgrounder will work on Windows; too many bash scripts holding things together)

Stephen