Error building conceptnet data

[Cristian Ruz]

Hi,

I'm following the instructions from https://github.com/commonsense/conceptnet5, to build the ConceptNet DB from raw data. However, the process (snakemake -j 8 --resource 'ram=16') fails in this step:

rule read_conceptnet4:
 input: data/raw/conceptnet4/conceptnet4_flat_2.jsons, data/db/wiktionary.db
 output: data/edges/conceptnet4/conceptnet4_flat_2.msgpack
 jobid: 215
 wildcards: num=2

Traceback (most recent call last):
 File "/usr/local/bin/cn5-read", line 11, in <module>
 load_entry_point('ConceptNet', 'console_scripts', 'cn5-read')()
 File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 722, in __call__
 return self.main(*args, **kwargs)
 File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 697, in main
 rv = self.invoke(ctx)
 File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 1066, in invoke
 return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 895, in invoke
 return ctx.invoke(self.callback, **ctx.params)
 File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 535, in invoke
 return callback(*args, **kwargs)
 File "/storage/datasets/conceptnet5/conceptnet5/readers/cli.py", line 23, in run_conceptnet4
 conceptnet4.handle_file(input, output)
 File "/storage/datasets/conceptnet5/conceptnet5/readers/conceptnet4.py", line 389, in handle_file
 builder.transform_file(input_filename, output_file)
 File "/storage/datasets/conceptnet5/conceptnet5/readers/conceptnet4.py", line 383, in transform_file
 for new_obj in self.handle_assertion(obj):
 File "/storage/datasets/conceptnet5/conceptnet5/readers/conceptnet4.py", line 371, in handle_assertion
 if not skip_assertion(source_dict, start, end):
 File "/storage/datasets/conceptnet5/conceptnet5/readers/conceptnet4.py", line 177, in skip_assertion
 if lemmatize_uri(start) in CONCEPT_BLACKLIST or lemmatize_uri(end) in CONCEPT_BLACKLIST:
 File "/storage/datasets/conceptnet5/conceptnet5/language/lemmatize.py", line 227, in lemmatize_uri
 return LEMMATIZER.lemmatize_uri(uri)
 File "/storage/datasets/conceptnet5/conceptnet5/language/lemmatize.py", line 196, in lemmatize_uri
 root, _form = self.lookup(language, text, pos)
 File "/storage/datasets/conceptnet5/conceptnet5/language/lemmatize.py", line 151, in lookup
 cursor.execute(QUERY, (language, word))
sqlite3.OperationalError: no such table: forms

Error in rule read_conceptnet4:
 jobid: 0
 output: data/edges/conceptnet4/conceptnet4_flat_2.msgpack
 
RuleException:
CalledProcessError in line 233 of /storage/datasets/conceptnet5/Snakefile:
Command 'cn5-read conceptnet4 data/raw/conceptnet4/conceptnet4_flat_2.jsons data/edges/conceptnet4/conceptnet4_flat_2.msgpack' returned non-zero exit status 1
 File "/storage/datasets/conceptnet5/Snakefile", line 233, in __rule_read_conceptnet4
 File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
Exiting because a job execution failed. Look above for error message

Actually this line:

cn5-read conceptnet4 data/raw/conceptnet4/conceptnet4_flat_2.jsons data/edges/conceptnet4/conceptnet4_flat_2.msgpack

is the one that fails with error:

sqlite3.OperationalError: no such table: forms

I've not been able to find a more precise cause of error. I'd appreciate any guide with this.