text2bin.py question

[gahu...@gmail.com]

I am very new to TF, and I'm still trying things out.

I was trying text2bin.py and came across an error:

struct.error: pack expected 50 items for packing (got 1)

for the code below:

        vecs_out.write(fmt.pack(vec))

I am wondering, if my 'vec' should be broken down to strings to match the number of expected item.

From the original model, 'vec' was a tuple in a list, so it looks like it was suppose to be 1.

Any help??

My full code is a below:

import getopt
import os
import struct
import sys

try:
  opts, args = getopt.getopt(sys.argv[1:], 'o:v:', ['output=', 'vocab='])

except getopt.GetoptError as e:
    print (e, file=sys.stderr)
    sys.exit(2)

opt_output = 'vecs.bin'
opt_vocab = 'vocab.txt'
for o, a in opts:
  if o in ('-o', '--output'):
    opt_output = a
  if o in ('-v', '--vocab'):
    opt_vocab = a
def go(fhs):
  fmt = None
  with open(opt_vocab, 'w+') as vocab_out:
    with open(opt_output, 'wb+') as vecs_out:
      for lines in list(zip(fhs)):
        parts = [str(line).split() for line in lines]
        token = parts[0][0]
        print(token)

        if any(part[0] != token for part in parts[1:]):
          raise IOError('vector files must be aligned')

        print(token, file = vocab_out)

        vec = [sum(float(x) for x in xs) for xs in zip(parts[0][1:])]

        if not fmt:
          fmt = struct.Struct('%df' % len(vec))
          print ('hi')

        print (vec)
        vecs_out.write(fmt.pack(vec))

if args:
  fhs = [open(filename) for filename in args]
  go(fhs)
  for fh in fhs:
    fh.close()
else:
  fhs = open('C:\\Users\\jhan\Desktop\\tensorflow\\swivel\\swivel output\\col_embedding.tsv')
  go(fhs)

[Chris Waterson]

Try fmt.pack(*vec)?

--
You received this message because you are subscribed to the Google Groups "Swivel Embeddings" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swivel-embeddings+unsubscribe@googlegroups.com.
To post to this group, send email to swivel-embeddings@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/swivel-embeddings/a7a7b153-7194-4f06-a566-dc0e46df4b5b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gahu...@gmail.com]

I'm trying to convert everything to Python 3, so I'm having little tough time here.

I read that TensorFlow only works on Python3.5.

'*' would give out tuple in python 2.7, but would it be same for 3.5?

Try fmt.pack(*vec)?

To unsubscribe from this group and stop receiving emails from it, send an email to swivel-embeddi...@googlegroups.com.
To post to this group, send email to swivel-e...@googlegroups.com.