error raised when trying to use normalize_table.py

[MushroomLady]

I'm trying to implement the normalize_table.py command but I keep getting the following errors: I've tried it with the otu table formatted in multiple ways. Is the OTU table supposed to be in biom, hdf5 or json format?

MacQIIME kosh-321-6:SOD $ normalize_table.py -i OTUs_2016/OTU_tables_Allfungi/otus_table_taxonomy_0.97_hdf5.biom -a CSS -o normalized_tables/

Traceback (most recent call last):

  File "/macqiime/anaconda/bin/normalize_table.py", line 157, in <module>

    main()

  File "/macqiime/anaconda/bin/normalize_table.py", line 140, in main

    normalize_CSS(input_path, out_path, output_CSS_statistics)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/normalize_table.py", line 36, in normalize_CSS

    temp_fh.write(tmp_bt.to_json('forR'))

  File "/macqiime/anaconda/lib/python2.7/site-packages/biom/table.py", line 3729, in to_json

    dumps(obs[2])))

  File "/macqiime/anaconda/lib/python2.7/json/__init__.py", line 243, in dumps

    return _default_encoder.encode(obj)

  File "/macqiime/anaconda/lib/python2.7/json/encoder.py", line 207, in encode

    chunks = self.iterencode(o, _one_shot=True)

  File "/macqiime/anaconda/lib/python2.7/json/encoder.py", line 270, in iterencode

    return _iterencode(o, 0)

UnicodeDecodeError: 'utf8' codec can't decode byte 0xeb in position 10: unexpected end of data

I just updated to Mac Os El Capitan and just re-installed macqiime 1.9 so it's possible that not everything is working properly. this is the output of print_qiime_config.py -t 

MacQIIME kosh-321-6:SOD $ print_qiime_config.py -t 

System information

==================

         Platform: darwin

   Python version: 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42)  [GCC 4.2.1 (Apple Inc. build 5577)]

Python executable: /macqiime/anaconda/bin/python

QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:

 https://github.com/biocore/qiime-default-reference/releases/tag/0.1.2

Dependency versions

===================

          QIIME library version: 1.9.1

           QIIME script version: 1.9.1

qiime-default-reference version: 0.1.2

                  NumPy version: 1.9.2

                  SciPy version: 0.15.1

                 pandas version: 0.16.1

             matplotlib version: 1.4.3

            biom-format version: 2.1.4

                   h5py version: 2.4.0 (HDF5 version: 1.8.14)

                   qcli version: 0.1.1

                   pyqi version: 0.3.2

             scikit-bio version: 0.2.3

                 PyNAST version: 1.2.2

                Emperor version: 0.9.51

                burrito version: 0.9.1

       burrito-fillings version: 0.1.1

              sortmerna version: SortMeRNA version 2.0, 29/11/2014

              sumaclust version: SUMACLUST Version 1.0.00

                  swarm version: Swarm 1.2.19 [Jun  2 2015 14:40:16]

                          gdata: Installed.

QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

 http://qiime.org/install/qiime_config.html

 http://qiime.org/tutorials/parallel_qiime.html

                     blastmat_dir: None

      pick_otus_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                         sc_queue: all.q

      topiaryexplorer_project_dir: None

     pynast_template_alignment_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

                  cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                     torque_queue: friendlyq

                    jobs_to_start: 1

                       slurm_time: None

            denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

                         temp_dir: /tmp/

                     slurm_memory: None

                      slurm_queue: None

                      blastall_fp: blastall

                 seconds_to_sleep: 60

QIIME base install test results

===============================

.........

----------------------------------------------------------------------

Ran 9 tests in 0.043s

OK

[abir...@gmail.com]

Hi--Hmm, it appears to me that the input is expected to be a hdf5 biom file, so I'm not sure what's amiss.  Let me query one of the developers; stay tuned!

Amanda

[abir...@gmail.com]

Hi, 

I contacted the developer for the normalize functionality, who said:

"Thanks for bringing this to my attention!  It is not readily apparent to me.  However, just as a check, can the user please specify full -i and -o paths?  Also, using an actual .biom table output would be helpful, e.g.

-o /full/path/to/otus_table_taxonomy_0.97_hdf5_output.biom

hdf5 biom format is preferred, but the script should work on either json or hdf5."

Also, could you please post the input biom table file you're using, so we can double-check that there are no issues with its format?

Thanks,

Amanda