using parsemap to create connectograms

[Tim Meehan]

Hello,

I am trying to follow the recipe in the tutorial for connectograms (8.19). I believe I have created the map and link files appropriately as indicated to use as input for the parsemap script. However, when I simply test whether the parsemap script works at all (i.e. by attempting to run it in the tutorial folder with the map and link files that came with the tutorial) I get an error:

command: parsemap -map map.txt -links map.links.txt

error: bash: parsemap: command not found

Can anyone offer advice on how to run the script correctly? Do I need to run it from perl as opposed to the linux command line?

Also, once I do get it to run correctly, I am confused as to the location I should run it from. I've organized separate directories for each image I wish to create as recommended in the best practices section, for example:

/home/me/circos/project1/data

/home/me/circos/project1/etc

Judging by the tutorial directory structure, I should have my map and links files in the project1 directory and run parsemap from there. Does this mean I need to copy parsemap to project1 so that it takes the correct inputs?

Thanks,

Tim

Using linux CentOS 5.5 distribution

[Martin Krzywinski]

Hi Tim,

Try running the script as

./parsemap -map ...

If you don't have "." in your PATH, even if the script is in your current directory the shell won't find it - you need the "./" prefix. If you're using the script frequently, you can put it in some central location, e.g. /usr/local/bin. It probably shouldn't be stored together with an images data/conf files. This is just a helper script - you don't need it if your data pipeline is already producing data in the right format.

The input files to the script are passed using the -map and -links flags, so the script doesn't need to be in the same location as the files.

I suggest the following directory structure

project1/

 raw/

  map.txt          # your parcellation map

  map.links.txt  # links between regions

 data/              # circos data files

 etc/                # circos conf files

Assuming you've put parsemap in /usr/local/bin,

cd project1

/usr/local/bin/parsemap -map raw/map.txt -links raw/map.links.txt

By default the script will put data files in data/ and configuration in etc/ under the directory from which it was run.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

--
You received this message because you are subscribed to the Google Groups "Circos" group.
To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.
To post to this group, send email to circos-data-...@googlegroups.com.
Visit this group at http://groups.google.com/group/circos-data-visualization?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Martin Krzywinski]

I will add this content to the tutorial text to clarify how the script works.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

[Tim Meehan]

Thanks so much for the quick reply.

I organized as you advised and used the correct syntax to run the parsemap script (clearly I don't do much bash scripting :) ). I get the following error:

Use of uninitialized value in printf at /home/meehan/software/circos/parsemap line 179, <FILE> line 11.

It looks like the error is in the heatmap section of the parsemap script, and I suspect it may have to do with the fact that I supplied no measurements with which to build heatmaps as I would like my connectogram to contain only lobes, parcellations, and links. Is this possible to do using parsemap?

Unfortunately, I can't attach files as I am using an out of date browser due to my old linux distribution. They're short enough that I can paste them below, however.

Thanks,
Tim

####map.txt#####
Fro FEF 255 204 0
Par aIPS 204 0 153
Par pIPS 102 0 0
D.Occ V3 0 255 204
D.Occ V3Ad 102 255 255
D.Occ V2d 0 255 255
D.Occ V1d 51 255 204
V.Occ V3Av 204 204 255
V.Occ VP 153 204 255
V.Occ V2v 153 153 255
V.Occ V1v 102 102 255

#####map.links.txt######
l aIPS l V1d 0 0.1830675
r aIPS r V1d 0 0.192829166666667
l FEF l V1d 0 0.23310875
r FEF r V1d 0 0.1888575
l pIPS l V1d 0 0.145930833333333
r pIPS r V1d 0 0.162990833333333
l aIPS l V1v 0 0.1771725
r aIPS r V1v 0 0.241859166666667
l FEF l V1v 0 0.2485775
r FEF r V1v 0 0.232760833333333
l pIPS l V1v 0 0.1385775
r pIPS r V1v 0 0.154633333333333
l aIPS l V2d 0 0.09450375
r aIPS r V2d 0 0.166645
l FEF l V2d 0 0.161285
r FEF r V2d 0 0.143714166666667
l pIPS l V2d 0 0.116401666666667
r pIPS r V2d 0 0.105245833333333
l aIPS l V2v 0 0.19673375
r aIPS r V2v 0 0.237115
l FEF l V2v 0 0.21999125
r FEF r V2v 0 0.218260833333333
l pIPS l V2v 0 0.174376666666667
r pIPS r V2v 0 0.15149
l aIPS l V3 0 0.112695
r aIPS r V3 0 0.164829166666667
l FEF l V3 0 0.13496125
r FEF r V3 0 0.131531666666667
l pIPS l V3 0 0.0948141666666667
r pIPS r V3 0 0.110899166666667
l aIPS l V3Ad 0 0.093905
r aIPS r V3Ad 0 0.1544425
l FEF l V3Ad 0 0.1432025
r FEF r V3Ad 0 0.146293333333333
l pIPS l V3Ad 0 0.120335
r pIPS r V3Ad 0 0.0879641666666667
l aIPS l V3Av 0 0.07651125
r aIPS r V3Av 0 0.176380833333333
l FEF l V3Av 0 0.12729
r FEF r V3Av 0 0.163821666666667
l pIPS l V3Av 0 0.0997841666666667
r pIPS r V3Av 0 0.0952258333333333
l aIPS l VP 0 0.19273125
r aIPS r VP 0 0.255470833333333
l FEF l VP 0 0.2586475
r FEF r VP 0 0.2344825
l pIPS l VP 0 0.202443333333333
r pIPS r VP 0 0.165338333333333

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visualization+unsub...@googlegroups.com.

[Martin Krzywinski]

I modified parsemap (attached) to include your case, where there are no heatmaps. A couple of other tweaks too.

I attach an example project directory for your image. To generate it, run the file 'run'

> ./run

This assumes that parsemap is in /usr/local/bin and that the circos binary is in your path. If not, adjust 'run'.

The configuration file etc/circos.conf is very similar to the example in the tutorial. I modified it slightly to allow for the fact that you don't have heatmaps. 

I commented out reference to the heatmaps. If you don't use them, you can remove these lines altogether.

Your link scores were in the range [0,0.25] so I adjusted the link color/thickness rules to use this interval rather than [0,1].

Let me know whether this works for you.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.

[Tim Meehan]

Thanks very much for the help, very much appreciated. The image you generated is exactly what I had in mind.

However, I don't see the modified version of parsemap in the directory you sent.

Maybe you can clarify something for me as well. Does the parsemap script generate a circos.conf file, or does that need to be generated independently?

Thanks,
Tim

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visualization+unsubsc...@googlegroups.com.

To post to this group, send email to circos-data-...@googlegroups.com.

Visit this group at http://groups.google.com/group/circos-data-visualization?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Martin Krzywinski]

You don't need root permissions - the 'run' script is just a batch file.

Make it executable by

> chmod +x run

or run it like this

> . ./run

I attached the new parsemap in my previous message. I fixed the script so that it no longer produces the error.

It doesn't create the etc/circos.conf file, only the data files (these include color file and segment order file). 

All the files you need, including etc/circos.conf, are included in the tim.tgz archive I attached in the previous file. The archive is self contained - you don't need to move any files or add any files.

The only thing to do is to move the parsemap script to some central location (e.g. /usr/local/bin).

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

On Fri, May 17, 2013 at 8:06 PM, Tim Meehan <time...@gmail.com> wrote:

Thank you for all the help, very much appreciated.

Unfortunately, I don't seem to have permission to use 'run'. I am running from my home folder. Even so I tried sudoing which did not work either.

When I am able to use the 'run' script, will parsemap not give the same error? And does parsemap generate the circos.conf file, or do I need to copy this into the appropriate project project/etc directory beforehand?

Thanks again,

Tim

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visualization+unsubsc...@googlegroups.com.

To post to this group, send email to circos-data-...@googlegroups.com.

Visit this group at http://groups.google.com/group/circos-data-visualization?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Circos" group.
To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visualization+unsub...@googlegroups.com.
To post to this group, send email to circos-data-...@googlegroups.com.
Visit this group at http://groups.google.com/group/circos-data-visualization?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Tim Meehan]

Sorry for the permission confusion, I sorted that out.

However, I still don't see the updated parsemap script. It does not seem to be in the directory you attached. Could you reattach it? Or can I simply comment out the heatmap section?

Thanks,

Tim

[Martin Krzywinski]

The script was attached to the message I sent as a separate file. It's not in the directory archive.

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.

[Tim Meehan]

I'm sorry, but I only see two attachments. One is the directory tar file, the other is a png file. I don't have any other messages with attachments.

[Martin Krzywinski]

Weird... it was the third attachment. Here it is again.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.

[Tim Meehan]

Thank you! I got it.

Still very bizarre though: I can't see the attachment in the group, but can see it in my private gmail account. Will keep that in mind for next time. Sorry for the back and forth, and thanks a lot for the help.

Tim

[Tim Meehan]

Hello again,

Was only just now able to test the new parsemap script. Seems that error is gone, but I get a new one from circos (this is using the same files included in the 'tim' directory):

[meehan@cnl10 tim]$ ./run

./run: line 6: /home/meehan/software/circos/parsemap: Permission denied

debuggroup conf 0.15s welcome to circos v0.62-1 25 Jun 2012

debuggroup conf 0.15s guessing configuration file

debuggroup conf 0.16s looking for conf file /home/meehan/software/circos/tim/etc/circos.conf

debuggroup conf 0.16s found conf file /home/meehan/software/circos/tim/etc/circos.conf

debuggroup summary 0.40s debug will appear for these features: summary

debuggroup summary 0.40s parsing karyotype and organizing ideograms

  *** CIRCOS ERROR ***

  Incorrectly formatted chromosomes_order field (or content of

  chromosomes_order_file). Tag [bst] is not associated with any chromosome.

  If you are having trouble debugging this error, use this tutorial to learn how

  to use the debugging facility

      http://www.circos.ca/tutorials/lessons/configuration/debugging

  If you're still stumped, get support in the Circos Google Group

      http://groups.google.com/group/circos-data-visualization

  Stack trace:

 at /home/meehan/software/circos/circos_main/bin/../lib/Circos/Error.pm line 332

Circos::Error::fatal_error('ideogram', 'orphan_tag', 'bst') called at /home/meehan/software/circos/circos_main/bin/../lib/Circos.pm line 5599

Circos::read_chromosomes_order() called at /home/meehan/software/circos/circos_main/bin/../lib/Circos.pm line 303

Circos::run('Circos') called at /home/meehan/software/circos/circos_main/bin/circos line 233

[Martin Krzywinski]

Remove the bst tag from the file in etc/ that is determining the order of segments.

The new parsemap script creates this file and shouldn't write this tag out ... I thought I fixed that.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.

[Tim Meehan]

Is there a way to remove the writing of that tag in the parsemap script? Since I need to create many images, it would be easier not to have to change each of the segment order files individually. However, if that is too involved, it is not a huge problem for me to make those individual changes.

[Martin Krzywinski]

The script writes the order to 

etc/segment.order.conf

The circos.conf file refers to this file with

chromosomes_order_file = etc/segment.order.conf

To not use the file that the script creates, use your own segment file and disregard the one generated by the script. 

chromosomes_order_file = etc/segment.order.custom.conf

Later, I can add a flag to omit this output.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.

[Tim Meehan]

Just to clarify -- seems that parsemap only creates some conf files, but does not create, for example, ideogram.conf, and I need to copy those from the directory you supplied into any new directory from which I want create images. This is judging from the fact that when I used the run script from a new directory there was no ideogram.conf file for circos to use in that path.

Is this correct? If so, which conf files do I need to copy because they are not created by parsemap?

[Martin Krzywinski]

Yes, parsemap only creates files that are relevant to the data. You need to provide the other configuration files (ideogram, ticks, circos).

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

To unsubscribe from this group and stop receiving emails from it, send an email to circos-data-visual...@googlegroups.com.

[Martin Krzywinski]

I've added debug output to parsemap (attached) that shows what files are being written.

> ./parsemap -map map.txt -links map.links.txt -debug
...
debug[1] wrote file etc/color.brain.conf
debug[1] wrote file data/segments.txt
debug[1] wrote file etc/segment.order.conf
debug[1] wrote file data/structure.label.txt
debug[1] wrote file data/measure.0.txt
debug[1] wrote file data/measure.1.txt
debug[1] wrote file data/measure.2.txt
debug[1] wrote file data/measure.3.txt
debug[1] wrote file data/measure.4.txt
debug[1] wrote file data/links.txt

I also amended the tutorial to clarify that the core of the configuration files are provided by the user and not generated by parsemap.

You'll need to supply the remaining configuration files that define how the image is organized — sample configuration can be found in etc/. These areetc/circos.conf, etc/ideogram*conf, etc/ticks*conf and etc/bands.conf.

Hopes this help clear things up.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca

[Tim Meehan]

Brilliant, thanks! I think for now I have figured out the protocol. Thank you for all your help.

You received this message because you are subscribed to a topic in the Google Groups "Circos" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/circos-data-visualization/lmFI7-2Skcc/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to circos-data-visual...@googlegroups.com.

[Martin Krzywinski]

You're welcome.

We're here to help.

Martin Krzywinski

science + art

http://mkweb.bcgsc.ca