problem loading cdf files

nm29

unread,

Jan 4, 2010, 3:50:14 PM1/4/10

to aroma.affymetrix

I would like to use FIRMA for analyzing Affy exon array data. I have
copied my session up to the point where it fails below:
> getwd()
[1] "/Users/Neel/Projects/HuR Binding Site/HuR Knockdown/Analysis/
FIRMA"
> library(aroma.affymetrix)
Loading required package: R.utils
Loading required package: R.oo
Loading required package: R.methodsS3
R.methodsS3 v1.0.3 (2008-07-02) successfully loaded. See ?R.methodsS3
for help.
R.oo v1.6.5 (2009-10-30) successfully loaded. See ?R.oo for help.
R.utils v1.2.4 (2009-10-30) successfully loaded. See ?R.utils for
help.
Loading required package: aroma.core
Loading required package: R.cache
R.cache v0.2.0 (2009-10-16) successfully loaded. See ?R.cache for
help.
Loading required package: R.filesets
Loading required package: digest
R.filesets v0.6.5 (2009-10-30) successfully loaded. See ?R.filesets
for help.
Loading required package: R.rsp
R.rsp v0.3.6 (2009-09-16) successfully loaded. See ?R.rsp for help.
Type browseRsp() to open the RSP main menu in your browser.
Loading required package: matrixStats
Loading required package: aroma.light
aroma.light v1.15.1 (2009-11-01) successfully loaded. See ?aroma.light
for help.
aroma.core v1.3.1 (2009-11-03) successfully loaded. See ?aroma.core
for help.
Loading required package: aroma.apd
Loading required package: R.huge
R.huge v0.2.0 (2009-10-16) successfully loaded. See ?R.huge for help.
Loading required package: affxparser
aroma.apd v0.1.7 (2009-10-16) successfully loaded. See ?aroma.apd for
help.
aroma.affymetrix v1.3.0 (2009-11-01) successfully loaded. See ?
aroma.affymetrix for help.
Patching /Users/Neel/.Rpatches/aroma.affymetrix/20091204/
AffymetrixCelSet.R
Patching /Users/Neel/.Rpatches/aroma.affymetrix/20091204/
AromaUflFile.R
> verbose <- Arguments$getVerbose(-8, timestamp=TRUE)
> chipType <- "HuEx-1_0-st-v2"
> cdf <- AffymetrixCdfFile$byChipType(chipType, tags="core,A20071112,EP")
> print(cdf)

*** caught segfault ***
address 0xfffffffc, cause 'memory not mapped'

Traceback:
1: .Call("R_affx_get_cdf_file_header", filename, PACKAGE =
"affxparser")
2: readCdfHeader(getPathname(this))
3: getHeader.AffymetrixCdfFile(this)
4: getHeader(this)
5: getChipType.AffymetrixCdfFile(this)
6: getChipType(this)
7: sprintf("Chip type: %s", getChipType(this))
8: as.character.AromaChipTypeAnnotationFile(x)
9: NextMethod("as.character", this, ...)
10: as.character.AffymetrixCdfFile(x)
11: as.character(x)
12: print(as.character(x))
13: print.Object(cdf)
14: print(cdf)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

Any help would be greatly appreciated.
Sincerely,
Neel

Henrik Bengtsson

unread,

Jan 4, 2010, 3:56:23 PM1/4/10

to aroma-affymetrix

Hi,

what does the following return:

library("aroma.affymetrix");
print(sessionInfo());

and

chipType <- "HuEx-1_0-st-v2"
pathname <- AffymetrixCdfFile$findByChipType(chipType,
tags="core,A20071112,EP");
print(file.info(pathname));

/Henrik

> --
> When reporting problems on aroma.affymetrix, make sure 1) to run the latest version of the package, 2) to report the output of sessionInfo() and traceback(), and 3) to post a complete code example.
>
>
> You received this message because you are subscribed to the Google Groups "aroma.affymetrix" group.
> To post to this group, send email to aroma-af...@googlegroups.com
> To unsubscribe from this group, send email to aroma-affymetr...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/aroma-affymetrix?hl=en

nm29

unread,

Jan 4, 2010, 3:58:49 PM1/4/10

to aroma.affymetrix

On Jan 4, 3:56 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:

> Hi,
>
> what does the following return:
>
> library("aroma.affymetrix");
> print(sessionInfo());

> library("aroma.affymetrix")
> print(sessionInfo())
R version 2.10.1 (2009-12-14)
i386-apple-darwin8.11.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods
base

other attached packages:
[1] aroma.affymetrix_1.3.0 aroma.apd_0.1.7
affxparser_1.18.0
[4] R.huge_0.2.0 aroma.core_1.3.1
aroma.light_1.15.1
[7] matrixStats_0.1.8 R.rsp_0.3.6
R.filesets_0.6.5
[10] digest_0.4.2 R.cache_0.2.0
R.utils_1.2.4
[13] R.oo_1.6.5 R.methodsS3_1.0.3

>
> and
>
> chipType <- "HuEx-1_0-st-v2"
> pathname <- AffymetrixCdfFile$findByChipType(chipType,
> tags="core,A20071112,EP");
> print(file.info(pathname));

> chipType <- "HuEx-1_0-st-v2"
> pathname <- AffymetrixCdfFile$findByChipType(chipType,ags="core,A20071112,EP")
> print(file.info(pathname))
Error in file.info(pathname) : invalid filename argument

Henrik Bengtsson

unread,

Jan 4, 2010, 4:08:15 PM1/4/10

to aroma-affymetrix

Hi,

you also need to have the working directory - getwd() - the same as
when you get the core dump. Right now it didn't find the CDF file at
all. This will help you:

library("aroma.affymetrix");

# Throws an explanatory error if you don't have the right directory structure
path <- Arguments$getReadablePath("annotationData/chipTypes/HuEx-1_0-st-v2/");

# Then:

chipType <- "HuEx-1_0-st-v2"
pathname <- AffymetrixCdfFile$findByChipType(chipType,
tags="core,A20071112,EP");
print(file.info(pathname));

# Alternatively:
cdf <- AffymetrixCdfFile$byChipType(chipType, tags="core,A20071112,EP");
pathname <- getPathname(cdf);
print(file.info(pathname));

/Henrik

nm29

unread,

Jan 4, 2010, 4:32:59 PM1/4/10

to aroma.affymetrix

Hi Henrik,
I continued with the analysis described here
http://groups.google.com/group/aroma-affymetrix/web/human-exon-array-analysis.
When I try to define the cel set I get the following error:
> cs <- AffymetrixCelSet$byName("HuRknockdown", cdf=cdf)
Error in method(static, ...) : object 'cdf' not found

The following describes my directory sturcture:
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA
annotationData rawData
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA/annotationData/
chipTypes
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA/annotationData/
chipTypes/
HuEx-1_0-st-v2
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA/annotationData/
chipTypes/HuEx-1_0-st-v2/
HuEx-1_0-st-v2,control,A20071112,EP.cdf HuEx-1_0-st-
v2,extended,A20071112,EP.cdf HuEx-1_0-st-v2,main,A20071112,EP.cdf
HuEx-1_0-st-v2,core,A20071112,EP.cdf HuEx-1_0-st-
v2,full,A20071112,EP.cdf
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA/rawData/
HuRknockdown
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA/rawData/
HuRknockdown/
HuEx-1_0-st-v2
Neelanjan-Mukherjees-iMac:Analysis Neel$ ls FIRMA/rawData/HuRknockdown/
HuEx-1_0-st-v2/
HuR_KD_A.CEL HuR_KD_B.CEL HuR_KD_C.CEL Neg_KD_A.CEL Neg_KD_B.CEL
Neg_KD_C.CEL

Is there something that I can do fix the working directory/core dump
problem?
Thanks for being so quick to respond!
Neel

On Jan 4, 4:08 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:
> Hi,
>

Henrik Bengtsson

unread,

Jan 4, 2010, 4:38:03 PM1/4/10

to aroma-affymetrix

Hi,

I am indeed trying to help you troubleshoot this, so please report the
output of what I ask for. If it gives errors, we have to figure out
why. It does not make sense to continue with the script - each step
is there for a reason.

/Henrik

nm29

unread,

Jan 4, 2010, 4:44:37 PM1/4/10

to aroma.affymetrix

Fair enough (I probably got ahead of things!)
Here is the output from the commands you suggested:

> path <- Arguments$getReadablePath("annotationData/chipTypes/HuEx-1_0-st-v2/")

> chipType <- "HuEx-1_0-st-v2"
> pathname <- AffymetrixCdfFile$findByChipType(chipType,tags="core,A20071112,EP")

> print(file.info(pathname))

size
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 40108891

isdir mode
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf FALSE 644

mtime
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 2007-12-03 17:26:56

ctime
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 2009-12-31 10:12:42

atime
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 2010-01-04 15:43:19

uid gid
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 503 20

uname
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf Neel

grname
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf staff
> print(file.info(pathname))

size isdir mode mtime
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 40108891 FALSE 644 2007-12-03 17:26:56

ctime atime uid gid uname
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf 2009-12-31 10:12:42 2010-01-04 15:43:19 503
20 Neel

grname
annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf staff

Next, I went on to defining the cel set (which is what the next
recommended step is) and got the following error:
> cs <- AffymetrixCelSet$byName("BCGC_2006", cdf=cdf)

Error in method(static, ...) : object 'cdf' not found

Neel

On Jan 4, 4:38 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:
> Hi,
>

> I am indeed trying to help you troubleshoot this, so please report the
> output of what I ask for. If it gives errors, we have to figure out
> why. It does not make sense to continue with the script - each step
> is there for a reason.
>
> /Henrik
>
>
>
> On Mon, Jan 4, 2010 at 1:32 PM, nm29 <neelanjanmukher...@gmail.com> wrote:
> > Hi Henrik,
> > I continued with the analysis described here

> >http://groups.google.com/group/aroma-affymetrix/web/human-exon-array-....

Henrik Bengtsson

unread,

Jan 4, 2010, 5:05:56 PM1/4/10

to aroma-affymetrix

Good. So, this narrows it down to be either (i) an issue with your
affxparser installation or (b) a corrupt CDF file. I suspect the
former, but let's check the latter first, because it is easier.

Q1. What does the following output:

library("aroma.affymetrix");
pathname <- AffymetrixCdfFile$findByChipType("HuEx-1_0-st-v2",
tags="core,A20071112,EP");
df <- GenericDataFile(pathname);
print(df);
print(getChecksum(df));

Q2. POSSIBLY A CLUE, if the file is corrupt: You CDF file is named
HuEx-1_0-st-v2,core,A20071112,EP.cdf. Although its filesize matches
known ones, the name does not. On the chip type page 'HuEx-1_0-st-v2'
[http://groups.google.com/group/aroma-affymetrix/web/huex-1-0-st-v2]
and subpage 'Affymetrix transcript clusters definitions' there are
HuEx-1_0-st-v2,coreR3,A20071112,EP.cdf (and
HuEx-1_0-st-v2,coreR2,A20070914,EP.cdf). Note the tag 'coreR3'
whereas your is 'core'. The filesizes are all 40108891 bytes. How
did you get hold of your file? (It might be that Elizabeth P or
someone else decided to add that 'R3' later; I don't follow the
history of those updated)

/Henrik

nm29

unread,

Jan 4, 2010, 5:19:16 PM1/4/10

to aroma.affymetrix

On Jan 4, 5:05 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:

> Good. So, this narrows it down to be either (i) an issue with your
> affxparser installation or (b) a corrupt CDF file. I suspect the
> former, but let's check the latter first, because it is easier.
>
> Q1. What does the following output:
>
> library("aroma.affymetrix");
> pathname <- AffymetrixCdfFile$findByChipType("HuEx-1_0-st-v2",
> tags="core,A20071112,EP");
> df <- GenericDataFile(pathname);
> print(df);
> print(getChecksum(df));

> pathname <- AffymetrixCdfFile$findByChipType("HuEx-1_0-st-v2",tags="core,A20071112,EP")
> df <- GenericDataFile(pathname)
> print(df)
GenericDataFile:
Name: HuEx-1_0-st-v2
Tags: core,A20071112,EP
Full name: HuEx-1_0-st-v2,core,A20071112,EP
Pathname: annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-
v2,core,A20071112,EP.cdf
File size: 38.25 MB (40108891 bytes)
RAM: 0.00 MB
> print(getChecksum(df))
[1] "e7b0bacd27699534d125b16266d7cc09"

>
> Q2. POSSIBLY A CLUE, if the file is corrupt: You CDF file is named
> HuEx-1_0-st-v2,core,A20071112,EP.cdf. Although its filesize matches
> known ones, the name does not. On the chip type page 'HuEx-1_0-st-v2'
> [http://groups.google.com/group/aroma-affymetrix/web/huex-1-0-st-v2]
> and subpage 'Affymetrix transcript clusters definitions' there are
> HuEx-1_0-st-v2,coreR3,A20071112,EP.cdf (and
> HuEx-1_0-st-v2,coreR2,A20070914,EP.cdf). Note the tag 'coreR3'
> whereas your is 'core'. The filesizes are all 40108891 bytes. How
> did you get hold of your file? (It might be that Elizabeth P or
> someone else decided to add that 'R3' later; I don't follow the
> history of those updated)

I believe I got the cdf file here:
http://groups.google.com/group/aroma-affymetrix/web/affymetrix-defined-transcript-clusters

> ...
>
> read more »

Henrik Bengtsson

unread,

Jan 4, 2010, 5:29:30 PM1/4/10

to aroma-affymetrix

Ok, this file is identical to HuEx-1_0-st-v2,coreR3,A20071112,EP.cdf
and correct; it gives the same checksum for me:

pathname <- AffymetrixCdfFile$findByChipType("HuEx-1_0-st-v2",tags="coreR3,A20071112,EP");
print(getChecksum(df));
[1] "e7b0bacd27699534d125b16266d7cc09"

TROUBLESHOOTING affxparser:

Now I suspect that you still get the core dump if you do the following:

Start a fresh R session and do:

library("affxparser");
pathname <- "annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-v2,core,A20071112,EP.cdf";
stopifnot(file.exists(pathname));

# Here is the line that I guess gives you the core dump
hdr <- readCdfHeader(pathname);

# If not...
str(hdr);

Confirm that you get a core dump above.

/Henrik

nm29

unread,

Jan 4, 2010, 5:44:44 PM1/4/10

to aroma.affymetrix

> library("affxparser")
> pathname <- "annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-v2,core,A20071112,EP.c df"

> stopifnot(file.exists(pathname))
Error: file.exists(pathname) is not TRUE
> hdr <- readCdfHeader(pathname)
Error in readCdfHeader(pathname) :
Cannot read CDF header. File not found: annotationData/chipTypes/
HuEx-1_0-st-v2/HuEx-1_0-st-v2,core,A20071112,EP.c df
> str(hdr)
Error in str(hdr) : object 'hdr' not found

On Jan 4, 5:29 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:

> >http://groups.google.com/group/aroma-affymetrix/web/affymetrix-define...

> ...
>
> read more »

Henrik Bengtsson

unread,

Jan 4, 2010, 6:39:44 PM1/4/10

to aroma-affymetrix

Please make sure to copy'n'paste things correctly; you have a space
character in your pathname!

/H

nm29

unread,

Jan 4, 2010, 6:47:59 PM1/4/10

to aroma.affymetrix

Sorry about that. I did get the core dump as you predicted:

> library("affxparser")
> pathname <- "annotationData/chipTypes/HuEx-1_0-st-v2/HuEx-1_0-st-v2,core,A20071112,EP.cdf"
> stopifnot(file.exists(pathname))

> hdr <- readCdfHeader(pathname)

*** caught segfault ***
address 0xfffffffc, cause 'memory not mapped'

Traceback:
1: .Call("R_affx_get_cdf_file_header", filename, PACKAGE =
"affxparser")

2: readCdfHeader(pathname)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

On Jan 4, 6:39 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:

> ...
>
> read more »

Henrik Bengtsson

unread,

Jan 4, 2010, 7:14:18 PM1/4/10

to aroma-affymetrix

Ok, now we have ruled out that it has anything to do with
aroma.affymetrix; it is an issue with *your installation* of
affxparser that fails to read the CDF. We will have to report this
problem/move this discussion to Bioconductor instead, but before doing
that let's work with a CDF that anyone can access.

1. Install the AffymetrixDataTestFiles Bioconductor package, i.e.

source("http://bioconductor.org/biocLite.R");
biocLite("AffymetrixDataTestFiles");

so that

library("AffymetrixDataTestFiles");

works!

2. Try the following:

library("AffymetrixDataTestFiles");
pathname <- "annotationData/chipTypes/Test3/1.XDA/Test3.CDF";
pathname <- system.file(pathname, package="AffymetrixDataTestFiles");
stopifnot(file.exists(pathname));
print(pathname);

library("affxparser");
# This Test3.CDF file should also core dump
hdr <- readCdfHeader(pathname);
str(hdr);

Does it core dump?

/Henrik

nm29

unread,

Jan 4, 2010, 8:04:07 PM1/4/10

to aroma.affymetrix

Core dump indeed:

> library("AffymetrixDataTestFiles")
> pathname <- "annotationData/chipTypes/Test3/1.XDA/Test3.CDF"
> pathname <- system.file(pathname, package="AffymetrixDataTestFiles")
> stopifnot(file.exists(pathname))
> print(pathname)

[1] "/Users/Neel/Library/R/2.10/library/AffymetrixDataTestFiles/
annotationData/chipTypes/Test3/1.XDA/Test3.CDF"
> library("affxparser")
> hdr <- readCdfHeader(pathname)

*** caught segfault ***
address 0xfffffffc, cause 'memory not mapped'

Traceback:
1: .Call("R_affx_get_cdf_file_header", filename, PACKAGE =
"affxparser")
2: readCdfHeader(pathname)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:

On Jan 4, 7:14 pm, Henrik Bengtsson <henrik.bengts...@gmail.com>
wrote:

> ...
>
> read more »

Henrik Bengtsson

unread,

Jan 4, 2010, 8:49:49 PM1/4/10

to aroma-affymetrix

This discussion was moved to the Bioconductor mailing list since it
has to do with affxparser and not aroma.affymetrix. See thread
'[BioC] affxparser::readCdfHeader() core dumps' (Tue Jan 5 02:27:23
CET 2010):