Notes on corpus annotations
3 views
Skip to first unread message
Sandra Garcia Blasco
Jun 21, 2011, 8:27:54 AM6/21/11
to ddiextra...@googlegroups.com, Santiago Mola
Dear all,
We have been analyzing the predictions of our system, specifically
those that were labeled as
true with high confidence. We spotted some cases where we think there
is an interaction, but
it is labeled in the result file as non interaction.
We would like to share some examples of them with you all, and see
what are your opinions.
Maybe this could be used in the future to improve the corpus in the
case they are in
fact interactions.
-------------------Sample 1--------------------------
DrugDDI.d250.s0.e0 DrugDDI.d250.s0.e1 0 File:
Isosorbide Dinitrate_ddi.xml
The vasodilating effects of isosorbide dinitrate may be additive with
those of other vasodilators.
It looks like there is an interaction here between the two drugs, with
an additive effect.
---------------------Sample 2------------------------
DrugDDI.d392.s10 File: Desipramine_ddi.xml
While all the selective serotonin reuptake inhibitors (SSRIs), e.g.,
fluoxetine, seriraline, and paroxetine, inhibit P450 2D6, they may
vary in the extent of inhibition.
<entity id="DrugDDI.d392.s10.e3" origId="s10.p128"
charOffset="117-126" type="drug" text="P450 2D6"/>
It looks like every drug here interacts with the enzyme P450 2D6.
However, since we are
talking about drug drug interaction, the problem would not be that
there are not interaction
labeled, but that the P450 2D6 should not be labeled as an entity of type drug.
---------------------Sample 3 -----------------------
DrugDDI.d354.s15 File: Chlorothiazide_ddi.xml
Lithium: Generally should not be given with diuretics.
It looks like Lithium and diuretics have dangerous interactions which
leads to this
recomendation of not administrating them together.
-----------------------------------------------------
We are interested in knowing if this are indeed interactions, or if we
have some
misunderstanding about the annotation criteria.
Thank you very much.
Best regards,
Sandra Garcia &
Santiago Mola
Stefania Rubrichi
Jun 21, 2011, 9:35:18 AM6/21/11
to ddiextra...@googlegroups.com
Dear Sandra and Santiago,
We totally agree with your remarks, however the occurrence of discrepancies through the test set doesn't surprise us. We found out the same kind of discrepancies in the training set, that we have reported to the organizers (you can find a copy of the e-mails we sent, in the following lines) using some examples to clarify. Unfortunately they fixed just the few examples we suggested...
Kind Regards,
Stefania Rubrichi &
Matteo Gabetta
------------------------------------------------------------------------------------------
Dear Stefania,
On 16 May 2011 19:03, Rubrichi Stefania <stefania...@unipv.it> wrote:
Dear Isabel,
Thank you for considering my remarks. I got the new version of the
corpus and I saw that you fixed the mistakes I reported. However, the
changes that I suggested in my email were just a few examples of the
discrepancy I found while reading the corpus. It might be beneficial
for the consistency of the corpus if you did a careful review of the
corpus in light of my findings.
Best regards,
Stefania
2011/5/15 Isabel Segura <iseg...@gmail.com>:
> Dear Stefania,
> We have fixed these errors. I send you the new version of the dataset.
> Thank you!!!
>
> On 14 May 2011 19:33, Isabel Segura <iseg...@gmail.com> wrote:
>>
>> Dear Stefania,
>> Drug names were automatically annotated by the MMTx tool. Unfortunately,
>> MMTx makes some mistakes.
>> DDIs were manually annotated by a pharmacist. Regarding your first
>> comment, our pharmacist decided that there was one only interaction in this
>> sentence. Regarding the rest of comments, I think that they are (possibly)
>> mistakes. We are going to review them and upload a new version as soon as we
>> can do.
>> Thank you for your comments.
>> Regards, Isabel
>>
>> On 14 May 2011 19:13, Rubrichi Stefania <stefania...@unipv.it>
>> wrote:
>>>
>>> Dear Dr. Segura Bedmar,
>>>
>>> I'm a Ph.D student from University of Pavia and I'm considering
>>> applying for the DDIExtraction2011 challenge. Going through the corpus
>>> DrugDDI_Unified I found out some discrepancies that I'm going to explain
>>> in the following lines. I'll use some examples to clarify.
>>>
>>> 1. Very often in sentences a drug can occur more than one time. From a
>>> semantic point of view, I think that if it is involved in an
>>> interaction with a special drug, the attribute "interaction" should
>>> always
>>> be "true" when the two drugs appear in a "pair".
>>> Consider as an example the following sentence s5 from Adenosine_ddi.xml.
>>> The drug entity Adenosine occurs 2 times and is involved in the
>>> interaction e0-e1.
>>> Wouldn't it be better to consider "true" the interaction attribute for
>>> the pair "DrugDDI.d419.s5.p1"?
>>>
>>> Document: Adenosine_ddi.xml sentence: s5
>>>
>>> ------------------------------
We totally agree with your remarks, however the occurrence of discrepancies through the test set doesn't surprise us. We found out the same kind of discrepancies in the training set, that we have reported to the organizers (you can find a copy of the e-mails we sent, in the following lines) using some examples to clarify. Unfortunately they fixed just the few examples we suggested...
Kind Regards,
Stefania Rubrichi &
Matteo Gabetta
------------------------------------------------------------------------------------------
Dear Stefania,
Currently, we are working very hard to improve the quality of the corpus.
I apologize again for any mistakes.
Regards,
Isabel
- Nascondi testo citato -
On 16 May 2011 19:03, Rubrichi Stefania <stefania...@unipv.it> wrote:
Dear Isabel,
Thank you for considering my remarks. I got the new version of the
corpus and I saw that you fixed the mistakes I reported. However, the
changes that I suggested in my email were just a few examples of the
discrepancy I found while reading the corpus. It might be beneficial
for the consistency of the corpus if you did a careful review of the
corpus in light of my findings.
Best regards,
Stefania
2011/5/15 Isabel Segura <iseg...@gmail.com>:
> Dear Stefania,
> We have fixed these errors. I send you the new version of the dataset.
> Thank you!!!
>
> On 14 May 2011 19:33, Isabel Segura <iseg...@gmail.com> wrote:
>>
>> Dear Stefania,
>> Drug names were automatically annotated by the MMTx tool. Unfortunately,
>> MMTx makes some mistakes.
>> DDIs were manually annotated by a pharmacist. Regarding your first
>> comment, our pharmacist decided that there was one only interaction in this
>> sentence. Regarding the rest of comments, I think that they are (possibly)
>> mistakes. We are going to review them and upload a new version as soon as we
>> can do.
>> Thank you for your comments.
>> Regards, Isabel
>>
>> On 14 May 2011 19:13, Rubrichi Stefania <stefania...@unipv.it>
>> wrote:
>>>
>>> Dear Dr. Segura Bedmar,
>>>
>>> I'm a Ph.D student from University of Pavia and I'm considering
>>> applying for the DDIExtraction2011 challenge. Going through the corpus
>>> DrugDDI_Unified I found out some discrepancies that I'm going to explain
>>> in the following lines. I'll use some examples to clarify.
>>>
>>> 1. Very often in sentences a drug can occur more than one time. From a
>>> semantic point of view, I think that if it is involved in an
>>> interaction with a special drug, the attribute "interaction" should
>>> always
>>> be "true" when the two drugs appear in a "pair".
>>> Consider as an example the following sentence s5 from Adenosine_ddi.xml.
>>> The drug entity Adenosine occurs 2 times and is involved in the
>>> interaction e0-e1.
>>> Wouldn't it be better to consider "true" the interaction attribute for
>>> the pair "DrugDDI.d419.s5.p1"?
>>>
>>> Document: Adenosine_ddi.xml sentence: s5
>>>
>>> ------------------------------
--------------------------------------------------
>>> <sentence id="DrugDDI.d419.s5" origId="s5" text="In the presence of
>>> these methylxanthines, larger doses of adenosine may be required or
>>> adenosine may not be effective.">
>>> <entity id="DrugDDI.d419.s5.e0" origId="s5.p62" charOffset="25-40"
>>> type="drug" text="METHYLXANTHINES"/>
>>> <entity id="DrugDDI.d419.s5.e1" origId="s5.p64" charOffset="58-67"
>>> type="drug" text="Adenosine"/>
>>> <entity id="DrugDDI.d419.s5.e2" origId="s5.p69" charOffset="58-67"
>>> type="drug" text="Adenosine"/>
>>> <pair id="DrugDDI.d419.s5.p0" e1="DrugDDI.d419.s5.e0"
>>> e2="DrugDDI.d419.s5.e1" interaction="True"/>
>>> <pair id="DrugDDI.d419.s5.p1" e1="DrugDDI.d419.s5.e0"
>>> e2="DrugDDI.d419.s5.e2" interaction="False"/>
>>> <pair id="DrugDDI.d419.s5.p2" e1="DrugDDI.d419.s5.e1"
>>> e2="DrugDDI.d419.s5.e2" interaction="False"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>>
>>>
>>> *******************************************************************************************************************************************
>>> 2. I found some portions of text labelled as "entity" that, in my
>>> opinion, should not be and vice versa.
>>>
>>> Document: Valdecoxib_ddi.xml sentence: s6
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d264.s6" origId="s6" text="In a parallel group
>>> drug interaction study comparing the intravenous prodrug form of
>>> valdecoxib at 40 mg BID (n=10) vs placebo (n=9), valdecoxib had no
>>> effect on in vitro aspirin-mediated inhibition of arachidonate- or
>>> collagen-stimulated platelet aggregation.">
>>> <entity id="DrugDDI.d264.s6.e0" origId="s6.p105"
>>> charOffset="85-95"
>>> type="drug" text="valdecoxib"/>
>>> <entity id="DrugDDI.d264.s6.e1" origId="s6.p106"
>>> charOffset="105-108" type="drug" text="BID"/>
>>> <entity id="DrugDDI.d264.s6.e2" origId="s6.p113"
>>> charOffset="128-132" type="drug" text="n=9"/>
>>> ...
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why have you labelled as entity the text "n=9" and "BID"?
>>>
>>> Document: Almotriptan_ddi.xml sentence: s1
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d91.s1" origId="s1" text="Because there is a
>>> theoretical basis that these effects may be additive, use of
>>> ergotamine-containing or ergot-type medications (like
>>> dihydroergotamine or methysergide) and AXERT? within 24 hours of each
>>> other should be avoided.">
>>> <entity id="DrugDDI.d91.s1.e0" origId="s1.p15"
>>> charOffset="80-90"
>>> type="drug" text="Ergotamine"/>
>>> <entity id="DrugDDI.d91.s1.e1" origId="s1.p18"
>>> charOffset="116-127"
>>> type="drug" text="MEDICATIONS"/>
>>> <entity id="DrugDDI.d91.s1.e2" origId="s1.p20"
>>> charOffset="134-151"
>>> type="drug" text="Dihydroergotamine"/>
>>> <entity id="DrugDDI.d91.s1.e3" origId="s1.p22"
>>> charOffset="155-167"
>>> type="drug" text="Methysergide"/>
>>> ...
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why has "AXERT" not been labelled as an entity?
>>>
>>> Document: Valdecoxib_ddi.xml sentence: s3
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d264.s3" origId="s3" text="In vitro studies
>>> indicate that valdecoxib is a moderate inhibitor of CYP 2C19 (IC50 = 6
>>> ?g/mL or 19 ?M) and 2C9 (IC50 = 13 ?g/mL or 41 ?M), and a weak
>>> inhibitor of CYP 2D6 (IC50 = 31 ?g/mL or 100 ?M) and 3A4 (IC50 = 44
>>> ?g/mL or 141 ?M )..">
>>> <entity id="DrugDDI.d264.s3.e0" origId="s3.p44"
>>> charOffset="31-41"
>>> type="drug" text="valdecoxib"/>
>>> <entity id="DrugDDI.d264.s3.e1" origId="s3.p47"
>>> charOffset="69-77"
>>> type="drug" text="CYP 2C19"/>
>>> <entity id="DrugDDI.d264.s3.e2" origId="s3.p65"
>>> charOffset="164-171"
>>> type="drug" text="CYP 2D6"/>
>>> <pair id="DrugDDI.d264.s3.p0" e1="DrugDDI.d264.s3.e0"
>>> e2="DrugDDI.d264.s3.e1" interaction="False"/>
>>> <pair id="DrugDDI.d264.s3.p1" e1="DrugDDI.d264.s3.e0"
>>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
>>> <pair id="DrugDDI.d264.s3.p2" e1="DrugDDI.d264.s3.e1"
>>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why have you labelled as an entity the text "CYP 2C19" but the text
>>> "2C9"?
>>>
>>>
>>> *******************************************************************************************************************************************
>>> 3. I found some interactions that in my opinion should not to be and vice
>>> versa.
>>>
>>> Document:Valdecoxib_ddi.xml sentence: s29
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d264.s29" origId="s29" text="Plasma exposure
>>> (AUC) to valdecoxib was increased 62% when coadministered with
>>> fluconazole and 38% when coadministered with ketoconazole.">
>>> <entity id="DrugDDI.d264.s29.e0" origId="s29.p415"
>>> charOffset="25-35" type="drug" text="valdecoxib"/>
>>> <entity id="DrugDDI.d264.s29.e1" origId="s29.p421"
>>> charOffset="79-90" type="drug" text="Fluconazole"/>
>>> <entity id="DrugDDI.d264.s29.e2" origId="s29.p426"
>>> charOffset="124-136" type="drug" text="Ketoconazole"/>
>>> <pair id="DrugDDI.d264.s29.p0" e1="DrugDDI.d264.s29.e0"
>>> e2="DrugDDI.d264.s29.e1" interaction="False"/>
>>> <pair id="DrugDDI.d264.s29.p1" e1="DrugDDI.d264.s29.e0"
>>> e2="DrugDDI.d264.s29.e2" interaction="False"/>
>>> <pair id="DrugDDI.d264.s29.p2" e1="DrugDDI.d264.s29.e1"
>>> e2="DrugDDI.d264.s29.e2" interaction="True"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> I think that the attribute "interaction" should be "true" for the
>>> pairs "DrugDDI.d264.s29.p0" and "DrugDDI.d264.s29.p1" and "false" for
>>> the pair "DrugDDI.d264.s29.p2".
>>>
>>> Document: Alfetanil_ddi.xml sentence: s0
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d564.s0" origId="s0" text="Both the magnitude
>>> and duration of central nervous system and cardiovascular effects may
>>> be enhanced when ALFENTA is administered in combination with other CNS
>>> depressants such as barbiturates, tranquilizers, opioids, or
>>> inhalation general anesthetics.">
>>> <entity id="DrugDDI.d564.s0.e0" origId="s0.p11"
>>> charOffset="106-113"
>>> type="drug" text="Alfenta"/>
>>> <entity id="DrugDDI.d564.s0.e1" origId="s0.p15"
>>> charOffset="156-171"
>>> type="drug" text="CNS Depressants"/>
>>> <entity id="DrugDDI.d564.s0.e2" origId="s0.p16"
>>> charOffset="180-192"
>>> type="drug" text="Barbiturates"/>
>>> <entity id="DrugDDI.d564.s0.e3" origId="s0.p17"
>>> charOffset="194-207"
>>> type="drug" text="TRANQUILIZERS"/>
>>> <entity id="DrugDDI.d564.s0.e4" origId="s0.p18"
>>> charOffset="209-216"
>>> type="drug" text="Opioids"/>
>>> <entity id="DrugDDI.d564.s0.e5" origId="s0.p20"
>>> charOffset="232-251"
>>> type="drug" text="general anesthetics"/>
>>> <pair id="DrugDDI.d564.s0.p0" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e1" interaction="True"/>
>>> <pair id="DrugDDI.d564.s0.p1" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e2" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p2" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e3" interaction="True"/>
>>> <pair id="DrugDDI.d564.s0.p3" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p4" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e5" interaction="True"/>
>>> <pair id="DrugDDI.d564.s0.p5" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e2" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p6" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e3" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p7" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p8" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p9" e1="DrugDDI.d564.s0.e2"
>>> e2="DrugDDI.d564.s0.e3" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p10" e1="DrugDDI.d564.s0.e2"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p11" e1="DrugDDI.d564.s0.e2"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p12" e1="DrugDDI.d564.s0.e3"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p13" e1="DrugDDI.d564.s0.e3"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p14" e1="DrugDDI.d564.s0.e4"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why is the "interaction" attribute of pairs "DrugDDI.d564.s0.p1",
>>> "DrugDDI.d564.s0.p3" "false"?
>>>
>>>
>>>
>>>
>>> I look forward to hearing your feedback on this issue.
>>>
>>> Best regards,
>>>
>>> Stefania
-------------------------------------------------------------------------------------->>> <sentence id="DrugDDI.d419.s5" origId="s5" text="In the presence of
>>> these methylxanthines, larger doses of adenosine may be required or
>>> adenosine may not be effective.">
>>> <entity id="DrugDDI.d419.s5.e0" origId="s5.p62" charOffset="25-40"
>>> type="drug" text="METHYLXANTHINES"/>
>>> <entity id="DrugDDI.d419.s5.e1" origId="s5.p64" charOffset="58-67"
>>> type="drug" text="Adenosine"/>
>>> <entity id="DrugDDI.d419.s5.e2" origId="s5.p69" charOffset="58-67"
>>> type="drug" text="Adenosine"/>
>>> <pair id="DrugDDI.d419.s5.p0" e1="DrugDDI.d419.s5.e0"
>>> e2="DrugDDI.d419.s5.e1" interaction="True"/>
>>> <pair id="DrugDDI.d419.s5.p1" e1="DrugDDI.d419.s5.e0"
>>> e2="DrugDDI.d419.s5.e2" interaction="False"/>
>>> <pair id="DrugDDI.d419.s5.p2" e1="DrugDDI.d419.s5.e1"
>>> e2="DrugDDI.d419.s5.e2" interaction="False"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>>
>>>
>>> *******************************************************************************************************************************************
>>> 2. I found some portions of text labelled as "entity" that, in my
>>> opinion, should not be and vice versa.
>>>
>>> Document: Valdecoxib_ddi.xml sentence: s6
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d264.s6" origId="s6" text="In a parallel group
>>> drug interaction study comparing the intravenous prodrug form of
>>> valdecoxib at 40 mg BID (n=10) vs placebo (n=9), valdecoxib had no
>>> effect on in vitro aspirin-mediated inhibition of arachidonate- or
>>> collagen-stimulated platelet aggregation.">
>>> <entity id="DrugDDI.d264.s6.e0" origId="s6.p105"
>>> charOffset="85-95"
>>> type="drug" text="valdecoxib"/>
>>> <entity id="DrugDDI.d264.s6.e1" origId="s6.p106"
>>> charOffset="105-108" type="drug" text="BID"/>
>>> <entity id="DrugDDI.d264.s6.e2" origId="s6.p113"
>>> charOffset="128-132" type="drug" text="n=9"/>
>>> ...
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why have you labelled as entity the text "n=9" and "BID"?
>>>
>>> Document: Almotriptan_ddi.xml sentence: s1
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d91.s1" origId="s1" text="Because there is a
>>> theoretical basis that these effects may be additive, use of
>>> ergotamine-containing or ergot-type medications (like
>>> dihydroergotamine or methysergide) and AXERT? within 24 hours of each
>>> other should be avoided.">
>>> <entity id="DrugDDI.d91.s1.e0" origId="s1.p15"
>>> charOffset="80-90"
>>> type="drug" text="Ergotamine"/>
>>> <entity id="DrugDDI.d91.s1.e1" origId="s1.p18"
>>> charOffset="116-127"
>>> type="drug" text="MEDICATIONS"/>
>>> <entity id="DrugDDI.d91.s1.e2" origId="s1.p20"
>>> charOffset="134-151"
>>> type="drug" text="Dihydroergotamine"/>
>>> <entity id="DrugDDI.d91.s1.e3" origId="s1.p22"
>>> charOffset="155-167"
>>> type="drug" text="Methysergide"/>
>>> ...
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why has "AXERT" not been labelled as an entity?
>>>
>>> Document: Valdecoxib_ddi.xml sentence: s3
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d264.s3" origId="s3" text="In vitro studies
>>> indicate that valdecoxib is a moderate inhibitor of CYP 2C19 (IC50 = 6
>>> ?g/mL or 19 ?M) and 2C9 (IC50 = 13 ?g/mL or 41 ?M), and a weak
>>> inhibitor of CYP 2D6 (IC50 = 31 ?g/mL or 100 ?M) and 3A4 (IC50 = 44
>>> ?g/mL or 141 ?M )..">
>>> <entity id="DrugDDI.d264.s3.e0" origId="s3.p44"
>>> charOffset="31-41"
>>> type="drug" text="valdecoxib"/>
>>> <entity id="DrugDDI.d264.s3.e1" origId="s3.p47"
>>> charOffset="69-77"
>>> type="drug" text="CYP 2C19"/>
>>> <entity id="DrugDDI.d264.s3.e2" origId="s3.p65"
>>> charOffset="164-171"
>>> type="drug" text="CYP 2D6"/>
>>> <pair id="DrugDDI.d264.s3.p0" e1="DrugDDI.d264.s3.e0"
>>> e2="DrugDDI.d264.s3.e1" interaction="False"/>
>>> <pair id="DrugDDI.d264.s3.p1" e1="DrugDDI.d264.s3.e0"
>>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
>>> <pair id="DrugDDI.d264.s3.p2" e1="DrugDDI.d264.s3.e1"
>>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why have you labelled as an entity the text "CYP 2C19" but the text
>>> "2C9"?
>>>
>>>
>>> *******************************************************************************************************************************************
>>> 3. I found some interactions that in my opinion should not to be and vice
>>> versa.
>>>
>>> Document:Valdecoxib_ddi.xml sentence: s29
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d264.s29" origId="s29" text="Plasma exposure
>>> (AUC) to valdecoxib was increased 62% when coadministered with
>>> fluconazole and 38% when coadministered with ketoconazole.">
>>> <entity id="DrugDDI.d264.s29.e0" origId="s29.p415"
>>> charOffset="25-35" type="drug" text="valdecoxib"/>
>>> <entity id="DrugDDI.d264.s29.e1" origId="s29.p421"
>>> charOffset="79-90" type="drug" text="Fluconazole"/>
>>> <entity id="DrugDDI.d264.s29.e2" origId="s29.p426"
>>> charOffset="124-136" type="drug" text="Ketoconazole"/>
>>> <pair id="DrugDDI.d264.s29.p0" e1="DrugDDI.d264.s29.e0"
>>> e2="DrugDDI.d264.s29.e1" interaction="False"/>
>>> <pair id="DrugDDI.d264.s29.p1" e1="DrugDDI.d264.s29.e0"
>>> e2="DrugDDI.d264.s29.e2" interaction="False"/>
>>> <pair id="DrugDDI.d264.s29.p2" e1="DrugDDI.d264.s29.e1"
>>> e2="DrugDDI.d264.s29.e2" interaction="True"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> I think that the attribute "interaction" should be "true" for the
>>> pairs "DrugDDI.d264.s29.p0" and "DrugDDI.d264.s29.p1" and "false" for
>>> the pair "DrugDDI.d264.s29.p2".
>>>
>>> Document: Alfetanil_ddi.xml sentence: s0
>>>
>>> --------------------------------------------------------------------------------
>>> <sentence id="DrugDDI.d564.s0" origId="s0" text="Both the magnitude
>>> and duration of central nervous system and cardiovascular effects may
>>> be enhanced when ALFENTA is administered in combination with other CNS
>>> depressants such as barbiturates, tranquilizers, opioids, or
>>> inhalation general anesthetics.">
>>> <entity id="DrugDDI.d564.s0.e0" origId="s0.p11"
>>> charOffset="106-113"
>>> type="drug" text="Alfenta"/>
>>> <entity id="DrugDDI.d564.s0.e1" origId="s0.p15"
>>> charOffset="156-171"
>>> type="drug" text="CNS Depressants"/>
>>> <entity id="DrugDDI.d564.s0.e2" origId="s0.p16"
>>> charOffset="180-192"
>>> type="drug" text="Barbiturates"/>
>>> <entity id="DrugDDI.d564.s0.e3" origId="s0.p17"
>>> charOffset="194-207"
>>> type="drug" text="TRANQUILIZERS"/>
>>> <entity id="DrugDDI.d564.s0.e4" origId="s0.p18"
>>> charOffset="209-216"
>>> type="drug" text="Opioids"/>
>>> <entity id="DrugDDI.d564.s0.e5" origId="s0.p20"
>>> charOffset="232-251"
>>> type="drug" text="general anesthetics"/>
>>> <pair id="DrugDDI.d564.s0.p0" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e1" interaction="True"/>
>>> <pair id="DrugDDI.d564.s0.p1" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e2" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p2" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e3" interaction="True"/>
>>> <pair id="DrugDDI.d564.s0.p3" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p4" e1="DrugDDI.d564.s0.e0"
>>> e2="DrugDDI.d564.s0.e5" interaction="True"/>
>>> <pair id="DrugDDI.d564.s0.p5" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e2" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p6" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e3" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p7" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p8" e1="DrugDDI.d564.s0.e1"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p9" e1="DrugDDI.d564.s0.e2"
>>> e2="DrugDDI.d564.s0.e3" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p10" e1="DrugDDI.d564.s0.e2"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p11" e1="DrugDDI.d564.s0.e2"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p12" e1="DrugDDI.d564.s0.e3"
>>> e2="DrugDDI.d564.s0.e4" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p13" e1="DrugDDI.d564.s0.e3"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> <pair id="DrugDDI.d564.s0.p14" e1="DrugDDI.d564.s0.e4"
>>> e2="DrugDDI.d564.s0.e5" interaction="False"/>
>>> </sentence>
>>>
>>> --------------------------------------------------------------------------------
>>> Why is the "interaction" attribute of pairs "DrugDDI.d564.s0.p1",
>>> "DrugDDI.d564.s0.p3" "false"?
>>>
>>>
>>>
>>>
>>> I look forward to hearing your feedback on this issue.
>>>
>>> Best regards,
>>>
>>> Stefania
2011/6/21 Sandra Garcia Blasco <sang...@gmail.com>
advaned
Jun 21, 2011, 11:18:28 AM6/21/11
to DDIExtraction2011
Dear Stefania,
Of course, there are some errors in our corpus. The annotation process
is a human labour, and to err is human.
This is the first time we have organized this task, and we have tried
to do it as best as possible.
We are very grateful to all participants who have contributed to
improve the task and the corpus.
Thanks, Isabel
On 21 jun, 15:35, Stefania Rubrichi <stef...@gmail.com> wrote:
> Dear Sandra and Santiago,
>
> We totally agree with your remarks, however the occurrence of discrepancies
> through the test set doesn't surprise us. We found out the same kind of
> discrepancies in the training set, that we have reported to the organizers
> (you can find a copy of the e-mails we sent, in the following lines) using
> some examples to clarify. Unfortunately they fixed just the few examples we
> suggested...
>
> Kind Regards,
> Stefania Rubrichi &
> Matteo Gabetta
>
> --------------------------------------------------------------------------- ---------------
>
> Dear Stefania,
>
> Currently, we are working very hard to improve the quality of the corpus.
> *I apologize again for any mistakes. *
> *************************************************************************** ****************************************************************>>> 2. I found some portions of text labelled as "entity" that, in my
> >>> "2C9"?
>
> *************************************************************************** ****************************************************************>>> 3. I found some interactions that in my opinion should not to be and
>
> leer más »
Of course, there are some errors in our corpus. The annotation process
is a human labour, and to err is human.
This is the first time we have organized this task, and we have tried
to do it as best as possible.
We are very grateful to all participants who have contributed to
improve the task and the corpus.
Thanks, Isabel
On 21 jun, 15:35, Stefania Rubrichi <stef...@gmail.com> wrote:
> Dear Sandra and Santiago,
>
> We totally agree with your remarks, however the occurrence of discrepancies
> through the test set doesn't surprise us. We found out the same kind of
> discrepancies in the training set, that we have reported to the organizers
> (you can find a copy of the e-mails we sent, in the following lines) using
> some examples to clarify. Unfortunately they fixed just the few examples we
> suggested...
>
> Kind Regards,
> Stefania Rubrichi &
> Matteo Gabetta
>
> --------------------------------------------------------------------------- ---------------
>
> Dear Stefania,
>
> Currently, we are working very hard to improve the quality of the corpus.
>
> Regards,
>
> Isabel
>
> - Nascondi testo citato -
>
> Regards,
>
> Isabel
>
> - Nascondi testo citato -
>
> On 16 May 2011 19:03, Rubrichi Stefania <stefania.rubri...@unipv.it> wrote:
> Dear Isabel,
>
> Thank you for considering my remarks. I got the new version of the
> corpus and I saw that you fixed the mistakes I reported. However, the
> changes that I suggested in my email were just a few examples of the
> discrepancy I found while reading the corpus. It might be beneficial
> for the consistency of the corpus if you did a careful review of the
> corpus in light of my findings.
>
> Best regards,
> Stefania
>
> 2011/5/15 Isabel Segura <isegur...@gmail.com>:
> Dear Isabel,
>
> Thank you for considering my remarks. I got the new version of the
> corpus and I saw that you fixed the mistakes I reported. However, the
> changes that I suggested in my email were just a few examples of the
> discrepancy I found while reading the corpus. It might be beneficial
> for the consistency of the corpus if you did a careful review of the
> corpus in light of my findings.
>
> Best regards,
> Stefania
>
>
>
>
>
>
>
>
> > Dear Stefania,
> > We have fixed these errors. I send you the new version of the dataset.
> > Thank you!!!
>
>
>
>
>
>
>
> > Dear Stefania,
> > We have fixed these errors. I send you the new version of the dataset.
> > Thank you!!!
>
> > On 14 May 2011 19:33, Isabel Segura <isegur...@gmail.com> wrote:
>
> >> Dear Stefania,
> >> Drug names were automatically annotated by the MMTx tool. Unfortunately,
> >> MMTx makes some mistakes.
> >> DDIs were manually annotated by a pharmacist. Regarding your first
> >> comment, our pharmacist decided that there was one only interaction in
> this
> >> sentence. Regarding the rest of comments, I think that they are
> (possibly)
> >> mistakes. We are going to review them and upload a new version as soon as
> we
> >> can do.
> >> Thank you for your comments.
> >> Regards, Isabel
>
> >> On 14 May 2011 19:13, Rubrichi Stefania <stefania.rubri...@unipv.it>
>
> >> Dear Stefania,
> >> Drug names were automatically annotated by the MMTx tool. Unfortunately,
> >> MMTx makes some mistakes.
> >> DDIs were manually annotated by a pharmacist. Regarding your first
> >> comment, our pharmacist decided that there was one only interaction in
> this
> >> sentence. Regarding the rest of comments, I think that they are
> (possibly)
> >> mistakes. We are going to review them and upload a new version as soon as
> we
> >> can do.
> >> Thank you for your comments.
> >> Regards, Isabel
>
> >>> opinion, should not be and vice versa.
>
> >>> Document: Valdecoxib_ddi.xml sentence: s6
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d264.s6" origId="s6" text="In a parallel group
> >>> drug interaction study comparing the intravenous prodrug form of
> >>> valdecoxib at 40 mg BID (n=10) vs placebo (n=9), valdecoxib had no
> >>> effect on in vitro aspirin-mediated inhibition of arachidonate- or
> >>> collagen-stimulated platelet aggregation.">
> >>> <entity id="DrugDDI.d264.s6.e0" origId="s6.p105"
> >>> charOffset="85-95"
> >>> type="drug" text="valdecoxib"/>
> >>> <entity id="DrugDDI.d264.s6.e1" origId="s6.p106"
> >>> charOffset="105-108" type="drug" text="BID"/>
> >>> <entity id="DrugDDI.d264.s6.e2" origId="s6.p113"
> >>> charOffset="128-132" type="drug" text="n=9"/>
> >>> ...
> >>> </sentence>
>
> --------------------------------------------------------------------------- ----->>> Why have you labelled as entity the text "n=9" and "BID"?
>
> >>> Document: Valdecoxib_ddi.xml sentence: s6
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d264.s6" origId="s6" text="In a parallel group
> >>> drug interaction study comparing the intravenous prodrug form of
> >>> valdecoxib at 40 mg BID (n=10) vs placebo (n=9), valdecoxib had no
> >>> effect on in vitro aspirin-mediated inhibition of arachidonate- or
> >>> collagen-stimulated platelet aggregation.">
> >>> <entity id="DrugDDI.d264.s6.e0" origId="s6.p105"
> >>> charOffset="85-95"
> >>> type="drug" text="valdecoxib"/>
> >>> <entity id="DrugDDI.d264.s6.e1" origId="s6.p106"
> >>> charOffset="105-108" type="drug" text="BID"/>
> >>> <entity id="DrugDDI.d264.s6.e2" origId="s6.p113"
> >>> charOffset="128-132" type="drug" text="n=9"/>
> >>> ...
> >>> </sentence>
>
>
> >>> Document: Almotriptan_ddi.xml sentence: s1
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d91.s1" origId="s1" text="Because there is a
> >>> theoretical basis that these effects may be additive, use of
> >>> ergotamine-containing or ergot-type medications (like
> >>> dihydroergotamine or methysergide) and AXERT? within 24 hours of each
> >>> other should be avoided.">
> >>> <entity id="DrugDDI.d91.s1.e0" origId="s1.p15"
> >>> charOffset="80-90"
> >>> type="drug" text="Ergotamine"/>
> >>> <entity id="DrugDDI.d91.s1.e1" origId="s1.p18"
> >>> charOffset="116-127"
> >>> type="drug" text="MEDICATIONS"/>
> >>> <entity id="DrugDDI.d91.s1.e2" origId="s1.p20"
> >>> charOffset="134-151"
> >>> type="drug" text="Dihydroergotamine"/>
> >>> <entity id="DrugDDI.d91.s1.e3" origId="s1.p22"
> >>> charOffset="155-167"
> >>> type="drug" text="Methysergide"/>
> >>> ...
> >>> </sentence>
>
> --------------------------------------------------------------------------- ----->>> Why has "AXERT" not been labelled as an entity?
> >>> Document: Almotriptan_ddi.xml sentence: s1
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d91.s1" origId="s1" text="Because there is a
> >>> theoretical basis that these effects may be additive, use of
> >>> ergotamine-containing or ergot-type medications (like
> >>> dihydroergotamine or methysergide) and AXERT? within 24 hours of each
> >>> other should be avoided.">
> >>> <entity id="DrugDDI.d91.s1.e0" origId="s1.p15"
> >>> charOffset="80-90"
> >>> type="drug" text="Ergotamine"/>
> >>> <entity id="DrugDDI.d91.s1.e1" origId="s1.p18"
> >>> charOffset="116-127"
> >>> type="drug" text="MEDICATIONS"/>
> >>> <entity id="DrugDDI.d91.s1.e2" origId="s1.p20"
> >>> charOffset="134-151"
> >>> type="drug" text="Dihydroergotamine"/>
> >>> <entity id="DrugDDI.d91.s1.e3" origId="s1.p22"
> >>> charOffset="155-167"
> >>> type="drug" text="Methysergide"/>
> >>> ...
> >>> </sentence>
>
>
> >>> Document: Valdecoxib_ddi.xml sentence: s3
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d264.s3" origId="s3" text="In vitro studies
> >>> indicate that valdecoxib is a moderate inhibitor of CYP 2C19 (IC50 = 6
> >>> ?g/mL or 19 ?M) and 2C9 (IC50 = 13 ?g/mL or 41 ?M), and a weak
> >>> inhibitor of CYP 2D6 (IC50 = 31 ?g/mL or 100 ?M) and 3A4 (IC50 = 44
> >>> ?g/mL or 141 ?M )..">
> >>> <entity id="DrugDDI.d264.s3.e0" origId="s3.p44"
> >>> charOffset="31-41"
> >>> type="drug" text="valdecoxib"/>
> >>> <entity id="DrugDDI.d264.s3.e1" origId="s3.p47"
> >>> charOffset="69-77"
> >>> type="drug" text="CYP 2C19"/>
> >>> <entity id="DrugDDI.d264.s3.e2" origId="s3.p65"
> >>> charOffset="164-171"
> >>> type="drug" text="CYP 2D6"/>
> >>> <pair id="DrugDDI.d264.s3.p0" e1="DrugDDI.d264.s3.e0"
> >>> e2="DrugDDI.d264.s3.e1" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s3.p1" e1="DrugDDI.d264.s3.e0"
> >>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s3.p2" e1="DrugDDI.d264.s3.e1"
> >>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
> >>> </sentence>
>
> --------------------------------------------------------------------------- ----->>> Why have you labelled as an entity the text "CYP 2C19" but the text
> >>> Document: Valdecoxib_ddi.xml sentence: s3
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d264.s3" origId="s3" text="In vitro studies
> >>> indicate that valdecoxib is a moderate inhibitor of CYP 2C19 (IC50 = 6
> >>> ?g/mL or 19 ?M) and 2C9 (IC50 = 13 ?g/mL or 41 ?M), and a weak
> >>> inhibitor of CYP 2D6 (IC50 = 31 ?g/mL or 100 ?M) and 3A4 (IC50 = 44
> >>> ?g/mL or 141 ?M )..">
> >>> <entity id="DrugDDI.d264.s3.e0" origId="s3.p44"
> >>> charOffset="31-41"
> >>> type="drug" text="valdecoxib"/>
> >>> <entity id="DrugDDI.d264.s3.e1" origId="s3.p47"
> >>> charOffset="69-77"
> >>> type="drug" text="CYP 2C19"/>
> >>> <entity id="DrugDDI.d264.s3.e2" origId="s3.p65"
> >>> charOffset="164-171"
> >>> type="drug" text="CYP 2D6"/>
> >>> <pair id="DrugDDI.d264.s3.p0" e1="DrugDDI.d264.s3.e0"
> >>> e2="DrugDDI.d264.s3.e1" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s3.p1" e1="DrugDDI.d264.s3.e0"
> >>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s3.p2" e1="DrugDDI.d264.s3.e1"
> >>> e2="DrugDDI.d264.s3.e2" interaction="False"/>
> >>> </sentence>
>
> >>> "2C9"?
>
> *************************************************************************** ****************************************************************>>> 3. I found some interactions that in my opinion should not to be and
> vice
> >>> versa.
>
> >>> Document:Valdecoxib_ddi.xml sentence: s29
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d264.s29" origId="s29" text="Plasma exposure
> >>> (AUC) to valdecoxib was increased 62% when coadministered with
> >>> fluconazole and 38% when coadministered with ketoconazole.">
> >>> <entity id="DrugDDI.d264.s29.e0" origId="s29.p415"
> >>> charOffset="25-35" type="drug" text="valdecoxib"/>
> >>> <entity id="DrugDDI.d264.s29.e1" origId="s29.p421"
> >>> charOffset="79-90" type="drug" text="Fluconazole"/>
> >>> <entity id="DrugDDI.d264.s29.e2" origId="s29.p426"
> >>> charOffset="124-136" type="drug" text="Ketoconazole"/>
> >>> <pair id="DrugDDI.d264.s29.p0" e1="DrugDDI.d264.s29.e0"
> >>> e2="DrugDDI.d264.s29.e1" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s29.p1" e1="DrugDDI.d264.s29.e0"
> >>> e2="DrugDDI.d264.s29.e2" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s29.p2" e1="DrugDDI.d264.s29.e1"
> >>> e2="DrugDDI.d264.s29.e2" interaction="True"/>
> >>> </sentence>
>
> --------------------------------------------------------------------------- ----->>> I think that the attribute "interaction" should be "true" for the
> >>> versa.
>
> >>> Document:Valdecoxib_ddi.xml sentence: s29
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d264.s29" origId="s29" text="Plasma exposure
> >>> (AUC) to valdecoxib was increased 62% when coadministered with
> >>> fluconazole and 38% when coadministered with ketoconazole.">
> >>> <entity id="DrugDDI.d264.s29.e0" origId="s29.p415"
> >>> charOffset="25-35" type="drug" text="valdecoxib"/>
> >>> <entity id="DrugDDI.d264.s29.e1" origId="s29.p421"
> >>> charOffset="79-90" type="drug" text="Fluconazole"/>
> >>> <entity id="DrugDDI.d264.s29.e2" origId="s29.p426"
> >>> charOffset="124-136" type="drug" text="Ketoconazole"/>
> >>> <pair id="DrugDDI.d264.s29.p0" e1="DrugDDI.d264.s29.e0"
> >>> e2="DrugDDI.d264.s29.e1" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s29.p1" e1="DrugDDI.d264.s29.e0"
> >>> e2="DrugDDI.d264.s29.e2" interaction="False"/>
> >>> <pair id="DrugDDI.d264.s29.p2" e1="DrugDDI.d264.s29.e1"
> >>> e2="DrugDDI.d264.s29.e2" interaction="True"/>
> >>> </sentence>
>
> >>> pairs "DrugDDI.d264.s29.p0" and "DrugDDI.d264.s29.p1" and "false" for
> >>> the pair "DrugDDI.d264.s29.p2".
>
> >>> Document: Alfetanil_ddi.xml sentence: s0
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d564.s0" origId="s0" text="Both the magnitude
> >>> and duration of central nervous system and cardiovascular effects may
> >>> be enhanced when ALFENTA is administered in combination with other CNS
> >>> depressants such as barbiturates, tranquilizers, opioids, or
> >>> inhalation general anesthetics.">
> >>> <entity id="DrugDDI.d564.s0.e0" origId="s0.p11"
> >>> charOffset="106-113"
> >>> type="drug" text="Alfenta"/>
> >>> <entity id="DrugDDI.d564.s0.e1"
>
> ...
> >>> the pair "DrugDDI.d264.s29.p2".
>
> >>> Document: Alfetanil_ddi.xml sentence: s0
>
> --------------------------------------------------------------------------- -----
>
>
>
>
>
>
>
> >>> <sentence id="DrugDDI.d564.s0" origId="s0" text="Both the magnitude
> >>> and duration of central nervous system and cardiovascular effects may
> >>> be enhanced when ALFENTA is administered in combination with other CNS
> >>> depressants such as barbiturates, tranquilizers, opioids, or
> >>> inhalation general anesthetics.">
> >>> <entity id="DrugDDI.d564.s0.e0" origId="s0.p11"
> >>> charOffset="106-113"
> >>> type="drug" text="Alfenta"/>
> >>> <entity id="DrugDDI.d564.s0.e1"
>
>
> leer más »
Reply all
Reply to author
Forward
0 new messages