While doing some processing I find that I have a list, 8 elements, some of which are empty. The final operation would be to concatenate all the elements into a single feature with coma separation (use listconcatenator). There might also be duplicates that can be removed with ListDuplicateRemover.
Does anyone know an easy way to remove the empty elements from a list before I do the concatenation?
Thanks
Richard Wilkinson Systems Analyst Resources - ICT Services Leicestershire County Council 0116 3057709 rwilkin...@leics.gov.uk
___________________________________________________________________________ Leicestershire County Council - rated a 'four-star' council by the Audit Commission ___________________________________________________________________________ This e-mail and any files transmitted with it are confidential. If you are not the intended recipient, any reading, printing, storage, disclosure, copying or any other action taken in respect of this e-mail is prohibited and may be unlawful. If you are not the intended recipient, please notify the sender immediately by using the reply function and then permanently delete what you have received. Incoming and outgoing e-mail messages are routinely monitored for compliance with Leicestershire County Council's policy on the use of electronic communications. The contents of e-mails may have to be disclosed to a request under the Data Protection Act 1998 and the Freedom of Information Act 2000. The views expressed by the author may not necessarily reflect the views or policies of the Leicestershire County Council. Attachments to e-mail messages may contain viruses that may damage your system. Whilst Leicestershire County Council has taken every reasonable precaution to minimise this risk, we cannot accept any liability for any damage which you sustain as a result of these factors. You are advised to carry out your own virus checks before opening any attachment.
Richard Wilkinson wrote: > While doing some processing I find that I have a list, 8 elements, some > of which are empty. The final operation would be to concatenate all the > elements into a single feature with coma separation (use > listconcatenator). There might also be duplicates that can be removed > with ListDuplicateRemover.
> Does anyone know an easy way to remove the empty elements from a list > before I do the concatenation?
To be honest, I'd try taking it out of the concatenated attribute using a StringReplacer. Replace ",," with "," would probabely do the trick... -- Hans van der Maarel Red Geographics
That was one of my thoughts but as there could be several empty elements
posible two/three together I would have to run the stringreplacer several times.
Simple case = A, , C, D, E, , G, H
Other case = , , , D, E, , G,
Or = A, , C, , , , G, H
On second thoughts maybe this could work. Attributetrimmer (both)
removes left & right comma. String replacer twice.
-----Original Message-----
From: fmetalk@googlegroups.com [mailto:fmetalk@googlegroups.com] On
Behalf Of Hans van der Maarel
Sent: 21 November 2008 15:32
To: fmetalk@googlegroups.com
Subject: [fme] Re: List Processing - How to Remove Empty Elements
Richard Wilkinson wrote:
> While doing some processing I find that I have a list, 8 elements, > some
> of which are empty. The final operation would be to concatenate all
the > elements into a single feature with coma separation (use > listconcatenator). There might also be duplicates that can be removed
> with ListDuplicateRemover.
> Does anyone know an easy way to remove the empty elements from a list
> before I do the concatenation?
To be honest, I'd try taking it out of the concatenated attribute using a StringReplacer. Replace ",," with "," would probabely do the trick...
-- Hans van der Maarel
Red Geographics
___________________________________________________________________________
Leicestershire County Council - rated a 'four-star' council by the Audit Commission
___________________________________________________________________________
This e-mail and any files transmitted with it are confidential. If you are not the intended recipient, any reading, printing, storage, disclosure, copying or any other action taken in respect of this e-mail is prohibited and may be unlawful. If you are not the intended recipient, please notify the sender immediately by using the reply function and then permanently delete what you have received.
Incoming and outgoing e-mail messages are routinely monitored for compliance with Leicestershire County Council's policy on the use of electronic communications. The contents of e-mails may have to be disclosed to a request under the Data Protection Act 1998 and the Freedom of Information Act 2000.
The views expressed by the author may not necessarily reflect the views or policies of the Leicestershire County Council.
Attachments to e-mail messages may contain viruses that may damage your system. Whilst Leicestershire County Council has taken every reasonable precaution to minimise this risk, we cannot accept any liability for any damage which you sustain as a result of these factors. You are advised to carry out your own virus checks before opening any attachment.
Richard Wilkinson wrote: > That was one of my thoughts but as there could be several empty elements > posible two/three together I would have to run the stringreplacer > several times.
> Simple case = A, , C, D, E, , G, H > Other case = , , , D, E, , G, > Or = A, , C, , , , G, H
> On second thoughts maybe this could work. Attributetrimmer (both) > removes left & right comma. String replacer twice.
> I'll have another go.
Try [ ,] as a regular expression. -- Hans van der Maarel Red Geographics
a bit of tcl will do the trick, somthing like this
proc concatList {listName attr} {
set resLst [list] set n [ FME_Execute NumElements $listName ] for { set i 0} { i < $n } { incr i } { set s [ FME_GetAttribute $listName{$i} ] if { $s ne "" } { lappend resLst $s } } FME_SetAttribute $attr [join $resLst ,]
}
the idear is 1. get the number of ellmenst in the list 2. loop trough the list and get the value 3. if the value is different from "" add it to the result tcl list 4. create the new attribute and join the lsit with ',' and add it to the new attribute
On Behalf Of peter laulund
Sent: 21 November 2008 21:24
To: fmetalk@googlegroups.com
Subject: [fme] Re: List Processing - How to Remove Empty
Elements
Hi Richard
a bit of tcl will do the trick, somthing like this
proc concatList {listName attr} {
set resLst [list]
set n [ FME_Execute NumElements $listName ]
for { set i 0} { i < $n } { incr i } {
set s [ FME_GetAttribute $listName{$i} ] if { $s ne "" } {
lappend resLst $s
} }
FME_SetAttribute $attr [join $resLst ,]
}
the idear is 1. get the number of ellmenst in the list
2. loop trough the list and get the value 3. if the value is different from "" add it to the result tcl
list
4. create the new attribute and join the lsit with ',' and add
it to the new attribute
___________________________________________________________________________
Leicestershire County Council - rated a 'four-star' council by the Audit Commission
___________________________________________________________________________
This e-mail and any files transmitted with it are confidential. If you are not the intended recipient, any reading, printing, storage, disclosure, copying or any other action taken in respect of this e-mail is prohibited and may be unlawful. If you are not the intended recipient, please notify the sender immediately by using the reply function and then permanently delete what you have received.
Incoming and outgoing e-mail messages are routinely monitored for compliance with Leicestershire County Council's policy on the use of electronic communications. The contents of e-mails may have to be disclosed to a request under the Data Protection Act 1998 and the Freedom of Information Act 2000.
The views expressed by the author may not necessarily reflect the views or policies of the Leicestershire County Council.
Attachments to e-mail messages may contain viruses that may damage your system. Whilst Leicestershire County Council has taken every reasonable precaution to minimise this risk, we cannot accept any liability for any damage which you sustain as a result of these factors. You are advised to carry out your own virus checks before opening any attachment.
Hm, I tried this. But not having used TCL there seems to be a problem. I have replaced the names to my correct table names and added the following to TCLCaller-parameters: Destination Attribute: resLst Attributes to Expose: listName{}
I can see the resLst in the Inspector, but it does not have any values. Am I missing some curlybrackets on for instance resLst? (resLst{0}?) or do I have to Expose for instance listName{}.Name and listName{}.Address for them to be available?
On Friday, 21 November 2008 22:24:20 UTC+1, peter laulund wrote:
> Hi Richard
> a bit of tcl will do the trick, somthing like this
> proc concatList {listName attr} {
> set resLst [list] > set n [ FME_Execute NumElements $listName ] > for { set i 0} { i < $n } { incr i } { > set s [ FME_GetAttribute $listName{$i} ] > if { $s ne "" } { > lappend resLst $s > } > } > FME_SetAttribute $attr [join $resLst ,]
> }
> the idear is > 1. get the number of ellmenst in the list > 2. loop trough the list and get the value > 3. if the value is different from "" add it to the result tcl list > 4. create the new attribute and join the lsit with ',' and add it to the > new attribute
OK, there is another way in 2013. It will sound odd and I am going to
file an enhancement request for reasons you'll see in a moment.
In 2013, we added an option to the AttributeSplitter to ignore empty
values.
So, if you can use 2013 you could:
- use ListConcatenator to get a comma separated attribute
- use AttributeSplitter to turn it back into a list, without the empty
fields
- use ListConcatenator again to get the final comma separated
attribute
Obviously we should also add the ignore null option to the
ListConcatenator. That's what I will file the PR for.
But I think the above should work fine just now.
Regards
Mark
Mark Ireland | Senior Product Specialist
Safe Software Inc.
Suite 2017, 7445-132nd Street, Surrey, BC, CANADA
T 604.501.9985 | F 604.501.9965
http://www.safe.com/support | Twitter @FMEDoctors | http://www.fmepedia.com
On May 4, 1:36 am, SigTill <shers...@gmail.com> wrote:
> Hm, I tried this. But not having used TCL there seems to be a problem. I
> have replaced the names to my correct table names and added the following
> to TCLCaller-parameters:
> Destination Attribute: resLst
> Attributes to Expose: listName{}
> I can see the resLst in the Inspector, but it does not have any values. Am
> I missing some curlybrackets on for instance resLst? (resLst{0}?) or do I
> have to Expose for instance listName{}.Name and listName{}.Address for them
> to be available?
> On Friday, 21 November 2008 22:24:20 UTC+1, peter laulund wrote:
> > Hi Richard
> > a bit of tcl will do the trick, somthing like this
> > proc concatList {listName attr} {
> > set resLst [list]
> > set n [ FME_Execute NumElements $listName ]
> > for { set i 0} { i < $n } { incr i } {
> > set s [ FME_GetAttribute $listName{$i} ]
> > if { $s ne "" } {
> > lappend resLst $s
> > }
> > }
> > FME_SetAttribute $attr [join $resLst ,]
> > }
> > the idear is
> > 1. get the number of ellmenst in the list
> > 2. loop trough the list and get the value
> > 3. if the value is different from "" add it to the result tcl list
> > 4. create the new attribute and join the lsit with ',' and add it to the
> > new attribute
Hm, unfortunately we have to run FME Server 2012, so doing this in 2013 is not possible. I will try to find another workaround. Nor sure why the TCL-does not work though, have to read up on it to better understand it I guess.
On Friday, 4 May 2012 18:04:27 UTC+2, Mark Ireland wrote:
> OK, there is another way in 2013. It will sound odd and I am going to > file an enhancement request for reasons you'll see in a moment.
> In 2013, we added an option to the AttributeSplitter to ignore empty > values. > So, if you can use 2013 you could:
> - use ListConcatenator to get a comma separated attribute > - use AttributeSplitter to turn it back into a list, without the empty > fields > - use ListConcatenator again to get the final comma separated > attribute
> Obviously we should also add the ignore null option to the > ListConcatenator. That's what I will file the PR for. > But I think the above should work fine just now.
> Regards
> Mark
> Mark Ireland | Senior Product Specialist > Safe Software Inc. > Suite 2017, 7445-132nd Street, Surrey, BC, CANADA > T 604.501.9985 | F 604.501.9965 > http://www.safe.com/support | Twitter @FMEDoctors | > http://www.fmepedia.com
> On May 4, 1:36 am, SigTill <shers...@gmail.com> wrote: > > Hm, I tried this. But not having used TCL there seems to be a problem. I > > have replaced the names to my correct table names and added the > following > > to TCLCaller-parameters: > > Destination Attribute: resLst > > Attributes to Expose: listName{}
> > I can see the resLst in the Inspector, but it does not have any values. > Am > > I missing some curlybrackets on for instance resLst? (resLst{0}?) or do > I > > have to Expose for instance listName{}.Name and listName{}.Address for > them > > to be available?
> > On Friday, 21 November 2008 22:24:20 UTC+1, peter laulund wrote:
> > > Hi Richard
> > > a bit of tcl will do the trick, somthing like this
> > > proc concatList {listName attr} {
> > > set resLst [list] > > > set n [ FME_Execute NumElements $listName ] > > > for { set i 0} { i < $n } { incr i } { > > > set s [ FME_GetAttribute $listName{$i} ] > > > if { $s ne "" } { > > > lappend resLst $s > > > } > > > } > > > FME_SetAttribute $attr [join $resLst ,]
> > > }
> > > the idear is > > > 1. get the number of ellmenst in the list > > > 2. loop trough the list and get the value > > > 3. if the value is different from "" add it to the result tcl list > > > 4. create the new attribute and join the lsit with ',' and add it to > the > > > new attribute
Did you try it with StringReplacer and AttributeTrimmer, using ,,* (comma,comma,asterisk) or ,+ (comma,plus) as regex in the StringReplacer? Both seem to cleanup the concatenated value for me, even with multiple consecutive commas.
J-----.
Op maandag 7 mei 2012 10:35:27 UTC+2 schreef SigTill het volgende:
> Hm, unfortunately we have to run FME Server 2012, so doing this in 2013 is > not possible. I will try to find another workaround. Nor sure why the > TCL-does not work though, have to read up on it to better understand it I > guess.
> On Friday, 4 May 2012 18:04:27 UTC+2, Mark Ireland wrote:
>> OK, there is another way in 2013. It will sound odd and I am going to >> file an enhancement request for reasons you'll see in a moment.
>> In 2013, we added an option to the AttributeSplitter to ignore empty >> values. >> So, if you can use 2013 you could:
>> - use ListConcatenator to get a comma separated attribute >> - use AttributeSplitter to turn it back into a list, without the empty >> fields >> - use ListConcatenator again to get the final comma separated >> attribute
>> Obviously we should also add the ignore null option to the >> ListConcatenator. That's what I will file the PR for. >> But I think the above should work fine just now.
>> Regards
>> Mark
>> Mark Ireland | Senior Product Specialist >> Safe Software Inc. >> Suite 2017, 7445-132nd Street, Surrey, BC, CANADA >> T 604.501.9985 | F 604.501.9965 >> http://www.safe.com/support | Twitter @FMEDoctors | >> http://www.fmepedia.com
>> On May 4, 1:36 am, SigTill <shers...@gmail.com> wrote: >> > Hm, I tried this. But not having used TCL there seems to be a problem. >> I >> > have replaced the names to my correct table names and added the >> following >> > to TCLCaller-parameters: >> > Destination Attribute: resLst >> > Attributes to Expose: listName{}
>> > I can see the resLst in the Inspector, but it does not have any values. >> Am >> > I missing some curlybrackets on for instance resLst? (resLst{0}?) or do >> I >> > have to Expose for instance listName{}.Name and listName{}.Address for >> them >> > to be available?
>> > On Friday, 21 November 2008 22:24:20 UTC+1, peter laulund wrote:
>> > > Hi Richard
>> > > a bit of tcl will do the trick, somthing like this
>> > > proc concatList {listName attr} {
>> > > set resLst [list] >> > > set n [ FME_Execute NumElements $listName ] >> > > for { set i 0} { i < $n } { incr i } { >> > > set s [ FME_GetAttribute $listName{$i} ] >> > > if { $s ne "" } { >> > > lappend resLst $s >> > > } >> > > } >> > > FME_SetAttribute $attr [join $resLst ,]
>> > > }
>> > > the idear is >> > > 1. get the number of ellmenst in the list >> > > 2. loop trough the list and get the value >> > > 3. if the value is different from "" add it to the result tcl list >> > > 4. create the new attribute and join the lsit with ',' and add it to >> the >> > > new attribute
Hi all
Just following up to let you know that the 2013 ListConcatenator now
has an option to ignore empty fields.
It will be in FME beta builds 13100 or greater.
I know it doesn't help in this case, if you can't upgrade to 2013, but
I just wanted to let you know it will be there when you can eventually
upgrade to 2013
Regards
Mark
Mark Ireland | Senior Product Specialist
Safe Software Inc.
Suite 2017, 7445-132nd Street, Surrey, BC, CANADA
T 604.501.9985 | F 604.501.9965
http://www.safe.com/support | Twitter @FMEDoctors | http://www.fmepedia.com
On May 8, 12:05 am, Jeroen Muris <j.mu...@schiedam.nl> wrote:
> Did you try it with StringReplacer and AttributeTrimmer, using ,,*
> (comma,comma,asterisk) or ,+ (comma,plus) as regex in the StringReplacer?
> Both seem to cleanup the concatenated value for me, even with multiple
> consecutive commas.
> J-----.
> Op maandag 7 mei 2012 10:35:27 UTC+2 schreef SigTill het volgende:
> > Hm, unfortunately we have to run FME Server 2012, so doing this in 2013 is
> > not possible. I will try to find another workaround. Nor sure why the
> > TCL-does not work though, have to read up on it to better understand it I
> > guess.
> > On Friday, 4 May 2012 18:04:27 UTC+2, Mark Ireland wrote:
> >> OK, there is another way in 2013. It will sound odd and I am going to
> >> file an enhancement request for reasons you'll see in a moment.
> >> In 2013, we added an option to the AttributeSplitter to ignore empty
> >> values.
> >> So, if you can use 2013 you could:
> >> - use ListConcatenator to get a comma separated attribute
> >> - use AttributeSplitter to turn it back into a list, without the empty
> >> fields
> >> - use ListConcatenator again to get the final comma separated
> >> attribute
> >> Obviously we should also add the ignore null option to the
> >> ListConcatenator. That's what I will file the PR for.
> >> But I think the above should work fine just now.
> >> Regards
> >> Mark
> >> Mark Ireland | Senior Product Specialist
> >> Safe Software Inc.
> >> Suite 2017, 7445-132nd Street, Surrey, BC, CANADA
> >> T 604.501.9985 | F 604.501.9965
> >>http://www.safe.com/support| Twitter @FMEDoctors |
> >>http://www.fmepedia.com
> >> On May 4, 1:36 am, SigTill <shers...@gmail.com> wrote:
> >> > Hm, I tried this. But not having used TCL there seems to be a problem.
> >> I
> >> > have replaced the names to my correct table names and added the
> >> following
> >> > to TCLCaller-parameters:
> >> > Destination Attribute: resLst
> >> > Attributes to Expose: listName{}
> >> > I can see the resLst in the Inspector, but it does not have any values.
> >> Am
> >> > I missing some curlybrackets on for instance resLst? (resLst{0}?) or do
> >> I
> >> > have to Expose for instance listName{}.Name and listName{}.Address for
> >> them
> >> > to be available?
> >> > On Friday, 21 November 2008 22:24:20 UTC+1, peter laulund wrote:
> >> > > Hi Richard
> >> > > a bit of tcl will do the trick, somthing like this
> >> > > proc concatList {listName attr} {
> >> > > set resLst [list]
> >> > > set n [ FME_Execute NumElements $listName ]
> >> > > for { set i 0} { i < $n } { incr i } {
> >> > > set s [ FME_GetAttribute $listName{$i} ]
> >> > > if { $s ne "" } {
> >> > > lappend resLst $s
> >> > > }
> >> > > }
> >> > > FME_SetAttribute $attr [join $resLst ,]
> >> > > }
> >> > > the idear is
> >> > > 1. get the number of ellmenst in the list
> >> > > 2. loop trough the list and get the value
> >> > > 3. if the value is different from "" add it to the result tcl list
> >> > > 4. create the new attribute and join the lsit with ',' and add it to
> >> the
> >> > > new attribute
On Friday, 11 May 2012 21:00:08 UTC+2, Mark Ireland wrote:
> Hi all > Just following up to let you know that the 2013 ListConcatenator now > has an option to ignore empty fields. > It will be in FME beta builds 13100 or greater.
> I know it doesn't help in this case, if you can't upgrade to 2013, but > I just wanted to let you know it will be there when you can eventually > upgrade to 2013
> Regards
> Mark
> Mark Ireland | Senior Product Specialist > Safe Software Inc. > Suite 2017, 7445-132nd Street, Surrey, BC, CANADA > T 604.501.9985 | F 604.501.9965 > http://www.safe.com/support | Twitter @FMEDoctors | > http://www.fmepedia.com
> On May 8, 12:05 am, Jeroen Muris <j.mu...@schiedam.nl> wrote: > > Did you try it with StringReplacer and AttributeTrimmer, using ,,* > > (comma,comma,asterisk) or ,+ (comma,plus) as regex in the > StringReplacer? > > Both seem to cleanup the concatenated value for me, even with multiple > > consecutive commas.
> > J-----.
> > Op maandag 7 mei 2012 10:35:27 UTC+2 schreef SigTill het volgende:
> > > Hm, unfortunately we have to run FME Server 2012, so doing this in > 2013 is > > > not possible. I will try to find another workaround. Nor sure why the > > > TCL-does not work though, have to read up on it to better understand > it I > > > guess.
> > > On Friday, 4 May 2012 18:04:27 UTC+2, Mark Ireland wrote:
> > >> OK, there is another way in 2013. It will sound odd and I am going to > > >> file an enhancement request for reasons you'll see in a moment.
> > >> In 2013, we added an option to the AttributeSplitter to ignore empty > > >> values. > > >> So, if you can use 2013 you could:
> > >> - use ListConcatenator to get a comma separated attribute > > >> - use AttributeSplitter to turn it back into a list, without the > empty > > >> fields > > >> - use ListConcatenator again to get the final comma separated > > >> attribute
> > >> Obviously we should also add the ignore null option to the > > >> ListConcatenator. That's what I will file the PR for. > > >> But I think the above should work fine just now.
> > >> Regards
> > >> Mark
> > >> Mark Ireland | Senior Product Specialist > > >> Safe Software Inc. > > >> Suite 2017, 7445-132nd Street, Surrey, BC, CANADA > > >> T 604.501.9985 | F 604.501.9965 > > >>http://www.safe.com/support| Twitter @FMEDoctors | > > >>http://www.fmepedia.com
> > >> On May 4, 1:36 am, SigTill <shers...@gmail.com> wrote: > > >> > Hm, I tried this. But not having used TCL there seems to be a > problem. > > >> I > > >> > have replaced the names to my correct table names and added the > > >> following > > >> > to TCLCaller-parameters: > > >> > Destination Attribute: resLst > > >> > Attributes to Expose: listName{}
> > >> > I can see the resLst in the Inspector, but it does not have any > values. > > >> Am > > >> > I missing some curlybrackets on for instance resLst? (resLst{0}?) > or do > > >> I > > >> > have to Expose for instance listName{}.Name and listName{}.Address > for > > >> them > > >> > to be available?
> > >> > On Friday, 21 November 2008 22:24:20 UTC+1, peter laulund wrote:
> > >> > > Hi Richard
> > >> > > a bit of tcl will do the trick, somthing like this
> > >> > > proc concatList {listName attr} {
> > >> > > set resLst [list] > > >> > > set n [ FME_Execute NumElements $listName ] > > >> > > for { set i 0} { i < $n } { incr i } { > > >> > > set s [ FME_GetAttribute $listName{$i} ] > > >> > > if { $s ne "" } { > > >> > > lappend resLst $s > > >> > > } > > >> > > } > > >> > > FME_SetAttribute $attr [join $resLst ,]
> > >> > > }
> > >> > > the idear is > > >> > > 1. get the number of ellmenst in the list > > >> > > 2. loop trough the list and get the value > > >> > > 3. if the value is different from "" add it to the result tcl > list > > >> > > 4. create the new attribute and join the lsit with ',' and add it > to > > >> the > > >> > > new attribute
