How to remove comma from a double quoted string using sed
srikanth
Can any one tell me how to remove the comma from the double quoted
string.
echo "Hello,""1,043"",TEST1" | sed -e "/\"[A-Z].*\,.*\"s/,/ /g"
echo "Hello,""1,043"",TEST1" | sed 's/^"\(.*\),\(.*\)\"/"\1 \2"/'
Current output: Hello, 1,043, ERLRW091
Here I want to remove the comma from the double quoted string "1,043"
to 1043
Thanks in advance.
Chris F.A. Johnson
You are not passing a double-quotes string. Try the echo command by
itself so you can see what it actually does.
What you want is:
echo '"Hello,""1,043"",TEST1"' | sed -e ...
--
Chris F.A. Johnson, author <http://cfaj.freeshell.org/shell/>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence
Tim Harig
Is the number coming from a variable? If so, it is easier to remove it
there:
var2=`echo var1 | tr -d ','`
otherwise:
echo "Hello,""1,043"",TEST1" | \
awk 'BEGIN {FS=","} {print $1 "," $2 $3 "," $4}'
Tim Harig
> On 2009-06-26, srikanth wrote:
>> Hi,
>>
>> Can any one tell me how to remove the comma from the double quoted
>> string.
> echo '"Hello,""1,043"",TEST1"' | sed -e ...
He may not want or need the quotation marks in the output. Most CSV
style files don't use them unless they need to encase text where the
limiter is or might be present.
srikanth
Hey Tim,
Thanks for your quick responding. Actually I have already tried with
awk now I am trying with sed to do.
'm not good at sed actually. By googling I have found below script
echo '"file,d1",field2,field3,field4' | sed 's/\(.*\)"\(.*\),\(.*\)"\
(.*\)/\1"\2\3"\4/'
Which removes the comma from the double quoted string. But It will
remove the comma from the first string only. When there is a comma in
other string it is failing. Now I want to incorporate this into with
my script to remove the comma. Any idea to implement?
Chris F.A. Johnson
Who knows? If he wants to remove the commas, there's no CSV.
If he wants to remove the commas and the quotes::
echo "Hello,""1,043"",TEST1" | sed 's/,//g'
If he want to remove the commas and keep the quotes:
echo '"Hello,""1,043"",TEST1"' | sed 's/,//g'
srikanth
wrote:
> On 2009-06-26, Tim Harig wrote:
>
Sorry I think you have mistaken my question. I am asking how to remove
comma from the double quoted string.
Ex: "1,043" to "1043"
pk
> Hi,
>
> Can any one tell me how to remove the comma from the double quoted
> string.
>
> echo "Hello,""1,043"",TEST1" | sed -e "/\"[A-Z].*\,.*\"s/,/ /g"
>
> echo "Hello,""1,043"",TEST1" | sed 's/^"\(.*\),\(.*\)\"/"\1 \2"/'
>
> Current output: Hello, 1,043, ERLRW091
No. Both your lines above output exactly
Hello,1,043,TEST1
Paste some real input data, along with the expected output you want to
get from that data. Make clear whether the double quotes are part of the
input or not. The best thing to do would be to paste the actual data sed
will have to act on, rather than using echo and confusing people.
Ed Morton
Is this the kind of thing you're really looking for:
$ cat file
Hello,"1,043",TEST1
$ awk -v FS= '{
for (i=1;i<=NF;i++) {
if ($i=="\"")
inFld=!inFld
else if (!($i=="," && inFld))
printf "%s",$i
}
print ""
}' file
Hello,1043,TEST1
If you want to leave the quotes in the output, just remove the "else"
before the second "if".
Ed.
sw0rdfish
awk 'BEGIN{OFS=FS="\"\""}{for(i=2;i<NF;i+=2){gsub(/,/,"",$i)}}1' file
srikanth
Here the exact input is echo "Hello,""1,043"",TEST1" and I want to
remove double quotes and comma fromt the string.So the output should
be like this
Hello,1043,TEST1
Ed Morton
Then you'll have to tell us how the tool you're piping that to can
tell the difference between the comma you want to remove and all the
other commas, because there's no quotation marks in the echo output
for the subsequent tool to use to make a distinction:
$ echo "Hello,""1,043"",TEST1"
Hello,1,043,TEST1
That just looks like 4 comma-separated fields after the echo.
Ed.
Ben Bacarisse
It may be hard for you to see right now, but you are repeatedly
obscuring the issue by using echo. It would be better to back up and
describe what you are actually doing because the previous processing
might have a bearing on the answer as well. It is certain that you
don't want to process the output of one echo command -- that would be
pointless. You are using it as an example, but it is a bad way to
give an example unless you know exactly what echo does and I think you
don't because
echo "Hello,""1,043"",TEST1"
is the same as
echo Hello,1,043,TEST1
but the latter is much less confusing.
When you post some of the real data, you need to explain exactly which
commas (and quotes) need to be removed. By this I mean you need to
give the logic behind the removal: is it the comma in the second field
or is it any comma between any two digits or it is something else?.
I suspect you want to remove commas used as thousands separators in
numbers, but you will still have to explain how one can tell the
difference between consecutive numbers separated by commas and a
number with a comma inside it (the answer to that might well become
clear once you remove the confusion about the quotes).
--
Ben.
srikanth
I have mentioned in my first mail regarding removing comma from the
second double quoted string and also mentioned the string as well.
Sorry If I have not given correct info.
pk
> I have mentioned in my first mail regarding removing comma from the
> second double quoted string and also mentioned the string as well.
There is NO quoted string whatsoever in your "input". That's what people are
trying to tell you from the start.
If you want sed to act on a file, paste REAL lines taken from that file.
Jon LaBadie
>
> I have mentioned in my first mail regarding removing comma from the
> second double quoted string and also mentioned the string as well.
>
> Sorry If I have not given correct info.
Let's try this one more time srikanth,
You are using as your input to the pipe and sed/awk the following:
echo "Hello,""1,043"",TEST1"
And you have been told several times, but have not heard, that
there are NO double quoted strings in your data as you present it.
This is because the shell removes those quotes and the output from
echo is simply:
Hello,1,043,TEST1
To confirm this simply run the echo command at your terminal.
As there are no quotes to key on, your problem needs to be
restated. Otherwise, how is the comma between the 1 and 0
different than those between the o and 1 or between the 3 and T?
That is the reason several have asked to see some of your
real input data, not simply this test data. Or give us a
better understanding how to distinguish the data that needs
to be operated upon from the data to leave alone. Second
column all the time? What determines it is second column?
Is it the only place where digits appear -- Probably not.
Give us more to work on, not just a pointer back to your
original question.
srikanth
Yes, The shell will remove the quotes from input. Ok, I will explain
my exact doubt here. I want to remove the comma between the numbers 1
and 0.
$echo "Hello,""1,043"",TEST1" | sed -e "/\"[A-Z].*\,.*\"s/,/ /g"
$Hello,1,043,TEST1
I want to remove the comma in between the number i.e 1,0 to 10 so the
output should look like this Hello,1043,TEST1
Marcel Bruinsma
> Yes, The shell will remove the quotes from input. Ok, I will explain
> my exact doubt here. I want to remove the comma between the numbers 1
> and 0.
>
> $echo "Hello,""1,043"",TEST1" | sed -e "/\"[A-Z].*\,.*\"s/,/ /g"
> $Hello,1,043,TEST1
>
> I want to remove the comma in between the number i.e 1,0 to 10 so the
> output should look like this Hello,1043,TEST1
$ echo "Hello,""1,043"",TEST1" | sed 's;1,0;10;'
Hello,1043,TEST1
--
printf -v email $(echo \ 155 141 162 143 145 154 142 162 165 151 \
156 163 155 141 100 171 141 150 157 157 056 143 157 155|tr \ \\\\)
# Live every life as if it were your last! #
Piotr KUCHARSKI
> Here the exact input is echo "Hello,""1,043"",TEST1" and I want to
> remove double quotes and comma fromt the string.So the output should
> be like this
> Hello,1043,TEST1
Had it not been "echo", but input file...
% cat foo
"Hello,""1,043"",TEST1"
% cat foo | awk -F'"' '{gsub(",","",$4); print}'
Hello, 1043 ,TEST1
(removing spaces left as an excercise)
p.
--
Beware of he who would deny you access to information, for in his
heart he dreams himself your master. -- Commissioner Pravin Lal
Ed Morton
OK, so you've been told about 100 times in this thread that:
echo "Hello,""1,043"",TEST1"
produces the same output as:
echo Hello,1,043,TEST1
which is that same as having an input file with just this one line in
it:
Hello,1,043,TEST1
so you totally understand that now, right?
You've told us that you want to remove one of the commas and that it's
the comma between the first "1" and the "0" so given the above input
file you'd want to produce this output file:
Hello,1043,TEST1
but you still haven't told us how to identify that removable comma in
general input vs in that specific one line of input. To move things
along, what would you want your output to be for this input file:
Hello,1,043,TEST1
Hello,1,043,537,TEST1
1,043,Hello,TEST1
Hello,TEST1,1,043
Hello,043,TEST1
Hello,2,143,TEST1
Regards,
Ed.
Michael Paoli
> Can any one tell me how to remove the comma from the double quoted
> string.
By "any one" if you intended "someone", the answer is most likely:
Yes.
(Otherwise, stricly interpreting your question, the answer may
likely be:
No.
)
Now, your question has been answered. ;-)
Folks in technical forums may take your questions rather/quite/"too"
literally.
Ambiguous specifications can greatly complicate matters - most notably
one may get something that meets one's specification(s), but doesn't
at
all meet one's requirements or expectations.
> echo "Hello,""1,043"",TEST1" | sed -e "/\"[A-Z].*\,.*\"s/,/ /g"
> echo "Hello,""1,043"",TEST1" | sed 's/^"\(.*\),\(.*\)\"/"\1 \2"/'
> Current output: Hello, 1,043, ERLRW091
> Here I want to remove the comma from the double quoted string "1,043"
> to 1043
As has been pointed out elsewhere on the "thread" from your post,
your use of echo here confuses your intent - i.e., reiterating, as
you've given your examples here, the shell strips the double quotes
before you pass the remaining data along to sed(1), so it's not too
surprising folks are rather confused as to what exactly it is you want
them to do.
It would likely be better to indicate how you want input data
transformed into output data. One would do well to specify
sufficiently
completely and unambiguously - so that a "solution" might actually
meet
one's desired objectives - or if one isn't looking for that complete
of a
solution, at least clearly focus on and unambiguously describe the bit
that one is trying to solve.
Some of the many questions your questions raise:
So, for input like:
"1,234"
should it be output as:
"1234"
or stripped to:
1234
This looks like CSV data, is it? CSV isn't an entirely unambiguous
"specification", but at least if one made clear that this is CSV data,
we could start to make a lot of fairly reasonable presumptions about
the
nature of the input data.
Whether it is CSV data or not,
How should we handle an imbalance of double quote characters?
Is there a permitted way to have an escaped double quote, e.g.
might we expect to see something like:
"comma: ,","double quote: \""
to represent the fields:
comma: ,
and:
double quote: "
And (as I've mostly presumed here) is comma in the input data
within double quotes part of the field data, rather than field
separator?
Are the double quotes mostly optional? E.g. do these represent the
same
data:
"1",2,apple
1,"2","apple"
So, for input data like:
"one,two,three","1,234","123 Main St.,City,State,Zip"
Should the output look like:
"one,two,three","1234","123 Main St.,City,State,Zip"
or:
"onetwothree","1234","123 Main St.CityStateZip"
or:
"onetwothree""1234""123 Main St.CityStateZip"
If we just want to strip the commas out of double quoted "numeric"
fields, how precisely do we define which fields are numeric?
Can they have a leading sign, including possibly +?
Can they have a decimal point? Only one?
Can they have exponent (e.g. "3.0E8")?
What is/are the allowed convention(s) for comma and decimal placement
format in the double quoted numeric fields? (note that this can vary
quite substantially from country to country).
Anyway, I've answered your question (at least the one you asked), and
hopefully now you'll also better know how to ask your question(s).
See also:
http://catb.org/esr/faqs/smart-questions.html
(The above also have lots of additional useful tips which I didn't
specifically cover here, many of which could also be usefully applied
here).