CSV file

Showing 1-13 of 13 messages
CSV file Chappman 2/24/12 2:13 AM
Hi folks,

I am having trouble trying to save a matrix P as a CSV file in Sage:

Def function(A,D):
    (sage syntax for creating a matrix P)
return P

and then using a function which opens up the CSV file and utilizes the
entires in the matrix P, from the CSV file.
Is there a method for this?

Kind Regards
Chappman
Re: CSV file Harald Schilly 2/24/12 10:03 AM


On Friday, February 24, 2012 11:13:53 AM UTC+1, Chappman wrote:
and then using a function which opens up the CSV file and utilizes the
entires in the matrix P, from the CSV file.
Is there a method for this?

uhm, i'm not sure if you ask about reading or writing. also, your "d" in def is uppercase.

reading from this file matrix.csv:
1,2,3
2,2,-1.1
0,0,1


works like this:
sage: import csv                           
sage: data = list(csv.reader(file("matrix.csv")))
sage: m = matrix([[ float(_) for _ in line] for line in data])
sage: m
[ 1.0  2.0  3.0]
[ 2.0  2.0 -1.1]
[ 0.0  0.0  1.0]

the other way around works like this (i print this, writing to the file is trivial)

for line in m.rows():
   print ','.join((str(_) for _ in line)) 

gives

1.0,2.0,3.0
2.0,2.0,-1.1
0.0,0.0,1.0


Re: CSV file Volker Braun 2/24/12 11:32 AM
We do get questions about "how to read matrix from csv" quite regularly. Of course its just a few lines of code, but I think it would be nice to have a matrix_from_file('fname.csv') function that imports csv and perhaps others (gnumeric/ooffice/excel). Any volunteers? ;-)
Re: CSV file Chappman 2/26/12 4:44 AM
Hi Harald,

You answered I should of phrased that question better, but lets say I
created a matrix in Sage, and I want to save it as a csv file how do I
do that?

The matrix is made from the following function below

Def function(A,D)
    (syntex for making matrix P)
return P

Kind Regards

Chappman
Re: CSV file Dima Pasechnik 2/26/12 5:09 AM
In gmane.comp.mathematics.sage.support, you wrote:
> Hi Harald,
>
> You answered I should of phrased that question better, but lets say I
> created a matrix in Sage, and I want to save it as a csv file how do I
> do that?
>
> The matrix is made from the following function below
>
> Def function(A,D)
>     (syntex for making matrix P)
> return P
>
documentation on reading/writing CSV files in Python is here:
http://docs.python.org/library/csv.html


it's not clear what these A and D are supposed to be,
and how this would be different
from creating a Sage matrix like this, for instance:
M=matrix([[1,2],[3,4]])
and then writing entries of M, i.e. M[0,0], M[0,1], etc,
into CSV file using csv.writer().

Dmitrii

> --
> To post to this group, send email to sage-s...@googlegroups.com
> To unsubscribe from this group, send email to sage-support...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-support
> URL: http://www.sagemath.org
>

Re: CSV file Jason Grout 2/27/12 6:26 AM
On 2/24/12 1:32 PM, Volker Braun wrote:
> We do get questions about "how to read matrix from csv" quite regularly.
> Of course its just a few lines of code, but I think it would be nice to
> have a matrix_from_file('fname.csv') function that imports csv and
> perhaps others (gnumeric/ooffice/excel). Any volunteers? ;-)
>

How about the matrix constructor reads from an iterator and recognizes
csv?  We could even use the numpy savetxt and loadtxt functions to more
sophisticated parsing.

So:

with open('mymatrix.csv','r') as f:
     m=matrix(f)

or

m.save('test.csv') # or m.save('test.csv', format='csv')

I'm not volunteering in these suggestions...

Thanks,

Jason


Re: CSV file Nils Bruin 2/27/12 11:41 AM
On Feb 27, 6:26 am, Jason Grout <jason-s...@creativetrax.com> wrote:
> How about the matrix constructor reads from an iterator and recognizes
> csv?  We could even use the numpy savetxt and loadtxt functions to more
> sophisticated parsing.

You can already do:

sage: import csv
sage: M=matrix(RR,list(csv.reader(open("m.csv"))))
sage: csv.writer(open("c.csv","w")).writerows(list(M^2))

I think it would be possible to accept matrix(R,<iterable>) as a
generalization of matrix(R,<list>), which would save 6 characters in
the second line above.

Personally, I think apart from the "list" call, the line
"M=matrix(RR,list(csv.reader(open("m.csv"))))" carries no extraneous
cognitive ballast. All steps are meaningful and leaving any of them
out requires "magic" functionality that tends to break in corner
cases. (csv files can be tricky beasts, so having an extra call that
tells how to interpret the file as a csv makes good sense; requiring
the ring also makes good sense, because it's difficult to see from a
string what ring exactly the elements should lie in). Another
advantage of this incantation is that it's easily discovered by
someone familiar with python and that the logic behind it is entirely
transferable. A novice will have to look up how to do it anyway, so if
we just use this as doc for how to read csv files into a matrix, we're
done.

The writer looks a little less logical to me. I don't have a good
alternative for that. Perhaps your "m.save(...,format='csv')" is a
reasonable alternative.

Re: CSV file Chappman 2/29/12 2:49 PM

Thank you very much for that code. Just to clarify, I'm using the sage
notebook (running on my machine), will the code still work? I.e. be
able to read where the files are on my machine even when using the
notebook interface?

Thank you very much for your help.
Chappman
Re: [sage-support] Re: CSV file Robert Bradshaw 3/2/12 12:28 AM
The difficulty with accepting an iterator (of strings) is that it is
unclear if each item corresponds to a row or an element. But I would
be in favor of rather liberal string parsing, so one could do

matrix(open("test.csv").read())

just like

matrix("""
1,2,3
4,5,5
""")

> --
> To post to this group, send email to sage-s...@googlegroups.com
> To unsubscribe from this group, send email to
> sage-support...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-support
> URL: http://www.sagemath.org

Re: CSV file Nils Bruin 3/2/12 6:19 PM
On Mar 2, 12:28 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:
> The difficulty with accepting an iterator (of strings) is that it is
> unclear if each item corresponds to a row or an element. But I would
> be in favor of rather liberal string parsing, so one could do

Why is it easier to decide if the items correspond to rows or elements
in

matrix( [ c for c in iterator ])

than it is in

matrix( ( c for c in iterator ) )

I agree that this problem arises if we would allow the c's themselves
to just be generic iterators, which would include strings.

(although you could argue that if no dimensions are given, matrix
should insist on [ [...], [...] , ... , [...]] rather than
interpreting [1,2,3] as [[1,2,3]]).

Of course, accepting iterators would be at odds with accepting strings
straight away, as in matrix("1,2,3\n4,5,6"), so we'd have to make a
choice between those two. Currently strings nor iterators are accepted
by a bare matrix. However, for related objects the choice has already
been made in favor of iterators:

sage: M22=MatrixSpace(Integers(),2,2)
sage: M22([1,2,3,4])
[1 2]
[3 4]
sage: M22( i for i in [1,2,3,4] )
[1 2]
[3 4]
sage: M22("1 2 3 4")
TypeError: entries has the wrong length
sage: M22("1234")
[1 2]
[3 4]
sage: matrix(QQ,2,2, (i for i in [1,2,3,4]) )
[1 2]
[3 4]

Especially the last example makes me hesitant about letting matrix
parse strings.
Re: [sage-support] Re: CSV file Robert Bradshaw 3/4/12 1:14 AM
On Fri, Mar 2, 2012 at 6:19 PM, Nils Bruin <nbr...@sfu.ca> wrote:
> On Mar 2, 12:28 am, Robert Bradshaw <rober...@math.washington.edu>
> wrote:
>> The difficulty with accepting an iterator (of strings) is that it is
>> unclear if each item corresponds to a row or an element. But I would
>> be in favor of rather liberal string parsing, so one could do
>
> Why is it easier to decide if the items correspond to rows or elements
> in
>
> matrix( [ c for c in iterator ])
>
> than it is in
>
> matrix( ( c for c in iterator ) )

That's not the difference.

> I agree that this problem arises if we would allow the c's themselves
> to just be generic iterators, which would include strings.

Yep. I don't like matrix(["1 2", "3 4"]) because that's impossible to
distinguish (using types alone) from matrix(["1", "2", "3", "4"]) (or
the same replacing lists with iterators). Iterating over open("file"))
iterates over the lines which would typically be entire rows.

Interpreting strings as a 1-digit decimal list seems more broken to me

sage: MatrixSpace(SR, 2, 2)("1234")
[1234    0]
[   0 1234]
sage: MatrixSpace(ZZ, 2, 2)("1234")
[1 2]
[3 4]

I think it's fair to test for strings first, trying to parse, before
testing if it's an iterator. This is consistant with many other
objects that try to "parse" their string representations.

sage: ZZ['x']([1,2,3])
3*x^2 + 2*x + 1
sage: ZZ['x']("123")
123
sage: ZZ['x']("x^2 + 5")
x^2 + 5

- Robert

Re: CSV file Nils Bruin 3/4/12 10:34 AM
On Mar 4, 1:14 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:
> I think it's fair to test for strings first, trying to parse, before
> testing if it's an iterator. This is consistant with many other
> objects that try to "parse" their string representations.
>
> sage: ZZ['x']([1,2,3])
> 3*x^2 + 2*x + 1
> sage: ZZ['x']("123")
> 123
> sage: ZZ['x']("x^2 + 5")
> x^2 + 5

Indeed. It's not quit consistent with duck typing, but well ...

I guess matrix(S) would do:

  if isinstance(S,basestring):
    <do our best to parse S as a string rep of a matrix>
  else:
    try:
      V=iter(S)
      #figure out how to interpret the elements of V
      #(are they rows or elements?)
      #should we accept strings at this point?
      #the CSV example above indicates that with a known base ring,
      #allowing elements of V to be iterables of strings representing
elements
      #seems a good idea.
    except TypeError:
      #do whatever we do with non-iterable matrix initializers
Re: [sage-support] Re: CSV file Robert Bradshaw 3/5/12 1:56 AM

Yep, that's the basic idea, though I might test for the non-iterable
matrix initializers first, so something like matrix(QQ['x'], 2, x^2)
becomes

[x^2, 0]
[0, x^2]

rather than using list(x^2), and an iterable of strings is always
considered an iterator of elements, not an iterator of rows.

- Robert