Another CSV related question! How do you search and delete a record within a CSV file?

[Michael Rivera]

Hey guys, I just wanted to ask if it is possible to search for a record in a csv file say through their ID field. once searched is it possible to delete that searched record through the encoding/csv package?

if anyone wants to see the code (which is a collective effort of various helpful users such as Carlos Castillo, petergo, and Matthew Zimmerman) here it is :

http://play.golang.org/p/XWY4X7XNzd

I will continue reading through the net if I find any solutions :)

[Gyepi SAM]

On Fri, Apr 18, 2014 at 02:12:53AM -0700, Michael Rivera wrote:
> Hey guys, I just wanted to ask if it is possible to search for a record in
> a csv file say through their ID field. once searched is it possible to
> delete that searched record through the encoding/csv package?

CSV files are usually used for data at rest so the common use cases are encoding and
decoding. That is to say, writing and reading. It's certainly possible to do
what you're asking but it's cumbersome because that's not what it's meant to
be used for.

In any case, if you're still interested in playing with it, here's how to do it:

To search for a particular record:
Read records sequentially and compare record fields to the target value(s).
You'll either get a match or run out of records, in which case, the desired
record is not in the set. Note that this can be slow if you have a lot of data
and there is overhead for parsing each record.

To delete records, you have to basically have to write a new CSV file and omit
the discarded records.
Read records sequentially and compare record fields to the target value(s).
If they don't match write the record to the new CSV.
When finished, rename the new CSV over the old one.
The reading and writing can be separated into two phases if necessary, if you
keep track of the offset and lengths of the records to be discarded and use
that data to drive the write phase.

-Gyepi

[Michael Rivera]

I know that this may be cumbersome but I'm just using this to see the capabilities of what the encoding/csv package can do. I know that the search can be done but I'm wondering how the search may work. Like lets say I search for the ID "123" do I take the 123 and just use it to compare the data line by line? If so how would I be comparing it? I mean how would I extract data from the csv in knowing which is which?


For the delete function, basically it would be like a new write function? A write function that would create a new csv. My question here would be is if I overwrite the data say I write it over under the same file name, will it still retain the "not deleted" data? I'm sorry for the weird questions sir

[Michael Rivera]

I'm trying to delete the whole row of lets say the row that contains the ID "123" is that possible? 

for searching a specific row of data, I want to search by ID, and just print the record that contains that data. How would be my approach for that?

On Friday, April 18, 2014 5:12:53 PM UTC+8, Michael Rivera wrote:

[Michael Rivera]

My main goal here is to know if it is possible to have basic crud functions using the console input and having the csv as the "database" I know this might be weird but it would really help me a lot. I'm sorry for the weird need for this program and I hope someone could help me. 

[Ingo Oeser]

The algorithm that works best here is
* open the csv file
* create a new, empty file
* read each line from the existing file
* find out, if it contains Id 123
* if if doesn't write the line to the new file
* replace the old file with the new one.

[Gyepi SAM]

On Fri, Apr 18, 2014 at 05:15:58AM -0700, Michael Rivera wrote:
> I know that this may be cumbersome but I'm just using this to see the
> capabilities of what the encoding/csv package can do. I know that the
> search can be done but I'm wondering how the search may work. Like lets say
> I search for the ID "123" do I take the 123 and just use it to compare the
> data line by line?

Yes. There is no other way without some kind of index or other preprocessing
step.

> If so how would I be comparing it? I mean how would I
> extract data from the csv in knowing which is which?

OUT:
for {
record, err = csv.Read()
switch {
case err == io.EOF:
break OUT
case err != nil:
log.Fatal(err)
case record[0] == "123":
//found
}
}

If you're only looking for one value, then 'break OUT' after finding it.
Obviously you'd have to convert types, etc if you're not just comparing
strings.

> For the delete function, basically it would be like a new write function? A
> write function that would create a new csv. My question here would be is if
> I overwrite the data say I write it over under the same file name, will it
> still retain the "not deleted" data?

You should not "write it over under the same file name". You could lose data.
Instead, you create a new file, under a different name and write to that.
When the writing is done, you rename the new file to the old file, replacing
the old data with the new data. Like I said, it is cumbersome.
In any case, it sounds like you'd want to use the code above as a function
returning a record number or file offset so you can use it for both
operations.

> I'm sorry for the weird questions sir

They are not weird at all, but it would be easier to help you if you show
the code you've written, even if it doesn't work. Otherwise, it's a bit of a
guessing game, especially since I don't know where you're heading.

-Gyepi

[Michael Rivera]

Gyepi, I'm still thinking how my logic would go before writing down code for my problem so far from your input I've been thinking if when deleting I'd be going to search through ID (sort it then compare probably) ----> then delete the searched item. Now I'm starting to think that this isn't possible. Or i'm just going through a mind block right now. Would you happen to know an easier way of having Add record, View records, Search record, Delete record, and update record? I'm trying to make a basic CRUD program without having the need to use SQL so I thought that maybe a CSV could suffice. I didn't forsee these problems I'm having now when I thought about it :(.

For now I'm still thinking how I would code the logic given to me by very helpful posters including you. Thank you guys!

-Michael 

[Michael Rivera]

Oh and if you're curious to see where I'm add right now I only have the basic functions of add and view. Ha ha here's the code

http://play.golang.org/p/d-XFK2aSii

[Gyepi SAM]

CSV is not the simpler alternative to SQL. An in-memory database is:

http://play.golang.org/p/COprNs3Hx5

Once you understand this, you can then translate it to CSV.

This brings to mind a quote from Jon Bentley regarding amateur astronomy.
To make a six inch mirror, it is easier to first make a four inch mirror.

-Gyepi

[Shawn Milochik]

On Fri, Apr 18, 2014 at 11:09 AM, Gyepi SAM <self-...@gyepi.com> wrote:

CSV is not the simpler alternative to SQL. An in-memory database is:

http://play.golang.org/p/COprNs3Hx5

Maybe I'm missing something you're trying to teach in the context of the CSV question. But wouldn't a map of [int]Student be better, giving you an immediate O(1) lookup vs an O(n) list iteration, and replacing a copy() of a list with a delete call on the map key? Yes, it duplicates the ID by storing it in the Student and the map key, but I think it's worthwhile.

 

[Gyepi SAM]

On Fri, Apr 18, 2014 at 05:09:21PM +0200, Gyepi SAM wrote:
> This brings to mind a quote from Jon Bentley regarding amateur astronomy.
> To make a six inch mirror, it is easier to first make a four inch mirror.

Turns out that I paraphrased Bentley's quote of Thomson's Rule for First-Time Telescope Makers:

"It is faster to make a four-inch mirror then a six-inch mirror than to make a six-inch mirror."

Presumably, that would be Allyn J. Thompson.

-Gyepi

[James Bardin]

On Friday, April 18, 2014 11:17:16 AM UTC-4, Shawn Milochik wrote:

Maybe I'm missing something you're trying to teach in the context of the CSV question. But wouldn't a map of [int]Student be better, giving you an immediate O(1) lookup vs an O(n) list iteration, and replacing a copy() of a list with a delete call on the map key? Yes, it duplicates the ID by storing it in the Student and the map key, but I think it's worthwhile.

The constant in that O(1) is large. Slices are faster than maps in many cases, esp with smaller amounts of data.

[Gyepi SAM]

True, but a slice has a more direct concept mapping to CSV, which will be
necessary when he does the conversion. Just thinking ahead ;)

Also, we haven't talked about maps and I wanted to stay in the same realm.

-Gyepi

[Michael Rivera]

I'm trying to learn using CSV cause it's more of a portability issue wherein you can just bring around the csv file. I understand the concept of what you posted but I'm wondering where the logic going in CSV comes in? I'm guessing that the logic you're trying to teach me is scanning the whole csv then comparing it to what you're looking for? Out of curiousity I am also wondering what maps are? I read from http://www.golang-book.com/6 that maps function similarly with arrays? I'm coming from a java perspective so maps and slices are new to me. Sorry for the numerous questions once again! I'm really liking golang as a programming language and I want to move on but I can't until I get this problem over with. I'll try to read more when I wake up, it's 2am here. Have a pleaseant weekend everyone!

-Michael

[Shawn Milochik]

On Fri, Apr 18, 2014 at 1:22 PM, Michael Rivera <mikeep...@gmail.com> wrote:

I'm trying to learn using CSV cause it's more of a portability issue wherein you can just bring around the csv file. I understand the concept of what you posted but I'm wondering where the logic going in CSV comes in?

In Go, writing to a CSV is just writing a slice to the writer. So code that deals with slices is 1:1 with dealing with CSV data.

All you need to do to write a slice to CSV is:

writer := csv.NewWriter(os.Stdout)

write.Write(mySlice)

 

[Michael Rivera]

Hey Shawn, 

I understand writing and reading very well. I'm trying to expand to the possibilities of searching through a csv with say a given value under "ID" for example. I'm looking into everyone's suggestions and I think the best option would be sorting then comparing the whole document then returning with the record with the common ID. My problem now is translating that to code (which I will be studying when I wake up) feel free to put a sample of it though, I need all the help I could get :) Good night for now everyone 

-Michael

[Shawn Milochik]

On Fri, Apr 18, 2014 at 1:38 PM, Michael Rivera <mikeep...@gmail.com> wrote:

Hey Shawn, 

I understand writing and reading very well. I'm trying to expand to the possibilities of searching through a csv with say a given value under "ID" for example. I'm looking into everyone's suggestions and I think the best option would be sorting then comparing the whole document then returning with the record with the common ID. My problem now is translating that to code (which I will be studying when I wake up) feel free to put a sample of it though, I need all the help I could get :) Good night for now everyone 

Okay. Well, then Gyepi already answered it. Read the file, checking each record for a match to "search," and write another copy of the file, skipping lines you want to "delete." You're trying to treat a CSV file like a database, which it's not. 

I don't know what your application is supposed to do. But if you really want to use a CSV file as a database, you could read it into an in-memory SQLite3 database, let your app do its thing, and write an updated copy of the CSV when your app is done. There are a lot of problems with that. The biggest problem is that your app is the only thing that can touch the file -- any other changes will be lost.

If you explain what the real task is maybe we can be of more help. It sounds like you're setting an arbitrary limitation of needing to use CSV because it's portable, but it really sounds like an x/y problem, and I'd like to know what y is.

Shawn 

[Michael Rivera]

Hello, sorry for the late reply. Well I'm trying to do make an app that would have the basic CRUD functions (create, read, update, delete, and search) without having to use a database such as SQL. User input won't be from an interface but from the console itself. I thought that a CSV would be good enough but then realized that it would have problems updating deleting and searching. Any help at this point would be very much appreciated :(

[Shawn Milochik]

On Fri, Apr 18, 2014 at 9:23 PM, Michael Rivera <mikeep...@gmail.com> wrote:

Hello, sorry for the late reply. Well I'm trying to do make an app that would have the basic CRUD functions (create, read, update, delete, and search) without having to use a database such as SQL. 

What is the benefit of "not having to" use a database? CRUD is basic database stuff. 

[Michael Rivera]

I wanted to get the knowledge of structs down like having an arraylist from java not using a database and just saving data there. Data doesn't have to be saved once the program has been terminated. I was thinking of maps and structs. CSV came in because I just wanted to write what data I wrote with the basic CRUD function after I use it (sort of like and export to csv thing). I'm sorry if this sounds weird but it's what I did when I learned java and python so I'm just following my lessons there. 

[Michael Rivera]

I don't have my python codes right now but here is what I'm trying to achieve

import numpy
a = numpy.asarray([ [1,2,3], [4,5,6], [7,8,9] ])
numpy.savetxt("foo.csv", a, delimiter=",")

Basically I want to make an array of user input data (with different fields like a record)  through the console and save it in an array and then write the array on a CSV file. Other than building up the array of multiple records, I want to know how to search within that array to find a certain record and update/delete the certain record. I'm sorry for the weird logic and questions

[Tamás Gulácsi]

How do you search your numpy array? Use for cycle!