GridFS pymongo delete files?

[kevin]

is it possible to delete files based on metadata stored? for ex something like:

fs.remove({'hits':{'$lt':5}})

[Michael Dirolf]

Currently the delete helper takes an ObjectId. So to do something like
that you'd have to loop through the query result and delete each file
individually. We could implement a helper for that, but the driver
will actually have to do the same step internally so it won't be any
more efficient. Feel free to open a case if it's something you'd like.

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>

[Valery]

Hi

Let me ask even simpler pymongo questions:

Q1: is it possible just to remove all files from a collection?
(it is possible to get file names, but how to transform them into
corresponding ObjectID?)

Q2: is there any "wipe" method that removes fast the whole GridFS
collection without a need to iterate slowly from a client?

thanks,

Valery

[Valery]

OK, as for

> Q2: is there any "wipe" method that removes fast the whole GridFS
> collection without a need to iterate slowly from a client?

the following seems to work:

db.drop_collection('test_collection.chunks')
db.drop_collection('test_collection.files')

--
Valery

[Eliot Horowitz]

Q1 - not sure what you mean.  do you mean removing all files that match a certain query?

--

[Valery Khamenya]

On Mon, Sep 6, 2010 at 4:58 AM, Eliot Horowitz <elioth...@gmail.com> wrote:

Q1 - not sure what you mean.  do you mean removing all files that match a certain query?

yes, and query should be an equivalent to "all files" 

regards

Valery

[Eliot Horowitz]

If you want to delete all files you should just drop the fs.files and fs.chunks collection.

To do other deletes has to be file by file so the meta data and data collections are cleaned.

--

[Joseph Wang]

MongoConnnection: add server ip-10-160-86-4 at port 10000
MongoConnnection: add server ip-10-161-50-230 at port 10000
...


SEVERE: switched to: ip-10-160-86-4:10000 but isn't master
Sep 7, 2010 12:58:04 AM com.mongodb.DBTCPConnector _pickInitial
SEVERE: can't pick initial master, using random one
com.mongodb.MongoException: can't find master
        at com.mongodb.DBTCPConnector._pickInitial(DBTCPConnector.java:422)
        at com.mongodb.Mongo.<init>(Mongo.java:177)

[Valery Khamenya]

On Tue, Sep 7, 2010 at 4:37 AM, Eliot Horowitz <elioth...@gmail.com> wrote:

If you want to delete all files you should just drop the fs.files and fs.chunks collection.

well, as for my Q2, I've found the answer already:

http://groups.google.com/group/mongodb-user/msg/7cd8bcde26cfe300?hl=en

But how to get an ObjectId from filename is still not clear.

regards

Valery

 

[Kristina Chodorow]

What does your code look like?  Are you calling "new Mongo(ip1, ip2)"?  If you're connecting to a slave, you should just say "new Mongo(ip1)".

[Michael Dirolf]

A lot of these operations become pretty clear when looking at the
underlying structure of a GridFS collection, even if there aren't
helper methods designed specifically for each task. In your case you'd
just query the fs.files collection for {"filename": your_filename},
which would return all matching documents. You could then get the _id
of one (or all) of the files with the matching name.

[Joseph Wang]

            String[] servers = server_list.split(",");
            ArrayList<ServerAddress> addr = new ArrayList<ServerAddress>();
            int serverCount = 0;
            for (int i = 0; i < servers.length; ++i) {
                String[] serverInfo = servers[i].split(":");

                try {
                    if (serverInfo.length == SERVER_INFORMATION_FIELD_SIZE) {
                        System.out.println("MongoConnnection: add server "
                                + serverInfo[SERVER_NAME_FIELD] + " at port "
                                + serverInfo[SERVER_PORT_FIELD]);
                        ServerAddress host = new ServerAddress(
                                serverInfo[SERVER_NAME_FIELD],
                                Integer.parseInt(serverInfo[SERVER_PORT_FIELD]));
                        addr.add(host);

                        serverCount++;
                    }
                } catch (Exception ex) {

                }
            }

            if (serverCount > 0) {
                m = new Mongo(addr, opt);
            }

---

From: Kristina Chodorow <kris...@10gen.com>
To: mongod...@googlegroups.com
Sent: Tue, September 7, 2010 7:16:19 AM
Subject: Re: [mongodb-user] Java Driver Failed to Connect to Slaves

[Kristina Chodorow]

If you pass in a list of servers, one must be master.  If you just want to connect to a slave, do "new Mongo(new ServerAddress(ip))".

[Joseph Wang]

we want to connect a list of replication servers. will open a jira.

---

From: Kristina Chodorow <kris...@10gen.com>
To: mongod...@googlegroups.com

Sent: Tue, September 7, 2010 7:54:01 AM

[Scott Hernandez]

The latest java driver (2.1) will be able to connect to the master of the replicaset if you provide any server in the set. If the master changes the driver will find the new master as well.

If you want to be able to query non-master nodes (slaves) automatically you will have to wait for version 2.2 or until these issues are resolved: http://jira.mongodb.org/browse/JAVA-147, http://jira.mongodb.org/browse/JAVA-144

Is this what you are asking for?

[Joseph Wang]

Similar, but not the same issue. All the servers in our connection are slaves.

---

From: Scott Hernandez <scotthe...@gmail.com>
To: mongod...@googlegroups.com
Sent: Tue, September 7, 2010 8:32:51 AM

--

[Scott Hernandez]

Yeah, that isn't going to work very well; I think you are going to have to wait for a new driver release. 

What do you expect to happen when you give all slave addresses? How do you plan to use those connections? What is the goal of this, to do queries against the slaves only (no writes)?

[Joseph Wang]

There is no write to slaves.

We've one master and multiple slaves. All indexes have background indexing enabled.
We send bulk updates/inserts to master. We want to have fast query performance.

---

From: Scott Hernandez <scotthe...@gmail.com>
To: mongod...@googlegroups.com

Sent: Tue, September 7, 2010 9:00:36 AM

[Scott Hernandez]

Okay, as I said, you most likely want this http://jira.mongodb.org/browse/JAVA-147 (or something very close to this when it is done).

Slave queries in the current driver are not supported well.

In the future you will be able to supply a list of servers (master and slaves, or a replicaset) and the driver can choose the master for writes, and the slaves for queries. The driver is not there yet. It *will* be a supported option.

[Valery Khamenya]

On Tue, Sep 7, 2010 at 4:26 PM, Michael Dirolf <mi...@10gen.com> wrote:

A lot of these operations become pretty clear when looking at the
underlying structure of a GridFS collection, even if there aren't
helper methods designed specifically for each task. In your case you'd
just query the fs.files collection for {"filename": your_filename},
which would return all matching documents. You could then get the _id
of one (or all) of the files with the matching name.

Hi, Michael, thanks!