Questions about iCommand

[David Benaben]

Hello,

I'm a newbie using irods icommand and I would like to share some
questions and impressions with you.
I use irods 3.3 (soon 3.3.1 and soon as possible 4.x).

Perhaps I missed somme points and maybe it's not the right place to
discuss about that but I will be happy to have your feedback.

* It's hard to know the size of a file.
I mean "ils -l" returns size in bytes and in case of a big file, it's
tricky.
I would appreciate options like "-h, --human-readable" to print sizes in
human readable format (e.g., 1K 234M 2G) or "-b,-k,-m,-g" to show the
output in bytes, KB, MB, or GB.
What do you think about that?

* Output Description
I believe that for some commands or options a description (eg field
names) would be great, for example with "ils -l".

* More explicit output?
With the iput command, if a file already exists, you need to force
overwrite.
It's not very important but i think that the error
"OVERWRITE_WITHOUT_FORCE_FLAG" deserves to be more clear (eg "File
already exists, please force overwrite with -f option").

* It's hard to know the size of a directory.
When I first used the solution, I wanted to check that my local
directory was the same in irods.
A basic check would be to confirm the directory size (local directory
size ~= irods collection size).
But i couldn't find a simple command like 'du' to estimate space usage
for a directory (with or without replication).

* File exists but is empty
When a transfer fails, a file can still be created (but has a size of
'zero' byte).
It was surprising for me and i think it's error prone.
Is there something in the configuration that I can change ?

* Check replicas
In the same way, i wanted to check that all my replicas were created.
In our instance, there is a rule to automatically create three replicas
(with acPostProcForPut).
I can't find an easy way to check these replicas (i use commands like
"ils -l ./dir/ | wc -l" or "ils -l ./dir/ | cut -d ' ' -f 29 | sort -n |
uniq -c | grep -v '^ 3'").
May be there is a better way to do that ?

I also have some errors but I will write another email to avoid a
massive text...

Hope to hear from you,
Kind regards,

--
David Benaben

[Jean-Yves Nief]

hello,

David Benaben wrote:
> * It's hard to know the size of a directory.
> When I first used the solution, I wanted to check that my local
> directory was the same in irods.
> A basic check would be to confirm the directory size (local directory
> size ~= irods collection size).
> But i couldn't find a simple command like 'du' to estimate space usage
> for a directory (with or without replication).

iquest "%s" "select sum(DATA_SIZE) where COLL_NAME like 'full_path%'"
the '%' is important if you want to count all the sub collections.

>
> * Check replicas
> In the same way, i wanted to check that all my replicas were created.
> In our instance, there is a rule to automatically create three
> replicas (with acPostProcForPut).
> I can't find an easy way to check these replicas (i use commands like
> "ils -l ./dir/ | wc -l" or "ils -l ./dir/ | cut -d ' ' -f 29 | sort -n
> | uniq -c | grep -v '^ 3'").
> May be there is a better way to do that ?

you can use iquest for this. For example, you can try this:
iquest --no-page "%s/%s %s" "select
COLL_NAME,DATA_NAME,count(DATA_REPL_NUM)"
cheers,
JY

[Bernardo Augusto García Loaiza]

Hi David, I've some time this document about of the iCommand, with the more representative commands that appear in the help in the cli.
https://docs.google.com/document/d/1yaqsqX7g4c4K28WLVz5tR5YSn5158BO-MHrUZZnfQnU/edit?usp=sharing

This document have comments in spanish language :( , but I want share with you just in case of this be useful for you.

Regards.

Bernardo Augusto García Loaiza
Ingeniero de Sistemas
Estudiante de Maestría en Ingeniería Informática - Universidad EAFIT

http://about.me/bgarcial

--
--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution"   https://www.irods.org

iROD-Chat:  http://groups.google.com/group/iROD-Chat

--- You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Benaben]

Hi all,

Thanks Bernardo and JY for your reply.
I'll try to learn more about iquest. The command is not as "simple" as I expected but it's interesting. Thanks.

Best regards,

-- 
David Benaben

To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat

---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.

To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.

[Terrell Russell]

Hi David,

Welcome!

I've replied inline...

On Wednesday, October 15, 2014 9:05:52 AM UTC-4, David Benaben wrote:

Hello,

I'm a newbie using irods icommand and I would like to share some
questions and impressions with you.
I use irods 3.3 (soon 3.3.1 and soon as possible 4.x).

Perhaps I missed somme points and maybe it's not the right place to
discuss about that but I will be happy to have your feedback.

* It's hard to know the size of a file.
I mean "ils -l" returns size in bytes and in case of a big file, it's
tricky.
I would appreciate options like "-h, --human-readable" to print sizes in
human readable format (e.g., 1K 234M 2G) or "-b,-k,-m,-g" to show the
output in bytes, KB, MB, or GB.
What do you think about that?

I've created a new issue for this: https://github.com/irods/irods/issues/2384

 

* Output Description
I believe that for some commands or options a description (eg field
names) would be great, for example with "ils -l".

I've created a new issue for this: https://github.com/irods/irods/issues/2385

 

* More explicit output?
With the iput command, if a file already exists, you need to force
overwrite.
It's not very important but i think that the error
"OVERWRITE_WITHOUT_FORCE_FLAG" deserves to be more clear (eg "File
already exists, please force overwrite with -f option").

I've created a new issue for this: https://github.com/irods/irods/issues/2383

 

* It's hard to know the size of a directory.
When I first used the solution, I wanted to check that my local
directory was the same in irods.
A basic check would be to confirm the directory size (local directory
size ~= irods collection size).
But i couldn't find a simple command like 'du' to estimate space usage
for a directory (with or without replication).

JY answered above.

 

* File exists but is empty
When a transfer fails, a file can still be created (but has a size of
'zero' byte).
It was surprising for me and i think it's error prone.
Is there something in the configuration that I can change ?

This is a remnant of being a distributed system that needs better locking.  We are actively investigating wholesale changes that would 'fix' this issue.  Yes, we agree, this is confusing and should change.  Unfortunately, there is no workaround or configuration at this time.

 

* Check replicas
In the same way, i wanted to check that all my replicas were created.
In our instance, there is a rule to automatically create three replicas
(with acPostProcForPut).
I can't find an easy way to check these replicas (i use commands like
"ils -l ./dir/ | wc -l" or "ils -l ./dir/ | cut -d ' ' -f 29 | sort -n |
uniq -c | grep -v '^      3'").
May be there is a better way to do that ?

JY answered above. 

Thank you!  Keep the questions coming.

Terrell

RENCI