text mode

[Mel]

Dan,

I looked at "current_textfile" and "current_opts" as you suggested
a while ago.  It seems the basics for text output are all there and
it just needs a good user interface.  A problem I have with the
current implementation is that there doesn't appear to be a way to
change 'current_opts' or 'current_textfile' for a running "etmqt"
(or am I missing something).

I thought of several kludges around this, like forcing the main
program to check for a new "etm.cfg" (or "usr.cfg" which btw doesn't
appear to be documented anywhere), letting a 'current_textfile'
that begins with a '|' be a pipe to a program (instead of a file),
or even having some sort of "remote control" to direct the main
program (kind of like in modern browsers).

But then I noticed the main() program in etmData.py and decided
to copy it and create my own little python program that I am
attaching to this post.  The only thing it does differently is
override 'current_opts' with it's command line args and sets
'current_textfile' to "/dev/stdout".

If main() or get_options() in etmData.py were modified to allow
arguments to override 'current_textfile' and 'current_opts', it
could easily be incorporated in etm_qt.py or etmQt/etmData.py
directly.

[Daniel Graham]

Comments below.

On Mar 4, 2013, at 12:22 AM, Mel <meln...@gmail.com> wrote:

Dan,

I looked at "current_textfile" and "current_opts" as you suggested
a while ago.  It seems the basics for text output are all there and
it just needs a good user interface.  A problem I have with the
current implementation is that there doesn't appear to be a way to
change 'current_opts' or 'current_textfile' for a running "etmqt"
(or am I missing something).

The current files including current_textfile are set in ~/.etm/etm.cfg which can, of course be edited using any text editor. current_opts, on the other hand, is set in the report view which is the user interface for creating reports.

I thought of several kludges around this, like forcing the main
program to check for a new "etm.cfg" (or "usr.cfg" which btw doesn't
appear to be documented anywhere), letting a 'current_textfile'
that begins with a '|' be a pipe to a program (instead of a file),
or even having some sort of "remote control" to direct the main
program (kind of like in modern browsers).

~/.etm/etm.cfg is read whenever etm starts and also when any changes are made in etm using Shift-Ctrl-O. There is no usr.cfg since any user setting you like can be set in etm.cfg. 

But then I noticed the main() program in etmData.py and decided
to copy it and create my own little python program that I am
attaching to this post.  The only thing it does differently is
override 'current_opts' with it's command line args and sets
'current_textfile' to "/dev/stdout".

If main() or get_options() in etmData.py were modified to allow
arguments to override 'current_textfile' and 'current_opts', it
could easily be incorporated in etm_qt.py or etmQt/etmData.py
directly.

Yes, I this wouldn't be difficult and, in fact, there would be no need to override current_textfile since the output could be sent to stdout directly.

In your attachment you call getViewData and then print the tree that corresponds to tag view but, if I understand you correctly, you should be calling getReportData with a string giving a report specification and then printing the tree that corresponds to this report. Am I correct in thinking that you would like to be able to give a report specification on the command line and have the resulting report sent to stdout?

-Dan

[Dan]

Mel,

2.0.124 is out and provides command line report generation. Please let me know if this is what you had in mind.

-Dan

[Mel]

Dan,

My point about changing "current_textfile" is that it appears
a change to "current_textfile" in "etm.cfg" doesn't take effect
in a running "etmqt".  But now with the new text mode, it doesn't
matter to me.

Also btw, if I have a "./etm/usr.cfg" which is empty
(or all lines commented out), I get an error in etmData.py:
    'NoneType' object is not iterable)
for
    options.update(custom_options)

Need,
    if custom_options:
        options.update(custom_options)

Yes I was confused about getViewData vs getReportData as I've never
used either of these before.  For my simple usage they seemed to
give the save results.  But yes you are correct that I want a report.

Perhaps also due to my misunderstanding I can't seem to get
"-s summary" to work (even in the gui).

In a ".txt" file I have the line:

        ^ Open Source Private Clouds MU @s Mar 14 2013 @k NYLUG

"c ddd MMM d yyyy -k NYLUG -w 40" shows me (correctly):

        ^ Open Source Private Clouds MU

But "c ddd MMM d yyyy -s Clouds -w 40"

doesn't find anything and just outputs a bunch of "res False"
lines to the terminal.  How do I use "-s summary" for a report?

I downloaded the latest 2.0.124 and the new command line report
feature looks great.  Much thanks, it is just what I had in mind.
And btw I have the same problems as above when I enter:
        c ddd MMM d yyyy -s Cloud

[Mel]

Following up on my previous post, I noticed the new text mode
options to etm_qt.py was taking about 4 times longer than my
previous kludge, for example when I did:

        etm-2.0.124/etm_qt.py c c ddd MMM d yyyy -k nylug

Looking at the source code to see why, in etm-2.0.124/etmQt/etmData.py
function main() has,

        rows, busytimes, busydays, alerts, dates = getViewData(bef, file2uuids, uuid2hash, options=options)

which unless it has some side effects that I'm not aware of,
appears not to be needed.  At least in my simple tests I get
the same output without it.

Also for kicks I tried,

    etm_qt.py a ddd MMM d yyyy -k NYLUG -w 40

and it failed with "ValueError: max() arg is an empty sequence"
in etmData.py mk_rep().  It probably should catch the case when
there are no actions (when someone does something dumb like that :-).

[Mel]

I have "events" (*) and "occasions" (^).  When I do a report for one day
it just shows the occasions.  And when I do a report that covers two or
more days, I see  "events" and "occasions" for all days EXCEPT the last
day.  For example:
    etm_qt.py c c ddd MMM d yyyy -b +0 -e +1
shows the occasions and events for the first day but only the occasions
for the second day.  This happens in the gui also.

[Daniel Graham]

From the report help tab:

-e END_DATE

Fuzzy parsed date. Limit the display of dated items to those with datetimes falling before this datetime. As with BEGIN_DATE relative month expressions can be used so that, for example, -b -1/1 -e +1 would include all items from the previous month. Default: None.

Displaying occasions (and tasks) which have starting times at midnight on the '-e' date is a bug and will be fixed in the next release. You can use, e.g., '-e +1 11:59p' if you want to include all items with starting times before 11:58pm tomorrow.

--
You received this message because you are subscribed to the Google Groups "Event and Task Manager" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eventandtaskman...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Dan]

comments below

Also btw, if I have a "./etm/usr.cfg" which is empty
(or all lines commented out), I get an error in etmData.py:
    'NoneType' object is not iterable)
for
    options.update(custom_options)

Need,
    if custom_options:
        options.update(custom_options)

usr.cfg was never documented and was intended for a completely different purpose that overriding the etm.cfg settings. It will be removed in the next release.

Perhaps also due to my misunderstanding I can't seem to get
"-s summary" to work (even in the gui).

In a ".txt" file I have the line:

        ^ Open Source Private Clouds MU @s Mar 14 2013 @k NYLUG

"c ddd MMM d yyyy -k NYLUG -w 40" shows me (correctly):

        ^ Open Source Private Clouds MU

But "c ddd MMM d yyyy -s Clouds -w 40"

doesn't find anything and just outputs a bunch of "res False"
lines to the terminal.  How do I use "-s summary" for a report?

This is a bug and will be fixed in the next release. Thanks for spotting it.

[Daniel Graham]

comments below:

Following up on my previous post, I noticed the new text mode
options to etm_qt.py was taking about 4 times longer than my
previous kludge, for example when I did:

        etm-2.0.124/etm_qt.py c c ddd MMM d yyyy -k nylug

Looking at the source code to see why, in etm-2.0.124/etmQt/etmData.py
function main() has,

        rows, busytimes, busydays, alerts, dates = getViewData(bef, file2uuids, uuid2hash, options=options)

which unless it has some side effects that I'm not aware of,
appears not to be needed.  At least in my simple tests I get
the same output without it.

The script you previously attached makes the same call

//

if messages:

    for msg in messages:

        print(msg)

rows, busytimes, busydays, alerts, dates =  \

                getViewData(bef, file2uuids, uuid2hash, options=options)

//

By 'kludge' do you mean something other than this script?

There is no way to avoid calling getViewData that I can imagine.  What I added (in ETMCmd.mk_rep) is a call to getReportData which combines the output from getViewData with the grouping and filters from the provided report specification to create the report.

The call to getReportData could be omitted but then the only output that would be possible is that which would be displayed in one of the gui views, e.g., day view, without any option of changing the grouping, applying any filters or limiting the display period. If you want to generate a report that uses such options, then I don't see how the getReportData call can be avoided. 

By the way, if you use 'etm_qt.py l' to start the etm shell loop, then the overhead of loading the data is only incurred once when starting the shell. Entering report specifications at the prompt incurs no further overhead. 

Also for kicks I tried,

    etm_qt.py a ddd MMM d yyyy -k NYLUG -w 40

and it failed with "ValueError: max() arg is an empty sequence"
in etmData.py mk_rep().  It probably should catch the case when
there are no actions (when someone does something dumb like that :-).

Good idea. This will be fixed in the next release.

Also planned for the command line in the next release:

1. An 's' option that will print the 'schedule view' that would be generated by Ctrl-S in the gui.

2. An 'm' option for using items from your report_specifications file. Entering '? m' or 'help m' would give a numbered list of your report specifications from this file and entering, e.g., 'm 3' would produce a report using the 3rd report specification from the list.

-Dan

[Mel]

    > The script you previously attached makes the same call

    > By 'kludge' do you mean something other than this script?

Yes I meant the script.  But as you pointed out I mistakenly called
getViewData when I really wanted getReportData.  After your response
I changed it to do that and noticed it was much faster.

    > The call to getReportData could be omitted but then the only output
    > that would be possible is that which would be displayed in one of
    > the gui views, e.g., day view, without any option of changing the
    > grouping, applying any filters or limiting the display period. If
    > you want to generate a report that uses such options, then I don't
    > see how the getReportData call can be avoided.

An option to just get the limited display would be nice,
(but it's ok as it is now too).

    > By the way, if you use 'etm_qt.py l' to start the etm shell loop,
    > then the overhead of loading the data is only incurred once when
    > starting the shell.

Yes, I realize that and it brings up an idea.
Instead of a separate "etmqt.py l" maybe just the gui "etmqt.py"
could open a socket for input of commands and output of results
and command line text mode would just be to write to and read
from it?  But feel to ignore this suggestion as my usage is atypical
and things are fine for me as they are now.  I was just thinking
how browsers let you do some things via "remote control" rather
than with the mouse or keyboard.

[Mel]

Something changed somewhere between etm-2.0.124 and etm-2.0.127

that effects my command line usage.  With the following .txt data file:

* Test 1 @s Mar 7 2013 2am @e +0 @r d &u Mar 15 2013 @d Mar 14 2013

* Test 2 @s 1 10pm @e +0 @a 10 @r m &w MO,TU,WE,TH,FR

* Test 3 @s Fri 7:30pm @e +3h0m @a 75 @r w

^ Test 4 @s Fri @r w

"etm-2.0.124/etm_qt.py c ddd MMM d yyyy -w60 -b+0 -e+1" gives:

--------------------------------------------------------------------

Fri Mar 8 2013

  ^ Test 4

  * Test 1                                                   2:00am

  * Test 3                                                   7:30pm

  * Test 2                                                   10:00pm

--------------------------------------------------------------------

But, "etm-2.0.127/etm_qt.py c ddd MMM d yyyy -w60 -b+0 -e+1" gives:

--------------------------------------------------------------------

Fri Mar 8 2013

  * Test 1                                                   2:00am

  * Test 2                                                   10:00pm

--------------------------------------------------------------------

Is something broken or what do I have to do to see all four lines now?

(BTW this was run on Friday Mar 8 about 10pm.)

[Daniel Graham]

Yes, I broke this trying to fix the "events and occasions" issue raised earlier. I'll post a fix today.

[Daniel Graham]

This should be fixed in 2.0.131.

-Dan

[Mel]

I now see all my events and occasions in text command line mode.  Thanks.

But I encounter an encoding problem for a ".txt" entry like:

    * + Columbia Café Science @s 1 6pm @e +0 @r m &w 2MO

and the command line,

    etm_qt-2.0.132/etm_qt.py c ddd MMM d yyyy -w60 -b mon -e+2

It occurs at etm_qt-2.0.132/etmQt/etmData.py", line 3997, in term_print
with the message:
        UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9'
        in position 18: ordinal not in range(128)

My 'term_encoding' is 'UTF-8' and I seem to get by fine with just,
        print(s)
instead of,
        print(str(s).encode(term_encoding))

But I assume you have that there for other cases?

BTW, what is the best way to test things?  I've been creating
a separate 'datadir' for a '.txt' file with lines to test
and setting the "datadir:" line in etm.cfg  to it.  Then it
seems I also have to remember to remove the '.etmdata.pkl" file.
Is there a better way?

[Mel]

Some more about "term encoding".  The problem I'm having with the line:

    * + Columbia Café Science @s 1 6pm @e +0 @r m &w 2MO

seems to have been introduced somewhere between etm-2.0.127 and etm_qt-2.0.132;

"etm_qt-2.0.132/etm_qt.py c ddd MMM d yyyy -w60 -b+0 -e+1" gives me:
    Traceback (most recent call last):
      File "etm_qt-2.0.132/etm_qt.py", line 24, in <module>
    . . .
      File "/home/jaym/python/etm/etm_qt-2.0.132/etmQt/etmData.py", line 3997, in term_print
        print(str(s).encode(term_encoding))

    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 18: ordinal not in range(128)

While "etm-2.0.127/etm_qt.py c ddd MMM d yyyy -w60 -b+0 -e+1" correctly outputs:
    Mon Mar 11 2013
      * + Columbia Café Science                                  6:00pm

[Daniel Graham]

Mel,

I'll release 2.0.134 today which, I truly hope, fixes all unicode python 2 vs 3 issues. Here is a sample of the command line output with your Columbia Café Science example included:

------------------------------------------------

Mon Mar 11 2013

  * Monday event                         9:00am

  * twice a day reminder                 2:00pm

  * once a day reminder                  3:00pm

  * twice a day reminder                 4:00pm

  * Columbia Café Science                6:00pm

Tue Mar 12 2013

  * Tuesday event                        10:00am

Wed Mar 13 2013

  * Wednesday event                      12:00pm

Thu Mar 14 2013

  * Thursday event                       1:00pm

  - Overdue skip task                    

  - Overdue keep task                    

  - Overdue restart task                 

Fri Mar 15 2013

  * Friday event                         9:00am

  * Tête-à-têtes with staff              2:00pm

Sat Mar 16 2013

  * Saturday event                       1:00pm

Sun Mar 17 2013

  ^ St Patrick's Day                     

------------------------------------------------

and here is part of the same output but under German:

Mo. Mär 11 2013

  * Monday event                         9:00am

  * twice a day reminder                 2:00pm

  * once a day reminder                  3:00pm

  * twice a day reminder                 4:00pm

  * Columbia Café Science                6:00pm

Di. Mär 12 2013

  * Tuesday event                        10:00am

Your question about testing is an excellent one - I wish I had as good an answer. You can see from the output above that my test data files have a wide range of entries aimed at catching everything I can think of. "Tête-à-têtes with staff", e.g., is my unicode test. The main difficulty has been getting things to work under both python 2.7 and python 3.3 with OS X and Linux. My drill is to run my test data under command line and GUI with python 2 and 3 on OS X 10.8 and Ubuntu 12.4 and check all the output and displays. 

I seem to recall that you're the one that suggested porting to python 3 in the first place. I may eventually appreciate that suggestion. ;-)

-Dan

[Mel]

Thanks, etm qt 2.0.134 does fix my unicode problem with "Columbia Café".

But you seem to have left "locale_test" in etm_qt-2.0.134/etmQt/etmData.py
    locale_test = (("de_DE","UTF-8"), QLocale.German, QLocale.Germany)
which caused failure and gave me:
      File "/usr/lib/python2.7/locale.py", line 539, in setlocale
        return _setlocale(category, locale)
    locale.Error: unsupported locale setting
Things seem to work fine when I have:
    locale_test = ()

I think I just suggested keeping python 3 in mind.
I was actually surprised that PyQt4 ran under python 3
and that you did get etmQt to do so also.  I'm sure
eventually you will be happy you did.

[Daniel Graham]

Oops, I'll remove German for the next release.

Actually, I'm already happy to have things working, when they do, under python 3.

-Dan

[Dan]

In case anyone else is interested in the command line "text mode",  here is the current usage:

Usage:

    etm_qt.py [arguments]

With no arguments, etm will use setting from the
configuration file ~/.etm/etm.cfg and open the GUI.

If the first argument is the path to a directory
which contains a file named etm.cfg, then settings
from that file will be used instead.

If the first argument, other than the optional path,
is either "?", "a", "c", "m", "s" or "l" (lower case
L), then the remaining arguments will be executed by
etm without opening the GUI.

- ?: display (this) command line help information.
- a: display an action report using the remaining
     arguments as the report specification.
- c: display a custom report using the remaining
     arguments as the report specification.
- m: display a report using the remaining argument,
     which must be a positive integer, to display a
     report using the corresponding entry from the
     file given by report_specifications in etm.cfg.
     Use ? m or m ? to display the numbered list of
     entries from this file.
- s: display the next few days from the day view
     combined with any items in the now and next
     views. This command uses no further arguments.
- l: begin an interactive shell in which the above
     commands are available and can be adjusted and
     run again without reloading the data files.

[Mel]

I was wondering why in text mode 'end_date' was a date
and not an extent like for "@e" for data and thought
I could just use fuzzy date parsing instead.  But
although something like "-e 3/14/13 +2" didn't give
an error, the "+2" was ignored.

For this case, in "etm_qt-2.0.138/etmQt/etmData.py" function
parse_datetime, debugging shows:

        dt=' 3/14/13 +2'
        days=2
        new_date='2013-03-16'           ## rel_date_regex.sub(new_date, dt)
        new_dt=' 3/14/13 2013-03-16'

It seems that 'rel_date_regex' is just handling something like "-e +2",
but it wouldn't be hard to handle something like "-e 3/14/13 +2" also.
In addition to fixing 'rel_date_regex' it would also be necessary to
use what comes before the "+" to set 'now' instead of the current time.
Would this break anything I'm not aware of?

Also, while testing out the new option to set etmdir on the command line,
I happen to test it with a directory that didn't have a ".txt" file
and got a fault:

  File "/home/jaym/python/etm/etm_qt-2.0.138/etmQt/etmData.py", line 3624, in updateCurrentFiles
    if not txt[0]:
IndexError: list index out of range

Probably should check for when someone does something dumb like that :-).

[Daniel Graham]

Comments below:

> I was wondering why in text mode 'end_date' was a date
> and not an extent like for "@e" for data and thought
> I could just use fuzzy date parsing instead. But
> although something like "-e 3/14/13 +2" didn't give
> an error, the "+2" was ignored.

Both 3/14/13 and +2 (two days from today) both represent dates. Either would be accepted but not both.

> For this case, in "etm_qt-2.0.138/etmQt/etmData.py" function
> parse_datetime, debugging shows:
>
> dt=' 3/14/13 +2'
> days=2
> new_date='2013-03-16' ## rel_date_regex.sub(new_date, dt)
> new_dt=' 3/14/13 2013-03-16'
>
> It seems that 'rel_date_regex' is just handling something like "-e +2",
> but it wouldn't be hard to handle something like "-e 3/14/13 +2" also.
> In addition to fixing 'rel_date_regex' it would also be necessary to
> use what comes before the "+" to set 'now' instead of the current time.
> Would this break anything I'm not aware of?

rel_date_regex handles many common use situations. For example:

1) "-b -1/1 -e 1": the previous month

2) "-b 1 -e +1/1": the current month

3) "-b +0 -e +7": the next seven days

4) "-b -7 -e +0: the past seven days

The date calculator (bound to F5), on the other hand, would convert "6/1 8a + 4d3h" to "2013-06-05 11am". If I understand you correctly, what you would like is for something like this behavior to also be possible for -b and -e. Correct? Since what you propose doesn't seem so easy to me, could you give an example or two of how this would be useful? In your example, just entering "-e 3/16/13" seems simpler to me than "-e 3/14/13 +2".

> Also, while testing out the new option to set etmdir on the command line,
> I happen to test it with a directory that didn't have a ".txt" file
> and got a fault:
>
> File "/home/jaym/python/etm/etm_qt-2.0.138/etmQt/etmData.py", line 3624, in updateCurrentFiles
> if not txt[0]:
> IndexError: list index out of range
>
> Probably should check for when someone does something dumb like that :-).

I'll fix this for the next release.

Thanks,
Dan

[Daniel Graham]

Thinking more about your relative date suggestion, would the following combination of old and new do what you want?

"-1/1": the first day of the previous month
"-1/1 + 2d7h": 7am on the 3rd day of the previous month
"mon + 7d": the 2nd monday on or after today
"+7": seven days from today

The general format would be

fuzzy_date [ (+|-) period string]

If so, it would still be nice to have an example of how this would be useful.

Thanks,
Dan

[Daniel Graham]

As you suggested, it turned out to be fairly easy. The following will be possible for -b and -e in 2.0.142:

> "-1/1": the first day of the previous month
> "-1/1 + 2d7h": 7am on the 3rd day of the previous month
> "mon + 7d": the 2nd monday on or after today
> "+7": seven days from today
>

> The general format is

>
> fuzzy_date [ (+|-) period string]

Note that "+2", for example, can now be regarded as a shortcut for "+0 + 2d".

-Dan

[Mel]

Dan,

Thanks, the changes for -b and -e in 2.0.142 work great.