upcoming new release 2.2.0

[hjebbers]

I am preparing a new version 2.2.0
overview of changes: see http://code.google.com/p/bots/issues/list?can=1&q=label%3Aversion2.2.0
this version is very upward compatible: no database changes, all existing configurations should work the same way
my tests all run OK now, but I still want to rerun all tests.
To my surprise performance of the translation process is much better; stress tests with lots of input/big input files are 2-3 times faster (in these tests there files are read from system).

Let me know if you have changes that should be included in this new versions.

Not included in the new version is the option of running bots-engine parallel.
This change is high on my list, but turns out to be quite a big change (->Bots 3.0.).

kind regards,
henk-jan

[BikeMike]

On Mar 30, 11:14 pm, hjebbers <hjebb...@gmail.com> wrote:
> Let me know if you have changes that should be included in this new
> versions.

Hi henk-jan,
I think most of my changes you have already mentioned and are in the
issues list.

There are a few others not in current source or on the issues list:
1. Alternate botslogo configurable in bots.ini
2. Run menu requires superuser (not sure if I added this or you have
removed it, maybe this is just relevant to me)
3. webserver prints start info to console, used for testing

Another thing I have been thinking about but not done any work for, it
would be good to have more granular control of security. Particularly
I would like to give non-superusers access to maintain specific
codelists. Also I don't understand the difference between "Active" and
"Staff status", it seems you need both to do anything. So there are
only 2 levels really, normal staff user and super user. The user
permissions are of no use as they relate to specific tables an the
admin interface, not specific bots functions. Maybe this topic is
something else for v3.

Kind Regards,
Mike

[BikeMike]

I looked more closely at user security, I understand it better now :-)

I see that "staff status" actually means "access to configuration
menu" but everything is access denied by default. Then you can give
add/change/delete access to specific tables. Looks like these are just
every table in the database, it's not clear what effect some of those
would have. I would like to get more specific than that, giving access
to a few codelists, but I can see this will be difficult. I would also
like to control who can access the various Run menu options.

Kind Regards,
Mike

[BikeMike]

started a new thread on security so as not to clutter this one.

[hjeb...@gmail.com]

On 03/31/2012 02:48 AM, BikeMike wrote:
> On Mar 30, 11:14 pm, hjebbers<hjebb...@gmail.com> wrote:
>> Let me know if you have changes that should be included in this new
>> versions.
> Hi henk-jan,
> I think most of my changes you have already mentioned and are in the
> issues list.
>
> There are a few others not in current source or on the issues list:
> 1. Alternate botslogo configurable in bots.ini

I agree with the feature. It is important to be able to better distinguish different environments in bots-monitor.
but.....I think this cna be done easier.
now one has to edit the picture. Editing pictures is not very nice, this should be easier.
an option would be to include an (optional?) marker to the right of the bots-logo.
this can be text, maybe it is possible to include html?

> 2. Run menu requires superuser (not sure if I added this or you have
> removed it, maybe this is just relevant to me)
> 3. webserver prints start info to console, used for testing

you are right, I will do that.

[BikeMike]

Hi henk-jan,
I agree it is not so easy to edit the logo for this, but the finished
result looks neater I think. As well as text you might want to add
your own company logo etc. It would be possible to do it with html,
perhaps overlay the right hand side of the logo with another div in
html? I will have a look at this.

Kind Regards,
Mike

[hjeb...@gmail.com]

On 04/02/2012 12:23 AM, BikeMike wrote:
> Hi henk-jan,
> I agree it is not so easy to edit the logo for this, but the finished
> result looks neater I think. As well as text you might want to add
> your own company logo etc.

yes, I know.
but IMHO this is different case: it is possible to replace the picture itself.
where ar talking about different environments for the same company

> It would be possible to do it with html,
> perhaps overlay the right hand side of the logo with another div in
> html? I will have a look at this.

that would be great!
I do do mind having both solutions...

henk-jan
> Kind Regards,
> Mike
>

[hjeb...@gmail.com]

On 03/31/2012 02:48 AM, BikeMike wrote:

> 3. webserver prints start info to console, used for testing

I just checked this change in.

henk-jan

[BikeMike]

Hi henk-jan
I've developed this further (hope you haven't done anything with this
yet!). I think this is better as it avoids html code in bots.ini.

Instead, botslogo is separated into it's own html template and included
in base.html. You can take a copy of botslogo.html to another file,
customise it to your needs, and specify it's name in bots.ini.
botslogo_html filter gets the name to include.

With a few tweaks in the html it works as a standalone file for testing
in a browser (see comments in botslogo-test.html)

bots.ini
----------
[webserver]
# Replace botslogo.html with a customised one
botslogo=bots/botslogo-test.html
----------

Kind Regards,
Mike

[BikeMike]

previous one doesn't work.

[hjeb...@gmail.com]

looks very good!

trying to find out an 'easy' option: display a text from bots.ini on the right side of the bots logo.

henk-jan

[BikeMike]

> trying to find out an 'easy' option: display a text from bots.ini on the right side of the bots logo.

That can be done in exactly the same way. Create a new filter in
botsfilters.py to get the text, use the filter in a html template
somewhere.

Kind Regards,
Mike

[BikeMike]

> That can be done in exactly the same way. Create a new filter in
> botsfilters.py to get the text, use the filter in a html template
> somewhere.

bots.ini
----------
[webserver]
botslogo_text=Mike's Test System
----------

[hjeb...@gmail.com]

yes, that should do the trick.

henk-jan

[hjeb...@gmail.com]

hi Mike,

looks like this is quite tricky, as it is dependent upon the resulotuin of the screen (on my screen it looks quite different, the text is not as nice displayed as on your screenshot).

henk-jan

[BikeMike]

Yes, I found the text position slightly lower down on my home pc than
on my work pc. Nots sure if it's screen resolution, browser
differences or font differences. That is why I left all the
positioning, font size etc, in the html and not in css. But it still
could be a problem with multiple users and different setups.

With a separate logo.html that you copy and modify, you still have the
option of renaming and editing the image if that works better.

Kind Regards,
Mike

[BikeMike]

maybe because it's specified in pixels, should be em's?

[hjeb...@gmail.com]

yes.
I never work with pixels, it give a much more screen-size independent layout.

attached is a screenshot where the environment is in text.
this can be changed in bots.ini.

other things, like changes in the logo are not environment related (but per installation).

I did this in (larger) black letters, to have the layout not to flashy.

hen-jan

[BikeMike]

hi henk-jan,
try these, using em instead of px

I tried in IE and firefox on several PCs, all now look the same or very
close.

Kind Regards,
Mike

[hjeb...@gmail.com]

On 04/03/2012 02:04 PM, BikeMike wrote:

> previous one doesn't work.

hi mike,

I checked this in

kind regards,
henk-jan

[hjeb...@gmail.com]

hi Mike,

tested bots in django 1.1.4
everything seems to work, but the botslogo is not displayed :-((

all changes are in the repository
(rename directory template1.1 to template,
rename media/js1.1 to media/js)

I do not think this is related to javascript issues: javascript is not used.

any idea?

henk-jan

[hjeb...@gmail.com]

On 04/05/2012 03:35 AM, BikeMike wrote:

if I have in base.html:
{% include "bots/botslogo.html" %}
instead of
{% include "bots/botslogo.html"|botslogo_html %}
it does work.

so it might be filter-related?

henk-jan

[BikeMike]

Hi henk-jan,
It would seem that a filter nested within an include block does not
work in Django 1.1, but does work in Django 1.3. I tried to find any
mention of this in the documentation but could not see it.

As a workaround, this works for both 1.1 and 1.3.

{% with "bots/botslogo.html"|botslogo_html as logo %}
{% include logo %}
{% endwith %}

Kind Regards,
Mike

[hjeb...@gmail.com]

hi Mike,

I have now:
{% if not is_popup %}

{% with "bots/botslogo.html"|botslogo_html as logo %}
{% include logo %}
{% endwith %}

<div id="environment">
<div>{{ ""|environment_text }}</div>
</div>
{% include "bots/menu.html" %}
{% endif %}

but that does not work here :-((
in bots.ini I have:
botslogo =

henk-jan

[BikeMike]

Yes. botslogo must be specified or omitted entirely.
botsglobal.ini.get only uses the default if the option is missing, not
if it has no value.

I don't really see this as a problem, but it could be "fixed" by
modifying the filter.

def botslogo_html(default_html):
return botsglobal.ini.get('webserver','botslogo',default_html) or
default_html

Kind Regards,
Mike

[hjeb...@gmail.com]

ok, I see.
I will use the default value in bots.ini, that works.
'problem' solved.

henk-jan

[BikeMike]

This is my modified version of grammarcheck.py

It allows you also pass the path to a grammar instead of editype and
messagetype; I use this to run a grammarcheck directly as a tool in my
text editor while creating grammars. Although the path is passed, it
must still be in the "correct" location. The code just splits up the
path to get editype and messagetype.

(I renamed it, but this could replace grammarcheck.py, it is backwards
compatible)

Kind Regards,
Mike

[hjeb...@gmail.com]

yes, will check this in as replacement/enhancement.

henk-jan

[BikeMike]

Hi henk-jan
I just did this small enhancement, this gives an easy way to run a
single route from the menu. This is good for testing, and also I can use
this to run "manual" routes that are active but not in default runs.

It is disabled by default as some users may have a large list of routes
and not want this functionality.

in bots.ini
-----------
[settings]
# Include an entry on the "Run" menu so that each active route can be
run individually. Default: False
menurunroutes=True
-----------

in bots_context.py build a list of active routes (line 26-34)
in menu.html add a run menu entry for each route (line 73-78)

Kind Regards,
Mike

[BikeMike]

also removed text-transform:capitalize; from bots.css
It was capitalizing my route names, which I did not want. Cannot see
any negative side effects of this.

[Eppye Bots]

hi MIke,

also very interesting for me ;-))

I sometimes use customized menu entries for customers.

I am on holidays this week, will look next week.

henk-jan

[BikeMike]

Small cosmetic improvement, I tried using <hr/> instead of ----- as a
menu divider (with some css added), I think this looks nicer? Also
slightly changed the order & grouping of systasks menu to make it more
logical.

Kind Regards,
Mike

[BikeMike]

Hi henk-jan,
Another small "cosmetic" change; to the dropdown (star icon) menus.

These menus are a mix of links and buttons, so the fonts and margins
varied a bit. This effect was worse in IE browser. Tried to fix this in
css without success, so instead I changed the html links on these menus
into buttons so now they look consistent. (7 lines changed in 4 files)

eg.
<li><a href="/detail/?outidta={{row.idta}}" class="botslink">{% trans
'View detail' %}</a></li>

change to
<li><button class="botsheader"
onclick="document.location.href='/detail/?outidta={{row.idta}}'; return
false;">{% trans 'View detail' %}</button></li>

Also I've been experimenting with the css to make the bots menus etc.
more consistent with the look-and-feel of Django admin. I will post that
as an alternate "theme" bots.css when I'm happy with it.

Kind Regards,
Mike

[BikeMike]

hi henk-jan,
This is my alternate bots.css file. I have made the bots menus look
similar to the django filter menus (screenshots attached). Also includes
hr menu dividers, dropdown menu as buttons, fixed botsheader background
to transparent, was white. I have tested in IE and Firefox.

Also I have a suggestion about writing plugins, but not sure how to do
it. I would like the browser to offer the plugin for download
immediately after creating it. As a workaround now I change the plugin
filename to save it in /media/plugins folder I created, then browse to
/media/plugins/myplugin_xxxxxxxx.zip to download it. pluglib.plugoutcore
still needs to write to local filesystem though, as I use this in daily
housekeeping scripts for backups.

Kind Regards,
Mike

[BikeMike]

Write plugin - download via the browser:

I made a couple of small changes to get this to work.
1. forms.py class PlugoutForm
Make filename a hidden field
Initialize it to botspath/media/plugin.zip
2. views.py def plugout
Redirect to /media/plugin.zip instead of showing message

I decided it was simpler to use a fixed filename rather than adding the
date. This means you won't slowly build up a collection of plugins in
the media folder that you need to clean up; it is just overwritten each
time you create a new plugin. When the browser offers "plugin.zip" for
opening/saving you can then save it in any location with with any name
you want.

Kind Regards,
Mike

[BikeMike]

Hope I'm not bombarding you with too many changes... let me know if
you want me to stop! :-)

Kind Regards,
Mike

[BikeMike]

Admin list sorting by multiple columns:

This is another issue that has bugged me for a while now... but I found
a solution. Awesome! http://djangosnippets.org/snippets/2110/

I just had to add default ordering for the first time before any headers
are clicked. I have applied this to ccode, confirmrule, routes,
translate in admin.py. These had ordering on multiple fields that did
not work.

Django 1.4 apparently fixes the default ordering, but special ordering
is still useful.

Kind Regards,
Mike

[henk-jan ebbers]

hi Mike,

it looks like there is also a solution for this in django 1.4
(I tested with django 1.4. Did some changes in code (think these a uncommented for now), it works but not all css-files are used (so layout is not always nice)
for now I think it is not good to solve something that will be solved in future.

waht do you think about this?

kind regards,
henk-jan

[henk-jan ebbers]

great change!

[henk-jan ebbers]

the plugin is saved in the 'media' directory.
it used to be saved in the botssys directory.
this was because botssys is always readable/writeable, and contains all 'data'. (this is more importatn in linux...)

is it possible to use the 'old place' (fixed name is OK)

kind regards,
henk-jan

On 05/08/2012 02:26 AM, BikeMike wrote:

[BikeMike]

On May 19, 1:25 am, henk-jan ebbers <eppye.b...@gmail.com> wrote:
> the plugin is saved in the 'media' directory.
> it used to be saved in the botssys directory.
> this was because botssys is always readable/writeable, and contains all 'data'. (this is more importatn in linux...)
>
> is it possible to use the 'old place' (fixed name is OK)

Hi henk-jan,
It must be in a location that can be accessed from the browser with a
URL. I found that anything in the media folder could be directly
accessed with a URL. Maybe setting change in Django could allow it to
be put in in botssys. I will have a look at this.

For the default ordering change, I'm happy for it to be left out if
it's fixed in Django 1.4. I can just keep it as a local change until I
upgrade.

Kind Regards,
Mike

[BikeMike]

Solved this.
* plugin is created as plugin_temp.zip in the botssys folder,
overwritten each time.
* download filename is initialised to plugin_<date>.zip, of course you
can change this and select where to save it.

Kind Regards,
Mike

[henk-jan ebbers]

yes, looks very good.
sure I do wnat to use this!

only thing I can see:
the menu's are somewhat 'grey' (instead of black).
this is often indication of 'disabled'.
any reason to not use black instead of grey?

kind regards,
henk-jan

[BikeMike]

Hi henk-jan,
I copied the same colours from the Django filter menus, but it could
be made a bit darker. The light text contrasts with the darker blue
when the mouse hovers over an option.
In fact you could have various "Bots Themes" available as plugins. The
original colours with background highlights could be used too (I will
create a new css for that)

Kind Regards,
Mike

[BikeMike]

Attached are 3 css variant "themes" as plugins.
Kind Regards,
Mike

[henk-jan ebbers]

hard to choose ;-)
but there is one issue I noticed:
when I use chrome, the font size of the dropdown menu looks bigger.
when I remove the 'font-size: 110%; ' it looks smaller.
it is OK when I use 'font-size: 100%; '

how does this work for IE?

for me, I would prefer 'Plugin bots theme - menus like django but darker'

kind regards,
henk-jan

[henk-jan ebbers]

hi Mike,

another way to implement this is to have an extra section in bots.ini with run-menu's.

advantage would be that is gives more room to experiment with different run option.
disadvantage is that is not automatic (if you add a route, it will not be automatic in the menu)

how do you think about this?

kind regards,
henk-jan

[BikeMike]

On May 21, 10:54 pm, henk-jan ebbers <eppye.b...@gmail.com> wrote:
> hard to choose ;-)
> but there  is one issue I noticed:
> when I use chrome, the font size of the dropdown menu looks bigger.
> when I remove the 'font-size:  110%; ' it looks smaller.
> it is OK when I use 'font-size:  100%; '

Hi henk-jan,
This was an attempt to make the main menus and "star" menus look the
same size... it looks about right in Firefox; in IE the main menus are
slightly larger. If you want to leave it as 100% I am not worried.

I think the idea of extra menu items configured in bots.ini is good.
Maybe a special value (all active routes) or manually configured as an
alternative?

Kind Regards,
Mike