test 1.7.0 please
Catherine Devlin
(Speaking of PyCon, online registration ends TODAY!)
... so I'd love some quick testing help on sqlpython 1.7.0, the fully-armed and operational (but buggy) battle station. This is the version that works with postgreSQL and MySQL as well as Oracle. Go us!
cd ~
hg clone http://hg.assembla.com/python-cmd2 cmd2
cd cmd2
sudo python setup.py develop
cd ~
hg clone http://hg.assembla.com/sqlpython sqlpython
cd sqlpython
sudo python setup.py develop
(or, if you've already done that, cd ~/cmd2; hg pull; cd ~/sqlpython; hg pull)
Andy has released Gerald 0.3.5, which is what we needed for the cross-RDBMS stuff. However, it needs a couple patches to work completely, which I just emailed to him a few minutes ago so you'd better apply them to your own Gerald for now. Here's what you do.
easy_install -UZ gerald
cd /usr/local/lib/python2.6/dist-packages/gerald/gerald (actually, I don't remember the exact path name)
cp ~/sqlpython/*.patch .
patch schema.py schema.patch
patch oracle_schema.py oracle_schema.patch
Now go play with your databases.
sqlpython --postgres mydatabasename myusername
sqlpython --mysql mydatabasename myusername
Let me know what you find! You can send bug reports to me by email, or use the trac page: http://trac-hg.assembla.com/sqlpython/
Thanks! See you in Atlanta!
--
- Catherine
http://catherinedevlin.blogspot.com/
*** PyCon * Feb 17-25, 2010 * Atlanta, GA * us.pycon.org ***
Menno Smits
> ... so I'd love some quick testing help on sqlpython 1.7.0, the fully-armed
> and operational (but buggy) battle station. This is the version that works
> with postgreSQL and MySQL as well as Oracle. Go us!
Your setup instructions were spot on.
> sqlpython --postgres mydatabasename myusername
Works for me for some definition of "works". I can connect to
a Postgresql DB and the cat command works. Many commands (ls, grep,
find, describe) don't appear to do anything. Still, this is a big step
forward. Awesome!
> Let me know what you find! You can send bug reports to me by email, or use
> the trac page: http://trac-hg.assembla.com/sqlpython/
Hmmmm, I created an Assembla account and logged in to Trac but I don't seem
to have permissions to create tickets.
Menno
Catherine
>
> > sqlpython --postgres mydatabasename myusername
>
> Works for me for some definition of "works". I can connect to
> a Postgresql DB and the cat command works. Many commands (ls, grep,
> find, describe) don't appear to do anything. Still, this is a big step
> forward. Awesome!
>
everything that's in your own schema (one with the same name as the
logged-in user). But, if the objects are in any other schema
(including the public schema), it doesn't see them.
Andy invented a new Gerald class called User in oracle_schema.py that
knows not only about its owner's own objects, but also about all other
database objects that the user has access to. To make sqlpython work
across schemas, we'll need the analogous class created in
postgres_schema.py, too. That might be too much of a job to ask Andy
to squeeze into Gerald 0.3.5.1 over these next few days. For the
moment, we might just have to add the caveat that all the metadata
goodies only work under postgreSQL when you connect as the schema
owner.
We might need that for MySQL, too. I don't know MySQL well enough to
know if it even *has* separate schemas.
> > Let me know what you find! You can send bug reports to me by email, or use
> > the trac page:http://trac-hg.assembla.com/sqlpython/
>
> Hmmmm, I created an Assembla account and logged in to Trac but I don't seem
> to have permissions to create tickets.
I hadn't thought of that. I think, if you tell me your assembla
account name, I can assign you permissions. (I tried putting "menno"
in "@editors", so if that's your username, maybe it'll work for you
now.)
Thanks!
Devin Venable
$ sqlpython
Traceback (most recent call last):
File "/usr/local/bin/sqlpython", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 2562, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 626, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 524, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: gerald>=0.3.5
If I just start up python, import gerald, and help(gerald), I see that I'm working with the correct version of gerald.
FILE
/usr/local/lib/python2.6/dist-packages/gerald-0.3.5-py2.6.egg/gerald/__init__.py
Any quick tips? I don't mess around with dist-packages and not sure why the version check is failing. Running a pretty basic Ubuntu 9.10 here...
Devin
--
You received this message because you are subscribed to the Google Groups "sqlpython" group.
To post to this group, send email to sqlp...@googlegroups.com.
To unsubscribe from this group, send email to sqlpython+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlpython?hl=en.
Menno Smits
> Hmm... which schema are your objects in? ``ls`` etc. should show you
> everything that's in your own schema (one with the same name as the
> logged-in user). But, if the objects are in any other schema
> (including the public schema), it doesn't see them.
That could be the issue then. All the tables in this DB are in a
different tablespace to the user's name.
That said, I've just created a new DB in a tablespace with the same
name as the user and I'm still seeing the same problem. I don't have
time to debug this further right now. I'll try to have a look later
on.
> Andy invented a new Gerald class called User in oracle_schema.py that
> knows not only about its owner's own objects, but also about all other
> database objects that the user has access to. To make sqlpython work
> across schemas, we'll need the analogous class created in
> postgres_schema.py, too. That might be too much of a job to ask Andy
> to squeeze into Gerald 0.3.5.1 over these next few days. For the
> moment, we might just have to add the caveat that all the metadata
> goodies only work under postgreSQL when you connect as the schema
> owner.
Understood.
> > > Let me know what you find! �You can send bug reports to me by email, or use
> > > the trac page:http://trac-hg.assembla.com/sqlpython/
> >
> > Hmmmm, I created an Assembla account and logged in to Trac but I don't seem
> > to have permissions to create tickets.
>
> I hadn't thought of that. I think, if you tell me your assembla
> account name, I can assign you permissions. (I tried putting "menno"
> in "@editors", so if that's your username, maybe it'll work for you
> now.)
It's mjs0 (obviously!).
Cheers,
Menno
Catherine Devlin
I just made an attempt to run through the tests, but hit a snag. I used easy_install to install gerald which dropped an egg in the dist-packages directory. I unzipped it, patched the files, re-zipped and ran sqlpython and got this exception:
(wait... setup.py distribute is supposed to check on that and do the upgrade, I thought...)
If you've been mucking with multiple versions of Python on one machine, it's also possible to accidentally have easy_install refer to (say) easy_install-2.5 when you're running python2.6. You could try explicitly running /usr/bin/easy_install-2.6.
My packaging-fu is weak, too; those are just two guesses.
Devin Venable
--
Catherine Devlin
That said, I've just created a new DB in a tablespace with the same
name as the user and I'm still seeing the same problem. I don't have
time to debug this further right now. I'll try to have a look later
on.
Uhhh... are postgreSQL tablespaces the same as schemas?
> I hadn't thought of that. I think, if you tell me your assemblaIt's mjs0 (obviously!).
> account name, I can assign you permissions. (I tried putting "menno"
> in "@editors", so if that's your username, maybe it'll work for you
> now.)
Thanks!
Menno Smits
> On Thu, Feb 11, 2010 at 12:04 PM, Menno Smits <me...@freshfoo.com> wrote:
>
> > That said, I've just created a new DB in a tablespace with the same
> > name as the user and I'm still seeing the same problem. I don't have
> > time to debug this further right now. I'll try to have a look later
> > on.
> >
>
> Uhhh... are postgreSQL tablespaces the same as schemas?
Uhhh, no. Sorry, got my wires crossed. I'll try again later :)
> > > I hadn't thought of that. I think, if you tell me your assembla
> > > account name, I can assign you permissions. (I tried putting "menno"
> > > in "@editors", so if that's your username, maybe it'll work for you
> > > now.)
> >
> > It's mjs0 (obviously!).
> >
> > OK, I added mjs0 to @editors... I think. Hopefully that will do the trick.
Thanks, that worked.
Menno
Devin Venable
sqlpython is working fine connecting to Oracle and issuing sql commands. I also did a cut and paste from a SQL log of my recent Oracle commands, pasted it directly into the sqlpython window, and ran the query. The good news: I was joining eight tables and the query contained two subqueries and...I got the right answer. The bad news: when I did the paste into the console, the console was very slow to complete the paste. I could see a line or two from the sql statement getting added every second. It took about 20 seconds to finish pasting. Once complete the query executed very quickly.
Looks like you are making great progress!
Devin
Catherine Devlin
sqlpython is doing what I would expect for the most part when using the mysql connection.
Hooray! I'm delighted to hear that. Thank you!
I'm able to connect, desc tables, and query for values. When using a native mysql client, I use the command 'show tables' to get a list of all tables. This doesn't seem to work in sqlpython.
Ah... great suggestion. 60-second fix, now in the trunk.
sqlpython is working fine connecting to Oracle and issuing sql commands. I also did a cut and paste from a SQL log of my recent Oracle commands, pasted it directly into the sqlpython window, and ran the query. The good news: I was joining eight tables and the query contained two subqueries and...I got the right answer. The bad news: when I did the paste into the console, the console was very slow to complete the paste. I could see a line or two from the sql statement getting added every second. It took about 20 seconds to finish pasting. Once complete the query executed very quickly.
Basically, I need to pipe each command through a big pyparsing jungle to figure out (among other things) whether the command is complete yet. If it isn't, I accept another line of input, append it to the now-longer command, and parse again. Repeat, with a bigger string to send to pyparsing each time. This Does Not Scale.
I don't think parsing just the final line can work, because there may be important information anywhere in the string.
SELECT name, -- ! ;;
comment as "what a ; great column; name;",
age /* Heaven knows
what ; could be \x in
comments */
FROM mytable
WHERE age >
25 \h > report.html
Figuring out that the \h is a terminator, the \x and ; are not, and the first > is part of the SQL but the second indicates output redirection? Ouch. pyparsing does it, but at the expense of those increasingly slow parses. (And a horrid-looking .parsed() method.)
There's a *partial* solution; sqlpython interprets REMARK BEGIN to mean "quit parsing now, I promise this is one big SQL statement until you hit a REMARK END statement". Bracketing your statement in those fixes the slow parsing problem. Great... but only if you *know* about it (and remember to use it, and your scripts use it).
I'm open for ideas... (though I doubt we can fix this by PyCon)
I suppose there could be a settable parameter, ``i_will_not_write_nasty_syntax_sql_just_parse_my_queries_fast_please``, that would assume the user. One problem there is that it would require a separate, simpler pyparsing grammar, which would have to be kept in synch with the primary (complex) one...
Hmm.
Catherine Devlin
On Feb 10, 6:41 pm, Catherine Devlin <catherine.dev...@gmail.com>
wrote:
snip
> Let me know what you find!
Hi Catherine
I've been testing sqlpython 1.7.0 and found this bug:
If the very first command I enter at the sqlpython prompt is just
<enter> or a comment (--) then it exits sqlpython. If the first
command is something else it works normally.
Ah, thank you for catching that! It's fixed in 1.7.1 (released today).
GHZ
Some things I noticed with the 1.7.0
1) If I start sqlpython, then press enter straight away, sqlpython
quits with no warning
2) I am connecting to oracle. I often see an error like: (maybe this
only happens when I connect as a DBA)
Traceback (most recent call last):
File "/opt/csw/lib/python2.6/threading.py", line 525, in
__bootstrap_inner
self.run()
File "/opt/csw/lib/python2.6/site-packages/sqlpython/
connections.py", line 276, in run
newgerald = gerald_classes[self.db_instance.rdbms]
(self.db_instance.username, self.db_instance.conn_data.gerald_uri())
File "/opt/csw/lib/python2.6/site-packages/gerald/schema.py", line
96, in __init__
self.schema = self._get_schema(self._cursor)
File "/opt/csw/lib/python2.6/site-packages/gerald/oracle_schema.py",
line 171, in _get_schema
schema[table_key] = Table(table_name, cursor, owner)
File "/opt/csw/lib/python2.6/site-packages/gerald/schema.py", line
314, in __init__
self._get_table(cursor)
File "/opt/csw/lib/python2.6/site-packages/gerald/oracle_schema.py",
line 290, in _get_table
raise AttributeError, "Can't get DDL for table %s" % uc_table_name
AttributeError: Can't get DDL for table _DEFAULT_AUDITING_OPTIONS_
3) running sqlpython from the command line with no arguments gives an
ugly exception.
4) After running sqlpython I am left with a large file called
'gerald.log'. Can logging be turned off?
5) redirecting output to a file, which has the same name as an
existing directory.. e.g. select 1 from dual; > name_of_existing_dir
results in sqlpython quitting with no warning.
Thanks,
Geraint.
On Feb 10, 7:41 pm, Catherine Devlin <catherine.dev...@gmail.com>
wrote:
> - Catherinehttp://catherinedevlin.blogspot.com/
Paul Moore
> 2) I am connecting to oracle. I often see an error like: (maybe this
> only happens when I connect as a DBA)
> Traceback (most recent call last):
> File "/opt/csw/lib/python2.6/threading.py", line 525, in
> __bootstrap_inner
> self.run()
> File "/opt/csw/lib/python2.6/site-packages/sqlpython/
> connections.py", line 276, in run
> newgerald = gerald_classes[self.db_instance.rdbms]
> (self.db_instance.username, self.db_instance.conn_data.gerald_uri())
> File "/opt/csw/lib/python2.6/site-packages/gerald/schema.py", line
> 96, in __init__
> self.schema = self._get_schema(self._cursor)
> File "/opt/csw/lib/python2.6/site-packages/gerald/oracle_schema.py",
> line 171, in _get_schema
> schema[table_key] = Table(table_name, cursor, owner)
> File "/opt/csw/lib/python2.6/site-packages/gerald/schema.py", line
> 314, in __init__
> self._get_table(cursor)
> File "/opt/csw/lib/python2.6/site-packages/gerald/oracle_schema.py",
> line 290, in _get_table
> raise AttributeError, "Can't get DDL for table %s" % uc_table_name
> AttributeError: Can't get DDL for table _DEFAULT_AUDITING_OPTIONS_
I can confirm this. One question that immediately comes to mind, is
> 4) After running sqlpython I am left with a large file called
> 'gerald.log'. Can logging be turned off?
I didn't see this however.
I note from the "gerald" website that the current version of gerald is
considered "alpha". Is there a way to "switch off" the use of gerald?
What features does it provide for sqlpython?
The setup.py for sqlpython requires setuptools. I hate setuptools and
avoid it whenever possible. Could setup.py be altered to at the very
least degrade gracefully if setuptools isn't present? The following
seems to do it:
try:
from setuptools import setup, find_packages
except ImportError:
from distutils.core import setup
def find_packages():
return ['sqlpython']
You lose generation of executables for the entry points, but I don't
want that anyway (my goal is to package sqlpython using py2exe).
(BTW, the same fix in the setup.py for cmd2 would also be welcome!
There's even less reason for the use of setuptools there, as it
doesn't use entry points).
Paul.
Catherine Devlin
On Mar 3, 12:02 pm, GHZ <geraint.willi...@gmail.com> wrote:
> raise AttributeError, "Can't get DDL for table %s" % uc_table_name
> AttributeError: Can't get DDL for table _DEFAULT_AUDITING_OPTIONS_
http://sourceforge.net/apps/trac/halfcooked/ticket/23
You're right, it only affects DBA-level users who can see the objects that Oracle itself creates with garbage names.
I can confirm this. One question that immediately comes to mind, is
> 4) After running sqlpython I am left with a large file called
> 'gerald.log'. Can logging be turned off?
I note from the "gerald" website that the current version of gerald is
considered "alpha". Is there a way to "switch off" the use of gerald?
What features does it provide for sqlpython?
Gerald does all the metadata connection; turning it off would be crippling.
The setup.py for sqlpython requires setuptools. I hate setuptools and
avoid it whenever possible. Could setup.py be altered to at the very
least degrade gracefully if setuptools isn't present? The following
seems to do it:
Okay, thanks for the patch! Incorporated on the trunk of sqlpython and cmd2.
Thanks,
--
- Catherine
Andy Todd
My bad, I had the logging level set to 'DEBUG' in the last release. It
will be fixed in the next release.
> I note from the "gerald" website that the current version of gerald is
> considered "alpha". Is there a way to "switch off" the use of gerald?
> What features does it provide for sqlpython?
Gerald is alpha, but I've been using it for about seven years. The
reason it *was* alpha is that the internals kept shifting. Now that the
internal API is locked down
(http://halfcooked.com/code/gerald/schema_api.html) and after the
thorough testing it has recently received I will be marking the next
release as Beta.
From there it should probably only be another seven or so years before
I release a 1.0 version ;-)
>
> The setup.py for sqlpython requires setuptools. I hate setuptools and
> avoid it whenever possible. Could setup.py be altered to at the very
> least degrade gracefully if setuptools isn't present? The following
> seems to do it:
>
> try:
> from setuptools import setup, find_packages
> except ImportError:
> from distutils.core import setup
> def find_packages():
> return ['sqlpython']
>
> You lose generation of executables for the entry points, but I don't
> want that anyway (my goal is to package sqlpython using py2exe).
>
> (BTW, the same fix in the setup.py for cmd2 would also be welcome!
> There's even less reason for the use of setuptools there, as it
> doesn't use entry points).
>
> Paul.
>
Regards,
Andy
--
From the desk of Andrew J Todd esq - http://www.halfcooked.com/