SQLAlchemy support

[Terrence Brannon]

It is understood that Reahl itself uses a database. And so a database

must be created But it is not understood how to configure what

database Reahl uses internally. For

example, when you start a Reahl app there are warnings about SQLite

being used making it impossible to be concurrent. So if one wanted

Reahl to use a postgres database instead, how would you do it? Did I

miss the docs on this?

More importantly, I now want to use Reahl's SQLAlchemy support to

persist my models. However, the 2 sections on persisting models:

- http://www.reahl.org/docs/3.1/tutorial/persistence.d.html

- http://www.reahl.org/docs/3.1/tutorial/housingmodels.d.html

do not discuss where to setup the engine connection info for

persisting my SQLAlchemy objects.

Finally, I find it unusual that a test for a Reahl app cannot use the

SQLAlchemy metadata that is used for the app itself to make its

connections. I would think it would be accessible to both. In fact,

anything else means the test might pass when the code might fail or

vice versa due to differences in test and live environment.

I have some suggestions for [this

page](http://www.reahl.org/docs/3.1/tutorial/persistence.d.html)

- where it says "a little Reahl help" it should like to the

  reahl.sqlalchemysupport docs.

- likewise when this same page says "some glue" that should also be a

  link to those docs.

- where it says "In a complete Reahl program, none of this database

  housekeeping is visible in code – Reahl provides tools that deal

  with it." there should be a link to the tools that deal with it.

[Craig Sparks]

Thanks Terrence,

You have raised some valid points which we shall address later.

The short answer would be to set the connect string in the config like so for reahl to use your postgresql database:

(Linux notation)
in your project directory edit(or create) etc/reahl.config.py:
change:
reahlsystem.connection_uri = 'postgresql://scott:tiger@localhost/mydatabase'

Use reahl-control as described here:
http://www.reahl.org/docs/3.1/tutorial/gettingstarted-examples.d.html#running-examples
to create the required user/db/tables etc. Reahl will use this for its internals, as well well as your domain.

Tip: To see a list of config you can set, you could run this command in your project directory:
reahl-control listconfig etc/

-Craig.

--
You received this message because you are subscribed to the Google Groups "Reahl discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reahl-discus...@googlegroups.com.
To post to this group, send email to reahl-...@googlegroups.com.
Visit this group at http://groups.google.com/group/reahl-discuss.
For more options, visit https://groups.google.com/d/optout.

[Terrence Brannon]

I appreciate your information on how to change the database that reahl uses internally. I would appreciate some information on how to make use of reahl's sqlalchemy support. I need to configure it. 

I looked through the code an dunderstand that the filename for this is sqlalchemy.config.py and the "config_key" is sqlalchemy.

However, does this file go in "etc/" also? 

I would like to see a complete example of using a database like postgres for the applications database... and I think most people doing serious work will want to see the same.

If nothing else, please supply such a config file and I should be able to take it from there as long as I know where it belongs.

[Iwan Vosloo]

He Terrence,

On 22/04/2015 03:39, Terrence Brannon wrote:

I appreciate your information on how to change the database that reahl uses internally. I would appreciate some information on how to make use of reahl's sqlalchemy support. I need to configure it.

I think you pointed out a glaring hole in the docs. We'll have to write something up.

Seems there are two issues here:

(a) how to use postgresql instead of sqlite; and
(b) something you want to configure regarding sqlalchemy

Craig answered (a).

But I don't understand what you need configurable about using sqlalchemy. There is nothing user configurable about reahl-sqlalchemysupport. (Some components, like sqlalchemysupport indeed have Configurations, but they only exist to do a bit of dependency injection behind the scenes, there are no config settings defined that you can set yourself.)

Finally, I find it unusual that a test for a Reahl app cannot use the

SQLAlchemy metadata that is used for the app itself to make its

connections. I would think it would be accessible to both. In fact,

anything else means the test might pass when the code might fail or

vice versa due to differences in test and live environment.

Are you talking about the tests in http://www.reahl.org/docs/3.1/tutorial/persistence.d.html ? This is just a simple test to show some basics, we normally do not run tests like that. This is easier to explain once you understand Fixtures: http://www.reahl.org/docs/3.1/tutorial/testing.d.html.

In short, some Fixtures are meant to be used as run fixtures - they're set up before all tests are run, and torn down after all tests have run. The two you'd be interested in are documented here: http://www.reahl.org/docs/3.1/devtools/tools.d.html (right at the top)

You'd generally want to use BrowserSetup (in inherits from CleanDatabase).

When you use a BrowserSetup as run fixture for a test, it will make sure all your database tables are created and all classes are instrumented before any test is run. It also connects to the database -- all based on the config read from ./etc. In short, it will execute all tests in the same environment your app also runs.

There is another Fixture, WebFixture which I now see is not documented. It is not a run fixture, see how it is used in:
 http://www.reahl.org/docs/3.1/tutorial/testing.d.html#testing-without-a-real-browser

Amongst other things, it ensures a transaction is started before each test and rolled back after each test.

I have some suggestions for [this

page](http://www.reahl.org/docs/3.1/tutorial/persistence.d.html)

- where it says "a little Reahl help" it should like to the

  reahl.sqlalchemysupport docs.

- likewise when this same page says "some glue" that should also be a

  link to those docs.

- where it says "In a complete Reahl program, none of this database

  housekeeping is visible in code – Reahl provides tools that deal

  with it." there should be a link to the tools that deal with it.

Thanks, noted. We appreciate all your feedback.

Regards
- Iwan

-- 
Reahl, the Python only web framework: http://www.reahl.org

[Terrence Brannon]

On Tuesday, April 21, 2015 at 11:47:41 PM UTC-7, Iwan Vosloo wrote:

He Terrence,

Hi Ivan :)

 

On 22/04/2015 03:39, Terrence Brannon wrote:

I appreciate your information on how to change the database that reahl uses internally. I would appreciate some information on how to make use of reahl's sqlalchemy support. I need to configure it.

I think you pointed out a glaring hole in the docs. We'll have to write something up.

Seems there are two issues here:

(a) how to use postgresql instead of sqlite; and
(b) something you want to configure regarding sqlalchemy

Craig answered (a).

No, Craig gave an answer to (a) regarding how to switch the database that Reahl uses internally for it's housekeeping. What I want is an answer to: "How can I allow Reahl to use SQLite for its internal tasks while I use postgresql for my application database usage"... 

If that is not possible, then I need to know how to use Postgresql for both my application and for Reahls internal housekeeping.

 

But I don't understand what you need configurable about using sqlalchemy.

You need to place the connection URL somewhere. Furthermore, there should be a way to flexibly switch between staging and production connection URLs.

[Iwan Vosloo]

Hi Terrence,

On 22/04/2015 12:08, Terrence Brannon wrote:

Seems there are two issues here:

(a) how to use postgresql instead of sqlite; and
(b) something you want to configure regarding sqlalchemy

Craig answered (a).

No, Craig gave an answer to (a) regarding how to switch the database that Reahl uses internally for it's housekeeping. What I want is an answer to: "How can I allow Reahl to use SQLite for its internal tasks while I use postgresql for my application database usage"... 

If that is not possible, then I need to know how to use Postgresql for both my application and for Reahls internal housekeeping.

Nope, that's not possible. There is only on database shared by your program and Reahl itself. The answer Craig gave thus applies to that one database.

 

But I don't understand what you need configurable about using sqlalchemy.

Furthermore, there should be a way to flexibly switch between staging and production connection URLs.

Anything that is different between different environments is put into configuration - thus, your ./etc directory. You would have a different ./etc in prod than you do in a staging environment.

[Terrence Brannon]

On Wednesday, April 22, 2015 at 4:58:59 AM UTC-7, Iwan Vosloo wrote:

Hi Terrence,

On 22/04/2015 12:08, Terrence Brannon wrote:

Seems there are two issues here:

(a) how to use postgresql instead of sqlite; and
(b) something you want to configure regarding sqlalchemy

Craig answered (a).

No, Craig gave an answer to (a) regarding how to switch the database that Reahl uses internally for it's housekeeping. What I want is an answer to: "How can I allow Reahl to use SQLite for its internal tasks while I use postgresql for my application database usage"... 

If that is not possible, then I need to know how to use Postgresql for both my application and for Reahls internal housekeeping.

Nope, that's not possible. There is only on database shared by your program and Reahl itself. The answer Craig gave thus applies to that one database.

I'm in something of a corporate environment and do not have the freedom to create database users and tables, etc. But the tables I need to run the application on are located in a MSSQL database. I've been manually handling database activity outside of the way Reahl does it. So SQLite is for Reahl and my application uses MSSQL.

 

-- 
Reahl, the Python only web framework: http://www.reahl.org

*a* Python-only web framework, others being:

- muntjac

- nagare

- flexx

- pyjs 

[Iwan Vosloo]

Hi Terrence,

On 22/04/2015 18:30, Terrence Brannon wrote:
>
> I'm in something of a corporate environment and do not have the
> freedom to create database users and tables, etc. But the tables I
> need to run the application on are located in a MSSQL database. I've
> been manually handling database activity outside of the way Reahl does
> it. So SQLite is for Reahl and my application uses MSSQL.

I see. I assume you would like to use something like Sqlalchemy's
support for vertical partitioning:
http://docs.sqlalchemy.org/en/latest/orm/persistence_techniques.html#simple-vertical-partitioning
?

We've never really played with that, although in principle it should be
possible to get it to work. Reahl manages database transactions for you,
and we also use nested transactions (implemented via savepoints via
sqlalchemy). My guess is that transaction handling across the two
backends may present some practical problems (like supporting two-phase
commit).

The bottom line is that we will need to think how to open up the
possibility for users to use the more advanced possibilities of
SQLAlchemy Session, mapper and engine configuration.

Regards
- Iwan

[Giuseppe Traficante]

Hi Craig,

I'm a new reahl's user and i think it's a good product,

but I can't configure it with Postgres, the steps i made was:

1. create a reahl virtualenv 
2. activate that virtual env
4. pip install reahl[declarative,postgresql,dev,doc]
6. reahl example tutorial.addressbook2
7. in this project doesn't exists reahl.config.py, so i've created this file in etc directory and inside I wrote one line reahlsystem.connection_uri = 'postgresql://myusername:password@localhost/mydb'
8. obviously myusername is a postgres user that have the grants to mydb database and password is its password
   
9. sudo reahl setup -- develop -N, i've added sudo because reahl can't acccess to some python directory else
10. reahl-control createdbuser etc, ok here is its output:
    

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

WARNING:reahl.component.config:web.default_http_port in etc/web.config.py is using a dangerous default setting: 8000

WARNING:reahl.component.config:web.encrypted_http_port in etc/web.config.py is using a dangerous default setting: 8363

WARNING:reahl.component.config:web.static_root in etc/web.config.py is using a dangerous default setting: /home/linuxuser/addressbook2

Traceback (most recent call last):

  File "/usr/local/bin/reahl-control", line 11, in <module>

    sys.exit(ProductionCommandline.execute_one())

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/shelltools.py", line 165, in execute_one

    return_code = commandline.execute_command(command, line, options, parser)

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/shelltools.py", line 178, in execute_command

    return self.command_named(command).do(line)

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/shelltools.py", line 107, in do

    return self.execute(options, args)

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/prodshell.py", line 128, in execute

    super(CreateDBUser, self).execute(options, args)

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/prodshell.py", line 44, in execute

    self.create_context(args[0])

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/prodshell.py", line 58, in create_context

    self.context.system_control = SystemControl(self.context.config)

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/dbutils.py", line 41, in __init__

    self.db_control = self.db_control_for_uri(self.connection_uri)

  File "/usr/local/lib/python2.7/dist-packages/reahl/component/dbutils.py", line 90, in db_control_for_uri

    raise CouldNotFindDatabaseControlException(url)

reahl.component.dbutils.CouldNotFindDatabaseControlException: postgresql://postgres:password@localhost/mydb

I can't understand where i'm wrong.

tnks in advance!

[Craig Sparks]

Hi Giuseppe,

we think Reahl is great too :->

Thanks for taking the time to provide the details of what you tried.

All your steps seem right, except step 9. sudo...
when you sudo, you exit the python environment created by virtualenv, and don't have access to all the python packges installed in that virtualenv. Your error suggest that the reahl-postgresqlsupport package is not installed, confirming that the sudo is causing issues.

Perhaps you could post the details of the error you get when you "reahl setup -- develop -N" (without the sudo). We need to fix that first.
Which OS are you running on?
If its Unix* like, could you run: which python in a shell where you have activated the virtualenv and post the output here?

Reahly,
Craig.

Visit this group at https://groups.google.com/group/reahl-discuss.

[Giuseppe Traficante]

Wow tnks for the quickly answer, 

i hope to start to develop asap using reahl!

ok, i've deleted virtualenv and project directories  and start from scratch:

1. create a reahl virtualenv 
   
2. activate that virtual env
   
3. pip install reahl[declarative,postgresql,dev,doc]
   
4. reahl example tutorial.addressbook2
   

1. reahlsystem.connection_uri = 'postgresql://myusername:password@localhost/mydb' -> etc/reahl.config.py 
   

1. reahl setup -- develop -N

I'm sorry i obtained another error, here is the error

REAHLWORKSPACE environment variable not set, defaulting to /home/linuxuser

Traceback (most recent call last):

  File "/home//reahl_env/bin/reahl", line 11, in <module>

    sys.exit(WorkspaceCommandline.execute_one())

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 165, in execute_one

    return_code = commandline.execute_command(command, line, options, parser)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/dev/devshell.py", line 582, in execute_command

    return self.command_named(command).do(line)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 107, in do

    return self.execute(options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/dev/devshell.py", line 371, in execute

    return super(ForAllParsedWorkspaceCommand, self).execute(options, self.saved_args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/dev/devshell.py", line 263, in execute

    results[i] = self.execute_one(i, options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/dev/devshell.py", line 205, in execute_one

    retcode = self.function(project, options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/dev/devshell.py", line 435, in function

    project.setup(args)

AttributeError: 'Project' object has no attribute 'setup'

I'm using ubuntu 14.04

(reahl_env) linuxuser@cpu:~$ which python

/home/linuxuser/reahl_env/bin/python

[Craig Sparks]

Hi Giuseppe,

After step 4, you should "cd addressbook2". This is where your newly created project(addressbook2) is.
Step 5 and 6 should be executed while in this directory.

The error is thrown because you are not in a project directory - I will log a bug for Reahl to report that more clearly :->

Great stuff, I think you are almost there!

Craig.

[Giuseppe Traficante]

I'm really sorry, in effect i don't know if executing the sudo command now i've compromised something, infact :

(reahl_env) linuxuser@CPU:~/addressbook2$ reahl setup -- develop -N

REAHLWORKSPACE environment variable not set, defaulting to /home/linuxuser

running develop

running egg_info

creating addressbook2.egg-info

writing requirements to addressbook2.egg-info/requires.txt

writing addressbook2.egg-info/PKG-INFO

writing namespace_packages to addressbook2.egg-info/namespace_packages.txt

writing top-level names to addressbook2.egg-info/top_level.txt

writing dependency_links to addressbook2.egg-info/dependency_links.txt

writing entry points to addressbook2.egg-info/entry_points.txt

writing manifest file 'addressbook2.egg-info/SOURCES.txt'

warning: manifest_maker: standard file 'reahl' not found

file setup.py (for module setup) not found

reading manifest file 'addressbook2.egg-info/SOURCES.txt'

writing manifest file 'addressbook2.egg-info/SOURCES.txt'

running build_ext

Creating /home/linuxuser/reahl_env/lib/python2.7/site-packages/addressbook2.egg-link (link to .)

Adding addressbook2 0.0 to easy-install.pth file

and finally executing reahl-control createdbuser etc

i've got the same error

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

WARNING:reahl.component.config:web.default_http_port in etc/web.config.py is using a dangerous default setting: 8000

WARNING:reahl.component.config:web.encrypted_http_port in etc/web.config.py is using a dangerous default setting: 8363

WARNING:reahl.component.config:web.static_root in etc/web.config.py is using a dangerous default setting: /home/linuxuser/addressbook2

Traceback (most recent call last):

  File "/home/linuxuser/reahl_env/bin/reahl-control", line 11, in <module>

    sys.exit(ProductionCommandline.execute_one())

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 165, in execute_one

    return_code = commandline.execute_command(command, line, options, parser)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 178, in execute_command

    return self.command_named(command).do(line)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 107, in do

    return self.execute(options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 128, in execute

    super(CreateDBUser, self).execute(options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 44, in execute

    self.create_context(args[0])

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 58, in create_context

    self.context.system_control = SystemControl(self.context.config)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/dbutils.py", line 41, in __init__

    self.db_control = self.db_control_for_uri(self.connection_uri)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/dbutils.py", line 90, in db_control_for_uri

    raise CouldNotFindDatabaseControlException(url)

reahl.component.dbutils.CouldNotFindDatabaseControlException: postgresql://myusername:password@localhost/mydb

[Craig Sparks]

No worries,

I'll try to help you.

could you execute the following and supply the output, while still in the directory addressbook2 (It will indicate to me whether the right package is installed in the virtualenv and provide some version info):

python -c 'import sys;print("%s -%s" % (sys.version,sys.prefix)); import pkg_resources; pkg_resources.require('reahl-postgresqlsupport')"


Craig.

[Giuseppe Traficante]

tnks a lot!

(reahl_env) linuxuser@CPU:~/addressbook2$ python -c 'import sys;print("%s -%s" % (sys.version,sys.prefix)); import pkg_resources; pkg_resources.require("reahl-postgresqlsupport")'

2.7.6 (default, Jun 22 2015, 17:58:13) 

[GCC 4.8.2] -/home/linuxuser/reahl_env

...

[Craig Sparks]

Sorry, I made a mistake. I forgot a print, could you you try again:

python -c 'import sys;print("%s -%s" % (sys.version,sys.prefix)); import pkg_resources; print(pkg_resources.require("reahl-postgresqlsupport"))'

Craig.

--

[Giuseppe Traficante]

python -c 'import sys;print("%s -%s" % (sys.version,sys.prefix)); import pkg_resources; print(pkg_resources.require("reahl-postgresqlsupport"))'

2.7.6 (default, Jun 22 2015, 17:58:13) 

[GCC 4.8.2] -/home/linuxuser/reahl_env

[reahl-postgresqlsupport 3.1.1 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), psycopg2 2.5.5 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), reahl-component 3.1.1 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), Babel 2.1.1 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), wrapt 1.10.6 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), six 1.10.0 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), python-dateutil 2.2 (/home/linuxuser/reahl_env/lib/python2.7/site-packages), pytz 2016.1 (/home/linuxuser/reahl_env/lib/python2.7/site-packages)]

tnks again!

...

[Craig Sparks]

Thanks,

It looks good.
Next please edit etc/reahl.config.py to look like this:

reahlsystem.connection_uri = 'postgresql://myusername:password@localhost/mydb'

print reahlsystem.databasecontrols

Then run:
reahl-control createdb etc

and show the output.

Craig.

--

[Giuseppe Traficante]

reahl-control createdb etc

Traceback (most recent call last):

  File "/home/linuxuser/reahl_env/bin/reahl-control", line 11, in <module>

    sys.exit(ProductionCommandline.execute_one())

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 165, in execute_one

    return_code = commandline.execute_command(command, line, options, parser)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 178, in execute_command

    return self.command_named(command).do(line)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 107, in do

    return self.execute(options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 154, in execute

    super(CreateDB, self).execute(options, args)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 44, in execute

    self.create_context(args[0])

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 53, in create_context

    self.context = ExecutionContext.for_config_directory(config_directory)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/context.py", line 56, in for_config_directory

    config.configure()

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/config.py", line 268, in configure

    self.read(ReahlSystemConfig)

  File "/home/linuxuser/reahl_env/local/lib/python2.7/site-packages/reahl/component/config.py", line 326, in read

    exec(compile(f.read(), file_path, 'exec'), globals(), locals_dict)

  File "etc/reahl.config.py", line 3

    print reahlsystem.databasecontrols

                    ^

SyntaxError: invalid syntax

maybe  should run only create table? in effect in posgres exists DB and user.

...

[Craig Sparks]

Giuseppe,

We (Iwan has joined in) see that there is an issue with your virtualenv because you are supposedly running python 2.7 in the virtualenv, yet using print without brackets causes a syntax error as it would if using Python3.

maybe  should run only create table? in effect in posgres exists DB and user.

- I understand what you are saying, but we are not there yet :->

We have found that some environment variables and older versions of virtualenv can cause similar issues to yours. To sort your issue out, we need you to:


Please edit etc/reahl.config.py again to look like this:

reahlsystem.connection_uri = 'postgresql://myusername:password@localhost/mydb'

print(reahlsystem.databasecontrols)
import sys
print(sys.prefix)
print(sys.path)


Then run:
reahl-control createdbtables etc ; echo $PATH ; virtualenv --version; pip --version; python -c 'import pkg_resources; print(pkg_resources.require("setuptools"))'

and show the output.

Craig.

--

[Giuseppe Traficante]

[<class 'reahl.postgresqlsupport.PostgresqlControl'>, <class 'reahl.component.dbutils.NullDatabaseControl'>]

/home/linuxuser/reahl_env

['etc', '/home/linuxuser/reahl_env/bin', '/home/linuxuser/addressbook2', '/home/linuxuser/reahl_env/lib/python2.7', '/home/linuxuser/reahl_env/lib/python2.7/plat-x86_64-linux-gnu', '/home/linuxuser/reahl_env/lib/python2.7/lib-tk', '/home/linuxuser/reahl_env/lib/python2.7/lib-old', '/home/linuxuser/reahl_env/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/home/linuxuser/reahl_env/local/lib/python2.7/site-packages']

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

WARNING:reahl.component.config:web.default_http_port in etc/web.config.py is using a dangerous default setting: 8000

WARNING:reahl.component.config:web.encrypted_http_port in etc/web.config.py is using a dangerous default setting: 8363

WARNING:reahl.component.config:web.static_root in etc/web.config.py is using a dangerous default setting: /home/linuxuser/addressbook2

/home/linuxuser/reahl_env/bin:/usr/bin/java_home/bin:/usr/bin/java_home/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

15.0.0

pip 8.1.0 from /home/linuxuser/reahl_env/local/lib/python2.7/site-packages (python 2.7)

[setuptools 20.2.2 (/home/linuxuser/reahl_env/lib/python2.7/site-packages)]

...

[Iwan Vosloo]

Hi Giuseppe,

Thanks for that info. It actually confirms that your virtualenv is fine,
you are running the correct python and all kinds of things we were
unsure about. We will need to dig a bit deeper now:

Can you please again send the output of running:

reahl-control createdbtables etc

But, using the following contents for etc/reahl.config.py

reahlsystem.connection_uri =
'postgresql://myusername:password@localhost/mydb'
print(reahlsystem.databasecontrols)

for i in reahlsystem.databasecontrols:
print(i.uri_regex_string)
print(i.matches_uri(reahlsystem.connection_uri))

Thanks!

[Iwan Vosloo]

Hi Guiseppe,

Another thing... can you additionally also send the output of:

reahl-control listconfig -v etc/

[Giuseppe Traficante]

Hi,

I've repeated the same step on home comnputer

and I've the same problem

I will print the output of last command you asked me

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ reahl-control createdbtables etc ; echo $PATH ; virtualenv --version; pip --version; python -c 'import pkg_resources; print(pkg_resources.require("setuptools"))'

[<class 'reahl.postgresqlsupport.PostgresqlControl'>, <class 'reahl.component.dbutils.NullDatabaseControl'>]

/home/peppicus/reahl_env

['etc', '/home/peppicus/reahl_env/bin', '/home/peppicus/addressbook2', '/home/peppicus/reahl_env/lib/python2.7', '/home/peppicus/reahl_env/lib/python2.7/plat-x86_64-linux-gnu', '/home/peppicus/reahl_env/lib/python2.7/lib-tk', '/home/peppicus/reahl_env/lib/python2.7/lib-old', '/home/peppicus/reahl_env/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/home/peppicus/reahl_env/local/lib/python2.7/site-packages']

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

WARNING:reahl.component.config:web.default_http_port in etc/web.config.py is using a dangerous default setting: 8000

WARNING:reahl.component.config:web.encrypted_http_port in etc/web.config.py is using a dangerous default setting: 8363

WARNING:reahl.component.config:web.static_root in etc/web.config.py is using a dangerous default setting: /home/peppicus/addressbook2

Traceback (most recent call last):

  File "/home/peppicus/reahl_env/bin/reahl-control", line 11, in <module>

    sys.exit(ProductionCommandline.execute_one())

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 165, in execute_one

    return_code = commandline.execute_command(command, line, options, parser)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 178, in execute_command

    return self.command_named(command).do(line)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 107, in do

    return self.execute(options, args)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 213, in execute

    super(CreateDBTables, self).execute(options, args)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 44, in execute

    self.create_context(args[0])

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 58, in create_context

    self.context.system_control = SystemControl(self.context.config)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/dbutils.py", line 41, in __init__

    self.db_control = self.db_control_for_uri(self.connection_uri)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/dbutils.py", line 90, in db_control_for_uri

    raise CouldNotFindDatabaseControlException(url)

reahl.component.dbutils.CouldNotFindDatabaseControlException: postgresql://fiberuser:f1b&rgest@localhost/fibergest

/home/peppicus/reahl_env/bin:/usr/bin/java_home/bin:/usr/bin/java_home/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

1.11.4

pip 1.5.4 from /home/peppicus/reahl_env/local/lib/python2.7/site-packages (python 2.7)

[setuptools 2.2 (/home/peppicus/reahl_env/lib/python2.7/site-packages)]

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ 

pip version is really different form the other PC :-S in both pc i've ubuntu 14.04

but python version is the same

now the new command output you asked me

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ reahl-control createdbtables etc 

[<class 'reahl.postgresqlsupport.PostgresqlControl'>, <class 'reahl.component.dbutils.NullDatabaseControl'>]

postgres(ql)?://(?P<user>\w+)(:(?P<password>\w+))?@(?P<host>\w+)(:(?P<port>\d+))?/(?P<database>\w+)$

False

False

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

WARNING:reahl.component.config:web.default_http_port in etc/web.config.py is using a dangerous default setting: 8000

WARNING:reahl.component.config:web.encrypted_http_port in etc/web.config.py is using a dangerous default setting: 8363

WARNING:reahl.component.config:web.static_root in etc/web.config.py is using a dangerous default setting: /home/peppicus/addressbook2

Traceback (most recent call last):

  File "/home/peppicus/reahl_env/bin/reahl-control", line 11, in <module>

    sys.exit(ProductionCommandline.execute_one())

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 165, in execute_one

    return_code = commandline.execute_command(command, line, options, parser)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 178, in execute_command

    return self.command_named(command).do(line)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/shelltools.py", line 107, in do

    return self.execute(options, args)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 213, in execute

    super(CreateDBTables, self).execute(options, args)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 44, in execute

    self.create_context(args[0])

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/prodshell.py", line 58, in create_context

    self.context.system_control = SystemControl(self.context.config)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/dbutils.py", line 41, in __init__

    self.db_control = self.db_control_for_uri(self.connection_uri)

  File "/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/component/dbutils.py", line 90, in db_control_for_uri

    raise CouldNotFindDatabaseControlException(url)

reahl.component.dbutils.CouldNotFindDatabaseControlException: postgresql://fiberuser:f1b&rgest@localhost/fibergest

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ 

[Giuseppe Traficante]

my new db connection parameter are

reahlsystem.connection_uri = 'postgresql://fiberuser:f1b&rgest@localhost/fibergest'

[Giuseppe Traficante]

the last command output you asked me

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ reahl-control listconfig -v etc/ 

Listing config for etc/

[<class 'reahl.postgresqlsupport.PostgresqlControl'>, <class 'reahl.component.dbutils.NullDatabaseControl'>]

postgres(ql)?://(?P<user>\w+)(:(?P<password>\w+))?@(?P<host>\w+)(:(?P<port>\d+))?/(?P<database>\w+)$

False

False

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

reahlsystem.connection_uri         postgresql://fiberuser:f1b&rgest@localhost/fibergest

reahlsystem.databasecontrols       [<class 'reahl.postgresqlsupport.PostgresqlControl'>, <class 'reahl.component.dbutils.NullDatabaseControl'>]

reahlsystem.debug                   True

reahlsystem.orm_control             <reahl.sqlalchemysupport.sqlalchemysupport.SqlAlchemyControl object at 0x7f2a2b2fcf90>

reahlsystem.root_egg               addressbook2

reahlsystem.serialise_parallel_requests False

reahlsystem.translation_packages   [<module 'reahl.messages' (built-in)>, <module 'reahl.messages' (built-in)>, <module 'reahl.messages' (built-in)>, <module 'reahl.doc.examples.tutorial.i18nexample.i18nexamplemessages' from '/home/peppicus/reahl_env/local/lib/python2.7/site-packages/reahl/doc/examples/tutorial/i18nexample/i18nexamplemessages/__init__.pyc'>, <module 'reahl.messages' (built-in)>]

web.cache_max_age                   600

web.default_http_port               8000

web.default_http_scheme             http

web.default_url_locale             en_gb

web.encrypted_http_port             8363

web.encrypted_http_scheme           https

web.frontend_libraries             <reahl.web.libraries.LibraryIndex object at 0x7f2a2c992f10>

web.guest_key_lifetime             31536000

web.guest_key_name                 reahl-guest

web.idle_lifetime                   7200

web.idle_lifetime_max               1209600

web.idle_secure_lifetime           3600

web.persisted_exception_class       <class 'reahl.webdeclarative.webdeclarative.PersistedException'>

web.persisted_file_class           <class 'reahl.webdeclarative.webdeclarative.PersistedFile'>

web.persisted_userinput_class       <class 'reahl.webdeclarative.webdeclarative.UserInput'>

web.session_class                   <class 'reahl.webdeclarative.webdeclarative.UserSession'>

web.session_key_name               reahl

web.session_lifetime               1209600

web.site_root                       <class 'addressbook2.AddressBookUI'>

web.static_root                     /home/peppicus/addressbook2

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ 

tnks a lot in advance!

[Giuseppe Traficante]

The user, pwd e db are correct infact i can connect successfully with dbeaver, as you can see below :-)

[Iwan Vosloo]

Thanks Giuseppe.

We use a regex to match the reahlsystem.connection_uri to a particular type of database backend.

It looks like our regex is too strict in what it allows for passwords. Below is my reasoning:

You have the correct database backend adapter from reahl installed, since you have PostgresalControl in your config:

reahlsystem.databasecontrols       [<class 'reahl.postgresqlsupport.PostgresqlControl'>, <class 'reahl.component.dbutils.NullDatabaseControl'>]

From the debugging output in your config file, the regex your reahlsystem.connection_uri should match is:

postgres(ql)?://(?P<user>\w+)(:(?P<password>\w+))?@(?P<host>\w+)(:(?P<port>\d+))?/(?P<database>\w+)$

(The following false shows it does not match)

Using your connection URI:

reahlsystem.connection_uri         postgresql://fiberuser:f1b&rgest@localhost/fibergest

I could play around in a Python shell and I get:

regex = r'postgres(ql)?://(?P<user>\w+)(:(?P<password>\w+))?@(?P<host>\w+)(:(?P<port>\d+))?/(?P<database>\w+)$'
re.match(regex, 'postgresql://fiberuser:f1b&rgest@localhost/fibergest')
(produces no match)

whereas (note I removed the ampersand):
re.match(regex, 'postgresql://fiberuser:f1brgest@localhost/fibergest')
(does match)

Can you perhaps change that password accordingly, and see if that works? If it does, we have a workaround in mind for you. Obviously we will need to change the regex too - this is an oversight on our part - thanks so much providing all this feedback.

Regards
- Iwan

[Giuseppe Traficante]

now i've created myusername, mydbname with pwd a and it works!

that is the problem, infact in the other pc the real password was password!

but I wrote by hand and i missed the "!", 

then surely it needs to change the regex to match any good pwd ;-)

ok, i launched directly createdbtables, and reahl creates the table correctly

(reahl_env)peppicus@peppicus-HP-Pavilion-dv6-Notebook-PC:~/addressbook2$ reahl-control createdbtables etc 

WARNING:reahl.component.config:reahlsystem.debug in etc/reahl.config.py is using a dangerous default setting: True

WARNING:reahl.component.config:reahlsystem.root_egg in etc/reahl.config.py is using a dangerous default setting: addressbook2

WARNING:reahl.component.config:web.default_http_port in etc/web.config.py is using a dangerous default setting: 8000

WARNING:reahl.component.config:web.encrypted_http_port in etc/web.config.py is using a dangerous default setting: 8363

WARNING:reahl.component.config:web.static_root in etc/web.config.py is using a dangerous default setting: /home/peppicus/addressbook2

ok, tnks a lot,

[Iwan Vosloo]

On 19/03/2016 12:10, Giuseppe Traficante wrote:
> now i've created myusername, mydbname with pwd a and it works!

Great Giuseppe!

I have logged a bug for this at:
https://bugs.launchpad.net/reahl/+bug/1559434

The workaround is to include the following lines in your reahl.config.py:

from reahl.postgresqlsupport import PostgresqlControl
PostgresqlControl.uri_regex_string =
r'postgres(ql)?://(?P<user>\w+)(:(?P<password>.*))?@(?P<host>\w+)(:(?P<port>\d+))?/(?P<database>\w+)$'

[Craig Sparks]

Great, seems you guys have been having fun without me. ;->

[Giuseppe Traficante]

;-)

--
You received this message because you are subscribed to a topic in the Google Groups "Reahl discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/reahl-discuss/E6W6Yg8YxRY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to reahl-discus...@googlegroups.com.
To post to this group, send an email to reahl-...@googlegroups.com.

[Giuseppe Traficante]

Hi guys,

now i'm writing a module fro scratch (helloReahl), so I can learn every aspect of reahl.

I'm trying to use session object and so I would create my own User object to attach to session.

I've tried the same in the tutorial.sessionscope, I've modified there and all run correctly,

now I've trying to do the same in helloReahl, but when I run the reahl-control createdbtables etc command

I receive this error:

  File "/home/gtraficante/reahl_env/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 393, in __new__

    "existing Table object." % key)

sqlalchemy.exc.InvalidRequestError: Table 'sessionscope_loginsession' is already defined for this MetaData instance.  Specify 'extend_existing=True' to redefine options and columns on an existing Table object.

Obviously the db is empty, if you want i can send you the code.

sábado, 19 de Março de 2016 às 19:43:36 UTC+1, Giuseppe Traficante escreveu:

;-)

Il 19 mar 2016 19:22, "Craig Sparks" <cr...@reahl.org> ha scritto:

Great, seems you guys have been having fun without me. ;->



On 19/03/2016 14:03, Iwan Vosloo wrote:

On 19/03/2016 12:10, Giuseppe Traficante wrote:

now i've created myusername, mydbname with pwd a and it works!

Great Giuseppe!

I have logged a bug for this at: https://bugs.launchpad.net/reahl/+bug/1559434

The workaround is to include the following lines in your reahl.config.py:

from reahl.postgresqlsupport import PostgresqlControl
PostgresqlControl.uri_regex_string = r'postgres(ql)?://(?P<user>\w+)(:(?P<password>.*))?@(?P<host>\w+)(:(?P<port>\d+))?/(?P<database>\w+)$'

Regards
- Iwan

--
You received this message because you are subscribed to a topic in the Google Groups "Reahl discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/reahl-discuss/E6W6Yg8YxRY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to reahl-discuss+unsubscribe@googlegroups.com.

[Iwan Vosloo]

Hi Giuseppe,

On 24/03/2016 09:49, Giuseppe Traficante wrote:
> Obviously the db is empty, if you want i can send you the code.

Please do so! You can email it to my address directly if you want, then
I'll respond on the list again.

[Iwan Vosloo]

Hi Giuseppe,

On 24/03/2016 10:06, Iwan Vosloo wrote:
> Please do so! You can email it to my address directly if you want,
> then I'll respond on the list again.

The problem is that in your helloReahl/.reahlproject, you still
reference the old sessionscope classes:

<persisted>
<class locator="sessionscope:User"/>
<class locator="sessionscope:LoginSession"/>
</persisted>


If you change that to point to your new helloReahl instead, the problem
is solved.

It would be nice to pick up on such a mistake and warn you explicitly,
but that is quite difficult to do. We'll scratch around and see if we
can perhaps do something like that!

Thanks

[Giuseppe Traficante]

Sorry,

really sorry!

I missed that.

Now it works.

tnks a lot

[Iwan Vosloo]

On 24/03/2016 11:21, Giuseppe Traficante wrote:
> Sorry,
> really sorry!
>
> I missed that.
>
> Now it works.
>
> tnks a lot

Great. I have logged a "wishlist" item for giving a clearer error:
https://bugs.launchpad.net/reahl/+bug/1561422

Thanks!