No Oracle database dialect

548 views
Skip to first unread message

Philipp Altmann

unread,
Jan 30, 2012, 12:28:28 PM1/30/12
to Repo and Gerrit Discussion
Hi,

there are anyone who uses Gerrit with an Oracle DB backend with JDBC?
The Problem is that we get an "No dialect known for jdbc:oracle"
message and when I look into the source [1] I see that an Oracle
Dialect is not implemented.


Cheers,
Philipp


[1]
http://code.google.com/p/gerrit/source/browse/src/main/java/com/google/gwtorm/jdbc/Database.java?repo=gwtorm&r=9de0323cf09c5fd8bd9ac9f90dd89279f6bcb160

Shawn Pearce

unread,
Jan 30, 2012, 12:53:09 PM1/30/12
to Philipp Altmann, Repo and Gerrit Discussion
On Mon, Jan 30, 2012 at 09:28, Philipp Altmann <alt...@gmail.com> wrote:
> there are anyone who uses Gerrit with an Oracle DB backend with JDBC?

No, because the dialect is not supported. :-)

> The Problem is that we get an "No dialect known for jdbc:oracle"
> message and when I look into the source [1] I see that an Oracle
> Dialect is not implemented.

Nope. Its probably not very difficult to add, based on the PostgreSQL
dialect. Gerrit Code Review uses very little SQL features so it should
be easy to support Oracle.

Philipp Altmann

unread,
Jan 30, 2012, 1:10:18 PM1/30/12
to Repo and Gerrit Discussion
Ok. Gerrit also not uses stuff like triggers etc. only update, delete,
create etc. also normal datebase stuff :-)


On 30 Jan., 18:53, Shawn Pearce <s...@google.com> wrote:

Shawn Pearce

unread,
Jan 30, 2012, 4:59:34 PM1/30/12
to Philipp Altmann, Repo and Gerrit Discussion
On Mon, Jan 30, 2012 at 10:10, Philipp Altmann <alt...@gmail.com> wrote:
> Ok. Gerrit also not uses stuff like triggers etc. only update, delete,
> create etc. also normal datebase stuff :-)

No triggers. :-)

The only really special thing Gerrit uses is sequence objects... which
Oracle supports. Its syntax for those differs from PostgreSQL when
obtaining the next value. So the dialect has to differ slightly to
handle fetching the next value from the sequence.

My Oracle is rusty, I haven't used an Oracle database since 8i came
out, and I didn't trust myself to write an Oracle dialect strictly
from memory with nothing to test against. So this has to come from the
community. :-)

Lars Klonowski

unread,
Feb 2, 2012, 4:39:57 AM2/2/12
to Repo and Gerrit Discussion
Hello,

Philipp Altmann was kind enough to ask this question on my behalf, as
I wondered if there was an easy way to connect Gerrit with an Oracle
database.
I finally managed to get Gerrit initialized with our Oracle database
and now wanted to share my implementation of this dialect [6].

I wrote a new test class and also had a few tests in the running
Gerrit instance and it all seems to function quite well. However, I'm
new to Gerrit and can't really tell if I missed something important.

Unfortunately, there are three Oracle issues which will probably
prevent this implementation from going public:

1. There is no open repository for the Oracle driver. Therefore, the
hosting (local) repository and the driver's version always have to be
added manually to the pom.xml.
2. For prepared statement batches, Oracle will not know the number of
affected rows and return state SUCCESS_NO_INFO (-2) instead of 1 (see
[1] chapter "Update Counts in the Oracle Implementation of Standard
Batching"). The best way to prevent unwanted exceptions I found, was
to change the method execute of the class
com.google.gwtorm.jdbc.JdbcAccess to also accept return state -2. [2]
3. Oracle does not allow identifiers longer than 30 characters for
columns and other names. The table ACCOUNTS however has two columns
with longer identifiers and there is no way to treat this in the
SqlDialect. I therefore had to change the method makeSqlFriendly of
the class com.google.gwtorm.schema.Util to cut names being too long.
[3]

Apart from these problems, the implementation went smoothly. I
registered the newly wrote dialect in the constructor of the class
com.google.gwtorm.jdbc.Database [4] and updated the method create of
the class com.google.gerrit.server.schema.SchemaCreator to run
index_generic.sql not only for H2 but also for Oracle. [5]

Best regards,
Lars Klonowski

[1]
http://docs.oracle.com/cd/E11882_01/java.112/e10589/oraperf.htm#autoId17

[2]
private static void execute(final PreparedStatement ps, final int
cnt)
throws SQLException, OrmConcurrencyException {
if (cnt == 0) {
return;
}

final int[] states = ps.executeBatch();
if (states == null) {
throw new SQLException("No rows affected; expected " + cnt + "
rows");
}
for (int i = 0; i < cnt; i++) {
if (states.length <= i || (states[i] != 1 && states[i] != -2)) {
throw new OrmConcurrencyException();
}
}
}

[3]
public static String makeSqlFriendly(final String name) {
final StringBuilder r = new StringBuilder(name.length() + 8);
boolean lastWasCap = true;
for (int i = 0; i < name.length(); i++) {
final char c = name.charAt(i);
if (Character.isUpperCase(c)) {
if (!lastWasCap) {
r.append('_');
lastWasCap = true;
}
r.append(Character.toLowerCase(c));
} else if (c == '_') {
lastWasCap = true;
r.append(c);
} else {
lastWasCap = false;
r.append(c);
}
}
String friendlyName = r.toString();
if (friendlyName.length() > 30) {
String identifier =
Integer.toHexString(friendlyName.hashCode());
friendlyName = friendlyName.substring(0, 21) + identifier;
}
return friendlyName;
}

[4]
...
final String url = c.getMetaData().getURL();
if (url.startsWith("jdbc:postgresql:")) {
dialect = new DialectPostgreSQL();

} else if (url.startsWith("jdbc:h2:")) {
dialect = new DialectH2();

} else if (url.startsWith("jdbc:mysql:")) {
dialect = new DialectMySQL();

} else if (url.startsWith("jdbc:oracle:")) {
dialect = new DialectOracle();

} else {
throw new OrmException("No dialect known for " + url);
}
...

[5]
...
final SqlDialect d = jdbc.getDialect();
if (d instanceof DialectH2 || d instanceof DialectOracle) {
index_generic.run(db);

}
...

[6]
package com.google.gwtorm.schema.sql;

import com.google.gwtorm.client.OrmException;
import com.google.gwtorm.client.StatementExecutor;
import com.google.gwtorm.schema.ColumnModel;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Types;
import java.util.HashSet;
import java.util.Set;

public class DialectOracle extends SqlDialect {

public DialectOracle() {
typeNames.put(Types.BIGINT, "NUMBER(19,0)");
typeNames.put(Types.LONGVARCHAR, "CLOB");
}

@Override
public Set<String> listTables(final Connection db) throws
SQLException {
Statement s = db.createStatement();
try {
ResultSet rs = s.executeQuery("SELECT table_name FROM
user_tables");
try {
HashSet<String> sequences = new HashSet<String>();
while (rs.next()) {
sequences.add(rs.getString(1).toLowerCase());
}
return sequences;
} finally {
rs.close();
}
} finally {
s.close();
}
}

@Override
public Set<String> listSequences(Connection db) throws SQLException
{
Statement s = db.createStatement();
try {
ResultSet rs = s.executeQuery("SELECT sequence_name FROM
user_sequences");
try {
HashSet<String> sequences = new HashSet<String>();
while (rs.next()) {
sequences.add(rs.getString(1).toLowerCase());
}
return sequences;
} finally {
rs.close();
}
} finally {
s.close();
}
}

@Override
public void renameColumn(StatementExecutor e, String tableName,
String fromColumn, ColumnModel col) throws OrmException {
StringBuffer sb = new StringBuffer();
sb.append("ALTER TABLE ");
sb.append(tableName);
sb.append(" RENAME COLUMN ");
sb.append(fromColumn);
sb.append(" TO ");
sb.append(col.getColumnName());
e.execute(sb.toString());
}

@Override
public String getNextSequenceValueSql(String seqname) {
return "SELECT " + seqname + ".nextval FROM dual";
}

@Override
public boolean selectHasLimit() {
return false;
}

}

Saša Živkov

unread,
Feb 2, 2012, 11:37:50 AM2/2/12
to Lars Klonowski, Repo and Gerrit Discussion
On Thu, Feb 2, 2012 at 10:39 AM, Lars Klonowski
<lars.kl...@software-friends.de> wrote:
> Hello,
>
> Philipp Altmann was kind enough to ask this question on my behalf, as
> I wondered if there was an easy way to connect Gerrit with an Oracle
> database.
> I finally managed to get Gerrit initialized with our Oracle database
> and now wanted to share my implementation of this dialect [6].
>
> I wrote a new test class and also had a few tests in the running
> Gerrit instance and it all seems to function quite well. However, I'm
> new to Gerrit and can't really tell if I missed something important.
>
> Unfortunately, there are three Oracle issues which will probably
> prevent this implementation from going public:
>
> 1. There is no open repository for the Oracle driver. Therefore, the
> hosting (local) repository and the driver's version always have to be
> added manually to the pom.xml.
> 2. For prepared statement batches, Oracle will not know the number of
> affected rows and return state SUCCESS_NO_INFO (-2) instead of 1 (see

The same is valid for MaxDB which we had to support at SAP.

> [1] chapter "Update Counts in the Oracle Implementation of Standard
> Batching"). The best way to prevent unwanted exceptions I found, was
> to change the method execute of the class
> com.google.gwtorm.jdbc.JdbcAccess to also accept return state -2. [2]

This was the first thing which a colleague of mine tried. I don't remember
all the details but although it seemed to work it started failing
later with some
scenarios which I can't remember right now.
You can take a look at the change [1] where the gwtorm support for JDBC drivers
returning SUCCESS_NO_INFO is prepared for review.
Actually, it would be great if you also review this change as it is
what you will
also need to easier add Oracle dialect.

[1] https://gerrit-review.googlesource.com/#/c/25647/


> 3. Oracle does not allow identifiers longer than 30 characters for
> columns and other names. The table ACCOUNTS however has two columns

The same issue with MaxDB. Ironically, this was a change from SAP
that introduced these long column names :-(
The best would be to change these field names to something shorter
than 30 characters.


Saša Živkov

Lars Klonowski

unread,
Feb 6, 2012, 2:38:13 AM2/6/12
to Repo and Gerrit Discussion


On 2 Feb., 17:37, Saša Živkov <ziv...@gmail.com> wrote:

> This was the first thing which a colleague of mine tried. I don't remember
> all the details but although it seemed to work it started failing
> later with some
> scenarios which I can't remember right now.

This is not exactly what I wanted to read, but thanks for the
warning :)

> You can take a look at the change [1] where the gwtorm support for JDBC drivers
> returning SUCCESS_NO_INFO is prepared for review.
> Actually, it would be great if you also review this change as it is
> what you will
> also need to easier add Oracle dialect.
>
> [1]https://gerrit-review.googlesource.com/#/c/25647/

I will definitely try it out, if it works now as it is. Thanks for the
link.

Best regards,
Lars Klonowski

Adrian Görler

unread,
Feb 6, 2012, 8:21:51 AM2/6/12
to Repo and Gerrit Discussion
Hi,

> if (states.length <= i || (states[i] != 1 && states[i] != -2)) {
> throw new OrmConcurrencyException();
> }

Although this will "work" on Oracle for INSERT, UPDATE and DELETE it
would not allow you to detect concurrent modifications. To detect
concurrent modification, you need to process the rows in the batch
individually. This is what the change https://gerrit-review.googlesource.com/#/c/25647/
proposes to do.

Also, this would not get you UPSERT running on Oracle. Currently,
gwtorm attempts to determine which rows are preexisting on the
database by executing a batch update. Due to the SUCCESS_NO_INFO-
issue, this is not feasible on Oracle. This issue is also tackled by
change 25647.

We are using the changes proposed in change 25647 productively on an
SUCESS_NO_INFO-returning MaxDB database since quite some time yet.

-Adrian

On Feb 6, 8:38 am, Lars Klonowski <lars.klonow...@software-friends.de>
wrote:

Edwin Kempin

unread,
Oct 17, 2012, 1:53:18 AM10/17/12
to Jeff Stephenson, repo-d...@googlegroups.com
Hi Jeff,

for my organization we also have the need to add support for other databases.
Since with upcoming Gerrit 2.5 plugins will be supported, we were thinking that it would
be nice to have an extension point in Gerrit that allows plugins to contribute support for new databases.
The pre-work is done in [1]. In addition it's needed that plugins can contribute init steps to do the
database configuration. This will be enabled by [2].
Once both changes pass the code review, we want to add this extension point.
At the moment this has not high prio for us, but once these changes are in it shouldn't be much work left
to finish it.

If we have this extension point, I think it would be very nice to have a public plugin which adds the support
for Oracle.

Edwin

[1] https://gerrit-review.googlesource.com/37341
[2] https://gerrit-review.googlesource.com/38041

2012/10/17 Jeff Stephenson <jeffrey.s...@gmail.com>


I see issue 25647 was merged which should now make the Oracle dialect 
changes even simpler.  My organization is interested in getting Gerrit to
work on Oracle as well.   Lars, did you proceed to use your patch locally
and if so, how has it been working?



Luca Milanesio

unread,
Oct 17, 2012, 3:10:19 AM10/17/12
to Edwin Kempin, Jeff Stephenson, repo-d...@googlegroups.com
Could be an interesting topic for the hackathon :-)
We have implemented Ms SQL Server as well (other than Oracle) but need a "plug-in" interface to publish it for review.

Luca.

Lars Klonowski

unread,
Dec 4, 2012, 10:56:48 AM12/4/12
to repo-d...@googlegroups.com
Hi Jeff,

sorry for the late reply. I didn't pay attention to this thread any more after all this time.
I set up my implementation on the SUCCESS_NO_INFO-Branch (https://gerrit-review.googlesource.com/#/c/25647/) as suggested and we've been running this successfully on production since then.

We're upgrading to Gerrit 2.5 now and it seems the patch (from version 2.2.2) only needs little changes to be applied. But it still needs to be tested.
I made patches for both versions, if you are interested.

Best regards,
Lars

Joseph D Carroll Jr

unread,
Jul 1, 2013, 11:16:15 AM7/1/13
to repo-d...@googlegroups.com
Its been a number of months since there has been any update on this thread and I was wondering if anyone has been able to make any progress.  We are in the testing phases of Gerrit (more of a POC to show people how simple it makes things), and are using a PostgreSQL db.  However, I already know that they will want to move to an Oracle DB once (if ever) we go live.

Not sure how valuable I would be at working something up.  But I would be happy to spend a little time with it over the coming months to try and work something up.

Thanks,

JD

Joseph D Carroll Jr

unread,
Jul 1, 2013, 11:20:03 AM7/1/13
to repo-d...@googlegroups.com
Just as an other thought....

Bugzilla currently supports Oracle DB (at least in R4.4), would any of their implementation translate to Gerrit?  I'm not sure what the difference in copyrights is (if any), but it could at least give a general framework on how to put something together.

Thanks,

JD


On Monday, January 30, 2012 11:28:28 AM UTC-6, Philipp Altmann wrote:

Shawn Pearce

unread,
Jul 3, 2013, 11:37:12 AM7/3/13
to Joseph D Carroll Jr, repo-discuss
Oracle and PostgreSQL are similar enough from Gerrit's perspective
that one could look at how the PostgreSQL dialect works in gwtorm and
duplicate it for Oracle. I don't have access to an Oracle installation
to test against, but would be willing to do the basic copy and pasting
needed to get it bootstrapped for someone else to test. :-)
> --
> ---
> You received this message because you are subscribed to the Google Groups
> "Repo and Gerrit Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to repo-discuss...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

David Ostrovsky

unread,
Jul 10, 2013, 4:30:17 PM7/10/13
to repo-d...@googlegroups.com, Joseph D Carroll Jr, repo-d...@googlegroups.com, Shawn Pearce


Am Mittwoch, 3. Juli 2013 17:37:12 UTC+2 schrieb Shawn Pearce:
Oracle and PostgreSQL are similar enough from Gerrit's perspective
that one could look at how the PostgreSQL dialect works in gwtorm and
duplicate it for Oracle. I don't have access to an Oracle installation
to test against, but would be willing to do the basic copy and pasting
needed to get it bootstrapped for someone else to test. :-)

done [1], even with running tests ;-)
I didn't try to run it against Gerrit, will try later.

[1] https://gerrit-review.googlesource.com/47691/

Saša Živkov

unread,
Jul 12, 2013, 7:47:58 AM7/12/13
to David Ostrovsky, repo-d...@googlegroups.com, Joseph D Carroll Jr, Shawn Pearce
Please also take a look at [2]. This is an attempt to make SqlDialects
and DataSourceTypes pluggable so we can support additional databases
without having to change the gwtorm and gerrit core.

It is NOT yet tested.

David Ostrovsky

unread,
Jul 12, 2013, 8:28:18 AM7/12/13
to repo-d...@googlegroups.com, Joseph D Carroll Jr, Shawn Pearce, repo-d...@googlegroups.com


On Friday, July 12, 2013 1:47:58 PM UTC+2, zivkov wrote:



On Wed, Jul 10, 2013 at 10:30 PM, David Ostrovsky <david.o...@gmail.com> wrote:


Am Mittwoch, 3. Juli 2013 17:37:12 UTC+2 schrieb Shawn Pearce:
Oracle and PostgreSQL are similar enough from Gerrit's perspective
that one could look at how the PostgreSQL dialect works in gwtorm and
duplicate it for Oracle. I don't have access to an Oracle installation
to test against, but would be willing to do the basic copy and pasting
needed to get it bootstrapped for someone else to test. :-)

done [1], even with running tests ;-)
I didn't try to run it against Gerrit, will try later.

[1] https://gerrit-review.googlesource.com/47691/

Please also take a look at [2]. This is an attempt to make SqlDialects
and DataSourceTypes pluggable so we can support additional databases
without having to change the gwtorm and gerrit core.


Thanks. I see your point. I personally think that gwtorm have broader spectrum as to be only the
ORM layer for Gerrit Review. It was called gwtorm and not gerrit-orm for a reason ;-)

So providing just Gerrit users the ability to use a db from vendor foo (through "proprietary" Gerrit plugins)
would unnecessary limit the potential user base of gwtorm. IOW to support more db vendors in gwtorm
natively has a value of its own.

Concerning the extension in Gerrit itself, for example for Oracle: 86 lines of code [1] were needed (well,
documentation and indexes extra). And no proprietary jars are needed to compile time, only to runtime
when Oracle is used. So who cares?

In any event for supporting Oracle and MaxDB in Gerrit we first need to migrate the ReviewDB schema
to satisfy the identifier length limitation [2]. And for that we need new gwtorm features, like rename
table [3] and rename (nested) column [4].


Saša Živkov

unread,
Jul 12, 2013, 9:01:31 AM7/12/13
to David Ostrovsky, repo-d...@googlegroups.com, Joseph D Carroll Jr, Shawn Pearce
On Fri, Jul 12, 2013 at 2:28 PM, David Ostrovsky <david.o...@gmail.com> wrote:


On Friday, July 12, 2013 1:47:58 PM UTC+2, zivkov wrote:



On Wed, Jul 10, 2013 at 10:30 PM, David Ostrovsky <david.o...@gmail.com> wrote:


Am Mittwoch, 3. Juli 2013 17:37:12 UTC+2 schrieb Shawn Pearce:
Oracle and PostgreSQL are similar enough from Gerrit's perspective
that one could look at how the PostgreSQL dialect works in gwtorm and
duplicate it for Oracle. I don't have access to an Oracle installation
to test against, but would be willing to do the basic copy and pasting
needed to get it bootstrapped for someone else to test. :-)

done [1], even with running tests ;-)
I didn't try to run it against Gerrit, will try later.

[1] https://gerrit-review.googlesource.com/47691/

Please also take a look at [2]. This is an attempt to make SqlDialects
and DataSourceTypes pluggable so we can support additional databases
without having to change the gwtorm and gerrit core.


Thanks. I see your point. I personally think that gwtorm have broader spectrum as to be only the
ORM layer for Gerrit Review. It was called gwtorm and not gerrit-orm for a reason ;-)
 
Theoretically correct. However, I am not aware of anything else using gwtorm except Gerrit.
 

So providing just Gerrit users the ability to use a db from vendor foo (through "proprietary" Gerrit plugins)
would unnecessary limit the potential user base of gwtorm. IOW to support more db vendors in gwtorm
natively has a value of its own.
 
I don't know if Shawn is interested into supporting gwtorm as a general purpose ORM library.
 
Concerning the extension in Gerrit itself, for example for Oracle: 86 lines of code [1] were needed (well,
documentation and indexes extra). And no proprietary jars are needed to compile time, only to runtime
when Oracle is used. So who cares?

Those who have to maintain that code :-)
Supporting Oracle in gwtorm and gerrit core is probably OK as long as someone is ready to maintain that.

In general we probably cannot add support for any new database platform into
gwtorm and gerrit core. There may be proprietary database platforms that no one
else would be interested to run Gerrit on except the owner of that platform.


In any event for supporting Oracle and MaxDB in Gerrit we first need to migrate the ReviewDB schema
to satisfy the identifier length limitation [2]. And for that we need new gwtorm features, like rename
table [3] and rename (nested) column [4].

David Ostrovsky

unread,
Jul 13, 2013, 2:35:07 PM7/13/13
to repo-d...@googlegroups.com, Joseph D Carroll Jr, Shawn Pearce, Edwin Kempin, David Pursehouse (Sony Mobile)

Gerrit is up and running on Oracle [1], with automatic installation process:

java -jar gerrit.war init -d gerrit_site
[...]

Database server type           [h2]: oracle

Gerrit Code Review is not shipped with Oracle jdbc driver 11g Release 2 (11.2.0)
**  This library is required for your configuration. **
Copy and install it now [Y/n]?
Copying file:///u01/app/oracle/product/11.2.0/xe/jdbc/lib/ojdbc6.jar ...Checksum ojdbc6.jar OK
Server hostname           [localhost]:
Server port                    [1521]:
Database name             [xe]:
Database username      [gerrit]:
gerrit's password             :
           confirm password :
[...]

ReviewDB schema was migrated to meet id length limitation:
2 tables, 1 column and 3 indexes were renamed. The migration for supported SQL dialects:
H2, MySQL & PostgreSQL is ready [2].

The only question is, if the migration for supported NoSQL dialect(s) can be done?

To test:

* patch, build and expose gwtorm, so that Gerrit can "see" the patched version
* patch, build and deploy Gerrit
* select Oracle during initialization


[1] https://gerrit-review.googlesource.com/#/q/status:open+topic:oracle,n,z
[2] https://gerrit-review.googlesource.com/#/c/47736/

Imre Rácz

unread,
Sep 17, 2013, 11:41:20 AM9/17/13
to repo-d...@googlegroups.com
Hi,

Could someone tell me how to apply gwtorm pluggin over gerrit in order to use oracle as database dialect while init phase?

Thanks
Imre 

Dave Borowitz

unread,
Sep 17, 2013, 12:49:18 PM9/17/13
to Imre Rácz, repo-discuss
Oracle support is in gwtorm 1.7, which was merged into Gerrit master at 32a0b583, which will not be released until Gerrit 2.8. If you want support for that sooner you'll have to build your own version of Gerrit from that commit or later.


--
Reply all
Reply to author
Forward
0 new messages