Options for model compare

[Thomas Kellerer]

Hi,

after using the Liquibase generator for a while, I'd like to enhance
it a bit.

Especially for DBMS that have non-transactional DDL (e.g. Oracle) it
is very useful to put every DDL statement into a separate "changeset"
in Liquibase (because it's easier to fix if something goes wrong which
cannot be rolled back).

Although that is the general recommendation from the Liquibase team,
I'd like to give the user the choice whether all statements should go
into a single changeset (as today) or if each DDL statement should be
enclosed into a separate changeset.

Currently the Compare Model dialog does not have a real concept of
selecting generator specific options (I could imagine generator
options for "pure" DDL as well)

Are there any plans to have unified options?

If yes, I'd simply adjust the Liquibase generator to emit a changeset
for each DDL (because that is the more general approach, and wouldn't
harm when used agains a DDL transaction capable DBMS) without an
option to choose this and wait for the "unified option" handling.

If there is no such, plan I'd simply add a new checkbox (similar to
"include indexes" or "Supress similarities") that would only be
enabled for Liquibase for the compare dialog and the "Forward
Engineer" dialog.

Any opinions?

Regards
Thomas

[Jonathan Fuerth]

Hi Thomas,

I've always imagined we would eventually implement a separate options GUI for each DDL generator. I'm happy that we've been able to push it off for as long as we have, because I think options are often unnecessary, even when you commonly see them in other tools.

However, the case you've presented here is a compelling reason to implement DDL options. There is no one best answer for the Liquibase changeset question, since it pretty much depends on your target platform.

As for where to set the options themselves (i.e. the data model that the options panel would work on), I think that's best done in the DDL generator instance itself.

So, by all means, go ahead and create a LiquibaseDDLOptionsPanel (a DataEntryPanel) for setting options on the LiquibaseDDLGenerator.

-Jonathan

--
You received this message because you are subscribed to the Google Groups "Architect Developers" group.
To post to this group, send email to architect-...@googlegroups.com.
To unsubscribe from this group, send email to architect-develo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/architect-developers?hl=en.

[Thomas Kellerer]

Hi Jonathan,

I finally have the time to tackle this....

> However, the case you've presented here is a compelling reason to implement
> DDL options. There is no one best answer for the Liquibase changeset
> question, since it pretty much depends on your target platform.
>
> As for where to set the options themselves (i.e. the data model that the
> options panel would work on), I think that's best done in the DDL generator
> instance itself.

Currently I'm thinking of a generic approach along the following
lines:

change DDLGenerator interface to include a new method
setGeneratorOptions(Map<String, Object> options)
create an empty implementation in GenericDDLGenerator

define a base DDLGeneratorOptionsPanel that returns a Map<String,
Object>
and pass those options to the instance of DDLGenerator used in the
Diff and forward engineer dialogs

For the integration into the UI I'm thinking of a OptionsPanelFactory
that creates a DDLGeneratorOptionsPanel based on the generator
instance passed to it.

Does this sound OK for you?

[Jonathan Fuerth]

On Fri, May 21, 2010 at 5:02 AM, Thomas Kellerer <google...@sql-workbench.net> wrote:

Hi Jonathan,

I finally have the time to tackle this....

Great!
 

Currently I'm thinking of a generic approach along the following
lines:

change DDLGenerator interface to include a new method
setGeneratorOptions(Map<String, Object> options)
create an empty implementation in GenericDDLGenerator

define a base DDLGeneratorOptionsPanel that returns a Map<String,
Object>
and pass those options to the instance of DDLGenerator used in the
Diff and forward engineer dialogs

Well, this approach has all the advantages and disadvantages of dynamic types. I think from Architect's point of view, the main advantage is that it would be really easy to persist the settings as long as they are confined to Strings, Integers, and Booleans.

But we already have a pretty easy mechanism for persisting JavaBeans properties in the project file (the old-school Jakarta Digester--ancient but still quite effective).

Feel free to make an argument more for dynamic properties, but I'm thinking JavaBeans properties are maybe a better choice in this particular setting.
 

For the integration into the UI I'm thinking of a OptionsPanelFactory
that creates a DDLGeneratorOptionsPanel based on the generator
instance passed to it.

Does this sound OK for you?

 Well, again, I have experimented with dynamically generating preferences UI's in other products (Power*Dashboard user preferences are defined dynamically and feature a dynamically generated UI for modifying those dynamic prefs). Although the system works fine, and it is definitely really easy to add new String, Boolean, and Multiple Choice (Combo Box) preference items, I have to say I found two big drawbacks:

1. It seemed as if every new preference I had to introduce required a new custom component (colour chooser, date chooser, custom nested dataset chooser, ...) so even though adding the property itself was easy, I ended up doing a lot of framework modifications to accommodate each new property.

2. The layout of the generated UI was not as easy to use and understand as it would have been if I'd laid it out according to the meanings and relationships between the preference items.


So I guess my suggestion would be to just code it "normally" (with Beans properties and custom-made DataEntryPanels). That will save you the time of building the framework, and (hopefully) give a higher-quality final result.

Again, I don't want to come across as saying "you must do it this way." Feel free to argue back if you disagree. :)

-Jonathan

[Thomas Kellerer]

Hi Jonathan,

> But we already have a pretty easy mechanism for persisting JavaBeans
> properties in the project file (the old-school Jakarta Digester--ancient but
> still quite effective).

Can you point me to an example in the code doing that?
Is that essentially what CompareDMSettings is about?

I'm not sure how those settings actually make it into the project
file, so if you can briefly outline how I can persist the Liquibase
options that would be nice.

> So I guess my suggestion would be to just code it "normally" (with Beans
> properties and custom-made DataEntryPanels). That will save you the time of
> building the framework, and (hopefully) give a higher-quality final result.

After starting with the DDLExportPanel I did feel the generic approach
might actually be a bit of an overkill, so I implemented a regular
DataEntryPanel and pass it the instance of the generator. Once more
generators have options, this can be changed to an interface on the
panel (similar to DataEntryPanel), then it's not necessary to pass
Maps back and forth.

I'm currently struggling with getting my panel into the
CompareDMPanel. Mainly because I have no experience with the
FormLayout and hiding and showing the panel depending on the radio
button is a bit more complex there compared to the DDLExportPanel

Btw: I wonder if the "English Description" Checkbox would be the next
candidator for such an option panel...

Regards
Thomas

[Jonathan Fuerth]

On Fri, May 21, 2010 at 12:05 PM, Thomas Kellerer <google...@sql-workbench.net> wrote:

Hi Jonathan,

> But we already have a pretty easy mechanism for persisting JavaBeans
> properties in the project file (the old-school Jakarta Digester--ancient but
> still quite effective).

Can you point me to an example in the code doing that?
Is that essentially what CompareDMSettings is about?

Yes, CompareDMSettings is essentially a configuration JavaBean which is persisted with the project's XML file and read back in using the Digester.

In a nutshell, Digester is a system for mapping XML elements to JavaBeans. It reads XML using SAX, and takes each SAX event and maps it to a rule. Rules are things like "create a new object for this XML element," "try to set all the properties on the current JavaBeans object based on this XML element's attributes," or "add the current object as a child of another object."

The component that handles mapping XML attributes to JavaBeans property values is Commons BeanUtils. All of its naming and conversion rules apply.

The entire architect project file (including SQLObjects and playpen components, as well as the various preference and state information) is parsed using Digester. See ca.sqlpower.architect.ProjectLoader#setupDigester() for the Digester configuration. Also, see the ca.sqlpower.architect.swingui.SwingUIProjectLoader.CompareDMSettingFactory for an example of how the CompareDMSettings object is retrieved from the project (rather than created directly by the Digester) when the corresponding XML element is encountered in the project file.

For the full story, see here: http://commons.apache.org/digester/commons-digester-1.8.1/docs/api/org/apache/commons/digester/package-summary.html#doc.Intro

All of the above pertains to loading only. For creating the XML in the first place, we take the extremely lightweight approach of just writing it out with print statements.

I'm not sure how those settings actually make it into the project
file, so if you can briefly outline how I can persist the Liquibase
options that would be nice.

Once you've got it in, we'd be happy to give you some specific guidance on that.

Right now, we only persist information about which type of DDL generator you last used, and what you typed in for Catalog and Schema name. We will probably want to enhance that by being able to remember separate preferences for each type of DDL generator. It's definitely doable. We're here to help!

 

> So I guess my suggestion would be to just code it "normally" (with Beans
> properties and custom-made DataEntryPanels). That will save you the time of
> building the framework, and (hopefully) give a higher-quality final result.

After starting with the DDLExportPanel I did feel the generic approach
might actually be a bit of an overkill, so I implemented a regular
DataEntryPanel and pass it the instance of the generator. Once more
generators have options, this can be changed to an interface on the
panel (similar to DataEntryPanel), then it's not necessary to pass
Maps back and forth.

I'm currently struggling with getting my panel into the
CompareDMPanel. Mainly because I have no experience with the
FormLayout and hiding and showing the panel depending on the radio
button is a bit more complex there compared to the DDLExportPanel

Hmm, yes. Maybe if there was always a placeholder panel there, and you could replace its contents with the appropriate DataEntryPanel? I'd just let the panel span most columns (have it stretch to fill the available space), and let the whole row take the panel's preferred height.

Remember you'll need to revalidate the whole CompareDM layout after swapping panels, but it should work.
 

Btw: I wonder if the "English Description" Checkbox would be the next
candidator for such an option panel...

Yeah, it would be nice to offer more types of output. We could also consider taking away all the output options from the CompareDM panel itself, and have the output window contain a bunch of tabs: one for DDL, one for English descriptions, and so on. That way, if you want to see several types of output, you only need to perform the comparison step once.

Enjoy your weekend.

-Jonathan

[Thomas Kellerer]

Hi,

> Right now, we only persist information about which type of DDL generator you
> last used, and what you typed in for Catalog and Schema name. We will
> probably want to enhance that by being able to remember separate preferences
> for each type of DDL generator. It's definitely doable. We're here to help!

As far as I can tell, I need to add this in two places:

- CompareDMSettings should have an instance of my LiqubaseSettings
- a "global" LiquibaseSettings that is used for forward engineering a
model.

For the latter I need to add this to ArchitectSwingSession if I'm not
mistaken. Would that be OK?

Thomas

[Jonathan Fuerth]

On Sat, May 22, 2010 at 5:23 AM, Thomas Kellerer <google...@sql-workbench.net> wrote:

> Right now, we only persist information about which type of DDL generator you
> last used, and what you typed in for Catalog and Schema name. We will
> probably want to enhance that by being able to remember separate preferences
> for each type of DDL generator. It's definitely doable. We're here to help!

As far as I can tell, I need to add this in two places:

- CompareDMSettings should have an instance of my LiqubaseSettings
- a "global" LiquibaseSettings that is used for forward engineering a
model.

For the latter I need to add this to ArchitectSwingSession if I'm not
mistaken. Would that be OK?

 Sure, the session is a good place for it. Since CompareDM and DDL generation are "core" features that do not depend on the Swing UI layer, ArchitectSession (not ArchitectSwingSession) would be the best place.

I think what you want is a method like:

public DDLGenerator getConfiguredDDLGenerator(Class<? extends DDLGenerator> generatorType)

in ArchitectSession.

It would instantiate the selected DDL generator type, configure it based on the persistent preferences, and return the new instance.

As for how to make the ArchitectSession aware of the preferences so they can be persisted, I guess the complementary method is also needed:

public void putDDLGeneratorConfiguration(DDLGenerator generator)

This would use BeanUtils.describe(generator) to create a map of name/value pairs, and associate that entire map with the generator's type (generator.getClass())

So in the config file, you'd get something like:

<ddl-generator type="ca.sqlpower.architect.ddl.PostgresDDLGenerator">
  <property name="usingUnlimitedLengthVarchar">true</property>
  <property name="usingQuotedIdentifiers">false</property>
</ddl-generator>

<ddl-generator type="ca.sqlpower.architect.ddl.SQLServerDDLGenerator">
  <property name="usingQuotedIdentifiers">false</property>
</ddl-generator>

<ddl-generator type="ca.sqlpower.architect.ddl.LiquibaseDDLGenerator">
  <property name="updateMode">INCREMENTAL</property>
</ddl-generator>

<default-ddl-generator>ca.sqlpower.architect.ddl.PostgresDDLGenerator</default-ddl-generator>


Of course, you'd be able to apply these preferences back to the DDLGenerator instances using BeanUtils. You'd probably just read them in as a Map<Class<? extends DDLGenerator>, Map<String, Object>> (maps DDL generator types to their bundle of options). You wouldn't need to create DDL generator instances until they're requested via ArchitectSession.getConfiguredDDLGenerator().

Does that make sense? It should be easy and flexible.

-Jonathan

[Jonathan Fuerth]

On Mon, May 24, 2010 at 11:42 AM, Jonathan Fuerth <fue...@sqlpower.ca> wrote:

So in the config file, you'd get something like:

<ddl-generator type="ca.sqlpower.architect.ddl.PostgresDDLGenerator">
  <property name="usingUnlimitedLengthVarchar">true</property>
  <property name="usingQuotedIdentifiers">false</property>
</ddl-generator>

<ddl-generator type="ca.sqlpower.architect.ddl.SQLServerDDLGenerator">
  <property name="usingQuotedIdentifiers">false</property>
</ddl-generator>

<ddl-generator type="ca.sqlpower.architect.ddl.LiquibaseDDLGenerator">
  <property name="updateMode">INCREMENTAL</property>
</ddl-generator>

Just to add to this, it would be more like the rest of the project file if we were to represent the same set of options like this:

<ddl-generator-config type="ca.sqlpower.architect.

ddl.PostgresDDLGenerator" usingUnlimitedLengthVarchar="true" usingQuotedIdentifiers="false" />

<ddl-generator-config type="ca.sqlpower.architect.

ddl.SQLServerDDLGenerator" usingQuotedIdentifiers="false" />

<ddl-generator-config type="ca.sqlpower.architect.

ddl.LiquibaseDDLGenerator" updateMode="INCREMENTAL" />

And I would definitely recommend this syntax if the goal was to create an instance of each type of DDL generator when the project is being loaded in.

In this case, though, we are actually just reading in "factory settings" for each type of DDL generator--the actual creation of the generators is deferred until they are needed.

Hmm, now that I think it through, perhaps the "standard Architect project syntax" (the style I'm mentioning in this email) really is best. The other "bunch of property elements" approach is perhaps a bit too generic compared to the rest of the Architect project file format.

-Jonathan

[Thomas Kellerer]

Hi Jonathan,

I have committed a first implementation.

For the model compare, the LiquibaseSettings are a property of the DMSettings (which is reflected in the XML)
For the "global" settings, I have a new tag (on the same level as ddl-generator), but I guess I can change that to an "embedded" option.

Btw: I have changes slightly the way the CompareDMSettings are written. Current they are only written if the saveFlag is true, which means that if DMSettings are read from a project file, but the user never compares the model, those settings will be lost.

My solution was to write a saveFlag="true" into the XML, thus the settings - once written to the project file - will always be persisted. I hope this is OK for you.

One thing that I'm not at all happy about currently, is the behaviour of the CompareDM dialog, when the Liquibase options are displayed. The status area is simply cut off. I tried to call pack() on the dialog, but that didn't fix the problem either :(

Regards
Thomas