Migrations Status & Feedback request

[Jeffrey Becker]

I'd like some feedback on how people feel about the current state of the migrations feature. I've used the existing code in a project and it seems to work pretty well.  That said, I haven't finished all the features that I originally wanted.  The key points I've hit so far are:

• Unified DDL generation framework (no weirdness like EF where different exports generate different sql)
  
• Migration Factory & Version Store
  
• Usable Configuration 
  
• Fluent Builder
  

That said, generating migrations auto-magically is turning out to be much more work than I'd originally anticipated.  I think the feature is definitely achievable, I just don't have the time to do it right now.  I'm proud of what the migrations code-base looks like right now.  I think its the right approach. I'm also concerned about having this sit unmerged for too long.  The longer I'm a fork the more work I have to keep track of the upstream. Pending some positive feedback and cleanup work I think I'd like to try to get this merged.  Are people comfortable with this?

[Jan Schubert]

I use the migration framework for my application. I don't finish my work, but currently it looks well! In the next few weeks, I will use some more features of the migration framework. Then I can make a better statement.

The framework is important for my application and I hope the branch will be merged next time.

[Jan Schubert]

I have a problem with the SchemaExport class. Without your changes for the migration framework, the tables wasn't quoted automaticly. So, if I use PostgreSQL as database, the table names are upper case. PostgreSQL enables uppercase names automaticly, if a name is quoted. I need lower case table names for PostgreSQL and CamelCase names for MSSQL table names. I have the same problem in my "IMigration" class.

Is it possible to use the auto-quote setting of NHibernate?

I try to fix the problem, but I can't do this with my insufficient knowledge.

PS: The last commit of your branch "migrations" is not compilable. Some files are missing.

Am Freitag, 14. November 2014 16:38:44 UTC+1 schrieb Jeffrey Becker:

[Jeffrey Becker]

I've found it.  I'm just stripping it out.

[Jeffrey Becker]

The behavior described seems to be centered in Table.GetThreePartName; Table.cs Line 404.  Stripping out the behavior is causing a lot of unit-test failures which I don't have time to address today.

[Jan Schubert]

What exactly would you change in line 404?

[Jeffrey Becker]

Fixes are in and your pull request is merged in the migrations branch.

[Ricardo Peres]

I think you should have placed your code in a different project, not integrated with NH.

RP

[Jan Schubert]

The fix works great! But the column names are still quoted. I changed this and pushed it as a pull request.

[Jeffrey Becker]

I'll fix this shortly.

--

---
You received this message because you are subscribed to a topic in the Google Groups "nhibernate-development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nhibernate-development/aIGssJRjHcw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nhibernate-develo...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gunnar Liljas]

I agree with Ricardo. I really like the idea, but it should be in a separate project.

/G

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

[Jeffrey Becker]

So here's my conundrum. I'd really like to break the majority of the code out into a separate library; there's a lot of ancillary stuff in there. Because of how baked-in and dialect dependent ddl generation is, I don't think its possible to implement this as purely a layer-on-top.

As I see it there are distinct several components in my work so far:

• The core ddl generation framework -- an OO framework for representing ddl operations
  
• The mapping diff engine -- Take two Configurations, look at their mappings and generate a set of operations which goes from A to B
  
• The migration framework & executor
  
• The fluent builder interface
  

I  feel like the first two are things which need to be in the core nhibernate where as the last two are stuff which is obviously better to separate out. 

To unsubscribe from this group and all its topics, send an email to nhibernate-development+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.

[Jan Schubert]

I try to set default values for my columns. My problem is that the default value is only a string. I use the migration framework for different databases.

Each database has an own dialect for a default value. E.g. Oracle use "EMPTY_BLOB()" for an empty binary column and MSSQL use "0x".

I think it is necessary to extend the SqlType class with a method like "GetDefaultValue(byte[] value) : DefaultValue" and the Dialect class should convert the DefaultValue to an SQL string.

Would my idea fit into the concept?

[Jan Schubert]

Another idea: Should I use the configuration "QuerySubstitutions" to replace the values?

Sample:

from -> ADD COLUMN CON0001 BIT DEFAULT false NOT NULL

to ->   ADD COLUMN CON0001 BIT DEFAULT 0 NOT NULL

[Jeffrey Becker]

So I finally got around to breaking the stuff I need in nhibernate-core and the migrations framework stuff out into separate repos.  Core work can be seen at:

https://github.com/jeffreyabecker/nhibernate-core/tree/operations-framework

Basically this work breaks schema generation up into a series of DdlOperation classes and corresponding DTOs which encapsulate the logic for building DDL.  From that point I can build all the migrations stuff out in a separate library. I prefer to keep the schema generation in the core framework for several reasons.  First, my coworkers who've gone from older EF schema generations to migrations have reported changes in the DDL generated.  Second, this allows me to leverage the existing test suites around schema generation to ensure I've kept things consistent. Third, some of the logic that this needs is appropriately private to Cfg.Configuration.

[Amro El-Fakharany]

Hi Jeffrey,

one thing I don’t quite understand is why do you want to write Migrations by hand while NHibernate already has all the Information you need for letting some sort of a framework do this automatically?

We already tell NHibernate what Tables and Columns are needed in terms of mappings.

I mean why not just read the mappings, extract the Schema from them and then let some framework do the rest without having to say things like Create.Table(…) etc.?

 

Let me rephrase my Question:

What are the benefits of your migrations framework compared to – let’s say – FluentMigrator?

 

Amro

To unsubscribe from this group and all its topics, send an email to nhibernate-develo...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

[Jeffrey Becker]

To answer your question succinctly, 

The benefit is that it allows people to write database independent migrations using the principles that NHibernate uses to ensure db independence. The whole framework is dialect-aware and therefore pretty darn database independent.  The data type and id generator specifications can be used exactly the same way as in the mappings.

The structure you proposed in terms of using the mappings is actually exactly what I'm shooting for.  My system uses mappings as NHibernate parses them in the Config class.  The work in repo specified above is all about splitting those mapping classes up to encapsulate the DDL generation framework in a reusable way.  From there my framework can pick up the mappings and generate that migration class.  The framework can also store some history about the mappings so it can generate differential updates and properly update data.  There's also facilities to auto-update the database based on these classes.

To unsubscribe from this group and all its topics, send an email to nhibernate-development+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.

[Amro El-Fakharany]

FluentMigrator is also db independent and I'm actually using it since some time to generate the schema for NHibernate.
What would truly be awesome is a framework which would inspect the mappings of nhibernate and automatically executes the appropriate operations against FluentMigrator (or any other migration "engine") to generate the schema.
This shouldn't require any changes to nhibernate and if you're going down that path you already have at least one volunteer :)

I don't know how others see this but my own opinion is that schema generation shouldn't be a responsibility of nhibernate at all.
And I'm saying this although I really like your concept of DdlOperation and the related code is much cleaner than what was there before!

Amro

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

[Jeffrey Becker]

From a practical perspective the DdlOperation code is needed just to properly expose the internal data structures of NHibernate in a consumable way. From there its largely irrelevant what migration engine is consuming them and generating coded migrations. The key point is that reading the hbm.xml files is extremely problematic for a number of reasons.  First, mapping by code and auto mapping eliminate the hbm.xml files.  Second, even if that were the case the Configuration needs to parse the whole set to make things coherent. Finally much of the information needed to generate the migrations against the engine is private to Configuration and rightly so. This is doubly true for comparing two Configurations and determining the deltas for the migration framework.  

To unsubscribe from this group and all its topics, send an email to nhibernate-development+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Amro El-Fakharany]

There is no need to read XML files or interpret mapping-by-code stuff.
All of the required mapping information ends up in PersistentClass which is exposed in Configuration through the Enumerable Property ClassMappings if I remember correctly.

[Jan Schubert]

Well done! Do you have any ideas to migrate data? In my project it is necessary to execute an UPDATE statement, after create new columns. If I release a new software to my customers and one of the customers overleaps a version, I can't use the current mapping.

E.g.

Version 1:

Table: User -> Columns: Name, Age

Version 2:

Table: User -> Columns: Name, Age, Gender

Version 3:

Table: User -> Columns: Name, Age

In version 1 the User table has no Gender column. For the migration to version 2 I want to execute an UPDATE statement like this:

UPDATE User SET Gender = 'm' WHERE Name LIKE 'John'

In version 3 I will drop the column. If a customer overleaps version 2 and install version 3 after version 1, I can't create the UPDATE statement because my POCO and the mapping file has no Gender property.

[Jeffrey Becker]

I did look into using the PersistentClass property but that doesn't seem to include information about identifier generators or auxiliary db objects. My first thought with this was to simply spider configuration and spit out a fluent migrator based class.  I went down the whole operations framework path after concluding that all the mapping information I needed just wasn't available.  

[Amro El-Fakharany]

Take a look at the implementation of IterateGenerators in the Configuration class (the last method in Configuration.cs).

It shows how to get hold of identifier generators.

 

As for auxiliary objects the easiest way I can ad hoc think of would be to subscribe to one of the Events BeforeBindMapping or AfterBindMapping of Configuration.

Both events gets HbmMapping as a property of the argument BindMappingEventArgs.

In the event handler you just iterate over HbmMapping.DatabaseObjects.

I won’t overestimate the importance of auxiliary objects though.

 

 

Von: nhibernate-...@googlegroups.com [mailto:nhibernate-...@googlegroups.com] Im Auftrag von Jeffrey Becker
Gesendet: Montag, 16. Februar 2015 14:26
An: nhibernate-...@googlegroups.com
Betreff: Re: [nhibernate-development] Re: Migrations Status & Feedback request

 

I did look into using the PersistentClass property but that doesn't seem to include information about identifier generators or auxiliary db objects. My first thought with this was to simply spider configuration and spit out a fluent migrator based class.  I went down the whole operations framework path after concluding that all the mapping information I needed just wasn't available.  

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeffrey Becker]

I'd expect the update statement to be included in the migration from v1 to v2 as a builder.Sql statement.  The binaries you ship should compile in all the migration classes.  The migrator will then find all the migrations it needs to run to get from the current version to the target version.  So in your case of a customer deploying v3, the migrator would find the migrations for both v2 and v3 and execute them in order.

[Jeffrey Becker]

Amaro, I started going down the path you suggested and I'm running into some pretty big road blocks. 

1) Resolving the type of a column requires access to the mapping which is private to Configuration

2) PersistentIdentifiers are black-boxes which keep their params private and spit out some sql.

These are both pretty big blockers which my code addresses through the operations framework.  I'm definitely much more interested in using FluentMigrator to do the actual migrations but at this point NHibernate doesn't expose the information FluentMigrator needs.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Amro El-Fakharany]

1) A PersistentClass holds a couple of Property iterators. I would first try “SubclassPropertyClosureIterator” (but I’m not really sure!).

Each Property instance holds a collection of Column objects called “ColumnIterator”.

To get the type of a column you just call: column.GetSqlTypeCode(mapping).

You get that “mapping” argument by calling BuildMapping() on a Configuration instance which you should do before accessing ClassMappings for the first time.

Here is a pseudo code example:

var mapping = nhConfiguration.BuildMapping()

foreach(var pc in nhConfiguration.ClassMappings)

{

foreach(var property in pc.SubclassPropertyClosureIterator)

{

  foreach(var col in property.ColumnIterator)

    var dbType = col.GetSqlTypeCode(mapping).DbType

}

}

 

2) As far as I can tell, the only true black box identifier is TableGenerator.

If I’m not completely mistaken the rest should be accessible through PersistenceClass.IdentifierProperty.ColumnIterator

 

Amro

 

Von: nhibernate-...@googlegroups.com [mailto:nhibernate-...@googlegroups.com] Im Auftrag von Jeffrey Becker
Gesendet: Dienstag, 10. März 2015 19:40
An: nhibernate-...@googlegroups.com
Betreff: Re: [nhibernate-development] Re: Migrations Status & Feedback request

 

Amaro, I started going down the path you suggested and I'm running into some pretty big road blocks. 

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

[Jeffrey Becker]

I remember having to modify the Enhanced stuff but I've never seen anyone use that so I'll just skip it.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.

[Jeffrey Becker]

Amaro,

Could I get your feedback on this Gist of how to extract the schema from Configuration?

[Amro El-Fakharany]

Jeffrey,

I would say that it looks good to my eyes in terms of pure schema extraction.

 

But this won’t work if you’re going down the path of determining changes in mappings and then automatically create migration expressions!

You will need to track entity and property mapping changes and thus extract the schema out of both not just the table associated with an entity.

For example, PropertyX mapped to ColumnX and then changed to ColumnY. You won’t be able to detect that this needs an alter column expression unless you extract the schema out of PropertyX.

 

We can work on this together if you are interested. Do you have a public repo for this?

 

Von: nhibernate-...@googlegroups.com [mailto:nhibernate-...@googlegroups.com] Im Auftrag von Jeffrey Becker
Gesendet: Mittwoch, 11. März 2015 18:24
An: nhibernate-...@googlegroups.com
Betreff: Re: [nhibernate-development] Re: Migrations Status & Feedback request

 

Amaro,

 

Could I get your feedback on this Gist of how to extract the schema from Configuration?

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-develo...@googlegroups.com.

[Jeffrey Becker]

Honestly I'd assumed that would be the something left to consuming developer to correct. Repo is up here.  Hit me up via email or at least lets start a new thread.

Amaro,

 

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "nhibernate-development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nhibernate-development+unsub...@googlegroups.com.

[Jan Schubert]

Hi Jeffrey,

what do you think when your changes are pushed to NHibernate?

[Robinreza khan]

So here's my conundrum. I'd really like to break the majority of the code out into a separate library; there's a lot of ancillary stuff in there. Because of how baked-in and dialect dependent ddl generation is, I don't think its possible to implement this as purely a layer-on-top.

buy mailing lists