[Doctrine] #1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be Performed Out of Order

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order
-----------------------------------+----------------------------------------
Reporter: deefour | Owner: jwage
Type: defect | Status: new
Priority: minor | Milestone:
Component: Migrations | Version: 1.1-DEV
Keywords: | Has_test: 0
Mystatus: Pending Core Response | Has_patch: 0
-----------------------------------+----------------------------------------
I am working with Doctrine_Migration via Doctrine_Cli, using the
Doctrine_Task_Migrate task.

I am using Doctrine 1.1, PHP 5.2.6, and MySQL 5.0.x

Locally I am running darwin (OS X Leopard).
On my server I am running CentOS 5.

Locally, I can run doctrine's migrate cli task without trouble. Having
initially generated my migration classes from an existing database, I have
32 migration classes currently; The first 31 contain the CREATE TABLE
statements for each table in the database, while the 32nd contains the
adding/dropping of all foreign keys.

On darwin, the RecusiveDirectoryIterator pulls the files from the
migration directory ordered properly, according the the prefixed unique
integer in the class filenames.

http://trac.doctrine-
project.org/browser/branches/1.1/lib/Doctrine/Migration.php#L116

On CentOS, the order which the files are pulled from the migration
directory is very different.

In my example,
Using Darwin, the foreign key class (# 32 when running `ls` in shell) is
properly set as version [b]32[/b] during the migration process.
Using CentOS, the foreign key class (# 32 when running `ls` in shell) is
[b]incorrectly[/b] set as version [b]26[/b] during the migration process.

The order that SPL's RecursiveDirectoryIterator (and I'd assume
DirectoryIterator too) grabs the files from the directory is OS-specific.
In the case of an OS like CentOS, this ordering does not follow the
natural sort order of the migration class filenames integer prefixes. This
causes the migrations to be performed out of order.

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688>
Doctrine <http://www.phpdoctrine.org>
PHP Doctrine Object Relational Mapper

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new
Priority: minor | Milestone:
Component: Migrations | Version: 1.1-DEV

Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Changes (by deefour):

* has_patch: 0 => 1

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:1>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV
Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Changes (by jwage):

* milestone: => 1.1.0

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:2>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV

Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Comment (by anj):

I have the same problem on Ubuntu 8.10, the console with the ls command
shows the files in correct order, but the order of the
RecursiveDirectoryIterator is completly different.

Also fixed it with the ksort, after reading the bunch of files.

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:3>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV

Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Comment (by anj):

Another solution:

Doctrine/Migration.php
Line about: 115

{{{
foreach ((array) $directory as $dir) {
$it = new RecursiveIteratorIterator(new
RecursiveDirectoryIterator($dir),
RecursiveIteratorIterator::LEAVES_ONLY);

// sort files by filename
$files = array();
foreach ($it as $file) {
$info = pathinfo($file->getFileName());
if (isset($info['extension']) && $info['extension'] ==
'php') {
$files[$info['filename']] = $file;
}
}
ksort($files);

foreach ($files as $file) {
$info = pathinfo($file->getFileName());
if (isset($info['extension']) && $info['extension'] ==
'php') {
require_once($file->getPathName());

$array = array_diff(get_declared_classes(), $classes);
$className = end($array);

if ($className) {
$this->loadMigrationClass($className,
$file->getPathName());
}
}
}
}
}}}

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:4>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV

Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Comment (by deefour):

Hi Anj-

Nice, consolidated solution.

I wonder which is faster, with mine having more array manipulation within
that Migration.php and yours having the extra foreach loop. I would think
in most cases the speed would be negligible.

Either way, I'm glad the issue is defined and good solutions have been
offered. I have my doctrine cli running via a Phing build process and
can't wait to get this last piece fixed up permanently to automate the
versioning of my production database.

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:5>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV

Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Comment (by anj):

Hi Deefour,

I was very happy about the migrations news with the new diff tool, but it
tooks me about one day to get it working :(
I do not know which solution is faster, but i thought this should be
simpler for other users to fix it, in the time this fix is not in
repository. I wonder if there is no option for the
"RecursiveIteratorIterator" function to set the sorting ... This should be
the best solution, i think.

best regards,
aNj

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:6>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------

Reporter: deefour | Owner: jwage
Type: defect | Status: new

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV

Resolution: | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Comment (by deefour):

Hi aNj-

I had the same question about RecursiveIteratorIterator, and asked in #php
on freenode. Someone there showed me in the source for PHP how
DirectoryIterator handles file listings, and it does not appear there is
an option to specify sorting. He mentioned that you should always do
proper sorting in your application when using DirectoryIterator.

Pending this patch being accepted I think I might try my hand at other
tickets in Trac here as time allows. I had a good time figuring out the
problem.

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:7>

[Doctrine]

#1688: RecursiveDirectoryIterator is OS-Specific - Causes Migrations to be
Performed Out of Order

-------------------------+--------------------------------------------------
Reporter: deefour | Owner: jwage
Type: defect | Status: closed

Priority: minor | Milestone: 1.1.0
Component: Migrations | Version: 1.1-DEV

Resolution: fixed | Keywords:

Has_test: 0 | Mystatus: Pending Core Response

Has_patch: 1 |
-------------------------+--------------------------------------------------
Changes (by jwage):

* status: new => closed
* resolution: => fixed

Comment:

In r5228 this was fixed.

--
Ticket URL: <http://trac.doctrine-project.org/ticket/1688#comment:8>