MachII and UNC Network Paths

[Shaun Byrnes]

Hi,

I have used MachII on numerous projects, but recently was tasked with
relocating some websites using the Framework to a new server.

The location of the Framework and all websites is on a network drive
and as such the root path looks something like:

//vmware-host/Shared Folders/sites/MachII
//vmware-host/Shared Folders/sites/website

Instead of previously being something like:

C:\inetpub\wwwroot\MachII
C:\inetpub\wwwroot\website

This seems to have caused a whole range of issues with the Framework
and there are many things that are not working. Firstly relative paths
in the config would not even load. This was fixed by adding the
following code at the end of the expandRelativePath() function in
MachII/util/Utils.cfc

<cfif Left(arguments.baseDirectory, 2) is "\\">
<cfset resolvedPath = "\\" & resolvedPath />
</cfif>

However then other things seem to be unworking - depending on the site
- one was using the MachII environments and refused to load past that
point. Another older site before MachII environments existed (had its
own environment system) was reporting errors with "event" being
undefined in arguments while using a filter. On that one I was able to
remove some filter code and get a page to load, but as soon as an
event was triggered I would get the same event undefined error.

My feeling on this is that events are not being passed through, likely
because of something like a hash id for memory storage being different
in several places (possible due to me changing that util code) and
thus it cant be looked up and found on subsequent execution flow. This
is simply a guess but one way or another MachII does not seem to play
nice with it being on a network drive.

Additionally other sites I have migrated that do not use MachII are
fine.

Help on this would be appreciated as all MachII sites wont' work with
a network drive setup :(

Best Regards,

Shaun Byrnes

[Peter Farrell]

I've heard numerous people using UNC paths...must be something different in your setup. We can work those out.

First, What version of Mach-II are you using?

Second is there a reason you decided to use UNC versus mapping that to a drive letter? ACF in general does wonky things with UNC paths.

I'd need an exception screenshot to help you out in the terms of environments. More than likely if you've moved servers that the configure is incorrect and no default enivornment is defined. Could you share the except and your environment xml?

As for the filter, could you share the code?

HTH,
.pjf

--
To post to this group, send email to mach-ii-for...@googlegroups.com
For more options and to unsubscribe, visit this group at http://groups.google.com/group/mach-ii-for-coldfusion?hl=en

SVN: http://svn.mach-ii.com/machii/
Wiki / Documentation / Tickets: http://trac.mach-ii.com/machii/

[Shaun]

Hi Peter,

Thanks for the quick response!

I see - I'm using 1.8.0

I agree - it would have been much easier to used a mapped drive letter, but CF does not seem to play nice with it. I already spent several hours trying to get it to understand a drive letting when creating a mapping in the administrator. I'm using CF10 and could not get the mappings to accept the mapped network drive. 

On the sites that were not using MachII some of them required CF mappings to the root to work, which seemed happy with those using UNC paths (the mapped drive would not work for me in the administrator).

On the MachII sites I was setting up I do not need any CF mappings at all for them to work - so nothing extra was done in the CF administrator. Should I have mapped one anyways? If so the drive letter mapping would still be an issue.

I checked the environment setup out for that same reason and the correct host names are already there.

For the filter there is numerous files involve - all these have been working perfectly in production environments for several years, when on the server's physical drive so I know there is no issues with that they are doing. I don't want to get into now trying/testing and updating code if this setup won't work as in example the site with the filter issue has tens of thousands of files and hundreds of thousands of lines of code, so it will be a nightmare! :)

If you have found people able to do this successfully in the past its likely I just need to update how I have set it up. Otherwise if its going to involve many changes to sites already working I may have to resetup the server to use it's physical drive, instead of a network one. :(

Best,

Shaun

[Peter Farrell]

The environment error is explaining the issue. You must have an environment property in a module which will try to inherit from the base app. Either the base app doesn't have an environment prop or the module env prop can't match the environment name. How is your app setup?

Id upgrade to 1.8.1 which has some bug fixes for environments.

Sounds like ACF10 messed up UNC paths in certain areas.

Do you have a screenshot for the filter that messes up? I suspect the method in the filter doesn't match up with the api...can you just share the filter cfc (you can send it to me directly if don't want to share it with the group).

--

[Shaun]

The app that is receiving the environment error is using no modules at all.

I tried upgrading to MachII 1.8.1 but unfortunately it breaks the generation of the .htaccess file for URL rewriting that works on my version 1.8.0 and as right now I am simply trying to migrate the sites was hoping to avoid getting back into the code on them and editing it from what was previously working (See attached file of working url code). 

The UNC paths seem to work fine on CF10 in all other non MachII sites - the immediate code that breaks the UNC paths in MachII is the expandRelativePath function which I was able to edit to ensure that the absolute path that is generated works fine:

<cffunction name="expandRelativePath" access="public" returntype="string" output="false"

hint="Expands a relative path to an absolute path relative from a base (starting) directory.">

<cfargument name="baseDirectory" type="string" required="true"

hint="The starting directory from which relative path is relative." />

<cfargument name="relativePath" type="string" required="true"

hint="The relative path to use." />

<cfset var combinedWorkingPath = arguments.baseDirectory & arguments.relativePath />

<cfset var pathCollection = 0 />

<cfset var resolvedPath = "" />

<cfset var hits = ArrayNew(1) />

<cfset var offset = 0 />

<cfset var i = 0 />

<!--- Unified slashes due to operating system differences and convert ./ to / --->

<cfset combinedWorkingPath = Replace(combinedWorkingPath, "\", "/", "all") />

<cfset combinedWorkingPath = Replace(combinedWorkingPath, "/./", "/", "all") />

<cfset pathCollection = ListToArray(combinedWorkingPath, "/") />

<!--- Check how many directories we need to move up using the ../ syntax --->

<cfloop from="1" to="#ArrayLen(pathCollection)#" index="i">

<cfif pathCollection[i] IS "..">

<cfset ArrayAppend(hits, i) />

</cfif>

</cfloop>

<cfloop from="1" to="#ArrayLen(hits)#" index="i">

<cfset ArrayDeleteAt(pathCollection, hits[i] - offset) />

<cfset ArrayDeleteAt(pathCollection, hits[i] - (offset + 1)) />

<cfset offset = offset + 2 />

</cfloop>

<!--- Rebuild the path from the collection --->

<cfset resolvedPath = ArrayToList(pathCollection, "/") />

<!--- Reinsert the leading slash if *nix system --->

<cfif Left(arguments.baseDirectory, 1) IS "/">

<cfset resolvedPath = "/" & resolvedPath />

</cfif>

<!--- Reinsert the trailing slash if the relativePath was just a directory --->

<cfif Right(arguments.relativePath, 1) IS "/">

<cfset resolvedPath = resolvedPath & "/" />

</cfif>

 <!--- If the base directory is a UNC network path add the removed slashes back to it so CF can find it --->

<cfif Left(arguments.baseDirectory, 2) is "\\">

<cfset resolvedPath = "\\" & resolvedPath />

</cfif>  

 

<cfreturn resolvedPath />

</cffunction>

The code works fine without network paths - I make the change to use network paths and all of a sudden I get the following error:

Type: Expression || Message: Element EVENT is undefined in ARGUMENTS. || Detail: || Base Template: \\vmware-host\Shared Folders\tim\Sites\srconstruction\filters\SecurityFilter.cfc at line 43 || Original Template: \\vmware-host\Shared Folders\tim\Sites\srconstruction\filters\SecurityFilter.cfc at line 43

Filter attached.

You mentioned earlier to use drive mappings in CF for network paths. How is this possible?

Best,

Shaun

On Friday, June 1, 2012 1:57:17 PM UTC+10, Shaun wrote:

[Kurt Wiersma]

The exception that you posted seems to be coming from your
SecurityFilter.cfc rather then from the Mach II framework. Thank you
for attaching SecurityFilter.cfc but I think to really understand what
is happening we will also need to see it's super class
BaseApplicationFilter.

I run several Mach II sites on a SAN that is accessed via UNC paths on
the server so I am pretty certain Mach II works with UNC paths. For us
UNC paths doesn't even come up because we path everything relative to
the root or use mappings which seems like generally a best practice.
Are you using that practice?

--Kurt

[Shaun]

I know it shows that it is coming from the security filter, then the base filter it extends from then ultimately the MachII filter - this was why I did not see a point in attaching just that one - all the code in the site is working fine in live production and the old testing setup and has been for over 2 years - both MachII and the application itself.

The only change is to move all sites and MachII onto a network drive - immediately I get errors such as event undefined in arguments which has never been a problem and which is always injected there from the framework - now suddenly missing from methods which have been working stably for years.

When the applications and MachII are on a physical drive, there is no need for the use of CF mappings for the root as the site (or MachII) folder name is simply used. Again this has never been a problem and IIRC is even suggested as the default setup in the wiki i.e. leave the MachII installation as "MachII" and no mapping is needed - all sites would access it. Again this has worked successfully for several years.

With the SAN setup you are describing, is it possible that it is different that what I am describing? In my case:

Windows Server

  |- IIS 7.5

        |- inetpub\wwwroot

              |- Virtual mappings to sites

  |- SQL Server

  |- ColdFusion 10

        |- inetpub\wwwroot    (all virtual mappings inherit CF access)

Network Drive

  |- site

  |- site2

  |- MachII


ColdFusion is fine with this setup and all non MachII sites work without any changes. The first issue that occurs with MachII is found in the file: MachII/util/Utils.cfc when expanding relative paths as I earlier explained. When the framework loads and CFFile is used to load config includes in MachII/framework/AppFactory.cfc an absolute path is expected and the following code is run:

<cfif Left(includeFilePath, 1) IS ".">

<cfset includeFilePath = variables.utils.expandRelativePath(arguments.parentConfigFilePathDirectory, includeFilePath) />

<cfelse>

<cfset includeFilePath = ExpandPath(includeFilePath) />

</cfif>

The expandRelativePath() code is one example of that which does not play nice with UNC paths as the "\\" which signify an absolute network path are replaced a single "/" which subsequently breaks relative config paths for starters. This would be the case on any setup loading from a UNC path. As I explained earlier, I found this, updated that util function to handle those types of paths too and tested against physical drive locations which handled that problem after I had restarted CF. Another Framework related problem with the filters (which are working perfectly on a physical drive setup) appeared.

All I am trying to say is that there is some points you *could* look into if you feel necessary to support a networked setup like I described. In my case I am going to drop this setup now and go for a standard physical drive setup as I dont have the spare time to spend doing a whole bunch of debug work on a proven working applications. Especially as the setup I am attempting does not appear to be common or tested much.

I guess its up to you guys if you feel it worth your time to investigate further on that setup, but there is a definite issue at the least with expandRelativePath() which I spotted and proposed a solution which works to fix it. I just dont have the extra time to continue drilling down other parts of the framework to spot anything else which doesn't gel well with that setup is all. I am also working from the stable datum that the old servers (using physical drive setup) has the exact same sites on them which work with any issue - UNC network setup is the only change - then all non MachII sites work without any changes (so unlikely to be CF).

Hope that helps & love the framework too btw!

Best,

Shaun

On Friday, June 1, 2012 1:57:17 PM UTC+10, Shaun wrote:

On Friday, June 1, 2012 1:57:17 PM UTC+10, Shaun wrote:

[Kurt Wiersma]

Could you send us a stripped down example app that shows the problem?
It would probably just consist of a index.cfm, application.cfc, and
mach-ii.xml along with the stack trace you get when you start up the
app.

I think with that we could reproduce and fix the issue.

--Kurt

[Shaun]

Hi Kurt/Peter,

Thanks for the reply! I have moved forward with this over the last 24 hours. In the server change over a number of different things have flushed up and I have isolated each one down except one now (for all the MachII sites).

1) The UNC paths would cause an issue in the framework when using expandRelativePath() and having the "\\" missing from the start of the path. Easy fix and MachII is happy.

2) The environment issue was handled with a change to the server config as suggested.

3) The event issue was caused by the other change I had introduced - moving from CF8 to CF10 (I could not even find CF8 hosted anymore so simply downloaded the latest version). After several hours of frustrating debugging in both my code and the MachII code I isolated why the error was being reported as "undefined event in arguments" when no code had been altered. Turns out I had a base listener/filter (which extended from the MachII listener/filter) which provided some utility functions other site listener/filters would extend from. This had a method "getDependancies()" which like:

<cffunction name="getDependancies" access="private" returntype="struct" output="false">

<!--- get references to the required model functionality --->

<cfset var local = {

i18nFacade = getEngine().getI18NFacade(),

sessionFacade = getEngine().getSessionFacade(),

logService = getEngine().getServiceFactory().getLogService(),

user = getEngine().getSessionFacade().getUser(),

role = getEngine().getSessionFacade().getUserRole()

} />

<cfreturn local />

</cffunction>

Then for example a listener extending the base and using this method like:

<cffunction name="getHomePage" access="public" returntype="void" output="false"

hint="Get home page content.">

<cfargument name="event" type="MachII.framework.Event" required="true" />

        <cfscript>

// get required dependancies

var local = getDependancies();

// set event args

arguments.event.setArg("metaTitle", local.i18nFacade.getResource("publicHomeMetaTitle"));

arguments.event.setArg("metaDescription", local.i18nFacade.getResource("publicHomeMetaDescription"));

arguments.event.setArg("metaKeywords", local.i18nFacade.getResource("publicHomeMetaKeywords"));

arguments.event.setArg("titleContent", local.i18nFacade.getResource("publicHomeTitle"));

arguments.event.setArg("headerContent", local.i18nFacade.getResource("publicHomeHeader"));

</cfscript>

</cffunction> 

This had always worked fine and suddenly I'm informed that arguments.event is undefined?! Makes no sense right. Well it turns out that in CF8 "local" is simply a common practice for "var scoped" function memory. In CF10 it is a CF scope - this I had even read about and knew had been added. Except I never suspected that the new "local" scope would now also by default have an "arguments" key which contained a reference to the function's arguments scope also. What this in turn resulted in was code that *had* been totally fine was broken as "var local = getDependancies();" would return all the other common variables needed just fine, but would store them back into the local scope and the empty "arguments" key/value in the struct from "getDependancies()" was overwriting the populated "arguments" scope from the main function! As such "event" was now gone - hence "undefined" errors".

I thought I would share that as it was quite unsuspected an may help someone out to know that. Below was the simple change which sorted that issue out:

<cffunction name="getDependancies" access="private" returntype="struct" output="false">

<!--- get references to the required model functionality --->

<cfset var dependancies = {

i18nFacade = getEngine().getI18NFacade(),

sessionFacade = getEngine().getSessionFacade(),

logService = getEngine().getServiceFactory().getLogService(),

user = getEngine().getSessionFacade().getUser(),

role = getEngine().getSessionFacade().getUserRole()

} />

<cfreturn dependancies />

</cffunction>

To just not use the "local" scope in the "getDependancies()" method, which avoids the "arguments" scope ever being returned (more elegant IMO than using "var local = { ... };" and then structDelete(local, "arguments"); as another possible solution).

This was also why, depending on the exact execution cycle per site, some would error in a filter and some in a listener - depending on whether it was a filter or listener run first (as they both have a similar "getDependancies()" helper).

4) Once all these were sorted the MachII sites were all happy - save for one particular issue. This I believe is another thing which has occurred because of going from CF8 to CF10. This relates to sessions and persistance of them; the code which had been working with CF8 was now not allowing sessions to persist from page to page.

The code the MachII sites are all currently using which relates to sessions is:

Applicaion.cfc

...

<cfset this.sessionManagement = true />

<cfset this.setClientCookies = false />

<cfset this.setDomainCookies = false />

...

<cffunction name="onSessionStart" access="public" returntype="void" output="false"

hint="Only runs when a session is created.">

<!--- Run super --->

<cfset super.onSessionStart() />

<!--- When the session starts, run the session facade's session start --->

<cfset getProperty("appManager").getEngine().getSessionFacade().onSessionStart() />

</cffunction>

<cffunction name="onRequestStart" access="public" returntype="void" output="true"

hint="Run at the start of a page request. (Output must be set to true).

In this case the super will run after the request start, rather than before it

as the request start contains certain processes such as security and request throttling

which should execute before anything else happens (including the mach-ii request cycle).">

<cfargument name="targetPage" type="string" required="true" />

<!--- When the request starts, run the request facade's request start --->

<cfset getProperty("appManager").getEngine().getRequestFacade().onRequestStart(arguments.targetPage) />

<!--- Run super --->

<cfset super.onRequestStart(targetPage = arguments.targetPage) />

</cffunction>

...

Then from the RequestFacade:

...

    <cffunction name="onRequestStart" access="public" output="false" returntype="void" 

hint="Should be fired when the request starts from the Application bootstrapper.">              

<cfargument name="targetPage" type="string" required="true" />        

  <!--- run specific actions --->

<cfset requestsManager() />

<cfset manageIllegalRequests(arguments.targetPage) />

<cfset ensureSessionCookiesExistIfNeeded() />

<cfset manageUrlRedirect() />

        <cfreturn />

    </cffunction>

...

<cffunction name="ensureSessionCookiesExistIfNeeded" access="private" returntype="void" output="false"

hint="I ensure that whenever a page is visited, if required, cookies will be set to memory

if they do not exist. This is to make sure that the session will persist multiple page requests

without writing cookies to the client machine (which has some security implications). This is

not required if J2EE session management is enabled as that form of session management works differently.">

<!--- Set per session cookies if not using J2EE session management --->

<cflock type="exclusive" name="#getUniqueLockId('sessionCookies')#" timeout="10">

<cfif structKeyExists(session, "cfid") and (not structKeyExists(cookie, "cfid") or not structKeyExists(cookie, "cftoken"))>

<cfcookie name="cfid" value="#session.cfid#" />

<cfcookie name="cftoken" value="#session.cftoken#" />

</cfif>

</cflock>

</cffunction>

...

Notice how the when onRequestStart() is called in the Application.cfc file and in turn calls for the facade to handle this, a private method ensureSessionCookiesExistIfNeeded() is then run. Also notice how "this.setClientCookies" is false in the Application.cfc

This is the code which now no longer works on CF10, although I am not certain why as the code which handles the session cookie differences - ensureSessionCookiesExistIfNeeded() was not devised by us, but rather found as a "best practice" someone online. I was *certain* I had long ago found this someone on the MachII wiki as a suggestion and had followed the recommendation, however I can find no record of it on there, or in google. I figured posting it here, if it had come from someone who contributed in the MachII team maybe some light can be shed on it - it seemed self explanatory to me up till this point; especially with the comments, yet now it is not working with CF10 I looked into why it was there further.

It seems that with this code running every page request gets new CFID/CFTOKENs on each request and as such the session is not passed page to page. If I change "this.setClientCookies" to true in the Application.cfc sessions now *seem* at a quick glance to work ok. Any thoughts on this point? Is the code contained in ensureSessionCookiesExistIfNeeded() now redundant in CF10? Or was it possibly always unneeded? I was just hesitant to make changes to what worked in CF8, without exploring possible consequences as the sites are still all live in production environments using CF8 and will continue to remain this way for the foreseeable future. Thus any code changes to ensure workability for the CF10 setup could possibly throw out working live sites if uploaded.

Hope all that makes sense!

Best,

Shaun

On Friday, June 1, 2012 1:57:17 PM UTC+10, Shaun wrote:

[Kurt Wiersma]

So it sounds like #1 below is the only Mach II related issue. Could we get a simple app that reproduces the error like I mentioned in the last email?

1) The UNC paths would cause an issue in the framework when using expandRelativePath() and having the "\\" missing from the start of the path. Easy fix and MachII is happy.

--Kurt

[Peter J. Farrell]

Kurt Wiersma said the following on 06/04/2012 03:28 PM:

So it sounds like #1 below is the only Mach II related issue. Could we get a simple app that reproduces the error like I mentioned in the last email?

1) The UNC paths would cause an issue in the framework when using expandRelativePath() and having the "\\" missing from the start of the path. Easy fix and MachII is happy.

Please file a ticket here about UNC paths and expandRelativePath() and I get just it fix:

https://github.com/Mach-II/Mach-II-Framework/issues/new

Or clone our GIT repo and make a pull request with the fix.

.pjf

-- 
Peter J. Farrell
OpenBD Steering Committee / Mach-II Lead Developer
pe...@mach-ii.com
p...@maestropublishing.com
http://blog.maestropublishing.com
Identi.ca / Twitter: @maestrofjp

[Shaun]

Ok guys - issue posted on github.

Best,

Shaun

[Kurt Wiersma]

Shaun,

Could you attach your Mach II config file to the issue as well anything else we need to reproduce it? Also we would love to get your patch incorporated if it fixes the issue. Could submit it as a git pull request? It looks like the code you tried to put in the issue didn't get saved.

https://github.com/Mach-II/Mach-II-Framework/issues/76

Thanks,

--Kurt

--