FTP Files Last Night

[Andrew Skinner]

Anyone using the FTP files as a data source and get some issues with a few log files last night?

I`ve got an hourly process, loading the log files generated in an hourly window, and then loading them into a database.  However, last night (seems like the first time since I started this about 2 weeks ago!) got 4 files that ended up in my rejected folder that won`t seem to load.

Anyone else have the same issue?

[petermount]

I just checked mine and they seemed to have worked - although I only use FTP for the Darwin reference files.

Peter

[Peter Hicks]

Hi Andrew

Why won't they load?  What's the error you get and where did it occur in the file?

Peter

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To post to this group, send email to openrail...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Skinner]

Hey Peter,

Well I`ve done an analysis of all 4 files with an issue, turned out there were 5 messages in these 4 files that had some malformed XML in them, unclosed XML tags essentially.  I loaded them as text into a ntext SQL Server column and then used the TRY_CAST function to test convert them to an XML datatype.  This allowed me to see the specific messages that weren`t properly formed XML.

 I`ve made some changes to my load process to filter any dodgy messages off somewhere.  It`s not something massive to worry about as this is the first time it`s happened.

[Peter Hicks]

Hi Andrew

> On 26 Aug 2016, at 14:01, Andrew Skinner <drkar...@gmail.com> wrote:
>
> Hey Peter,
>
> Well I`ve done an analysis of all 4 files with an issue, turned out there were 5 messages in these 4 files that had some malformed XML in them, unclosed XML tags essentially. I loaded them as text into a ntext SQL Server column and then used the TRY_CAST function to test convert them to an XML datatype. This allowed me to see the specific messages that weren`t properly formed XML.

Can you post the malformed messages here? If there is an issue, it would be good to get it fixed.


Peter

[Andrew Skinner]

Hi Peter,

Been doing some further analysis and it looks like my processing (it`s only a week old!) has hit a message type (TrainAlerts) that contains some strange characters, and probably carriage returns, which breaks the way I am processing the file.  So the file is actually ok.  I`m ignoring those now and hope nobody asks for them!

Thanks for the help though!