echo-ln as output, line-iterator as input

[argherna]

In main.grammar, the echo-ln task is an EchoTask with a suffix set to
a carriage-return line-feed (the default for Windows). If you write a
file with output from an echo-ln task (using print-stream) that is
meant to be input for another script that is read by line-iterator, an
exception is thrown because cernunnos complains about the carriage-
return. For example:

<!-- This script generates a csv file for input to another script -->
<print-stream file="input-to-batch-process2.csv">
<sql-datasource>
<sql-query>
<sql>select * from SOME_USER_INFO_TABLE</sql>
<subtasks>
<echo-ln>${1},${2},${3}</echo-ln>
</subtasks>
</sql-datasource>
</print-stream>

<!-- This script uses the csv file output from the previous script as
input -->
<line-iterator string="input-to-batch-process2.csv">
<with-attribute key="args" value="${groovy([])}">
<token-iterator regex=",">
<groovy><script>args.add('${Attributes.STRING}')</script></groovy>
</token-iterator>
<with>
<attribute key="$1">${groovy(args[0])}</attribute>
<subtasks>
<print-stream file="netids">
<crn location="get-netid.crn"/>
</print-stream>
</subtasks>
</with>
</with-attribute>
</line-iterator>

The second script will fail.

There are some easy work arounds for this (e.g. change echo-ln to echo
and specify a suffix of &#10; (\n) or change line-iterator to a token-
iterator and specify &#10; (\n) as the regex) but this can catch you
off-guard. But should the task definitions in main.grammar be changed
to have echo-ln and line-iterator be consistent? Another question is
would it be possible to change the task entry in echo-ln and line-
iterator to default to the system's line.separator by default?

[Eric Dalquist]

Perhaps both should use:

System.getProperty("line.separator");

-Eric

argherna wrote:

In main.grammar, the echo-ln task is an EchoTask with a suffix set to
a carriage-return line-feed (the default for Windows).  If you write a
file with output from an echo-ln task (using print-stream) that is
meant to be input for another script that is read by line-iterator, an
exception is thrown because cernunnos complains about the carriage-
return. For example:

<!-- This script generates a csv file for input to another script -->
<print-stream file="input-to-batch-process2.csv">
	<sql-datasource>
		<sql-query>
			<sql>select * from SOME_USER_INFO_TABLE</sql>
			<subtasks>
				<echo-ln>${1},${2},${3}</echo-ln>
			</subtasks>
	</sql-datasource>
</print-stream>

<!-- This script uses the csv file output from the previous script as
input -->
<line-iterator string="input-to-batch-process2.csv">
	<with-attribute key="args" value="${groovy([])}">
		<token-iterator regex=",">
			<groovy><script>args.add('${Attributes.STRING}')</script></groovy>
		</token-iterator>
		<with>
			<attribute key="$1">${groovy(args[0])}</attribute>
			<subtasks>
				<print-stream file="netids">
					<crn location="get-netid.crn"/>
				</print-stream>
			</subtasks>
		</with>
	</with-attribute>
</line-iterator>

The second script will fail.

There are some easy work arounds for this (e.g. change echo-ln to echo
and specify a suffix of &#10; (\n) or change line-iterator to a token-
iterator and specify &#10; (\n) as the regex) but this can catch you
off-guard.  But should the task definitions in main.grammar be changed
to have echo-ln and line-iterator be consistent?  Another question is
would it be possible to change the task entry in echo-ln and line-
iterator to default to the system's line.separator by default?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Cernunnos Discussion" group.
To post to this group, send email to cernunnos-...@googlegroups.com
To unsubscribe from this group, send email to cernunnos-discus...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/cernunnos-discussion?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Eric Dalquist]

I was looking at how to fix this and I think we'll need to wait for Drew to get back. The echo-line task is the same task as echo with a default suffix of \r\n. Perhaps we add a new property on EchoTask that is a boolean for a OS dependent line ending so that we can use the system property?

I'd also like to propose switching the expected type of the content of an echo or echo-line from String to Object. That way you don't get ugly warnings when you pass an object in that you just want to do a toString on. The code already functions correctly you just get a type-check warning in the logs.

-Eric

Eric Dalquist wrote:

Perhaps both should use:

System.getProperty("line.separator");

-Eric

argherna wrote:

In main.grammar, the echo-ln task is an EchoTask with a suffix set to
a carriage-return line-feed (the default for Windows).  If you write a
file with output from an echo-ln task (using print-stream) that is
meant to be input for another script that is read by line-iterator, an
exception is thrown because cernunnos complains about the carriage-
return. For example:

<!-- This script generates a csv file for input to another script -->
<print-stream file="input-to-batch-process2.csv">
	<sql-datasource>
		<sql-query>
			<sql>select * from SOME_USER_INFO_TABLE</sql>
			<subtasks>
				<echo-ln>${1},${2},${3}</echo-ln>
			</subtasks>
	</sql-datasource>
</print-stream>

<!-- This script uses the csv file output from the previous script as
input -->
<line-iterator string="input-to-batch-process2.csv">
	<with-attribute key="args" value="${groovy([])}">
		<token-iterator regex=",">
			<groovy><script>args.add('${Attributes.STRING}')</script></groovy>
		</token-iterator>
		<with>
			<attribute key="$1">${groovy(args[0])}</attribute>
			<subtasks>
				<print-stream file="netids">
					<crn location="get-netid.crn"/>
				</print-stream>
			</subtasks>
		</with>
	</with-attribute>
</line-iterator>

The second script will fail.

There are some easy work arounds for this (e.g. change echo-ln to echo
and specify a suffix of &#10; (\n) or change line-iterator to a token-
iterator and specify &#10; (\n) as the regex) but this can catch you
off-guard.  But should the task definitions in main.grammar be changed
to have echo-ln and line-iterator be consistent?  Another question is
would it be possible to change the task entry in echo-ln and line-
iterator to default to the system's line.separator by default?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Cernunnos Discussion" group.
To post to this group, send email to cernunnos-...@googlegroups.com

[Andrew Wills]

Hey folks,

Sorry it took me so long to post back to the list; it's been a
challenging week, getting back from Italy.

On Fri, Jan 30, 2009 at 1:08 PM, Eric Dalquist
<eric.d...@doit.wisc.edu> wrote:
> I was looking at how to fix this and I think we'll need to wait for Drew to
> get back. The echo-line task is the same task as echo with a default suffix
> of \r\n. Perhaps we add a new property on EchoTask that is a boolean for a
> OS dependent line ending so that we can use the system property?
>

To recap... <line-iterator> specifies "&#10;" whereas <echo-ln>
specifies "&#13;&#10;" -- "\n" and "\r\n" respectively -- for line
termination. Yes, this is an opportunity for improvement.

I think we can handle it like this (in main.grammar):

<entry name="line-iterator"
impl="org.danann.cernunnos.core.TokenIteratorTask">
..
<content-model
regex="${org.danann.cernunnos.core.SystemPropertyPhrase(line.separator)}"/>
</entry>

Done. I created Issue #74 to track it:
http://code.google.com/p/cernunnos/issues/detail?id=74

> I'd also like to propose switching the expected type of the content of an
> echo or echo-line from String to Object. That way you don't get ugly
> warnings when you pass an object in that you just want to do a toString on.
> The code already functions correctly you just get a type-check warning in
> the logs.
>

This is also a good idea -- done.

I created Issue #73 to track it:
http://code.google.com/p/cernunnos/issues/detail?id=73

drew wills

PS: I'd really like to give Issue #70 a crack; I hope to take a good
look here soon.