Re: Help getting an AppleScript to run as a text filter

[Rich Siegel]

On Wednesday, January 16, 2013, Adam Engst <a...@tidbits.com> wrote:

>The problem is that the AppleScript works fine when run from
>AppleScript Editor, but throws an error when run as a text
>filter from within BBEdit. [...] Any suggestions on how to fix
>this script so I can embed it in a text factory? Thanks!

Wait. Text *filter*, or text *factory*?

It's an important difference because while there's a little bit
of flexibility in how text *filter* scripts may be called, a
text *factory* script must conform to a specific calling
convention (because of the runtime requirements of text factories).

If you can clear that up, I bet someone can give you an informed
and accurate answer. :-)

R.
--
Rich Siegel Bare Bones Software, Inc.
<sie...@barebones.com> <http://www.barebones.com/>

Someday I'll look back on all this and laugh... until they
sedate me.

[John Delacour]

On 16/01/2013 17:32, Adam Engst wrote:

> (This paragraph is an example of lazylinks - the two links should be
> numbered 1 and 2.)
>
> [*]: http://tidbits.com/
> [*]: https://leanpub.com/

Adam,

I’m afraid I find AppleScript impossibly verbose for this kind of simple
text munging. I don’t know what your docs look like but, supposing each
link is on a separate line, then the Perl script below, saved as a text
filter, will change all the asterisks to the appropriate number. If
not, then the script can almost certainly be modified to do the job,
however the doc is laid out. This script presumes that the docs have
UNIX line endings. If they don’t then again it can easily be modified.
If you post a snippet from a typical doc then it will only take a few
moments to write a working filter


#!/usr/bin/perl
while (<>) {
if (m~http://tidbits~) { s~\[\*\]~[1]~ }
if (m~https://leanpub~) { s~\[\*\]~[2]~ }
print;
}

JD

[Christopher Stone]

Hey Adam,

Hmm.  That script doesn't work for me.  [I believe I've figured it out.]

Your script runs fine from BBEdit's script menu.

  ~/Library/Application Support/BBEdit/Scripts/Number Markdown * Links.scpt

On Jan 16, 2013, at 11:32, Adam Engst <a...@tidbits.com> wrote:

But every now and then I need to number the links for compatibility with a system that doesn't understand lazylinks, such as [Leanpub][*]. (This paragraph is an example of lazylinks - the two links should be numbered 1 and 2.)

Okay.  I think I figured it out:

You're trying to link unenumerated markers in article text to their associated footnoted URLs (again unenumerated)?  (This wasn't completely clear from the example you posted - to me at least.)

-------------------------------------------------------------------------------------------

Input:

-------------------------------------------------------------------------------------------

A TidBITS Reference in the article body [*]

A LeanPub Reference in the article body [*]

...

[*]: http://tidbits.com/

[*]: https://leanpub.com/

-------------------------------------------------------------------------------------------

Output:

-------------------------------------------------------------------------------------------

A TidBITS Reference in the article body ][1]

A LeanPub Reference in the article body ][2]

...

[1]: http://tidbits.com/

[2]: https://leanpub.com/

The problem is that the AppleScript works fine when run from AppleScript Editor, but throws an error when run as a text filter from within BBEdit.

Applescripts run from the script menu.  They do not run as 'Text Filters', and their utility is constrained when run as an 'Applescript Filter' from within a 'Text Factory' to INPUT-->OUTPUT — so you cannot process piecemeal as you're doing in the script — you must instead generate the entire output.

  on ApplyTextTransform (fileData)

      -- do something to fileData

      return fileData -- or some reasonable facsimile thereof

  end

The only reason to add an Applescript to a Text Factory is if you need capabilities in the TF that aren't available directly to Applescript.  Is that true in your use-case?

If you really need to do this I can probably help.

It looks like JD missed the boat with his Perl script, but I'm sure he can fix it in a trice.  It certainly would be more efficient than using Applescript to iterate through the ref-links in BBEdit.

If I was going to use Applescript I'd probably use the Satimage.osax to provide more options.  For instance I would do some basic error-checking and make certain the number of references and their targets matched up.

This basically emulates what you've done in BBEdit, but it's faster.  (Tested with 50 ref-links.)

-------------------------------------------------------------------------------------------

try

  

  tell application "BBEdit" to set _text to text of front text window

  

  set refList to find text "\\[\\*\\]([^:]|$)" in _text with regexp, all occurrences and string result

  set targetList to find text "\\[\\*\\]:" in _text with regexp, all occurrences and string result

  

  if length of refList ≠ length of targetList then

    error "The number of references and links are unequal."

  else

    repeat with i from 1 to length of refList

      set {sPos, _len} to {matchPos, matchLen} of (find text "\\[\\*\\]([^:]|$)" in _text with regexp)

      set _text to change "\\[\\*\\]" into "][" & i & "]" in _text starting at sPos for _len with regexp

      set {sPos, _len} to {matchPos, matchLen} of (find text "\\[\\*\\]:" in _text with regexp)

      set _text to change "\\*" into (i as text) in _text starting at sPos for _len with regexp

      

    end repeat

    

    tell application "BBEdit" to set text of front text window to _text

    

  end if

  

on error e number n

  set e to e & return & return & "Num: " & n

  tell me to set dDlg to display dialog e with title "ERROR!" buttons {"Cancel", "Copy", "OK"} default button "OK"

  if button returned of dDlg = "Copy" then set the clipboard to e

  

end try

-------------------------------------------------------------------------------------------

--

Take Care,

Chris

[Adam Engst]

On Wed, Jan 16, 2013 at 5:06 PM, John Delacour <johnde...@gmail.com> wrote:
> I’m afraid I find AppleScript impossibly verbose for this kind of simple
> text munging. I don’t know what your docs look like but, supposing each
> link is on a separate line, then the Perl script below, saved as a text
> filter, will change all the asterisks to the appropriate number. If not,
> then the script can almost certainly be modified to do the job, however the
> doc is laid out. This script presumes that the docs have UNIX line endings.
> If they don’t then again it can easily be modified. If you post a snippet
> from a typical doc then it will only take a few moments to write a working
> filter

Thanks, John! I've never learned any perl, so I can't really parse
what your script is doing, but it's not doing quite what I need. Take
a look at these two files for a before and after view:

https://dl.dropbox.com/u/574336/AutoRip-original.md

and

https://dl.dropbox.com/u/574336/AutoRip-numbered.md

Basically, for the first Markdown source/destination pair (the source
in the text, the destination with a URL after the paragraph), the
asterisks need to be replace by 1, and then for the next pair, 2, and
so on, sequentially.

cheers... -Adam

[Ronald J Kimball]

On Thu, Jan 17, 2013 at 11:10:05AM -0500, Adam Engst wrote:
> Thanks, John! I've never learned any perl, so I can't really parse
> what your script is doing, but it's not doing quite what I need. Take
> a look at these two files for a before and after view:
>
> https://dl.dropbox.com/u/574336/AutoRip-original.md
>
> and
>
> https://dl.dropbox.com/u/574336/AutoRip-numbered.md
>
> Basically, for the first Markdown source/destination pair (the source
> in the text, the destination with a URL after the paragraph), the
> asterisks need to be replace by 1, and then for the next pair, 2, and
> so on, sequentially.

Like John, I write Perl, but not AppleScript, so I'll offer this:

#!perl

my $src_counter = my $dest_counter = 0;

while (<>) {
# replace source references
s/\]\[\*\]/ '][' . ++$src_counter . ']' /eg;

# replace destination references
s/^\s*\[\*\]:/ '[' . ++$dest_counter . ']:' /e;

print;
}

The first substitution finds the source references, which always look like
'][*]', and replaces each one with an incremented counter.

The second substitution finds the destination references, which always look
like '[*]:' at the beginning of a line, and replaces each one with another
incremented counter.

\s* allows for optional whitespace, in case you decide to indent the
destination references.

/e tells Perl to execute the replacement as Perl code.

/g tells Perl to repeat the substitution until it stops matching. There
are never two destination references on the same line, so /g is unnecessary
on the second substitution.

HTH,
Ronald

P.S. Thank you for all you do with TidBITS!

[John Delacour]

On 17/01/2013 16:10, Adam Engst wrote:
> ...Thanks, John! I've never learned any perl, so I can't really parse

> what your script is doing, but it's not doing quite what I need. Take
> a look at these two files for a before and after view:
> https://dl.dropbox.com/u/574336/AutoRip-original.md and
> https://dl.dropbox.com/u/574336/AutoRip-numbered.md

Right, I see what you want now. The text filter below does it for files
formatted as AutoRip-original.md. You can save it as arip.pl in your
text filters folder. The script will replace the _first_, and only the
first, occurrence in the line of «[*]» with «[n]» and add 1 to n every
time it does so.

#!/usr/bin/perl
my $n = 1; # set n to 1
while (<>) { # read the file line by line
$n++ if s~\[\*\]:~[$n]~ ;
# if found [*], replace with [n] and set n to n+1
print; # write the line
}

Let me know if you have any difficulty with it. I think you’ll agree
AppleScript looks a little verbose in comparison!

JD

[Adam Engst]

Sorry for the delayed response here - Google Groups apparently decided not to send me email despite the Email Updates To Me setting, but since John cc'd me on his replies, I thought he was the only who had written back...

On Wednesday, January 16, 2013 2:36:04 PM UTC-5, Rich Siegel wrote:

On Wednesday, January 16, 2013, Adam Engst <a...@tidbits.com> wrote:

>The problem is that the AppleScript works fine when run from
>AppleScript Editor, but throws an error when run as a text
>filter from within BBEdit. [...] Any suggestions on how to fix
>this script so I can embed it in a text factory? Thanks!

Wait. Text *filter*, or text *factory*?

It's an important difference because while there's a little bit
of flexibility in how text *filter* scripts may be called, a
text *factory* script must conform to a specific calling
convention (because of the runtime requirements of text factories).

If you can clear that up, I bet someone can give you an informed
and accurate answer. :-)

Yes, that seems to be more important than I'd realized. Sorry!

What I'm trying to do is create a text _factory_ that does a few standard grep replacements, and then calls something else to do this renumbering (since there's no other way to do the renumbering from within a text factory that I can think of). In the text factory, there are options for AppleScript Filter and Unix Filter, and I naively assumed those meant text _filters_.

So while the AppleScript I tweaked originally and Ronald's perl script both work, and Ronald's perl script even works from the Text Filters menu, neither of them can be called from within a text _factory_. 

So the question becomes, how would such a perl script or AppleScript need to be written to work from within a text _factory_?

cheers... -Adam

[Christopher Stone]

On Jan 23, 2013, at 11:16, Adam Engst <a...@tidbits.com> wrote:
> What I'm trying to do is create a text _factory_ that does a few standard grep replacements, and then calls something else to do this renumbering (since there's no other way to do the renumbering from within a text factory that I can think of).

______________________________________________________________________

Hey Adam,

The most efficient way to do this would be to do all the replacements in Perl.

Ronald, or JD, or another prop-jet-head can give you a template for the find/replace that you can replicate and edit yourself.

Absent that I'd write it all as one Applescript (which can contain the Perl) rather than futz around with the pieces of a TextFactory.

--
Take Care,
Chris

[Adam Engst]

On Wed, Jan 23, 2013 at 1:33 PM, Christopher Stone
<listm...@suddenlink.net> wrote:
> On Jan 23, 2013, at 11:16, Adam Engst <a...@tidbits.com> wrote:
>> What I'm trying to do is create a text _factory_ that does a few standard grep replacements, and then calls something else to do this renumbering (since there's no other way to do the renumbering from within a text factory that I can think of).
>

> The most efficient way to do this would be to do all the replacements in Perl.
>
> Ronald, or JD, or another prop-jet-head can give you a template for the find/replace that you can replicate and edit yourself.

Great - hopefully they'll know the trick for getting it to work as a
Unix filter within a text factory.

> Absent that I'd write it all as one Applescript (which can contain the Perl) rather than futz around with the pieces of a TextFactory.

I'm just being bloody-minded here, since I believe this should be
doable entirely within a text factory. :-) If it's not, I'll just have
Keyboard Maestro call the text factory and then run one of the scripts
I already have. Besides, much as I love Keyboard Maestro, gotta learn
new things...

cheers... -Adam

[Rich Siegel]

On Wednesday, January 23, 2013, Adam Engst <a...@tidbits.com> wrote:

> So the question becomes, how would such a perl script or AppleScript need
> to be written to work from within a text _factory_?

Here's a quick excerpt from the manual:

When you use the Run AppleScript Filter operation, your script
should be written with an entry point named “ApplyTextTransform”.
The input parameter to this entry point is a Unicode string
containing the file’s contents. This entry point should return the
file’s contents as a Unicode string (or something which can be
directly coerced to one):

on ApplyTextTransform (fileData)
-- do something to fileData
return fileData -- or some reasonable facsimile thereof
end

• Unix filters run from text factories get their input on STDIN, and
should write any content they want passed to the next filter stage
to STDOUT. (Unlike Unix filters run from the Shebang menu, text
factories do not pass a temp file in argv[1].)

Enjoy,

[John Delacour]

On 23/01/2013 17:16, Adam Engst wrote:

> What I'm trying to do is create a text _factory_ that does a few
> standard grep replacements, and then calls something else to do this

> renumbering...So the question becomes, how would such a perl script or

> AppleScript need to be written to work from within a text _factory_?

I think the question is rather, why not do the whole thing with a Perl
text filter? Then your text filter to do the whole lot would be
something like the script below. This can be run, with or without a key
shortcut, from the Text Filters palette:

#!/usr/bin/perl
my $n = 1; # set n to 1
while (<>) { # read the file line by line

s~q~q~g; # your “a few standard grep replacements”
s~x~x~g; # ie. replace all ‘x’ in the line with ‘x’
s~y~y~g;
s~a~a~g;

$n++ if s~\[\*\]:~[$n]~ ;
# if found [*], replace with [n] and set n to n+1
print;
}

JD

[Adam Engst]

On Thu, Jan 24, 2013 at 5:10 AM, John Delacour <johnde...@gmail.com> wrote:
> I think the question is rather, why not do the whole thing with a Perl text
> filter? Then your text filter to do the whole lot would be something like
> the script below. This can be run, with or without a key shortcut, from the
> Text Filters palette:

Man, you're going to make me learn perl grep syntax, aren't you? :-)

I see roughly how to do it, and I can do simple substitutions in perl,
but I'm having trouble getting the more complex ones to run. I've
figured out that you need to dot-combine literal characters with grep
in the replacements, but I'm getting SCALAR and some numbers instead
of my grep replacements, and there's some distinction between /n and
/r I'm not yet understanding (since things work better with \n than
\r, but I always use \r in BBEdit).

s/^Title: / '#' /e;
s/^Blurb: (.+)/ 'A>' . \1/e;
s/^(by.+)/ '_' . \1 . '_'/ie;
s/^\*\*(.+)\*\* (-*)( *)/ '## ' . \1 . \r . \r/e;

cheers... -Adam

[Ronald J Kimball]

On Thu, Jan 24, 2013 at 09:51:25AM -0500, Adam Engst wrote:
> Man, you're going to make me learn perl grep syntax, aren't you? :-)
>
> I see roughly how to do it, and I can do simple substitutions in perl,
> but I'm having trouble getting the more complex ones to run. I've
> figured out that you need to dot-combine literal characters with grep
> in the replacements, but I'm getting SCALAR and some numbers instead
> of my grep replacements, and there's some distinction between /n and
> /r I'm not yet understanding (since things work better with \n than
> \r, but I always use \r in BBEdit).
>
> s/^Title: / '#' /e;
> s/^Blurb: (.+)/ 'A>' . \1/e;
> s/^(by.+)/ '_' . \1 . '_'/ie;
> s/^\*\*(.+)\*\* (-*)( *)/ '## ' . \1 . \r . \r/e;

/e is only necessary if you need to execute Perl code in the replacement.
If the replacement is just a string, you don't need it.

In the replacement, $1 is preferred over \1.

Perl uses \n to mean 'the native line-ending', so you should generally use
that instead of \r.

s/^Title: /#/;
s/^Blurb: (.+)/A>$1/;
s/^(by.+)/_${1}_/i;
s/^\*\*(.+)\*\* -* */## $1\n\n/;


By the way, the SCALAR followed by some numbers was caused by the
combination of /e and \1. \1 *as an expression* is a reference to the
scalar value 1. When you stringify a reference, you get the type followed
by the memory address.

Ronald

[John Delacour]

On 24/01/2013 14:51, Adam Engst wrote:

> Man, you're going to make me learn perl grep syntax, aren't you? :-)

Perl regular expressions are the basis of the PCRE used in BBEdit, so
there’s very little to learn if you’re familiar with PCRE. The main
difference is that Perl uses $&, $1, $2 for PCRE’s \0, \1, \2 ; but
you’re probably not using those anyway.

> I see roughly how to do it, and I can do simple substitutions in
> perl, but I'm having trouble getting the more complex ones to run.
> I've figured out that you need to dot-combine literal characters with
> grep in the replacements, but I'm getting SCALAR and some numbers

> instead of my grep replacements.

The trouble you’re having is due to your use of ///e, which treats the
substitution pattern as an executable string. Forget about using that,
almost ever. For one thing, servers won’t allow you to use it in CGI
scripts because you could use the trick to blow up their machines or the
world. I’d also forget about using ///x to allow spaces. Just get used
to reading the things. See
http://docstore.mik.ua/orelly/perl/perlnut/ch04_06.htm or read perlretut.

> ...and there's some distinction between

> /n and /r I'm not yet understanding (since things work better with \n
> than \r, but I always use \r in BBEdit).

You are reading line by line so \r and \n don’t come into it. Every
line ends with a line-ending (!!) and if you want to get rid of it, just
do ;chomp; but you probably don’t. I also use ‘~’ as the delimiter for
RE, because I very rarely have ‘~’ in a document but often have ‘/’s and
don’t want to have to escape them.

> s/^Title: / '#' /e;

s~Title: ~#~

> s/^Blurb: (.+)/ 'A>' . \1/e;

s~^Blurb: (.+)~'A>'$1~ # ???

I can’t work out quite what you mean by these replacements but it looks
as if you’re making things far more complicated for yourself than you
need to. You know how to use RE patterns, so forget about the few
differences between PCRE and Perl RE, use ‘~’ for convenience instead of
‘/’ and get on with it.

You can make a test filter as below and run it on a frontmost test
document to see what it does. The changes made by the filter can be
undone with ⌘Z. Make sure you save the changes in the filter script
before running it from the palette.

while (<>) {
s~~~g; # g means substitute all
s~~~i; # i means disregard caSe
s~~~g;
}

Hope this is readable. I hate Thunderbird and I hate Mail even more.
10.8 has robbed me of the only mail program that was ever any good.

JD

[Patrick Woolsey]

At 14:46 -0500 01/23/2013, Adam Engst wrote:

>I'm just being bloody-minded here, since I believe this should be
>doable entirely within a text factory. :-)

It is :-) and either John's or Ronald's scripts should work when applied
from within a text factory. (NB: You will need to edit the shebang line in
Ronald's script to specify the full path to the 'perl' binary.)

I'll also follow up to your prior post with some additional info in a few
minutes.


Regards,

Patrick Woolsey
==
Bare Bones Software, Inc. <http://www.barebones.com/>

[Patrick Woolsey]

At 09:16 -0800 01/23/2013, Adam Engst wrote:
[...]

>What I'm trying to do is create a text _factory_ that does a few standard
>grep replacements, and then calls something else to do this renumbering
>(since there's no other way to do the renumbering from within a text
>factory that I can think of).

That's correct; there are no native actions which can directly do this.

>In the text factory, there are options for AppleScript Filter and
>Unix Filter, and I naively assumed those meant text _filters_.

They do :-) but since the terminology can get a bit convoluted, in hopes of
clearing things up a bit:

A _text filter_ is any executable code that meets the conditions given in
the manual (under "Apply Text Filter" on pg. 112), and which you can apply
to a document's contents via the Text -> Apply Text Filter submenu.

A _script_ is any executable code that you can run from BBEdit's Scripts menu.

Therefore, although a script like your original AppleScript may also be
capable of modifying text, it is not a text filter, i.e. it doesn't meet
the specific conditions needed for stand-alone use.

Beyond that:

You *can* use Unix text filters directly within a text factory, via the
'Run Unix Filter' action.

You *may* be able to use an AppleScript text filter within a text factory,
depending on how it's structured (i.e. it must contain the
"ApplyTextTransform" entry point).

In general, however, you *cannot* run Scripts menu code (either AppleScript
or Unix) within a text factory.

>So while the AppleScript I tweaked originally and Ronald's perl script
>both work, and Ronald's perl script even works from the Text Filters menu,
>neither of them can be called from within a text _factory_.

Ronald's Perl script should work when called from within a text factory if
you just edit its shebang line to use the full path to 'perl':

#!/usr/bin/perl
...

Your AppleScript, however, won't work because it requires access to a
document. (Please see my comments below regarding this.)

>So the question becomes, how would such a perl script or AppleScript need
>to be written to work from within a text _factory_?

A Unix filter (e.g. a Perl script) must accept input on STDIN and write the
modified text to STDOUT.

An AppleScript must follow the rules given in the manual (as Rich also
mentioned in his previous post):

* When you use the Run AppleScript Filter operation, your script

should be written with an entry point named "ApplyTextTransform".
The input parameter to this entry point is a Unicode string
containing the file's contents. This entry point should return
the file's contents as a Unicode string (or something which can
be directly coerced to one):

====

on ApplyTextTransform (fileData)
-- do something to fileData
return fileData
-- or some reasonable facsimile thereof
end

====

It's important to keep in mind here that such an AppleScript has no
document to work with; it must instead directly manipulate the text (i.e.
the string) it receives from BBEdit.

[LuKreme]

On 24 Jan 2013, at 12:04 , Patrick Woolsey <pwoo...@barebones.com> wrote:
> It's important to keep in mind here that such an AppleScript has no
> document to work with; it must instead directly manipulate the text (i.e.
> the string) it receives from BBEdit.

This has turned into one of the most interesting threads on the list in a long time. I've learned tons, so I've flagged it so maybe at some point in the future I will refer back to it when I need to do something really interesting and complicated with text factories.

--
Always be sincere, even if you don't mean it.

[Adam Engst]

On Thu, Jan 24, 2013 at 5:17 PM, LuKreme <kre...@kreme.com> wrote:
> This has turned into one of the most interesting threads on the list in a long time. I've learned tons, so I've flagged it so maybe at some point in the future I will refer back to it when I need to do something really interesting and complicated with text factories.

I'd like to second this - thanks to Ronald and John and Patrick, I now
know a lot more about how BBEdit works (and even a tiny bit about
perl) such that I'll be even more ambitious next time I have some
funky text-munging to do. Thanks, guys!

As an exercise, I got both the text factory (using the perl text
filter) and and perl text filter on its own to work.

I feel like I've eaten my tech wheaties for the day!

BBEdit and perl FTW!

cheers... -Adam

[John Delacour]

On 24/01/2013 23:26, Adam Engst wrote:

> BBEdit and perl FTW! cheers...

As a postscript to this thread I think it’s worth emphasizing how
wonderfully simple it is to use Perl RE in text filters by explaining
just what is happening. BBEdit has set things up so that the input to
the text filter is the whole text of the front document, unless a
selection is highlighted, in which case the selection alone is taken as
the input and the remainder of the text left intact.

A text filter saved with the suffix .pl or beginning with the shebang
#!/usr/bin/perl will be recognized as a Perl filter.

The array (list) of lines in the text is written <STDIN> (= lines in
Standard Input) or simply <>, and the ‘while’ block iterates through
this array item by item; that is to say it processes the text line by line.

we have:
#!/usr/bin/perl
use strict;
while (<>) {
# do this; do that; do the other
s~^(\s*)([a-z])~\U$1$2~; # capitalize the first letter in the line
print;
}

This means: for each line of input, do these things and write the
(modified) line.

Without needing to learn any Perl at all more than this it is thus easy
for anybody who has gone to the trouble of learning how to use PCRE in
BBEdit to write Perl text filters to do substitutions using what is
_almost_ the same lingo. Anyone who does this is bound to get a craving
for a little bit more, and again it will be necessary to learn only a
tiny bit more Perl syntax at a time and not the whole lot.

JD