Re: Sort and disply duplicate lines wtih number in front

[Christopher Stone]

On Aug 26, 2012, at 18:11, Jim Walker <tvcn...@gmail.com> wrote:
> We lost the feature available through Linesort in previous versions of BBedit it appears.
>
> I have a list of 1 million IP addresses and I need to sort, remove duplicates and display to left the number of times the IP address appears in file.

______________________________________________________________________

Hey Jim,

LineSort was a nice plugin, but most of its functionality is available in BBEdit these days.

You can run something like this from the Terminal or from a BBEdit worksheet.

sort ~/Desktop/sort_test.txt | uniq -c

--
Best Regards,
Chris

[Bucky Junior]

Jim,

Chris' terminal line works great. I use something similar but create a new sorted file so I don't have to deal trying to get the info from the screen. The BBEdit worksheet will capture it.

sort tempfile0 | uniq -c | sort >tempfile1

> --
> --
> You received this message because you are subscribed to the
> "BBEdit Talk" discussion group on Google Groups.
> To post to this group, send email to bbe...@googlegroups.com
> To unsubscribe from this group, send email to
> bbedit+un...@googlegroups.com
> For more options, visit this group at
> <http://groups.google.com/group/bbedit?hl=en>
> If you have a feature request or would like to report a problem,
> please email "sup...@barebones.com" rather than posting to the group.
> Follow @bbedit on Twitter: <http://www.twitter.com/bbedit>
>
>
>

[Steve Kalkwarf]

On Aug 31, 2012, at 6:11 PM, Jim Walker <tvcn...@gmail.com> wrote:

> I'm not 100% clear what you mean by BBedit worksheet.

File->New->Shell Worksheet

It's a hybrid document/shell session, inspired by Apple's old MPW editor

> I see there is a script editor option with some sort options there.

Sure, but that's a lot of work.

> Would be cool if there was a sort and number dupes.

Text->Sort Lines
Text->Number Lines

It you want to do this many times, File->New->Text Factory, and build the transform yourself out of the two operations.

Steve

[Maarten Sneep]

On 1 sep. 2012, at 00:11, Jim Walker <tvcn...@gmail.com> wrote:

> I'm not 100% clear what you mean by BBedit worksheet.
>

> I see there is a script editor option with some sort options there.

> Would be cool if there was a sort and number dupes.

1) File > New > Shell Worksheet
If you've never used it, this should open a worksheet with some default examples.

2) Type

osascript -e 'tell application "BBEdit" to get contents of text document 1' | tr '\r' '\n' | sort | uniq -c | bbedit --clean

into the worksheet.

3) Open the log-file in BBEdit, make sure it is the frontmost text document.
4) go back to the shell worksheet and select the line you just typed.
5) press 'enter' (⌅) or 'command+return' (⌘+⏎).

A new text document should open with the unique lines, prepended with the number of occurrences.
Of course, you could use the terminal, but this worksheet can be saved and reused.

The command can be broken up into:

# ask BBEdit to dump the contents of text window 1 to 'standard output'
osascript -e 'tell application "BBEdit" to get contents of text document 1'

# BBEdit uses \r internally, while a unix toolchain expects \n. tr translates:
tr '\r' '\n'

# sort the lines
sort

# find all unique entries. uniq requires its input to be sorted.
uniq -c

# Open the result in a new window in BBEdit.
bbedit --clean


The shell can be very powerful, with many tools that can be strung together in ways that even Bare Bones can't imagine.

Now a few things would be nice:

a) an option for the BBEdit commandline tool to dump the contents of a specified window to stdout, including file-based line endings (see tr).
b) Adding a unique count to the unique lines/sort routine.

I'll file a feature request

Maarten

[Marshall Clow]

On Mar 23, 2013, at 5:39 PM, Jim Walker <tvcn...@gmail.com> wrote:

Can I pay someone to write a script which I can added to BBedit
so I can run the scripts, like linesort used to work, and get the totals like below?

is there some reason that 

sort | uniq -c 

won't do what you want?

-- Marshall

-JIm

On Sunday, August 26, 2012 4:11:54 PM UTC-7, Jim Walker wrote:

We lost the feature available through Linesort in previous versions of BBedit it appears.

I have a list of 1 million IP addresses and I need to sort, remove duplicates and display to left the number of times the IP address appears in file.

Like:
1   52.197.114.194
25   32.300.114.194
128  92.197.114.190
254   12.197.3004.194
942   22.197.114.194

Any way to this in the "new" version of BBedit?

Thanks,
Jim

--
--
You received this message because you are subscribed to the
"BBEdit Talk" discussion group on Google Groups.
To post to this group, send email to bbe...@googlegroups.com
To unsubscribe from this group, send email to
bbedit+un...@googlegroups.com
For more options, visit this group at
<http://groups.google.com/group/bbedit?hl=en>
If you have a feature request or would like to report a problem,
please email "sup...@barebones.com" rather than posting to the group.
Follow @bbedit on Twitter: <http://www.twitter.com/bbedit>
 

---
You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[TJ Luoma]

On Sat, Mar 23, 2013 at 8:39 PM, Jim Walker <tvcn...@gmail.com> wrote:

Can I pay someone to write a script which I can added to BBedit
so I can run the scripts, like linesort used to work, and get the totals like below?

Oh goodness, don't pay anyone to write simple shell scripts for you when the Internet is chock full of nerds who will do that for free ;-)

Here's the script you need for a Text Filter in BBEdit:

#!/bin/sh

/usr/bin/sort --numeric-sort "$@" |\

/usr/bin/uniq -c |\

/usr/bin/sort --numeric-sort

exit 0

#EOF

(What That Script Does: line 1 will sort the contents of the file, line 2 will combine duplicate lines and prefix it with a count of how many lines matched that line, and line 3 will sort that again so that the lines which have the most duplicates will be at the bottom of the list.)

To use this:

* Copy the above into a new BBEdit document and make sure that the "#!/bin/sh" line is the first line in the file 

* Save it to ~/Application Support/BBEdit/Text Filters/

(or "~/Dropbox/Application Support/BBEdit/Text Filters/" if you sync via Dropbox)

* name it something like "Linesort With Totals.sh"

* find it under the Text Filters menu:

http://images.luo.ma/bbedit/linesort/linesort-as-text-filter.jpg

NOTE:

TextFilters will REPLACE the content of your file.

If you want to leave the contents of the file alone and have the output of "Linesort With Totals.sh" to appear in another BBEdit file, you should use it as a BBEdit _Script_ which means:

Use this script instead:

#!/bin/sh

/usr/bin/sort --numeric-sort "$BB_DOC_PATH" |\

/usr/bin/uniq -c |\

/usr/bin/sort --numeric-sort

exit 0

#EOF

* put it into "~/Application Support/BBEdit/Scripts/" (or "~/Dropbox/Application Support/BBEdit/Scripts/"

* find it under the 'scripts' menu:

http://images.luo.ma/bbedit/linesort/linesort-as-script.jpg

BTW I have created a script which can be used as _either_ a Text Filter _or_ a BBEdit Script. You can find it here:

http://images.luo.ma/bbedit/linesort/Linesort%20With%20Totals.sh

(to download it, go to http://images.luo.ma/bbedit/linesort/ and then control+click on the ".sh" file)

No guarantees, use at your own risk, if it breaks you get to keep both parts, but it worked for me.

TjL

(Shell Script Nerd since the early 1990s)

[TJ Luoma]

On Sat, Mar 23, 2013 at 11:51 PM, Marshall Clow <mtc...@gmail.com> wrote:

On Mar 23, 2013, at 5:39 PM, Jim Walker <tvcn...@gmail.com> wrote:

Can I pay someone to write a script which I can added to BBedit
so I can run the scripts, like linesort used to work, and get the totals like below?

is there some reason that 

sort | uniq -c 

won't do what you want?

If I had to guess, I'd say that Jim was probably looking for a solution he could use inside BBEdit, without having to go into the Terminal.

I used BBEdit for years before I realized how easy it was to run scripts on BBEdit documents, and if you've never done it, it can seem pretty intimidating.

(As my old Comp Sci prof used to say: "Everything is easy when you know what you're doing.")

TjL

[Maarten Sneep]

On 24 mrt. 2013, at 05:15, TJ Luoma <luo...@gmail.com> wrote:

> If I had to guess, I'd say that Jim was probably looking for a solution he could use inside BBEdit, without having to go into the Terminal.
>
> I used BBEdit for years before I realized how easy it was to run scripts on BBEdit documents, and if you've never done it, it can seem pretty intimidating.
>
> (As my old Comp Sci prof used to say: "Everything is easy when you know what you're doing.")

Enter the manual:
Page 36 ("Text Filters") in the description of BBEdit's Application Support folder.
In short:
* Make a folder "$HOME/Library/Application Support/BBEdit/Text Filters" if it doesn't exist already.
* Create a text file with the following script:

#!/bin/sh

# read from stdin,
# sort by number,
# count number of unique occurrences,
# sort on the number of occurrences and finally write to stdout

/usr/bin/sort --numeric-sort | /usr/bin/uniq -c | /usr/bin/sort --numeric-sort

exit 0
# EOF

* Save this in "$HOME/Library/Application Support/BBEdit/Text Filters" under a recognizable name ('count unique IP addresses.sh')

Page 111 (Text Transformations -> Apply Text Filter). This is where your command will end up.

Open the file you need to analyze, call the script from the Text -> Apply Text Filter submenu.

Done.

Maarten