AI Command Shell

[SeH]

i was thinking of a command shell, or an extension to existing shell like BASH that analyzes your command history and automatically suggests macros that cover frequently used command patterns.

for example, if i end up typing commands like:

find | fgrep '.c'

find | fgrep 'y'

it would suggest or create a new macro:

find_p_fgrep $x

then allow me to rename it to a more convenient name like 'findit'

this could also work across a network, pooling everyone's most frequently used commands to generate a dictionary of macros that everyone can benefit from.

what i've found so far:

The Learning Shell: Automated Macro Construction

http://www.springerlink.com/content/ed2rpwv7f64tfykq/

By analysing sequences of actions performed by a user, one can find frequent subsequences that can be suggested as macro (script) definitions. However, often these ‘actions’ have additional features. In this paper we combine an algorithm to detect frequent subsequences with an inductive logic programming system to automatically generate for each frequent subsequence the most specific ‘template’ for these additional features that is consistent with the observed frequent subsequences. The resulting system is implemented and used in an application where we automatically generate macros from logs of the use of a Unix command shell.

[Matt Mahoney]

On Thu, May 17, 2012 at 11:49 AM, SeH <seh...@gmail.com> wrote:
> i was thinking of a command shell, or an extension to existing shell like
> BASH that analyzes your command history and automatically suggests macros
> that cover frequently used command patterns.
>
> for example, if i end up typing commands like:
>
> find | fgrep '.c'
> find | fgrep 'y'
>
>
> it would suggest or create a new macro:
>
> find_p_fgrep $x
>
>
> then allow me to rename it to a more convenient name like 'findit'
>
> this could also work across a network, pooling everyone's most frequently
> used commands to generate a dictionary of macros that everyone can benefit
> from.

That's an interesting idea. One problem might be ending up with more
macros than you can remember. Another approach might be auto-complete
by comparing what you are typing with commands you entered previously.
In either case, it's similar to the problem of compressing your
command history.


-- Matt Mahoney, mattma...@gmail.com

[james northrup]

training against strace dumps of any program, shell or otherwise would be trivial to provide for a stream of entity actions to classify

[SeH]

That's an interesting idea. One problem might be ending up with more macros than you can remember. Another approach might be auto-complete by comparing what you are typing with commands you entered previously.
In either case, it's similar to the problem of compressing your command history.
-- Matt Mahoney

absolutely, command auto-completion suggestions could certainly be involved.  

i certainly recognize the relevant application of compression and also the inductive logic solution applied in the mentioned paper.

On Thu, May 17, 2012 at 3:53 PM, james northrup wrote:

training against strace dumps of any program, shell or otherwise would be trivial to provide for a stream of entity actions to classify

yes! i think the strace or ltrace traces for a given command could indicate:

• variations in called functions and their parameters resulting from similar commands
• other possible sequences and combinations of traces that could produce novel programs
• identifying different commands that produce the same result (share similar trace segments, especially towards the "center" of a program's execution)

[YKY (Yan King Yin, 甄景贤)]

This is a good idea I think ... forward-chaining should be very suitable for this kind of processing.  It can also parse commands and give English explanations.  There's also fuzzy auto-complete in some IDEs...

I'm going to a local startup competition to introduce Genifer to some local Hong Kong people... which lasts for the weekend... will get back to this when I have time =)

KY

[Abram Demski]

SeH,

I was actually thinking about a similar idea, in the context of programming IDEs. It often happens that I write the same kind of structure twice or more, either within the same function or in several functions. Usually when I notice I'll make a function out of it, but not always. It's slightly annoying to go find all the instances of the pattern and replace them. An automatic subroutine suggestion would be nice, especially if it automatically re-factored my code.

--Abram

--
Abram Demski
http://lo-tho.blogspot.com/

[SeH]

this is one program that can compare similar code  segments..

http://dickgrune.com/Programs/similarity_tester/

whether this happens in a command shell or IDE is pretty much the same thing since this application would blur the lines between them.

for example, the strace/ltrace features Jim suggested would be analogous to JVM stack traces in an IDE's debugger.

i suppose it could also work in Mailpool to suggest common phrases and sentences that can be posted, helping to reduce the dimensionality of the problem space to jumpstart it in its early stages.

[Robert Swaine]

KY,

Good luck at making impressions for Genifer and AGI at the start-up convention!!

-RS

---

From: "YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com>
To: general-in...@googlegroups.com
Sent: Thursday, May 17, 2012 9:28 PM
Subject: Re: [GI] AI Command Shell

[Steve Richfield]

SeH,

The route to n log n speed would seem to be:
1.  Do some (rudimentary) grammatical analysis to identify potentially interesting phrases.
2.  Create sort keys from the phrases by reducing words to concepts, e.g. strip superlatives and weak adjectives, change less common words to similar but more common words, etc.
3.  Sort all the phrases along with records containing the original postings, resulting in similar statements sorting together.
4.  Analyze sequential records having identical sort keys.

Steve
==================

--
Full employment can be had with the stoke of a pen. Simply institute a six hour workday. That will easily create enough new jobs to bring back full employment.

[SeH]

i guess it's possible to form a kernel that contains the ltrace results of shell commands executed from an ever-growing networked distributed shell log.  it would be compressible since it would be exposed to many similar sequences of command characters and their resulting ltrace function calls.  

what would the advantages of such a kernel be?  

• could it optimize the system (for memory or speed) as a whole in a way that the current unix system architecture hasn't allowed us to consider?
• could it develop into a more natural (as in NLP) command shell by mapping semantics to dynamic shell macros?  this would gradually eliminate the need for a lot of programmed software in non-natural computer languages like sh, python, c++, java, javascript, etc...
• what kinds of new software could it "evolve" from the "ecosystem" of program fragments that are "learned" by the AI shell?
• could it learn arbitrary natural language commands for interacting with social networks, and Mailpool itself?

jim has suggested using a C++ compiler to dynamically recompile a high-performance knowledgebase module for this and other projects.  unless i'm mistaken, i think this could result in the fastest read-only KB (for our modern commodity CPU architecture, atleast).  does it provide any other advantages?

[james northrup]

I recently read about disruptor pattern ( http://martinfowler.com/articles/lmax.html  )

i stumbled upon ring buffer performance improvements in java a while back and really didnt have a way to articulate what i discovered to a degree that was useful for the amount of income runway i had to  pursue it to a close.  im glad others came to the same discovery, that java jit can be a monster with planned data locality (the thing that c++ programmers take shots at is java's large heap object footprint)

i think the access to a ringbuffer strategy using strongly compiled logic and ontology template metaprogramming against input is as optimal as technology permits for the folks in garages playing with kb's.