Availability of AGI 21 Papers?

[theiman]

Hi Y'all, 

I just wondered if the papers contributed to the AGI 2021 conference would be available? Thank you!!

Sincerely,

tom

[Linas Vepstas]

Hi!

The slide deck that I presented is available at

https://github.com/opencog/learn/blob/master/learn-lang-diary/recognizing-patterns.pdf

and a transcript of what I was going to say is at

https://github.com/opencog/learn/blob/master/learn-lang-diary/recognizing-patterns-notes

There's a youtube link in there too.

-- Linas

--
You received this message because you are subscribed to the Google Groups "opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencog+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/5ece07c0-2dba-4ac9-8c8b-73187f199c7an%40googlegroups.com.

--

Patrick: Are they laughing at us?

Sponge Bob: No, Patrick, they are laughing next to us.

 

[Amirouche Boubekki]

Le sam. 30 oct. 2021 à 06:17, Linas Vepstas <linasv...@gmail.com> a écrit :
>
> Hi!
>
> The slide deck that I presented is available at
>
> https://github.com/opencog/learn/blob/master/learn-lang-diary/recognizing-patterns.pdf
>
> and a transcript of what I was going to say is at
>
> https://github.com/opencog/learn/blob/master/learn-lang-diary/recognizing-patterns-notes

Very interesting. What are those acronyms:

- MI = Mutual Information?

- MST parses = Maximum Spanning Tree, according to wikipedia: a
spanning tree is "In the mathematical field of graph theory, a
spanning tree T of an undirected graph G is a subgraph that is a tree
which includes all of the vertices of G.", the maximum spanning tree
will be the spanning tree that goes through most edges or vertices. It
looks similar to a space filling curve somehow, except it is
structured.

- GUE


I am wondering why the algorithm only takes into account adjacent word
pairs. Unlike Link Grammar that draws connections across a sentence
jumping through intermediate words... Oops! Then you mention
skip-grams (https://en.wikipedia.org/wiki/N-gram#Skip-gram) so my
guess, unlike what is written in Combinatory Linguistics by Cem
Bozşahin, that stress the need to build a phrase structure grammar
with adjacent words
(https://en.wikipedia.org/wiki/Phrase_structure_grammar) vs. a
dependency grammar (https://en.wikipedia.org/wiki/Dependency_grammar)
but that is a categorical grammar? It is unclear to me what is what,
and whether that matters.

Quoting the transcript:
> * We can learn the rules of reasoning; they are not God-given (aka
> hard-coded by some programmer.)
> * They can be learned, and I've described an algorithm for learning
> them.

Awesome.

To summarize the presentation: you claim that it is possible with a
Machine Learning algorithm to build, in a completely *unsupervised*
way, that is without annotations, by mining existing corpus materials,
a grammar for natural languages, hence creating links between the
words forming a tree or graph. That graph is annotated somehow with
words, hence is explainable. You also claim the algorithm may be used
to infer grammars from other sources such as audio, video, etc... You
also claim that it is a very simple, walked path in terms of math,
already used in the industry. As far as I understand you joined the
dots, but there are still known unknowns such as a normal distribution
that appears out-of-the-blue.

In simpler words, you shed light (structures) into the void (the unstructured).

Let me know if I got this correctly.

Thanks for sharing.

[Linas Vepstas]

On Sat, Oct 30, 2021 at 2:47 AM Amirouche Boubekki <amirouche...@gmail.com> wrote:

Le sam. 30 oct. 2021 à 06:17, Linas Vepstas <linasv...@gmail.com> a écrit :
>
> Hi!
>
> The slide deck that I presented is available at
>
> https://github.com/opencog/learn/blob/master/learn-lang-diary/recognizing-patterns.pdf
>
> and a transcript of what I was going to say is at
>
> https://github.com/opencog/learn/blob/master/learn-lang-diary/recognizing-patterns-notes

Very interesting. What are those acronyms:

- MI = Mutual Information?

Yes.

- MST parses = Maximum Spanning Tree, according to wikipedia: a
spanning tree is "In the mathematical field of graph theory, a
spanning tree T of an undirected graph G is a subgraph that is a tree
which includes all of the vertices of G.", the maximum spanning tree
will be the spanning tree that goes through most edges or vertices. It
looks similar to a space filling curve somehow, except it is
structured.

Yes.

- GUE

Gaussian Unitary Ensemble.  It's complicated. Ignore it.

I am wondering why the algorithm only takes into account adjacent word
pairs.

Which algorithm? Not mine. Nowhere does it say "adjacent".

Unlike Link Grammar that draws connections across a sentence
jumping through intermediate words... Oops! Then you mention
skip-grams (https://en.wikipedia.org/wiki/N-gram#Skip-gram) so my
guess, unlike what is written in Combinatory Linguistics by Cem
Bozşahin, that stress the need to build a phrase structure grammar
with adjacent words
(https://en.wikipedia.org/wiki/Phrase_structure_grammar) vs. a
dependency grammar (https://en.wikipedia.org/wiki/Dependency_grammar)
but that is a categorical grammar? It is unclear to me what is what,
and whether that matters.

Given a dependency grammar, one can algorithmically convert it to a phrase-structure grammar.  To a combinatory grammar, to a categorial grammar. These are all equivalent formulations of the same concepts.  Now, linguists will argue strongly about this, as they all have their favorite ideas. From where I am, none of these arguments matter very much, as all these systems are inter-convertible.

What does matter then, for me, is

* how small is the representation?

* Is it easy to write algorithms that manipulate it?

* Are those algorithms efficient and fast?

Taking those into account, a dependency grammar, using the jigsaw-puzzle-piece paradigm, appears to be the simplest approach.

Given a lexis of jigsaw pieces, it can be converted to a phrase structure grammar or a combinatorial grammar or whatever, but I currently do not see the utility of performing those conversions.

Quoting the transcript:
> * We can learn the rules of reasoning; they are not God-given (aka
> hard-coded by some programmer.)
> * They can be learned, and I've described an algorithm for learning
> them.

Awesome.

To summarize the presentation: you claim that it is possible with a
Machine Learning algorithm to build, in a completely *unsupervised*
way, that is without annotations, by mining existing corpus materials,
a grammar for natural languages, hence creating links between the
words forming a tree or graph.

Yes. That project was started circa 2014 and finally worked "acceptably well" circa 2017. Where the bar for "acceptability" was set rather low.  I've made many improvements since then; it's an ongoing project.

That graph is annotated somehow with
words, hence is explainable.

Uhh, that the algorithm can be applied to obtain the references between words in text and objects in images, or patterns in audio, so that the when someone says "I hear whistling in the distance", the word "whisteling" can be associated with a particular collection of audio-processing filters, that an audio digital-signal processing expert would recognize as filters the select for a whistling sound. Thus, the word "whistling" is grounded in a particular set of audio filters that select for whistling.

 

You also claim the algorithm may be used
to infer grammars from other sources such as audio, video, etc...

Yes.

You
also claim that it is a very simple, walked path in terms of math,
already used in the industry.

No, I do not. Or, rather, I use a collection of concepts that are relatively well known to those who are versed in the state of the art, but these concepts remain confusing and generally misunderstood by many.

The situation I find myself in is kind of claiming that, in a vacuum, a cannonball and a feather will drop at the same rate, when common-sense experience clearly contradicts that. There are many people who will argue this, and argue details both large and small. It is difficult to have a meaningful conversation, due to the overall confusion about the situation.

As far as I understand you joined the
dots, but there are still known unknowns

There are always unknowns. If you've built a steam engine, or a glider, or a vacuum tube, there are unknowns. There are ways to make them better, more efficient, bigger, smaller, cheaper, faster.

 

such as a normal distribution
that appears out-of-the-blue.

Sure. I have made hundreds of different graphs of distributions of all sorts of variables, plotted in all kinds of different relationships. The directory in which the presentation slides are found also contains other PDF's, and a diary of research results, showing such figures.

The point is that some of these figures are sort-of "obvious" -- Zipfian distributions, and so on. Others are utterly unexplained.  Here's one on wikipedia: to the best of my knowledge, there is no scientific explanation whatsoever, of this graph:

https://en.wikipedia.org/wiki/Wikipedia:Does_Wikipedia_traffic_obey_Zipf%27s_law%3F

I did not create the original wikipedia page, but I did create the January 2020 update to it.  I have observed exactly the same graph in genome distribution, and in proteome distribution, and in reactome distribution. (I've placed a PDF in github somewhere with those graphs)

Again: I am not aware of any theoretical explanation of any of these graphs, either of the wikipedia hits, or the genome distribution, or the distributions I observe in natural language. It appears to be a completely open and utterly unexplored corner of network theory.

I think it has something to do with gaussian unitary ensembles. But that is an extremely vague and incomplete thought, at this time.

In simpler words, you shed light (structures) into the void (the unstructured).

Yes.

Let me know if I got this correctly.

Thanks for sharing.

Welcome!

[theiman]

Thank you!!!

-th

[Amirouche Boubekki]

There is one major missing piece, I do not know how to word it properly. Let's try: Linas' algorithm does build a model in a way that is unsupervised that can predict the linguistic structure of a sentence. With that model, Every word part a sentence can be attached to half a directed link, in the spirit of Link Grammar, then when the sentence make sense, there is complete linkage of the sentence with full directed links. Where does a link label come from?

ref: https://www.link.cs.cmu.edu/link/dict/index.html

[Linas Vepstas]

On Tue, Nov 9, 2021 at 3:01 AM Amirouche Boubekki
<amirouche...@gmail.com> wrote:
>
> There is one major missing piece, I do not know how to word it properly. Let's try: Linas' algorithm does build a model in a way that is unsupervised that can predict the linguistic structure of a sentence. With that model, Every word part a sentence can be attached to half a directed link, in the spirit of Link Grammar, then when the sentence make sense, there is complete linkage of the sentence with full directed links. Where does a link label come from?

The label is whatever you want it to be. The algo doesn't care.

I have no idea why you think this is a "missing piece", never mind it
being "major".

BTW, I'm convinced the algo works for audio and video, too. I'm
convinced that it also works for higher layers of abstraction, too.

--linas

[Amirouche Boubekki]

Le mar. 9 nov. 2021 à 18:18, Linas Vepstas <linasv...@gmail.com> a écrit :
>
> On Tue, Nov 9, 2021 at 3:01 AM Amirouche Boubekki
> <amirouche...@gmail.com> wrote:
> >
> > There is one major missing piece, I do not know how to word it properly. Let's try: Linas' algorithm does build a model in a way that is unsupervised that can predict the linguistic structure of a sentence. With that model, Every word part a sentence can be attached to half a directed link, in the spirit of Link Grammar, then when the sentence make sense, there is complete linkage of the sentence with full directed links. Where does a link label come from?
>
> The label is whatever you want it to be. The algo doesn't care.
>
> I have no idea why you think this is a "missing piece", never mind it
> being "major".

It is a missing piece to be able to tell whether the linkage make
sense or not. #explainableai

>
> BTW, I'm convinced the algo works for audio and video, too. I'm
> convinced that it also works for higher layers of abstraction, too.

Yeah, but then how do you build e.g. semantic frames or pragmatics
from the linkage?

[Linas Vepstas]

On Thu, Nov 11, 2021 at 8:55 AM Amirouche Boubekki <amirouche...@gmail.com> wrote:

Le mar. 9 nov. 2021 à 18:18, Linas Vepstas <linasv...@gmail.com> a écrit :
>
> On Tue, Nov 9, 2021 at 3:01 AM Amirouche Boubekki
> <amirouche...@gmail.com> wrote:
> >
> > There is one major missing piece, I do not know how to word it properly. Let's try: Linas' algorithm does build a model in a way that is unsupervised that can predict the linguistic structure of a sentence. With that model, Every word part a sentence can be attached to half a directed link, in the spirit of Link Grammar, then when the sentence make sense, there is complete linkage of the sentence with full directed links. Where does a link label come from?
>
> The label is whatever you want it to be. The algo doesn't care.
>
> I have no idea why you think this is a "missing piece", never mind it
> being "major".

It is a missing piece to be able to tell whether the linkage make
sense or not. #explainableai

Did you read through the slides? I think the title of the talk was "explainable AI"/

>
> BTW, I'm convinced the algo works for audio and video, too. I'm
> convinced that it also works for higher layers of abstraction, too.

Yeah, but then how do you build e.g. semantic frames or pragmatics
from the linkage?

Slide 7 or 8 or 9 of the talk.