Q: Theories, and formalizing natural language
Antti Ylikoski
I have in my hands the following discussion.
There is a problem Pr there, which I do not constrain here in other
ways than that Pr is a real-world object, which can be described with
the natural language, and this description has a finite length.
From Pr, we can (can we?) form the formal description of the problem
DSC = DSC(Pr)
where DSC is a discrete symbol string, whose lenght is finite.
DSC could for example be the description of the problem with
predicate logic, or it could be the description with a LISP symbolic
expression (S-expression).
Now, from DSC we will form a description of the problem in such a
form that it can be given to an information processing being to solve
Pr. This description could be for example
LISP-DSC = LISP(DSC)
ie. DSC having been coded so that (as a LISP S-expression) the
problem Pr can be given to a LISP program to be solved, or the
description could be eg.
TM-DSC = TM(DSC)
the description of the problem Pr represented as a finite symbol
string, which can be written onto the tape of a Turing machine, for
the solution of Pr.
(Here if the language of strings {TM-DSC} is in the class NP, we say
that the problem Pr is in the class NP. Similarly, if {TM-DSC} is
in the class P, we say that Pr is in the class P.)
In the above, the point is really describing the reality with a model.
The description of the reality with a theory is a question which, I
understand, has been extensívely studied by philosophy. Here this
theory is a formal model -- because the intention is to solve the
problem with a machine.
I discussed this matter in the Finnish Internet and got some pointers,
such as the pointer to the philosophy of language. But I'm not
totally satisfied by that discussion. Could someone give some
scientific pointers to this problem here? I would like to ask these
individuals to post the answers into the newsgroup and not answer to
me personally, because I feel that the problem might be interesting to
other researchers here.
There is a difficult point in the above discussion -- the question of
formalizing natural lanaguage. Years ago, I read the book
Lewis-Papadimitriou: Elements of the Theory of Computation, and the
book mentions that "it may be impossible to ever formalize natural
language." But the book does not discuss the point, nor does it give
any pointers.
Could somebody knowledgeable give some pointers to this question,
formalizing natural language, here in the newsgroup? I feel that this
question has some general significance so I would ask these
individuals to answer here in the newsgroup and not personally to me.
kind regards, Antti J. Ylikoski
Helsinki University of Technology
Helsinki, Finland, Northern Europe
The European Union
Wolf K
[...]
> There is a difficult point in the above discussion -- the question of
> formalizing natural lanaguage. Years ago, I read the book
> Lewis-Papadimitriou: Elements of the Theory of Computation, and the
> book mentions that "it may be impossible to ever formalize natural
> language." But the book does not discuss the point, nor does it give
> any pointers.
>
> Could somebody knowledgeable give some pointers to this question,
> formalizing natural language, here in the newsgroup? I feel that this
> question has some general significance so I would ask these
> individuals to answer here in the newsgroup and not personally to me.
>
>
> kind regards, Antti J. Ylikoski
> Helsinki University of Technology
> Helsinki, Finland, Northern Europe
> The European Union
>
Formalising really means defining a subset of natural language, and
limiting each element in it to one meaning or function. Thus, "computer
languages", even when they look like English, are not English. Thus also
the exact translation of OS terminology (eg, in menus) from one language
to another.
What makes natural languages impossible to formalise is metaphor.
Metaphor is the chief method of extending meanings as needed or desired
(some linguists say it's the only method.) You would have to know all
possible ways a term could be used metaphorically in order to formalise
extension of meaning, which is another way of saying that you would have
to know or all possible future metaphors. That's impossible. Etc.
Then there's translation. Read Quine on translation, for a logician's
take on the problems of natural language. Formal language expressions
mean whatever they are defined to mean no matter how they are
pronounced. But natural language expressions are often untranslatable -
the best one can do in translating from language A to language B is to
paraphrase the meaning expressed in A. Read George Steiner's After Babel
for a subtle and far ranging discussion of these and related issues.
Since you are translating from Finnish, you will have noticed that the
more technical the discourse, the more exact the translation. The reason
is that technical jargon is a kind of formal language.
Note that "formal English" is a subset of English with a narrow grammar
and a limited lexicon, and quite rigid usage rules. This is a general
principle that applies to all natural languages. In fact, in many
languages, there are a variety of formal languages available, each of
which is limited to a specific social context. The technical term for
such subsets/usages of a natural language is "register". Translations
often fail at using the correct register, BTW, and the results can be
quite funny when they are not appallingly rude (which is also funny,
actually. ;-))
There is a lot more that could be said. I trust the above provides some
starting points and provokes fruitful questions.
HTH
wolf k.
Neil W Rickert
>There is a problem Pr there, which I do not constrain here in other
>ways than that Pr is a real-world object, which can be described with
>the natural language, and this description has a finite length.
The basic problem is that descriptions, as used by people, are
instruments for conveying semantics. In order to use in a formal
setting, you need to have a set of rules of inference. And semantics
does not easily map into rules of inference.
>In the above, the point is really describing the reality with a model.
>The description of the reality with a theory is a question which, I
>understand, has been extensívely studied by philosophy.
Well, yes. But one cannot conclude that philosophy has solved
the problem.
>There is a difficult point in the above discussion -- the question of
>formalizing natural lanaguage. Years ago, I read the book
>Lewis-Papadimitriou: Elements of the Theory of Computation, and the
>book mentions that "it may be impossible to ever formalize natural
>language." But the book does not discuss the point, nor does it give
>any pointers.
I agree with that skeptical view, that it cannot be done. However,
I cannot prove that I a right. Many people think it can be
done, but they cannot give a proof either, and IMO the evidence
is against them.
Brian Martin
To paraphrase your extensive discussion:
1 - there is a problem Pr
2 - you are able to describe this problem Pr using some acceptable
notation DSC(Pr)
So what on earth were all the other words for ?
Sorry, trying to be pragmatic here.
Essentially you have said nothing at all.
Neil W Rickert
>OK folks, here's my problem with you theoretical types ...
>To paraphrase your extensive discussion:
>1 - there is a problem Pr
>2 - you are able to describe this problem Pr using some acceptable
>notation DSC(Pr)
>So what on earth were all the other words for ?
>Sorry, trying to be pragmatic here.
>Essentially you have said nothing at all.
To clarify, the actual problem here is that you have said nothing
at all. You have introduced "DSC(Pr)", but it is unclear what
that means. You talk of describing to a formal system, but it is
unclear what that means.
We have tried to guess at what you are asking. Presumably we
failed. So the situation reverts to one where you have not
clearly expressed your question.
chris
<snip>
>
> What makes natural languages impossible to formalise is metaphor.
> Metaphor is the chief method of extending meanings as needed or desired
> (some linguists say it's the only method.) You would have to know all
> possible ways a term could be used metaphorically in order to formalise
> extension of meaning, which is another way of saying that you would have
> to know or all possible future metaphors. That's impossible. Etc.
>
Not really. Given the work in neurosciences we can identity the
properties of our filtering system as a neuron-dependent species. All
specialist perspectives relabel that filtering system to deal with
specialist contexts and as such create metaphors/analogies.
We can identify the source of meaning in the neurology's use of
recursion to derive categories of meaning and from there comes, with
depth in the recursion, the development of the ability to use the
categories figuratively as well as literally - and such we see the
emergence of languages where such reflect local context customisations
of the filtering system categories.
The metaphor nature is in the different specialist perspectives being
interchangable in interpreting reality and as such bringing out a
grounding in emotional assessments of reality and the focus on
symmetry as a ground for communication through pattern matching.
Thus, given the recursion of the neurology we can map out the set of
POSSIBLE patterns out of which analogies/metaphors are made.
This dynamic allows us to come up with an abstract domain model for
all meaning processing - see example in
http://members.iimetro.com.au/~lofting/myweb/AbstractDomain.html -
this allows for the translation of one specialist metaphor into
another in that both will share the abstract domaibn model where ITS
source is in the neurology.
Chris
http://members.iimetro.com.au/~lofting/myweb/introIDM.html
Ian Parker
wrote:
>
> Could somebody knowledgeable give some pointers to this question,
> formalizing natural language, here in the newsgroup? I feel that this
> question has some general significance so I would ask these
> individuals to answer here in the newsgroup and not personally to me.
>
> kind regards, Antti J. Ylikoski
> Helsinki University of Technology
> Helsinki, Finland, Northern Europe
> The European Union
http://groups.google.co.uk/group/sci.math/browse_frm/thread/55fc34426fe64685?hl=en
If a language could be expressed formally it would no longer be a
natural language it would be QED. There are a number of threads in the
mathematics user groups which discuss just this topic.
http://groups.google.co.uk/group/creatingAI/browse_frm/thread/76177aca03c158a8?hl=en
http://groups.google.co.uk/group/sci.math.research/browse_frm/thread/38b4981f397713e9?hl=en
Although Natural Language is not formalizable in general there are a
number of expressions that can be treated formally. Note Arabic is
quoted in Buckwalter transliteration. Arabic script is given in the
Website.
http://sites.google.com/site/aitranslationproject/deepknowled
mDrwbA fY ArbEp ADEAf drjp HrArp sTHh Ay = Multiplied by four times
the temperature of any surface (Google). drjp - degree is totally
ignored. Determined by multiplying by the temperature degree four for
any surface (correct). Four times the temperature is NOT the Stephan
Bolzmann law.
Irony is, a Google search not only gives the correct law, but I can
also find out what happens when my "box" not only contains radiation
but particles (Big Bang hot). this is perhaps something to think
about.
(AmA bAlnsbh lIHjAm Alnjwm fYtm AltErf ElYhA mn xlAl tTbYq qAnwn
stYfAn bwltzmAn fY AlA$EAE)
This sentence is what we might define as a "deep" one. It is deep in
the sense that we are calling on the Stephan Boltzmann law. Now the
point I am making is that expressions like "Spephan Boltzmann" and
"Sobolov embedding" another example given in sci.math.research are
uniquely defined expressions. They are in fact part of a Von Neumann
of context free language.
Most of Natural language cannot be expressed formally although there
are a number of expressions which are formal and (potentially at
least) QED compatible. It should be pointed out that mathematics may
be defined as the study of formal systems, so that a formal expression
language can be given mathematical form. Most of Physical Science is
expressible in QED terms.
The question is if mathematicians go ahead with QED and if as Marc
Nardmann suggests students put sections into QED as projects leading
to their degree, there is a question as to where this leaves Natural
Language. It should be trivial to put QED into Arabic, English or
Finnish. The irony is that the author of the Arabic text has quoted
the Stephan Boltzmann formula correctly. If we had QED - > Arabic and
QED -> English we could ignore the Arabic and simple translate the QED
into English.
I hope this is helpful. I think EU interest would be extremely
helpful. However the essentially mathematical nature of QED should not
be ignored. Will QED improve translation? Yes emphatically, the words
which are unambiguous will all be translated accurately. No nonsense
about radiation increasing linearly with temperature.
One bee I have in my bonnet. AI has very much an interdisciplinary
nature. Ray Kurtzweil has boasted that Google Translate did not employ
a single Arabist. Google Translate is winning all the competitions. In
fact you need both. One point though is that the job of an Arabist is
to teach people Arabic. Journalists going to the Middle East for
example. The fact that similar ideas have crossed threads so much is
indicative of interdisciplinary requirements. Now what journalist ever
thinks to himself that a tunnel is a torus and that at for a single
tunnel under Raffah at least seven colors are needed for the map.
http://www.math.uic.edu/~kauffman/SevenColors.pdf
This expresses 2 cultures rather well.
- Ian Parker
Brian Martin
My reference to the "problem", an "acceptable notation", and the
notation "DSC(Pr)" are all sourced from Antti's original question.
This is not terminology or notation that I use myself.
I have introduced no new notation or terminology,
I'm simply saying (perhaps irreverently), that the original question is
somewhat pointless.
He essentially says (paraphrased) that he has a problem (Pr) which he is
able to describe using some notation (referred to as DSC(Pr)) that is
generally acceptable to the field of NLP participants.
I am saying : jolly good, you have a problem, which you can describe,
that's great, so what specifically is your point ?
Sorry, maybe I'm missing something here, but it seems like a lot of
words to say nothing at all.
I'm very interested in semantic modelling and problem description,
but either I don't understand Antti's question, or it means nothing.
Antti Ylikoski
Maybe the process of solving a real-world problem formally and with a
computer is so familiar to professional scientists that my description
of that process appears totally trivial and frivolous.
That description, trivial as it may be, (to a certain extent) points to
scientific questions -- semantic modelling and formalizing real-world
natural language. This is why I presented that description here.
Sorry if I have wasted your precious time by something -- as I said --
trivial and frivolous.
But -- some philosophers have written quite nontrivial answers to my
problem setting.
kind regards, Antti Ylikoski
Antti Ylikoski
No -- I think you did not fail at all. Rickert gave me a very good
answer when explaining that semantics does not easily map into rules of
inference.
regards, Antti "Andy" Ylikoski
PS. Is there a good definition of a "formal system"? A computer
scientist or a mathematician will know one when seeing it... But how
about the precise definition?
Ian Parker
wrote:
>
On Feb 1, 3:29 pm, Antti Ylikoski
<antti.yliko...@elisanet.fi.invalid>
wrote:
- Hide quoted text -
- Show quoted text -
> Neil W Rickert kirjoitti:
> > Brian Martin <brianNOS...@futuresoftware.com.auNOSPAM> writes:
> regards, Antti "Andy" Ylikoski
> about the precise definition?- Hide quoted text -
A formal system is simply a system that is built up using defined
rules. Ultimately any computer system is "formal" as computer
languages are formal.
In Natural Language there are words that have a formal definitition
and other words which do not. NL is statistical in the sense that
words have different meanings according to context.
Un barco a través de una cerradura - "lock" can have different
meanings and
that meaning can only be defined statistcally. A formal language need
not be a Von Neumann (context free) language although conversion into
a VN language in a series of finite steps is a requirement.
BTW I looked up Finnish. "lutticus" seems to have the same ambiguity.
- Ian Parker
Wolf K
[...]
> PS. Is there a good definition of a "formal system"? A computer
> scientist or a mathematician will know one when seeing it... But how
> about the precise definition?
Read a logic text, or take a course. If you want to do it yourself, you
could start with three concepts; well-formed formula, axiom, and rule of
inference. Any strictly formal definition of "formal system" will itself
be a formal system, BTW. It ain't easy. ;-)
Informally speaking, a formal system consists of variables, operators,
statements, rules of syntax, and rules of inference. A rule of syntax
specifies how variables and operators may be combined to produce
statements. A rule of inference specifies how one or more statements may
be restated to yield other statements. These rules are a "meta
language", they are about the formal system. (They are themselves a
formal system, actually). If the rules of inference are applied
correctly, the final statement in the sequence is said to be proved.
That is, the statements in the sequence are consistent with each other. Etc.
A formal system is abstract and content-free (even when its construction
is motivated by a desire to formalise a content-rich language). All that
matters is which possible statements can be proved within the system.
Interpreting the system means applying it to some content. Thus Boolean
algebra may be applied to a collection of natural language statements
about the world, and then becomes a method of deciding whether and how
these statements fit together to produce a logically consistent
description of that part of the world that it refers to. Etc.
I think that's enough for you to go on. ;-)
HTH
PS on computer languages:
In principle, a computer language is a formal system. A computer program
would then be a proof, in the formal sense: all its statements would be
consistent with each other. A bug-free program would be a correct proof,
and vice versa. Thus, a method of proving programs formally would be
very nice to have. AFAIK, there aren't any such methods, leastways not
for the computer languages that are actually used. The tests for bugs in
essence amount to varying the inputs to the program until you find some
that break it. Then you fix the code in question, and try again. But
fixing the code in question may introduce an inconsistency, ie, a new
bug. That's because actual computer languages are not formal systems,
but a hodge podge of object and meta languages, the latter often at two
or more levels (ie meta-meta languages). Etc.
That's how I understand it.
wolf k.
Lotzi Boloni
>
> In principle, a computer language is a formal system. A computer program
> would then be a proof, in the formal sense: all its statements would be
> consistent with each other. A bug-free program would be a correct proof,
> and vice versa.
I don't think that this is the definition of a programming language,
except that some theoretical minded people might wish that it would
be.
In practice, you can not predict what the output of a program will be
when executed on a certain computer, because it potentially depends on
the state of the computer (including things such as inputs, both
initial), its resources, other programs, events happening during the
execution, both high level (such as user input) and low level
(interrupts) etc.
For instance, I am writing this mail in GMail, from Linux. I don't
think that there is any formal system which would have predicted the
final document you are reading RIGHT NOW (including my text, and
whatever font and format you are using) based on the initial state of
my computer when I booted it an hour ago). (Ok, unless that formal
system also models the state of the internet, the state of the mind of
the previous posters on this mail, and the state of my mind).
This is not much different from the fact that the semantics of a NLP
text depends on the state of the reader. (And yes, I mean the reader,
not some probabilies inherent in the language).
Lotzi
Wolf K
a) I think you are agreeing with me; but
b) for the wrong reasons.
But your last comment is IMO significant.
Cheers,
wolf k.
Brian Martin
As I said, either I'm missing the point,
OR,
you are saying that you (a) have a problem, and (b) can describe it.
I take a more pragmatic and less theoretical view.
My own view is that semantic modelling of the real world is much more
important (and productive) than attaining perfection in syntax analysis
and parsing. I could of course be wrong.
Best wishes in your research goals.
Ian Parker
The original contributor is in the EU. I feel he should be aware of
the following thread.
http://groups.google.co.uk/group/sci.math.research/browse_frm/thread/38b4981f397713e9?hl=en#
Note particularly contrbution 14. Anything "formal" is a branch of
mathematics.
There are a number of systems like HOL light and Mizar. They are
mutually incompatible. I feel the EU should start thinking about a
system combining the two systems. If you have systems which are
"formal" there should be a relationship between them.
- Ian Parker
Wolf K
[...]
> Note particularly contrbution 14. Anything "formal" is a branch of
> mathematics.
> - Ian Parker
My prejudice is to say that anything formal is a branch of logic...
Thanks for the link.
wolf k.
goanna
>On 2 Feb, 15:46, Wolf K <weki...@sympatico.ca> wrote:
>> Antti Ylikoski wrote:
>>
>> [...]
>>
>The original contributor is in the EU. I feel he should be aware of
>the following thread.
>http://groups.google.co.uk/group/sci.math.research/browse_frm/thread/38b498=
>1f397713e9?hl=3Den#
You should probably be aware of this one too:
http://groups.google.com/group/alt.politics.org.nsa/browse_thread/thread/2064bd4cef292a63
Ian Parker
> Ian Parker <ianpark...@gmail.com> writes:
> >On 2 Feb, 15:46, Wolf K <weki...@sympatico.ca> wrote:
> >> Antti Ylikoski wrote:
>
> >> [...]
>
> >The original contributor is in the EU. I feel he should be aware of
> >the following thread.
> >1f397713e9?hl=3Den#
>
> You should probably be aware of this one too:
>
What's that supposed tp mean? sci.math.research has discussions of an
extremely high standards. What is this one supposed to do?
- Ian Parker
zzbu...@netscape.net
I hard lot more has been said about natural language already.
Since Converstaional English is why the people with language
brains even invented Digital. And idiiot IBM and QWERTY
is the people with actual computer brains invented Optical
Computers,
C++, RISC, Mini-Harddisks, Parallel Processing, Holograms, USB,
Anti-Spam,
Post GM Robotics, Drones, GPS, DVD-rom, DVD-ram, HDTV programming,
and XML.
>
> HTH
>
> wolf k.- Hide quoted text -
Don Stockbauer
wrote:
You left out the pulley and the horse-drawn cart.
pipeDream
I think your problem statement should really be worded as
follows.
Given a set of real world objects(including events as objects of
some sort)--Pr
and a description of the problem DSC(Pr)
Pr could be a set of predicates describing objects in the
universe of discourse and their relations and DSC(Pr) is a statement
in formal logic to be proved perhaps with constructive inference.
Your problem(the way I have understood) appears to be
can we represent any arbitrary problem description into this system.
To my mind it appears there are two issues
1.Representing arbitrary set of natural language statements (of finite
length-say 10 pages) in a formal system.
2.Solving them on a computer.
Without a clear start in the former any talk of NP hard is
meaningless.You cannot talk complexity of a problem without knowing
what the problem is.
All technical operations in natural language processing derive from
Formal logic and this in turn derives from the philosophy of language.
If you are talking formal logic it is obvious that there are
problems that cannot be solved - courtesy kurt godel. In
otherwords there is no solution for arbitrary problems. There are
solutions for a limited class of problems.
I guess you may wish to see if there is any other mathematical
apparatus other than formal logic which can handle your problem.
I am assuming that mathematics cannot be reduced to formal logic
whereas all computer operations can be.
If you wish to solve your problem there is little choice other than a
crash course in the philosophy of language followed by a search for
alternative paradigms.
Ian Parker
>
> To my mind it appears there are two issues
> 1.Representing arbitrary set of natural language statements (of finite
> length-say 10 pages) in a formal system.
> 2.Solving them on a computer.
>
> Without a clear start in the former any talk of NP hard is
> meaningless.You cannot talk complexity of a problem without knowing
> what the problem is.
It cannot be NP hard otherwise no one could speak any NL.
> All technical operations in natural language processing derive from
> Formal logic and this in turn derives from the philosophy of language.
> If you are talking formal logic it is obvious that there are
> problems that cannot be solved - courtesy kurt godel. In
> otherwords there is no solution for arbitrary problems. There are
> solutions for a limited class of problems.
>
> I guess you may wish to see if there is any other mathematical
> apparatus other than formal logic which can handle your problem.
> I am assuming that mathematics cannot be reduced to formal logic
> whereas all computer operations can be.
>
I am basically a mathematical physicist rather than a linguist or
philosopher. How would you class statistics and the secong law of
thermodynamics? I would say myself that statistics is definitely based
on a set of axioms. Likewise the 2nd law. Statistics is not though
certain. There is a chance (infinitessimally small) that heat will
pass from ice into my hand.
Entropy is an essential part of the 2nd law which states that entropy
always increases. What has this got to do with language? you might
ask. Well Marcus Hutter
http://www.google.co.uk/search?hl=en&q=marcus+hutter+papers&meta=
has suggested, that the quality of AI is dependent solely on
compression. Now the length of a binary string (compressed length) is
the number of states. Understanding language is a matter of spotting
nonsense. Nonsense is therefore increasing our compressed size.
Eliminating nonsense is therefore compressed size or entropy.
If we take our thermodynamic analogy we can describe NL as a
Hamiltonian. Combinations of words are more of less likely. We can say
that a pair of words at a given separation has a given probability.
Finding the most probable words therefore is basically an annealing
problem.
> If you wish to solve your problem there is little choice other than a
> crash course in the philosophy of language followed by a search for
> alternative paradigms.
>
Hutter has in my view proviided a theoretical paradigm, we need look
no further. A formal language is injective onto Natural Language. This
is to use the right juju of set theory. It means we can describe any
formal system using natural language, but we cannot necessarily
describe NL formally.
The importance of injection in AI is that we can derive from NL a
subset of formal terms. In our Arabic translation example the Stephan
Boltzman law is formal. The law of radiation is describable in terms
of a formal system such as Mizar. Scientific language is on the whole
formal. A black hole is an object arising out of the Riemann metric.
(differential geometry is on Mizar). The Black Hole of Calcutta may be
expessible in some formal way but it is not the Mizar Riemann space
definition!
How can I tell whether a word is part of a formal system or not? Only
by the context. I can assign a tag to each formal word. I can say that
Stephan Boltzmann law and Black Hole (Riemann) are formal words. If a
formal word provides Maximum Entropy we deem it to be present. A
formal word can trigger AI and that is why it is important. Words used
informally cann ot trigger.
This to me is the only possible paradigm.
- Ian Parker
Neil W Rickert
>I think your problem statement should really be worded as
>follows.
>Given a set of real world objects(including events as objects of
>some sort)--Pr
>and a description of the problem DSC(Pr)
>Pr could be a set of predicates describing objects in the
>universe of discourse and their relations and DSC(Pr) is a statement
>in formal logic to be proved perhaps with constructive inference.
And that highlights the problem. In fact we are not given a set of
real world objects. We are given a world, and we divide it into
"objects" for our own convenience. But there is no canonical way
of dividing the world into objects, and thus different people do
it differently. They might even do it differently on Wednesday
from how they did it on Tuesday.
Hmm, I suppose I could summarize that paragraph as "ontology is crap".
Natural language developed to deal with such a situation. Logic and
formal languages cannot cope with it. And that's the mismatch
between natural languages and formal languages. It is also the
mismatch between philosophy and reality, and it is the mismatch
between AI and real intelligence.
Wolf K
> pipeDream <Gopalara...@gmail.com> writes:
>
>> I think your problem statement should really be worded as
>> follows.
>> Given a set of real world objects(including events as objects of
>> some sort)--Pr
>> and a description of the problem DSC(Pr)
>> Pr could be a set of predicates describing objects in the
>> universe of discourse and their relations and DSC(Pr) is a statement
>> in formal logic to be proved perhaps with constructive inference.
>
> And that highlights the problem. In fact we are not given a set of
> real world objects. We are given a world, and we divide it into
> "objects" for our own convenience. But there is no canonical way
> of dividing the world into objects, and thus different people do
> it differently. They might even do it differently on Wednesday
> from how they did it on Tuesday.
As any multi-lingual person can tell you.
Read Quine.
> Hmm, I suppose I could summarize that paragraph as "ontology is crap".
>
> Natural language developed to deal with such a situation. Logic and
> formal languages cannot cope with it. And that's the mismatch
> between natural languages and formal languages. It is also the
> mismatch between philosophy and reality, and it is the mismatch
> between AI and real intelligence.
Formal languages are subsets of natural languages. More strongly: the
Formal Language of which all math and logic is an instance, is that
subset of natural language that all natural languages have in common.
That's why you can't formalise NL.
That's the conclusion I've come to after 50+ years of mulling it over,
doing math and logic, computer programming, writing fiction and
non-fiction, translating into and out of English, etc. FWIW, scientific
and technical language is nearly as universal as subset of natural
language as math/logic.
Anecdote:
Many years ago, I gave a short paper, followed by a long discussion, on
creating a dictionary of NL words as preliminary to processing NL text.
I proposed that each word in a NL dictionary have three tags:
1) its part of speech;
2) a list of synonyms;
3) a list of allowable semantic contexts (ie, pointers to other words)
Inflected forms would consist of the word plus a pointer to the main entry.
Well, it seemed to make sense at the time. ;-)
Cheers,
wolf k.
Brian Martin
> Anecdote:
> Many years ago, I gave a short paper, followed by a long discussion, on
> creating a dictionary of NL words as preliminary to processing NL text.
> I proposed that each word in a NL dictionary have three tags:
> 1) its part of speech;
> 2) a list of synonyms;
> 3) a list of allowable semantic contexts (ie, pointers to other words)
>
> Inflected forms would consist of the word plus a pointer to the main entry.
>
> Well, it seemed to make sense at the time. ;-)
Still does make sense. Wordnet synsets essentially implement your
proposal, with some extensions.
Brian Martin
Well said !
Entropy corresponds to how much actual information is in the text.
Compression algorithms recognise statistical patterns at the character,
word, phrase level and encode them with a shorthand symbol if they tend
to have a higher rate of occurence (frequency).
e.g. if I say "the time is 12 ..." the next word is most likely either
"Noon" or "O'clock".
Antti Ylikoski
> Not at all, my time is certainly not "precious".
>
> As I said, either I'm missing the point,
> OR,
> you are saying that you (a) have a problem, and (b) can describe it.
>
> I take a more pragmatic and less theoretical view.
>
> My own view is that semantic modelling of the real world is much more
> important (and productive) than attaining perfection in syntax analysis
> and parsing. I could of course be wrong.
>
> Best wishes in your research goals.
I wanted to be precise about what it means for a real-world problem to
be eg. "in the class NP" or "NP-complete".
There is a formal definition for a formal language L being in the class
NP. If there exists a nondeterministic Turing machine which decides L
in a polynomial time then the language L is in the class NP. An
extremely well-known definition exists for L being NP-complete.
But -- suppose that we visit an air force base with a number of
supersonic nuclear bombers, and they want to solve some of their
problems with a computer. What does it mean for one of these problems
to me "NP-complete"?
That discussion of mine, about solving Pr with a computer, stems from
this consideration.
I simply wanted everything to be precisely defined.
kind regards, Antti J. Ylikoski
Helsinki University of Technology
Helsinki, Finland, Nothern Europe
The European Union
pipeDream
> On 6 Feb, 13:43, pipeDream <Gopalarao.madd...@gmail.com> wrote:
>
>
>
> > To my mind it appears there are two issues
> > 1.Representing arbitrary set of natural language statements (of finite
> > length-say 10 pages) in a formal system.
> > 2.Solving them on a computer.
>
> > Without a clear start in the former any talk of NP hard is
> > meaningless.You cannot talk complexity of a problem without knowing
> > what the problem is.
>
> It cannot be NP hard otherwise no one could speak any NL.
-->I think the term NP hard from the computational theory lexicon
should be
restricted to problems that are posed formally for solution on a
computer.
Natural language as spoken and written (arbitrary samples) is as of
date not
well posed problem or alteast so I believe.
----------------------------
> > All technical operations in natural language processing derive from
> > Formal logic and this in turn derives from the philosophy of language.
> > If you are talking formal logic it is obvious that there are
> > problems that cannot be solved - courtesy kurt godel. In
> > otherwords there is no solution for arbitrary problems. There are
> > solutions for a limited class of problems.
>
> > I guess you may wish to see if there is any other mathematical
> > apparatus other than formal logic which can handle your problem.
> > I am assuming that mathematics cannot be reduced to formal logic
> > whereas all computer operations can be.
>
> I am basically a mathematical physicist rather than a linguist or
> philosopher. How would you class statistics and the secong law of
> thermodynamics? I would say myself that statistics is definitely based
> on a set of axioms. Likewise the 2nd law. Statistics is not though
> certain. There is a chance (infinitessimally small) that heat will
> pass from ice into my hand.
--> Some exposure to concepts of philosophy is a must for anyone
seriously
interested in natural language processing and espescially the
arbitrary
variety referred to as natural language understanding. Contemporary
linguistics uses the notions of semantics drawn the ideas of Bertrand
russel
et al. It is any body's guess how any one can advance the state of
the
without knowing the background.
The entire body of physics(including second law , mathematical physics
etc)
in my scheme of things is a formal system but not a formal logical
system.You
might take a look at
http://formalsystemsphilosophy.blogspot.com/
to get a better idea of what I mean
---------------------------------------
> Entropy is an essential part of the 2nd law which states that entropy
> always increases. What has this got to do with language? you might
> ask. Well Marcus Hutter
>
> http://www.google.co.uk/search?hl=en&q=marcus+hutter+papers&meta=
>
> has suggested, that the quality of AI is dependent solely on
> compression. Now the length of a binary string (compressed length) is
> the number of states. Understanding language is a matter of spotting
> nonsense. Nonsense is therefore increasing our compressed size.
> Eliminating nonsense is therefore compressed size or entropy.
-->As far as I can tell reinforcement learning algorithms have nothing
to do
with arbitrary language processing-they can at best cater to a small
subset.
Any machine learning technique has to use statistical processing of
one sort
or the other and to put it crudely "statistical processing is tossing
a coin
to decide if the human in front of you is male or female".
Before you do number crunching using Maxent or CRF or SNLP you need to
know
what numbers you are crunching.At one point of time Statistical
parsing was
supposed to be the ultimate. And Today people are talk in terms of
using it
in conjuntion with HPSG etc.The remarks apply to all statistical
techniques
irrespective of the domain. Einstein was supposed to have said "God
doesn't
play dice" in the context of quantum mechanics.
The only justification for a statistical technique is that a normal
law is
subsumed by technique with a probability of 1. But in general we are
trying
to induce the law using some sort of statistical processing where we
define
the terms of reference e.g the grammar or the attributes and so on.
Goof it
up and you goof up the whole thing. Domain knowledge is more important
than
statistics or number crunching to figure out the supposed law.
"Garbage in garbage out" as I learnt ages back from a copmuter text.
"Nonsense is therefore increasing our compressed size.
Eliminating nonsense is therefore compressed size or entropy." goes
totally against any account of observed natural language phenomenon.
The way we communicate we try to cut the crap. Eliminate redundancy
except when we use rhetoric. We use background and context to lessen
our burden. So I am of the view that the statement is nonsensical.
---------------------------------------------------
> If we take our thermodynamic analogy we can describe NL as a
> Hamiltonian. Combinations of words are more of less likely. We can say
> that a pair of words at a given separation has a given probability.
> Finding the most probable words therefore is basically an annealing
> problem.
>
> > If you wish to solve your problem there is little choice other than a
> > crash course in the philosophy of language followed by a search for
> > alternative paradigms.
>
> Hutter has in my view proviided a theoretical paradigm, we need look
> no further. A formal language is injective onto Natural Language. This
> is to use the right juju of set theory. It means we can describe any
> formal system using natural language, but we cannot necessarily
> describe NL formally.
--> well this is tantamount to saying that Hutter paradigm deals with
a subset of natural language. albeit with a little correction "A
formal language is injective onto Natural Language." onto is
technically incorrect.I assume it was used inadvertently and that into
was meant .Note that the quality of any AI/natural Intelligence
depends on context sensitivity not on any "right juju of set theory"
----------------------------------------------
> The importance of injection in AI is that we can derive from NL a
> subset of formal terms. In our Arabic translation example the Stephan
> Boltzman law is formal. The law of radiation is describable in terms
> of a formal system such as Mizar. Scientific language is on the whole
> formal. A black hole is an object arising out of the Riemann metric.
> (differential geometry is on Mizar). The Black Hole of Calcutta may be
> expessible in some formal way but it is not the Mizar Riemann space
> definition!
>
> How can I tell whether a word is part of a formal system or not? Only
> by the context. I can assign a tag to each formal word. I can say that
> Stephan Boltzmann law and Black Hole (Riemann) are formal words. If a
> formal word provides Maximum Entropy we deem it to be present. A
> formal word can trigger AI and that is why it is important. Words used
> informally cann ot trigger.
>
> This to me is the only possible paradigm.
-->I think you should not close your options so fast. Take a good hard
look at all available evidence. I would rather use Peter Drucker's
dictum "It is more important to do the right thing than doing the
thing right". This cannot be implemented first taking a topdown view
of existing approaches. MIZAR to me was Bizarre until I realized that
it "formal language derived from the mathematical vernacular" meant to
do some tasks--domain specific. I am looking at the broader picture.
If MIZAR does what you need that is perhaps the only possible
paradigm that you know of and is perfect if it solves your specific
problems. Even for that narrow domain there might be others.
You can take a look at the dozen or so theorem provers for possible
paradigms. But they are all traceable to FOL.
It is better to spread the net wide if breadth and depth are desired.
-----------------------------------
> - Ian Parker
Brian Martin
(well or poorly :) ) in a "reasonable" timeframe and with a large
but finite number of processing resources (neurons). But it takes about
20 years to train a new one fresh from the factory.
I'm not sure (or care really) whether what problems we solve day in day
out are NP-complete or not, but they can definitely be solved with
finite resources in reasonable time. Of course our real life problems
could probably be solved better, or sooner, and there are of course
whole realms of problems which we fail to solve, individually,
collectively, and globally.
Ian Parker
Statistical processing is NOT simply tossing a coin. You are making a
choice based on probabilities. If NL is to convey any meaning at all
these probabilities must, as a rule, amount to virtual certainties.
- Ian Parker
pipeDream
quite true.
The choice of the qualifier "crudely" was deliberate and there is no
denying that statistical methods are useful. Having said that in
Information Extraction for example it becomes necessary to weigh out
the pros and cons of statistical vs rule based models. speed ,
disambiguation, validity across corpora.. The last of these requires a
new look at what exactly is a rule(virtual certainty if you like) and
how it relates to the statistical model.
you may take look at http://smartai.blogspot.com/ for a little more
elaboration.
GR.
Wolf K
Thinking sideways --
For any given set of words, a variety of collocations are syntactically
possible. Of these, a subset are semantically possible (ie "have
meaning"). However, in general only a subset of semantically possible
collocations are "acceptable usage". Collections NL texts and speech
represent these acceptable usages, most of which are acceptable in
specific (and sometimes extremely narrow) contexts. Etc. IOW, Many rules
and constraints operate on the choice of collocations, which means that
absent a complete rule and meta-rule set, statistical description is the
best we can produce.
"Mastering" a language consists of internalising some set of
rules/meta-rules such that one chooses the collocation acceptable in a
given context. In every society (and social group) both linguistic and
on-linguistic cues are used to place an utterance in its intended
context, to signal deliberate violation of context, etc. In general,
meta-linguistic cues are more important than linguistic ones, a fact
that drives the "one word = one meaning" reformers nuts.
Cheers,
wolf k.
Brian Martin
Great summation of the issues Wolf :}
(notation: < = subset of)
1. Acceptable usage < Semantically possible
< Syntactically possible
< Random collocations
2. real usage includes context AND deliberate violations of context,
signalled by intonation, emphasis, or non-verbal clues.
Ian Parker
> elaboration.
> GR.
I had a look at your blog. It is perfectly true that there is no
contradiction between statistical methods and grammatical
considerations. Indeed it is possible to have a maximum entropy
approach whereby grammar and meaning are both present in a
"Hamiltonian". I am in fact in the process of wring a program which
does just this.
When I use the word "Hamiltonian" I am using it in the sense that a
tranaslation from one language to another (say Arabic-English) is a
relaxation process similar to annealing.
When you talk about "formal systems" there are formal words which have
got a very specific meaning. The irony is that Google can translate
into Formal words.
http://sites.google.com/site/aitranslationproject/deepknowled
If you have something like Google meaning is available. The Stefan
Boltzmann law is found by the statistical methods of Google Translate.
This is a FORMAL word. I can define Stefan Boltzmann in a system like
Mizar. Google produces the correct law when either "Stefan Boltzmann"
or "Black Body" is searched for. Why can't Google Translate label
formal terms.
Just for fun I translated "Stefan Boltzman law" into French, German,
Spanish, Russian and Arabic. All except French produced a correct
translation. French said "Stefan loi de Boltzmann" clearly wrong. I
sometimes wonder why an obviously formal concept is not translated
correctly in ALL languages. I would have thought too the Spanish and
French would produce similar answers, but they don't.
I think it would be a good idea if Google Translate would start
flagging formal words. These formal words would be translated by a
human into the various languages.
- Ian Parker
pipeDream
> On 11 Feb, 15:37, pipeDream <Gopalarao.madd...@gmail.com> wrote:
>
>
>
>
>
> > On Feb 9, 4:41 pm, Ian Parker <ianpark...@gmail.com> wrote:
>
> > > You have made a number of points. One issue I have with you.
> > > Statistical processing is NOT simply tossing a coin. You are making a
> > > choice based on probabilities. If NL is to convey any meaning at all
> > > these probabilities must, as a rule, amount to virtual certainties.
>
> > > - Ian Parker
>
> > quite true.
> > The choice of the qualifier "crudely" was deliberate and there is no
> > denying that statistical methods are useful. Having said that in
> > Information Extraction for example it becomes necessary to weigh out
> > the pros and cons of statistical vs rule based models. speed ,
> > disambiguation, validity across corpora.. The last of these requires a
> > new look at what exactly is a rule(virtual certainty if you like) and
> > how it relates to the statistical model.
>
> - Show quoted text -
I assumed (perhaps unrealistically) that Natural language
Understanding was meant since he said "I do not constrain...".
My messages should be viewed in this perspective. If this is not your
interest you may skip the rest.
Peter Drucker's dictum or it's equivalent "Look before you leap" or
"Take a top down view before getting bogged down in nitty gritty" is
useful in view of time constraints for everybody.
Let me start by saying that like others I have gone through
generations of Natural language processing tools conceptual and worked
through code of others in addition to creating my own each time saying
"this is it. Eureka!!", until I was dissuaded by a host of materials
the starting point being John F. Sowa's The Challenge of Knowledge
Soup" see http://www.jfsowa.com/pubs/. So today I , like most others
who have spent time recognise that the problems here are currently
unsolved and likely to remain so for a long long time.Many actually
beleive that an artificial mind is what is needed.
That said it is fascinating and a worthwhile Mind sport and in view of
it's use I would be surprised if the biggies-IBMs,Microsoft etc are
not at it.
The least we can do is to undestand the limitations of our current
systems and How we might overcome them. To understand the limitations
is to uncover the assumptions- often unstated that underlie the
current systems. In a word make the implicit into explicit. This is
more in the nature of a philosophers task and that is the reason why I
said back to a philosophy class.
All the messages I happened to look at seemed to take for granted what
I would call a language philosophy model=bag of words. and a reasoning
philosophy model=predictate logic.
A little elaboration - Nobody wants to use NLP to do only pos tagging
and parsing. One wants to use it in practical applications to do that
the most common approach is the use of predicate logic or it's
variants. So we really have Some assumptions about language. This set
of assumptions about the language can be summed up by the term
language philosophy model.
we view language
as a finite set of words and word collocations.
words are related to other words
words can be categorised
allowable word sequences are governed by a grammar
An alternate view of language(language philosophy model=concepts)
could be that it is a set of concepts not words. one difference that
there is no polysemy here.Bank as in finacial institution is different
concept from bank as in the bank of ganges(river) or a bank of
radars.You would have different wordnet senses.But in a concept net
they would be different concepts.
Similar remarks hold for reasoning philosophy model
Is there an alternative reasoning to predicates well yes. Take a look
at http://web.media.mit.edu/~hugo/conceptnet/ He doesn't use predicate
logic. I am not endorsing that work. All I am saying is don't shut
your eyes to alternate models. Now try and figure out the relevance of
NPHard in relation to conceptnet-No predicate logic simple node
search. So before you categorise a problem know what the problem is
and the terms in which you have described it.
Actually there exist a range of models (models in the sense of
engineering/physical science models not tarski type model) depending
on the philosophical beliefs embodied in
1.language philosophy model
2.reasoning philosophy model
Once you are through with this phase and taken position you will want
to use the models to capture capture Laws/Rules or regularities of
language and logic. The exact terms of reference in the law will
depend on the two models as above. Every body knows what these are for
the bag of words model. You look for regularity in terms of Noun
phrase preceding verb phrase and so on. your terms of refernce will be
words,word order,phrases,phrase order..etc. It is at this point that
you might wonder wether to use a rule based search of the corpus or a
statistical approach to capture the regularities. Or quite simply
generate the rules using volunteers to find the regularities.
I would not venture into the number and quality of the models other
than to say well they are there find them time permitting.
Another thing worth mentioning is Analog reasoning and the work of
Douglas Hofstadter http://en.wikipedia.org/wiki/Douglas_Hofstadter. I
dont particularly beleive in the utility of cognitive science but yes
it is worth a dekko.
you may also look at http://formalsystemsphilosophy.blogspot.com/
Some Author specific messages
Wolf K
You are right when you say that "What makes natural languages
impossible to formalise is metaphor". But Need we Formalize it in the
sense of bottling it in a Formal Logic sytem which was originally
designed for a different purpose-Viz the Axiomatization of
mathematics.
"For any given set of words, a variety of collocations are
syntactically.."
You are absolutely right within what I am referring to as language
philosophy model=bag of words. Change the Axioms( I am using the word
very loosely for belief in a set of views) and you will arrive at a
different conclusion and perhaps a new set of problems. As the above
examples show there are alternatives to a statistical description.
Neil W Rickert
"And semantics does not easily map into rules of inference." Reason is
that we are doing something we perhaps should not be doing . Semantics
in a formal logic system would be defined in terms of the
interpretation of the terms and compositionality where as I would
rather define sentence as meaningful if it makes sense to a human.
"Green Ideas sleep furiously" might not make sense to Noam Chmosky.
But it does to me I interpret it as meaning Newly developed ideas lie
there sleeping for a long while before being put to use.
Poetry,Rhetoric and Metaphor cannot be modelled by formal logic
systems.
"ontology is crap". I can understand your frustration. But
categorisation thinking in terms of hierarchy are useful tools. I
noticed that SUMO puts the same entity at two places in the tree. My
Solution was to add a tree basis i.e to use a hierarchy of entities
specified not just by entity but by another attribute = tree basis.
e.g. powerset(the company) will appear in the tree of Microsoft
Subsidaries (=basis) (and Sub Subsidaries). It can also appear in the
tree of Industries (basis=Industry). You could add the time element
also.
Ian Parker
"mathematics may be defined as the study of formal systems"
This is one view and certainly not the current one. see for example
http://en.wikipedia.org/wiki/Penelope_Maddy
Translation in general is part of NLU rather than NLP though you could
approximate translations using statistical or other techniques.
The word "formal systems" is used by a majority to mean formal logic
systems. I prefer to use it more broadly.
I am not too clear on How "it is possible to have a maximum entropy
approach whereby grammar and meaning are both present in a
"Hamiltonian"."
Any way you seem to have missed the main point viz that we have to
look beyond grammar and predicate logic- My be I have not communicated
effectively.
Lotzi Boloni
Computer Language is a formal system - is not a definition of
programming Language. It simply means that the Language can be
modelled using Formal Logic which ensures that the programmer is
assured of a correct output under all logical circumstances - by
simulation of the language on a computer.Abstraction removes such
things as font,format etc . You are right when you say that it does
not fully model the real world.
Brian Martin
I think I agree with you that Semantic modelling is more important and
perhaps use Syntax only as an adjunct. Perhaps we could do direct
semantic parsing.Semantic Role labelling uses parse as one feature but
even that is not essential if an alternate set of features could be
put in place.
Thanks to everyone for their views and comments
GR
Neil W Rickert
>Let me start by saying that like others I have gone through
>generations of Natural language processing tools conceptual and worked
>through code of others in addition to creating my own each time saying
>"this is it. Eureka!!", until I was dissuaded by a host of materials
>the starting point being John F. Sowa's The Challenge of Knowledge
>Soup" see http://www.jfsowa.com/pubs/.
Sowa's concern is with knowledge representation (KR). And IMO that is
never going to work.
>The least we can do is to undestand the limitations of our current
>systems and How we might overcome them. To understand the limitations
>is to uncover the assumptions- often unstated that underlie the
>current systems. In a word make the implicit into explicit.
I have been doing that. It seems that nobody is interested. People
prefer to cling to their implicit assumptions.
> So we really have Some assumptions about language. This set
>of assumptions about the language can be summed up by the term
>language philosophy model.
>we view language
> as a finite set of words and word collocations.
> words are related to other words
> words can be categorised
> allowable word sequences are governed by a grammar
I no longer make such assumptions.
>An alternate view of language(language philosophy model=concepts)
>could be that it is a set of concepts not words.
That's fine. But you would need to develop a clear idea of what
concepts are, and where they come from.
>Is there an alternative reasoning to predicates well yes. Take a look
>at http://web.media.mit.edu/~hugo/conceptnet/
In my judgement, that won't work either, and deep down it really is
based on the same kind of implicit assumptions as the "set of words"
view.
>Some Author specific messages
>Neil W Rickert
>"ontology is crap". I can understand your frustration.
That was not said in frustration. Rather, it was part of my
questioning the implicit assumptions that people make. And,
judging by your response, you are clinging to those implicit
assumptions with great tenacity.
> But
>categorisation thinking in terms of hierarchy are useful tools.
Thinking is prerequisite to categorization thinking. Hence the
problem of "thinking" needs to be solved first. People do not depend
on categorization to nearly the extent that is commonly assumed.
Rather, other more basic and more important cognitive activity is
mistakenly attributed to categorization.
Ian Parker
> Ian Parker
> "mathematics may be defined as the study of formal systems"
> This is one view and certainly not the current one. see for examplehttp://en.wikipedia.org/wiki/Penelope_Maddy
> Translation in general is part of NLU rather than NLP though you could
> approximate translations using statistical or other techniques.
Mizar does not enter into philosophical discussion. Mizar simply asks
whether my train of reasoning fulfils certain rules. You might ask the
question "Is Mizar right?" Wittenstein might argue that it wasn't.
However what I am talking about here is my Arabic translion. About "4
times the temperature". Mizar is perfectly capably of encoding Stefan
Boltzmann in, if not a toally correct philosophical way, at least a
way that would satisfy me as a physicist.
>
> The word "formal systems" is used by a majority to mean formal logic
> systems. I prefer to use it more broadly.
>
> I am not too clear on How "it is possible to have a maximum entropy
> approach whereby grammar and meaning are both present in a
> "Hamiltonian"."
I am aware that Maximum Entropy is a bit of a handful. I am using
analogies with Physics here. If we have a physical system, whether a
droplet of water, a galaxy or whatever it can be readily appreciated
that Entropy is a maximum. Let me take 2 words "bank" and "bonus".
What sort of a bank have we? "Bonus" tells me that is is probably a
financial institution, quite possibly one that has been "bailed out"
by the taxpayer. But I am not certain about this, suppose I receive
news of my bonus while I am walking along a river bank. We have a
number of words here, all of which affect the probability of the type
of "bank" we have. We can postulate that we are translating into a
larguage where they are different. German = Uefer (river), Bank
(financial). We translate in accordance with statistics.
Hutter states that AI is defined purely by compression. This is
synomous with entropy and we need and we need a technique whereby we
can evaluate the effect (fuzzy) of every word on every other. In
Physics this is called a Hamiltonian. We have a concept of "energy"
which is defined as exp(-E/kT).
> Any way you seem to have missed the main point viz that we have to
> look beyond grammar and predicate logic- My be I have not communicated
> effectively.
>
Let us take a concrete example - Translation. Every year there is a
competition in machine translation. Arabic, Chinese and Urdu have to
be translated into English. Google that operates on a strictly
statistical method has consistenly won year on year. The article which
I was discussing was an article in an arabic lanuage science magazine
on the main sequence. The argument was the relationship between size,
luminosity and color. The odd thing was that Google got the Stefan
Boltzmann law, but then misquoted it in its translation.
Perhaps I have not made myself clear. If I state the SB law I have
gone beyond grammar and syntax. A question I would like to ask you is
how do you propose to go beyond grammar and syntax? I can only do that
if I have some sort of model of the real world. I can I suppose draw a
Pert chart for building a dam.
Google translated "wsT jw AlmErkp" as "central air battle". Correct is
"the climatic environmental battle" or a more free translation would
be "the battle against climate and environment". Nobody seems to talk
about the difference between pouring concrete and fighting Israel in
Turing/Loebner competitions. AS I said a Pert chart is some sort of
description. At least there is no war.
I agree with you in part, but there needs to be some flesh on the
bone.
- Ian Parker
pipeDream
Basically I just wanted to point out
-That present methods are limited
-Pointers to some present alternatives like concept Net,CBR
I think the general problem is big even for the biggies let alone
individuals.To me it is a Mental Sport or a puzzle to be solved during
leisure. So If the there is no concrete proposal don't blame me. May
be the apple of "Enlightenment" will fall on some 21st century newton.
Antti might benefit from the results of Project Halo referred to in
Sowa's Challenge. You may wish to see Formalization in what sense
would be useful.
"Sowa's concern is with knowledge representation (KR). And IMO that
is never going to work."
Any General purpose NLU has to have KR of one sort or the other.I have
called it reasoning philosophy model because I wanted to include
general purpose reasoner including such things as procedural
attachments.
"That was not said in frustration. Rather, it was part of my
questioning the implicit assumptions that people make. And,
judging by your response, you are clinging to those implicit
assumptions with great tenacity."
Sorry about the lax usage of "frustration".
Some observations on language A boy or girl saying "I love you"
certainly does not think of love as " A series of occurent and
dispositional states."
Nobody sits with a book of grammar and style in the normal course of
life.Yet majority of the utterances do make sense in context.
So I was rather lax in my usage of "frustration".
In a technical institute/University the emphasis should be on creating
value addition as a product/body of knowledge that can better fulfill
the needs of society. The work of a technical person would be judged
by it's usefullness in relation to the context desired.
That is one reason why I did not like the phrase "Quality of AI" used
by Ian. To recall a useful definition of quality - "is fitness for
use."One judges quality only in relation to it's intended use. Google
is certainly nothing as an NLP/NLU but as a search tool it is
outstanding.
Yes I am clinging to the old because there is nothing either in my
stable or that of others which does a better job. I am happy to note
that you took the time off to examine the assumptions.
"In my judgement, that won't work either, and deep down it really is
based on the same kind of implicit assumptions as the "set of words"
You are right.Concept net is a semantic network and it is well known
that FOL subsumes a semantic network in the sense that every semantic
network can be expressed in FOL. So it really is part of the socalled
guarded fragment of logic.
But it does show that there could be other alternatives. Yes it treats
words and word collocations as concept nodes. There could be other
ways. But in terms of the goal of using common sense reasoning it does
a better job than OpenCyc.
"I have been doing that. It seems that nobody is interested. People
prefer to cling to their implicit assumptions."
Once again I am glad that you are interested. I might cling to
assumptions on deliverable mode but not in frewheeling mode.
"Hence the problem of "thinking" needs to be solved first."
I would rephrase it as "thinking" needs to be modelled. We are not
really concerned with human brain-the neurons,synapse et al. We need
to replicate the black box behaviour. The aeroplane does not fly the
way birds do. I am not so much concerned wether people depend on
categorization. I am concerned wether categorization and hierarchical
thinking can be modelled to produce useful results.
Ian
My knowledge of Maximum Entropy methods is based on "A Maximum Entropy
Approach to Natural Language Processing" By
Adam L. Berger,Stephen A. Della Pietray,Vincent J. Della Pietray
my problem is there is no Hamiltonian in there. Besides it open with
an example of translating from english to french.
I don't see too much merit in Hutter's work in relation to my current
interests.
"how do you propose to go beyond grammar and syntax? "-No pat answers.
Collecting data shifting evidence... but I do beleive that even in
practical systems depending on the language and reasoning models there
is a wide spectrum ranging from Analogue methods like CBR,Semantic
networks with procedural attachments( hacks?) Analogue reasoning
theorem provers in addition to the regular FOL methods.
Ian Parker
> Ian
> My knowledge of Maximum Entropy methods is based on "A Maximum Entropy
> Approach to Natural Language Processing" By
> Adam L. Berger,Stephen A. Della Pietray,Vincent J. Della Pietray
> my problem is there is no Hamiltonian in there. Besides it open with
> an example of translating from english to french.
>
> I don't see too much merit in Hutter's work in relation to my current
> interests.
>
I hope I am not giving pat answers. I try to give careful considered
answers to these very difficult questions. The question of
Hamiltonians first of all. As I have said I am a physicist by traing
so I immediately look at compression and think of the 2nd law of
thermodynamics.
Lets take a very simple probability evaluation to start with. The
frequency with which words occur in the English (or any other for that
matter) language. These probabilities are influenced by context. Bonus
and bank (financial). Fat cats are getting bonuses for failure. I
could say that a walk along the left bank (rive gauche) is an
additional bonus. If I am translating into French what is the
proability of "bank" and what is the probability of "rive". If depends
on the surrounding words. I am going to say that "bonus" affects P
(bank) and P(rive). Suppose initially they are both 0.e. After bonus
rive=0.25 bank=0.75. I now define the energy of rive-bonus to be -log
(2) and bank-bonus to be log(2).
In evaluating probabilities I need a logarithmic expression. The
innovation that I have made is use the term "energy" / "hamiltonian".
Energy gives us exp(-E/kT) thus E/kT is a logarithmic expression. A
system which is in contact with a large heat reservoir at temperature
T partitions itself in EXACTLY the way "bank" and "rive" do. You say
the two languages were English and French.
> "how do you propose to go beyond grammar and syntax? "-No pat answers.
This goes beyong grammar and syntax. People do however have bad
grammar sometimes. There is a very famous Frenxch film "La cage aux
folles". This was about a gay nightclub; Now on the fact of it "aux
folles" does not make sense. "aux fous" would be what I would expect.
I you twig that it is about men like women, attracted to other men,
then it makes sense. You need to go quite deep for that though.
Without such knowledge all you have is a hamiltonian. aux folles = of
mad women.
> Collecting data shifting evidence... but I do beleive that even in
> practical systems depending on the language and reasoning models there
> is a wide spectrum ranging from Analogue methods like CBR,Semantic
> networks with procedural attachments( hacks?) Analogue reasoning
> theorem provers in addition to the regular FOL methods
>
Google is almost getting there with its large database and statistical
methods. However, as demonstated by the Spefam Boltzmann law. It does
not use its search power to improve its translation.
http://sites.google.com/site/aitranslationproject/deepknowled
Another rather deep question is this. "Can we reconcile a deep
knowledge (understanding) picture with statistical methods?" Yes we
can. The deep meaning is of course in itself statistical.
- Ian Parker