no response to me. no response to Eric Jacobson (whose tone was much kinder
than mine, as if that would do any good) nor multiple other posters from
comp.dsp . they just want the issue to go away and for us DSPers to forget
about it.
'smatter MathWorks? finding out that your lame-ass excuses don't fly?
i know that there's Octave and Python (and MathCad and, hell, we can always
code it in C or C++) but, like Micro$hit, MATLAB has become the defacto
standard in modeling signal processing systems (as well as other systems).
the question is, does MathWorks plan to deal with their lead position in
this particular market the same way that Micro$oft does or do they respond
to their customer base and do their best, or at least what they can do to
make their product the best it can be or at least as good as its promise?
--
r b-j
Wave Mechanics, Inc.
45 Kilburn St.
Burlington VT 05401-4750
tel: 802/951-9700 ext. 207 http://www.wavemechanics.com/
fax: 802/951-9799 rob...@wavemechanics.com
--
Clay
p.s. A business man doesn't do something unless it either makes him
money or saves him money.
> Hello Robert,
> I do the consumer thing here and use other packages such as MathCad and
> Mathematica.
MATLAB still seems to be the standard in which to share or demonstrate DSP
concepts to remote parties.
> p.s. A business man doesn't do something unless it either makes him
> money or saves him money.
probably mostly true (we do things here at Wave Mechanics when customers
scream, because we wanna keep those customers) but not very enlightened.
>looks like they sent out Nabeel with the anemic and hollow "We care"
>response and, again, absolutely no response to the technical problem that
>was put forth. and now, it appears that their hoping that it blows over
>again.
>
>no response to me. no response to Eric Jacobson (whose tone was much kinder
>than mine, as if that would do any good) nor multiple other posters from
>comp.dsp . they just want the issue to go away and for us DSPers to forget
>about it.
>
>'smatter MathWorks? finding out that your lame-ass excuses don't fly?
>
>i know that there's Octave and Python (and MathCad and, hell, we can always
>code it in C or C++) but, like Micro$hit, MATLAB has become the defacto
>standard in modeling signal processing systems (as well as other systems).
>the question is, does MathWorks plan to deal with their lead position in
>this particular market the same way that Micro$oft does or do they respond
>to their customer base and do their best, or at least what they can do to
>make their product the best it can be or at least as good as its promise?
A big difference in this case is that there are plenty of
alternatives, some of which you've listed already. If the Mathworks
continues to ignore these issues, they'll just lose more sales to the
competition in the DSP community. If the people I've worked with and
around over the last 15 years are any indication, they've already lost
a tremendous amount of business, and I think it'll only get worse as
the alternatives get better and more numerous. IMHO the Mathworks
ignores this at their own peril. Their business may still grow (which
would be my guess), but they'll be underserving a significant market,
and leaving a lot of business untapped. The business is theirs to
lose.
Eric Jacobsen
Minister of Algorithms, Intel Corp.
My opinions may not be Intel's opinions.
http://home.earthlink.net/~ejacobsensprint
Hi,
Don't tell me you're really surprised ! The Mathworks has had a contemptuous
attitude with their customers for years now. The answer to your question
is YES : The MathWork$ is just like Micro$oft. They have to deal with
DSPs because there is a market. As they already are a de facto standard for
numerical simulation, people buy their (expensive) products without any
doubt... and are disapointed most of the time, of course.
But who cares ? If customers are ready to pay a whole lot of money for
software tools that are not worth the third, why should Mathworks not
go on ?
As for me, when I had to do DSP stuff, I used other tools : other code
generators like Mustig or Hypersignal (both are better than Matlab's)
or manual C programming. Even for numerical simulation, we now have less
Matlab products than we had a couple of years ago...
Ben
--
---------------------------------------------------
Benoit LEPRETTRE Signal Processing Engineer
Schneider Electric SA, Grenoble, France
Note : The opinions here are mine, not my company's
---------------------------------------------------
Speaking of alternatives, I get a lot of usefulness out of Scilab. Indexing
is one-based, but hey, it's free! (development is funded by the French
government...maybe the French are ok after all). It's not nearly as
full-featured as MATLAB but I just can't bring myself to spend that kind of
money (n * $1000) for my home projects. At work, I have need of such a tool
too infrequently to justify the expense. If they would make a relatively
cheap base product (< $500), then charge big bucks for the "toolboxes", I'd
be first in line. For now, I'll stick to Scilab and C++.
Rick Armstrong
Note: (the "attached" email address is bogus)
What d'ya mean "after all" ?!? :o)))
>It's not nearly as
> full-featured as MATLAB but I just can't bring myself to spend that kind of
> money (n * $1000) for my home projects. At work, I have need of such a tool
> too infrequently to justify the expense. If they would make a relatively
> cheap base product (< $500), then charge big bucks for the "toolboxes", I'd
> be first in line. For now, I'll stick to Scilab and C++.
There are also a lot of toolboxes for Scilab, but surely not as many as for
Matlab. Enough for most users, though...
"robert bristow-johnson" <rob...@wavemechanics.com> schrieb im Newsbeitrag
news:t3oR7.133$8m1...@nwrddc01.gnilink.net...
Bob
--
"Things should be described as simply as possible, but no simpler."
A. Einstein
////////////////////////////////////////\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
To contribute your unused processor cycles to the fight against cancer:
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\///////////////////////////////////////
> Robert, would you mind summarizing the issue under discussion with those
> of us in comp.dsp that don't go over there routinely?
Here are three threads--from newest to oldest...
<http://groups.google.com/groups?hl=en&threadm=MXbQ7.14%24s71.17614%40nwr
ddc02.gnilink.net&rnum=3&prev=/groups%3Fq%3Drobert%2Bbristow-johnson%26hl
%3Den%26group%3Dcomp.soft-sys.matlab%26rnum%3D3%26selm%3DMXbQ7.14%2524s71
.17614%2540nwrddc02.gnilink.net>
<http://groups.google.com/groups?hl=en&threadm=3882B39D.734E%40viconet.co
m&rnum=5&prev=/groups%3Fq%3Drobert%2Bbristow-johnson%26hl%3Den%26group%3D
comp.soft-sys.matlab%26rnum%3D5%26selm%3D3882B39D.734E%2540viconet.com>
<http://groups.google.com/groups?hl=en&threadm=3896aaf2.1458218958%40news
.dtu.dk&rnum=1&prev=/groups%3Fq%3Drobert%2Bbristow-johnson%26hl%3Den%26gr
oup%3Dcomp.soft-sys.matlab%26rnum%3D1%26selm%3D3896aaf2.1458218958%2540ne
ws.dtu.dk>
Google is a wonderful thing,
KP
> Robert, would you mind summarizing the issue under discussion with those
> of us in comp.dsp that don't go over there routinely?
From my POV, the issue centers around a basic promise of MATLAB:
"MATLAB is a high-performance language for technical computing. It
integrates computation, visualization, and programming in an easy-to-use
environment where problems and solutions are expressed in familiar
mathematical notation." (Getting Started with MATLAB, v. 5)
Or from an older MATLAB reference guide (ca. 1992):
"MATLAB integrates numerical analysis, matrix computation, signal
processing, and graphics in an easy-to-use environment where problems and
solutions are expressed just as they are written mathematically - ... "
Note the phrases in claims: "familiar mathematical notation" and "just as
they are written mathematically". I submit that those claims are false in a
sense that is salient particularly for those of use who use MATLAB for
(digital) signal processing.
In Discrete-Time Signal Processing (otherwise known as "Oppenheim and
Schafer") p. 531, Eqs. (8.57) and (8.58), the Discrete Fourier Transform
(DFT) and inverse DFT are defined as:
N-1
X[k] = SUM{ x[n] * exp(-j*2*pi*n*k/N) }
n=0
N-1
x[n] = (1/N)*SUM{ X[k] * exp(+j*2*pi*n*k/N) }
k=0
Whereas MathWorks defines these for MATLAB to be:
N
X[k] = SUM{ x[n] * exp(-j*2*pi*(n-1)*(k-1)/N) }
n=1
N
x[n] = (1/N)*SUM{ X[k] * exp(+j*2*pi*(n-1)*(k-1)/N) }
k=1
Likewise on p. 21 Eq. (2.39) O&S define discrete convolution as:
+inf
y[n] = SUM{ x[k] * h[n-k] }
k=-inf
which is equivalent to:
+inf
y[n] = SUM{ x[n-k] * h[k] }
k=-inf
If the impulse response were causal and finite (i.e. a causal FIR where N =
# of taps), that equation would degenerate to:
N-1
y[n] = SUM{ x[n-k] * h[k] }
k=0
But MATLAB says it's this:
N
y[n] = SUM{ x[n-1-k] * h[k] }
k=1
or
N-1
y[n+1] = SUM{ x[n-k] * h[k+1] }
k=0
and, in fact, makes excuses for that in the Signal Processing Toolbox manual
(ca. 1993) p. 1-12:
"Note: The filter coefficients start with subscript 1 rather than 0. This
reflects MATLAB's standard indexing scheme for vectors."
and p. 2-56:
"The series is indexed from n+1 and k+1 instead of the usual n and k because
MATLAB vectors run from 1 to n instead of 0 to n-1."
The DFT and convolution summations are not the only examples (the way IIR
coefficients are ordered is another that occurs to me), but are *so*
fundamental that for a company whose product expresses mathematical
instructions "just as they are written mathematically", lot's of red flags
should have been flying up when they were essentially being forced to change
or contrive those well established expressions into their fixed 1-based
indexing scheme. The fact they didn't do this 10 years ago is no excuse for
not fixing the problem now ASAP, because their product has already become
the defacto standard used by the signal processing community to concisely
exchange ideas and algorithms (in addition to modeling and emulating these
algorithms for themselves).
So will MathWorks deal with this the same way Micro$oft does or will they do
the right thing? Will they use their market power to replace the
mathematically natural definitions of fundamental DSP concepts that exist in
*all* of the textbooks with the unnatural (offset by 1) definitions that
MATLAB currently has? Will Oppenheim and Schafer eventually have to change
their representation of convolution and the DFT (and the hundreds of
formulae based on those definitions) to the MATLAB version? Or will we
continue to have to juggle between the conflicting definitions in our work?
Is it consistent with the goal of MATLAB to provide an "easy-to-use
environment where problems and solutions are expressed just as they are
written mathematically" for us to juggle between these two non-equivalent
definitions of the same fundamental DSP formulae that nearly *every* DSP
concept is based on? I can't think of hardly anything in DSP that is not
fundamentally based on convolution or the DFT somewhere in the operation.
In a later post, I will present (again, since I did this a year ago) a
perfectly backward compatible solution that, for some odd reason, MathWorks
apparently refuses to consider or to debate.
In article <e5qS7.1887$Ga....@nwrddc01.gnilink.net>,
robert bristow-johnson <rob...@wavemechanics.com> replied (in part):
> ...
>
>Is it consistent with the goal of MATLAB to provide an "easy-to-use
>environment where problems and solutions are expressed just as they are
>written mathematically" for us to juggle between these two non-equivalent
>definitions of the same fundamental DSP formulae that nearly *every* DSP
>concept is based on? I can't think of hardly anything in DSP that is not
>fundamentally based on convolution or the DFT somewhere in the operation.
>
>In a later post, I will present (again, since I did this a year ago) a
>perfectly backward compatible solution that, for some odd reason, MathWorks
>apparently refuses to consider or to debate.
>
>r b-j
Hi --
Like many people at MathWorks, I have been reading all the posts in
this thread, but have hesitated to get involved because the level of
discussion has been so unpleasant and nontechnical. But here Robert
has made a compelling argument that deserves a civil reply.
I agree that indexing in DSP is zero-based and I agree that it would
be desirable to reflect that in MATLAB notation. But MATLAB is based
on linear algebra where indexing is conventionally one-based. As
a result, instead of defining the FFT by
> N
> X[k] = SUM{ x[n] * exp(-j*2*pi*(n-1)*(k-1)/N)
> n=1
I would use
N-1
X(k+1) = SUM x(n+1) * exp(-j*2*pi*n*k/N) , k = 0:N-1
n=0
Or, better yet, use matrix-vector multiplication:
% The Finite Fourier Transform
omega = exp(-2*pi*j/N)
k = 0:N-1
F = omega.^(k'*k)
X = F*x
But that's avoiding the issue of zero-based subscripts.
The real problem is backwards compatibility. The MATLAB community,
including MathWorks and customers around the world, has millions of
lines of code using one-based subscripts. What happens to all that
code if we extend MATLAB notation to also allow zero- or variable-based
subscripts? What does "find" return? How do I write a matrix library
if I don't know where the subscripts start? How do I refer to the leading
element of a string? In fact, is it the "first" element, or the
"zeroth" element?
We don't refuse to consider this issue. There are few issues that
we have considered more. And we don't refuse to debate. But if
you want our participation, please make it a debate, not a diatribe.
-- Cleve Moler
mo...@mathworks.com
I might have been unpleasant, but have always been quite technical regarding
this issue.
> But here Robert
> has made a compelling argument that deserves a civil reply.
>
> I agree that indexing in DSP is zero-based and I agree that it would
> be desirable to reflect that in MATLAB notation. But MATLAB is based
> on linear algebra where indexing is conventionally one-based. As
> a result, instead of defining the FFT by
>
>> N
>> X[k] = SUM{ x[n] * exp(-j*2*pi*(n-1)*(k-1)/N)
>> n=1
>
> I would use
> N-1
> X(k+1) = SUM x(n+1) * exp(-j*2*pi*n*k/N) , k = 0:N-1
> n=0
But that is not what *any* DSP textbook uses for it's definition of the DSP,
is it Cleve? Slipping that "+1" into the subscript of the array looks
reasonably innocuous but what happens if you say:
[Xmax, Fmax] = max(abs(X));
to find the value of the peak frequency, Fmax? We must juggle your
definition with the natural and well-established definition in all of the
DSP textbooks and literature. We must remember to subtract 1 from Fmax,
otherwise it will be wrong. In a large and complicated MATLAB program (say,
a souped-up phase-vocoder), it is not unheard of (I can testify to this
directly) to lose hours of time searching for an obscure bug where one has
forgotten to subtract that 1 in computing frequency. But it's hard for me
to accept blame for that bug when the root of it is the non-standard
definition of the DFT that is forced upon us by the inflexible (and also
incorrect or at least unconventional) indexing scheme in MATLAB.
> Or, better yet, use matrix-vector multiplication:
>
> % The Finite Fourier Transform
> omega = exp(-2*pi*j/N)
> k = 0:N-1
> F = omega.^(k'*k)
> X = F*x
>
> But that's avoiding the issue of zero-based subscripts.
>
> The real problem is backwards compatibility.
I've disposed of that last year. I hope that you are willing to debate this
to its conclusion rather than just going silent when dispose of it this
time. That is why I got so frustrated and less civil. You state that
backwards compatibility is a problem and when I show exactly why it is not,
there was nothing but silence from you, Cleve, and everyone else at
MathWorks.
> The MATLAB community,
> including MathWorks and customers around the world, has millions of
> lines of code using one-based subscripts.
Of course. No product enhancement should hurt any of that code or the basic
function of it. I fully agree that it should be perfectly backward
compatible.
> What happens to all that
> code if we extend MATLAB notation to also allow zero- or variable-based
> subscripts?
NOTHING DOES. The reason why we know this is that 0 is the "identity vector
under the operation of addition". As I've suggested last year, the
perfectly backward compatible fix is to add to each MATLAB variable, right
beside the internal vector that defines the length of each dimension of the
MATLAB array, a vector of identical length (the length is equal to the
number of dimensions) that contains offsets to the indices of each
dimension. The implied "base" index is 1-offset for that dimension and THE
OFFSETS ALWAYS DEFAULT TO ZERO when a new array is created from scratch or
as the result of any operation on other 1-based (or 0-offset) arrays.
That's what makes it perfectly backward compatible. No legacy code will
break and I challenge anyone to come up with an example that does. I am
completely confident about this because I know that when you add or subtract
0 to any number, the result is unchanged.
> What does "find" return?
the indices returned by min(), max(), find(), sort(), ind2sub() (and any
other MATLAB function that returns indices) would have to adjust its
returned values by subtracting that very same offset for the corresponding
dimension.
> How do I write a matrix library
> if I don't know where the subscripts start?
But you *do*! Because of at least two new facilities that would have to be
created, one for changing the base (similar to reshape()), let's call it
"rebase()" and another for determining the base (similar to size()), let's
call that "base()". Rather than making you go back in Google Groups and
read what I posted a year ago, I will try to post in a few hours exactly
what the proposed fix is. Then nearly all of these questions will be
answered.
> How do I refer to the leading element of a string?
The same as you do now, except that MATLAB subtracts the offset parameter
that is defaulted or initialized to zero.
> In fact, is it the "first" element, or the "zeroth" element?
If it's a 1-based array, it's the "first" element (and that is always the
default and doesn't change unless the MATLAB user specifically changes the
base). If it's a 0-based array, it's the "zeroth" element. If it's a -128
based array, its index or subscript is -128.
> We don't refuse to consider this issue. There are few issues that
> we have considered more. And we don't refuse to debate. But if
> you want our participation, please make it a debate, not a diatribe.
I will try. I ask in return that silence or non-responsiveness is not used
as a debating technique when I *do* address your concerns of backward
compatibility. And I would also ask that the concept of "backward
compatibility" is not changed nor obfuscated from its simple and
straight-forward definition (that is, old code written before this
enhancement continues to work after the enhancement without changes).
Give me a couple of more hours and I'll post the main details of the
proposed fix (which i did last year) that I assert is perfectly backward
compatible.
robert bristow-johnson wrote:
>
> In a later post, I will present (again, since I did this a year ago) a
> perfectly backward compatible solution that, for some odd reason, MathWorks
> apparently refuses to consider or to debate.
Ah, the old zero/one origin indexing argument. I look forward to your
proposal. The last time this discussion went around that I was in
attendance the compatibility question nailed the coffin shut on the
argument and there wasn't a whole lot of argument even presented for the
wisdom of the original decision.
As an APL user for more than 30 years I am very annoyed by the 1-origin
only architecture of Matlab but APL was designed from the ground up to
deal with dual origins so that function authors knew they had to be
aware that their functions could be called from either context and that
those considerations extended to the data itself. Being very familiar
with what that meant I can see why a change at this point could have
catastrophic impact on existing libraries but if you see a way to do it
without requiring _any_ change to the existing Matlab code base I am all
ears.
> Like many people at MathWorks,
> I have been reading all the posts in this thread
> but have hesitated to get involved
> because the level of discussion
> has been so unpleasant and nontechnical.
> But Robert has made a compelling argument
> that deserves a civil reply.
>
> I agree that indexing in DSP is zero-based and I agree that
> it would be desirable to reflect that in MATLAB notation.
> But MATLAB is based on linear algebra
> where indexing is conventionally one-based.
> As a result, instead of defining the FFT by
>
> N
> X[k] = SUM{ x[n] * exp(-j*2*pi*(n-1)*(k-1)/N)
> n=1
>
> I would use
>
> N-1
> X(k+1) = SUM x(n+1) * exp(-j*2*pi*n*k/N) , k = 0:N-1
> n=0
Actually, the index base has nothing to do
with either linear algebra or digital signal processing.
Fortran programmers use 1 based indexing for both
and C and C++ programmers use 0 based indexing for both.
> Or, better yet, use matrix-vector multiplication:
>
> % The Finite Fourier Transform
> omega = exp(-2*pi*j/N)
> k = 0:N-1
> F = omega.^(k'*k)
> X = F*x
>
> But that's avoiding the issue of zero-based subscripts.
>
> The real problem is backwards compatibility.
> The MATLAB community
> including MathWorks and customers around the world
> has millions of lines of code using one-based subscripts.
> What happens to all that code if we extend MATLAB notation
> to also allow zero- or variable-based subscripts?
> What does "find" return? How do I write a matrix library
> if I don't know where the subscripts start?
> How do I refer to the leading element of a string?
> In fact, is it the "first" element, or the "zeroth" element?
You could introduce a new indexing operator .( )
which is zero based.
> We don't refuse to consider this issue.
> There are few issues that we have considered more.
> And we don't refuse to debate.
> But if you want our participation,
> please make it a debate and not a diatribe.
I strongly recommend that you filter out the diatribes
and respond only to compelling arguments
if you are going to respond at all.
The comp.dsp newsgroup has more than its share
of indigenous trolls and battle hardened flame warriors.
You will soon learn who they are
and that it is best to simply ignore them.
> robert bristow-johnson wrote:
>>
>> In a later post, I will present (again, since I did this a year ago) a
>> perfectly backward compatible solution that, for some odd reason, MathWorks
>> apparently refuses to consider or to debate.
>
> Ah, the old zero/one origin indexing argument.
It's not just 1 vs. 0 . An array index should be able to be bounded by
virtually any pair of finite integers. Arrays should be able to start with
1, 0, 10, -10 or -128 (the negative indices make it possible to cleanly
model acausal FIR filters, something that is useful for doing interpolation
or sample rate conversion or windowing).
> I look forward to your proposal.
It be below.
> The last time this discussion went around that I was in
> attendance the compatibility question nailed the coffin shut on the
> argument
I fail to see how. They (MathWorks) brought it up, I showed why there would
be no difference on legacy code (except for a very small increase in
execution time due to one additional small operation done to the indices),
MathWorks responded with utter silence on one occasion and on another
responded with straw men that were not truly problems of backward
compatibility.
> and there wasn't a whole lot of argument even presented for the
> wisdom of the original decision.
There was none except for saying MATLAB was originally envisioned as a
concise matrix processing language and all of the math textbooks give the
upper left corner of a matrix the indices (1,1). I agree with that, but it
does not preclude having the ability to create arrays with indices that do
not begin with 1.
> As an APL user for more than 30 years I am very annoyed by the 1-origin
> only architecture of Matlab but APL was designed from the ground up to
> deal with dual origins so that function authors knew they had to be
> aware that their functions could be called from either context and that
> those considerations extended to the data itself.
I've never used APL but I do program in C a lot and you can create a pointer
to an array that can mimic an array name and offset it in such a way that
the base of the array (the index of the "leading element") can be whatever
number you want. This is because C doesn't bound-check the array indices
during run-time as MATALB or Fortran (at least the Fortran of my day) does.
That bound checking is a necessary thing for MATLAB, in my opinion, because
if I'm modeling some process where I compute an index, I want to know if my
index went out of bounds before I code this into a DSP and an airplane falls
from the sky as the result of such an error.
> Being very familiar with what that meant I can see why a change at this point
> could have catastrophic impact on existing libraries
Not if you make the default array base the same as before (and do not write
code that uses any of the new re-basing feature in which legacy code cannot
do, of course, unless somebody has a time machine). Then there is no danger
of any "catastrophic impact on existing libraries".
> but if you see a way to do it without requiring _any_ change to the existing
> Matlab code base I am all ears.
Here it is:
Not being a MathWorks insider (I can't imagine why not) I have to guess a
little at the structure of a MATLAB variable:
enum MATLAB_class {text, real, complex};
// I don't wanna cloud the issue considering other classes.
typedef struct
{
void* data; // pointer to actual array data
char* name; // pointer to the variable's name
enum MATLAB_class type; // class of MATLAB variable (real, complex,...)
int num_dimensions; // number of array dimensions >= 2
long* size; // points to a vector with the number of rows, columns, etc.
} MATLAB_variable;
name[32]; // MATLAB names are unique to 31 chars
size[num_dimensions];
if (type == text)
{
char data[size[0]*size[1]*...*size[num_dimensions-1]];
}
else if (type == real)
{
double data[size[0]*size[1]*...*size[num_dimensions-1];
}
else if (type == complex)
{
double data[size[0]*size[1]*...*size[num_dimensions-1][2];
}
When an element, A(n,k), of a 2 dimensional MATLAB array A is accessed,
first n and k are confirmed to be integer value (not a problem in C), then
confirmed to be positive and less than or equal to size[0] and size[1],
respectively. It those constraints are satisfied, the value of that element
is accessed as:
data[(n-1)*size[1] + (k-1)];
For a 3 dimensional array, A(m,n,k), it would be the same but now:
data[((m-1)*size[2] + (n-1))*size[1] + (k-1)];
I realize that the pointer to "data" can be judiciously offset so that the
subtraction of 1 from the MATLAB indices to create the C indices would not
be necessary. I think any modern Fortran does this. Also I realize that
the MATLAB variable structure may have other internal fields that are not
described above and that I don't know about, but I don't see any reason what
that would affect the issues here.
What is proposed is to first add a new member to the MATLAB variable
structure called "offset" which is a vector of the very same length
(num_dimensions) as the "size" vector. The default value for all elements
of the offset[] vector would be 0 with only the exceptions outlined below.
This is what makes this backwards compatible, in the strictest sense of the
term.
typedef struct
{
void* data;
char* name;
enum MATLAB_class type;
int num_dimensions;
long* size;
long* offset; // points to a vector with index offset for each dimension
} MATLAB_variable;
name[32];
size[num_dimensions];
offset[num_dimensions];
Now immediately before each index is checked against the bounds for that
dimension ( > 0 and <= size[dim] where 0<=dim<num_dimensions), the offset
for that particular dimension (offset[dim]) is added to the index and then
the bounds comparison is made, and the element is accessed. Since the
default is 0, this will have no effect, save for the teeny amount of
processing time need to add the offset, on existing MATLAB legacy code.
The base index for an array dimension, dim, would be 1-offset[dim] and of
course the offset for that dimension would be 1-base[dim] .
Okay, how someone like myself would use this to do something different is
that there would be at least two new MATLAB facilities similar to size() and
reshape() that I might call "base()" and "rebase()", respectively. Just
like MATLAB size() function returns the contents of the size[] vector,
base() would return in MATLAB format, the contents of the offset[] vector
except it would be 1-offset. And just like reshape() changes (under proper
conditions) the contents of the size[] vector, rebase() would change the
contents of the offset[] vector except that the offset[dim] values would be
1-base[dim]. Since rebase() does not exist in legacy MATLAB code (oh, i
suppose someone could have created a function named that, but that's a
naming problem that need not be considered here), then there is no way for
existing MATLAB programs to change the offsets from their default values of
0 making this fix perfectly backward compatible.
Now just as there are dimension compatibility rules that exist now for
MATLAB operations, there would be a few natural rules that would be added so
that "rebased" MATLAB arrays could have operations applied to them in a
sensible way.
ARRAY ADDITION and SUBTRACTION and element-by-element ARRAY MULTIPLICATION,
DIVISION, POWER, and ELEMENTARY FUNCTIONS:
Currently MATLAB insists that the number of dimensions are equal and the
size of each dimension are equal (that is the same "shape") before adding or
subtracting matrices or arrays. The one exception to that is adding a
scaler to an array in which a hypothetical array of equal size and shape
with all elements equal to the scaler is added to the array. The resulting
array has the same size and shape as the input arrays.
The proposed system would, of course, continue this constraint and add a new
constraint in that index offsets for each dimension (the offset[] vector)
would have to be equal for two arrays to be added. The resulting array
would have the same shape and offset[] vector as the input arrays.
"MATRIX" MULTIPLICATION:
A = B*C;
Currently MATLAB appropriately insists that the number of columns of B are
equal to the number of rows of C (we shall call that number K). The
resulting array has the number of rows of B and the number of columns of C.
The value of a particular element of A would be:
K
A(m,n) = SUM{ B(m,k) * C(k,n) }
k=1
The proposed system would, of course, continue this constraint and add a new
constraint in that index offsets must be equal for each dimension where the
lengths must be equal. That is the number of columns of B are equal to the
number of rows of C and the base index of the columns of B are equal to the
base index of the rows of C. The resulting array has the number of rows of
B and the number of columns of C and the base index of the rows of B and
the base index of the columns of C. The value of a particular element of A
would be:
base+K
A(m,n) = SUM{ B(m,k) * C(k,n) }
k=base
where base = 1-offset[0] for the B array and 1-offset[1] for the C array
which must be the same number.
Both of these definitions are degenerations of the more general case where:
+inf
A(m,n) = SUM{ B(m,k) * C(k,n) }
k=-inf
where here you consider B and C to be zero-extended to infinity in all four
directions (up, down, left, and right). It's just that the zero element
pairs do not have to be multiplied and summed.
Probably matrix powers and exponentials (on square matrices) can be defined
to be consistent with this extension of the matrix multiply, but I can deal
with it at the moment.
CONCATINATION:
This would also be a simple and straight-forward extension of how MATLAB
presently concatinates arrays. When we say:
A = [B C];
The number of rows of B and C must be equal, but the number of columns of B
and C can be anything. The first columns of A are identical with the
columns of B and then also must the indices of those columns. And
independent of what the column indices of C are, they just pick up where the
column index of B left off. This rule extension defaults to what MATLAB
presently does if B and C are both 1-based arrays. A similar rule extension
can be made for A = [B ; C]; In all cases the upper left corner of A is
identical to the upper left corner of B, both in value but also in
subscripts (so A(1,1) becomes B(1,1) just like it does now in MATLAB).
MATRIX DIVISION ('/' and '\'):
I have to think about that a little, but I'm pretty sure a backward
compatible extension to the operations can be figgered out. If not, it
would be an illegal operation unless the bases were 1.
FUNCTIONS THAT RETURN INDICES (min(), max(), find(), sort(), ind2sub(), and
any others that I don't know about):
It must be internally consistent (and certainly can be made to be). The
indices returned would be exactly like the 1-based indices returned
presently in MATLAB except that the offset for the corresponding dimension
(that defaults to zero) would be subtracted from each index. That is, just
like now in MATLAB:
[max_value, max_index] = max(A);
This must mean that A(max_index) is equal to max_value.
I think that this is easy enough to define. The only hard part is to
identify all MATLAB functions that search through an array and modify them
to start and end at indices that might be different than 1 and size[dim] as
are the search bounds today. It would instead search from 1-offset[dim]
(the base) to size[dim]-offset[dim] which would default to the current
operation if the offset equals zero.
FOR ALL OTHER MATLAB OPERATIONS, until a reasonable extended definition for
non 1-based arrays is thunk up, MATLAB could either bomb out with an illegal
operation error if the base is not 1 or could, perhaps, ignore the base.
Either way it's still backwards compatible.
In a later post, I will suggest a few new DSP functions (they would actually
be new versions of existing DSP functions but with somewhat different names
so that there is backward compatibility) that would be made *far* more
elegant if these changes/extensions are implemented in MATLAB. I would
begin with convolution and the DFT.
Cleve, I hope that you and some of your "young Turks" at MathWorks will
seriously and thoughtfully look this over and respond. I might also suggest
that "straw men" counter examples to backwards compatibility requirement be
avoided in your response and that you folks stick closely to the strict
definition of "backwards compatibility". That strict definition means that
any code previously written in MATLAB will run with this extension exactly
as it had before the extension (except a minute amount slower due to the
offset addition).
welcome back, E.Bob! as in the W (p)residency, we can derive a little humor
from your presence. or perhaps you'ld like to provide some substantive and
useful technical content to the forum. that would be welcome also.
r b-j
In article <3C1A6F48...@jpl.nasa.gov> , Edwin Robert Tisdale
<E.Robert...@jpl.nasa.gov> wrote:
> Cleve Moler wrote:
>
>> Like many people at MathWorks,
>> I have been reading all the posts in this thread
>> but have hesitated to get involved
>> because the level of discussion
>> has been so unpleasant and nontechnical.
>> But Robert has made a compelling argument
>> that deserves a civil reply.
>>
>> I agree that indexing in DSP is zero-based and I agree that
>> it would be desirable to reflect that in MATLAB notation.
>> But MATLAB is based on linear algebra
>> where indexing is conventionally one-based.
>> As a result, instead of defining the FFT by
>>
>> N
>> X[k] = SUM{ x[n] * exp(-j*2*pi*(n-1)*(k-1)/N)
>> n=1
>>
>> I would use
>>
>> N-1
>> X(k+1) = SUM x(n+1) * exp(-j*2*pi*n*k/N) , k = 0:N-1
>> n=0
>
> Actually, the index base has nothing to do
> with either linear algebra or digital signal processing.
except convenience and consistency to the literature.
> Fortran programmers use 1 based indexing for both
> and C and C++ programmers use 0 based indexing for both.
and C and C++ programmers (like me) have made arrays or "pseudo-arrays" in
which we define any base we want so that our code looks clean and we don't
have to offset any negative indices each and every time they may come up.
that, my Mr. Tisdale, is how to model acausal FIR filters.
ARGGHHHHHH!
YOU CAN'T IGNORE US!!!!
ARGGHHHHHH! (jumping up and down furiously waiting for little Billy-Goat
Gruff.)
WHO'S THAT WALKING OVER MY BRIDGE?!! ARGGHHHHHH!
Cleve,
I think the reason why everyone is so upset is that MATLAB is heavily
advertised as being a DSP tool, but it doesn't let users use normal DSP
notation. This is really frustrating. For example, if you look in the
manual at the prony command, it refererences the Parks & Burrus book,
but the manual and the book use different notation.
Nearly all DSP related papers use zero-based arrays. It does not make
sense to publish a paper with a new algorith in it, and have the paper
and the example code (which is generally MATLAB) use different
conventions, but it happens all the time.
We're not asking to break backward compatibility, just an extension to
have the ability to specify the starting base of our arrays.
--
Matthew Donadio (m.p.d...@ieee.org)
>Hi --
>
>Like many people at MathWorks, I have been reading all the posts in
>this thread, but have hesitated to get involved because the level of
>discussion has been so unpleasant and nontechnical.
Personally, I find this a little insulting, but perhaps we're talking
about different things.
>But here Robert
>has made a compelling argument that deserves a civil reply.
>
>I agree that indexing in DSP is zero-based and I agree that it would
>be desirable to reflect that in MATLAB notation. But MATLAB is based
>on linear algebra where indexing is conventionally one-based.
When this came up a few years ago I tried to make the point that most
users (of most technical systems) couldn't care less about how the
tool is implemented or what the legacy is, they care about how it will
help them get their work done. The fact that MATLAB is based on
linear algebra may be a reason for the origin of one-based indexing,
but it is a poor excuse or rationale for continual justification in
the context of attempting to be a generalized tool. As a DSP and
communications person, I -really-, _really_, *really* don't care why
MATLAB was implemented with one-based indexing. What I do care about
is the additional time it takes me to translate, code, and debug
MATLAB routines because of this forced disconnect between standard
mathematical notation, it's connection to physical meaning, and how I
have to adjust all of that to code it (and debug it) in MATLAB.
...
>The real problem is backwards compatibility. The MATLAB community,
>including MathWorks and customers around the world, has millions of
>lines of code using one-based subscripts. What happens to all that
>code if we extend MATLAB notation to also allow zero- or variable-based
>subscripts?
Nothing if you come up with a suitable fix that is backward
compatible. There were several suggestions a few years ago that
seemed to go a long ways toward addressing the problem and preserving
backward compatibility (since that isn't lost on the people who are
concerned about this, many of us have a lot of legacy code, too), and
that's just from knowledgable concerned users. I'd guess that your
engineers and coders are creative enough and talented enough to come
up with similar or better means to achieve those ends.
>What does "find" return? How do I write a matrix library
>if I don't know where the subscripts start? How do I refer to the leading
>element of a string? In fact, is it the "first" element, or the
>"zeroth" element?
As I mentioned, this was addressed with several possible solutions by
several different people a few years ago when this came up. Since
Google has all of that stuff easily available now, the details of the
suggestions could be easily looked up.
>We don't refuse to consider this issue. There are few issues that
>we have considered more. And we don't refuse to debate. But if
>you want our participation, please make it a debate, not a diatribe.
And try, please, to not insult us. The folks who rant likely do so
because they care. They have a problem with using your product that
they'd like to see addressed because it causes them difficulty. As
the vendor of that product if you disenfranchise them for being
frustrated with your product, the likely result may be that they go
use something else. We can vote with our wallets, and in my
experience I think you'd be surprised how many have voted against you
because of this single issue.
I know I'd feel a lot better if I heard something from you guys other
than "How should we fix this?" (like the above, which could be
interpreted to indicate you're not up to the task yourself), reasons
why it hasn't been, can't be, or shouldn't be fixed, or how it *may*
or will definitely be fixed in an upcoming release, which then doesn't
happen. I'd much prefer a vendor that listens to what roadblocks I'm
encountering in getting productivity out of their tool and then tries
to figure out how to remove those roadblocks, than one that tries to
explain away the roadblock or tell me they won't even participate in a
discussion of the roadblock if they don't like how I describe it.
One way is a path to more productivity for me and my company, which
we're willing to pay for, and one way is a path to continued
frustration for the users which leads us to look for alternatives (or,
in many cases, to continue using the alternatives and not purchase
MATLAB).
Clearly the call is yours in how to handle the situation, but don't
think people don't make purchasing decisions based on this. I know of
personal cases because I've hit this wall on this very issue with many
departments in several different companies, and in my experience it
has cost you a lot of business. If my experience is even remotely
exemplary then it would seem to me that the resources needed to fix
this issue would be easily paid for in increased sales. I'd use it
more, and I'd be able to recommend it more (or even start recommending
it in cases where I've recommended against it for this reason).
But perhaps my tone is not up to your liking and you didn't even read
this far.
robert bristow-johnson wrote:
>
>
> It be below.
Thanks, Robert. I don't often print usenet postings but did this one
and will study it.
Some of us actually like 1 based indexing. A lot of us
even.
So TMW, if you do decide to fix their problems, please do
it so it does not blow the rest of us out of the water.
John D'Errico
--
> The real problem is backwards compatibility. The MATLAB community,
> including MathWorks and customers around the world, has millions of
> lines of code using one-based subscripts. What happens to all that
> code if we extend MATLAB notation to also allow zero- or variable-based
> subscripts? What does "find" return? How do I write a matrix library
> if I don't know where the subscripts start? How do I refer to the leading
> element of a string? In fact, is it the "first" element, or the
> "zeroth" element?
I know it has been brought up before but how about creating a new discrete
signal object? I haven't seen a definitive argument yet for why this
wouldn't work. It wouldn't have to break any older code (see below); it
would only require overloading many of the builtin objects.
The disignal object could (at a minimum) have the following fields:
%---
dsignal.data: array to hold the actual data.
dsignal.origin: the index of the data array that holds the origin. this
should be allowed to point outside the available data as well. Default
would be 1 which implies that dsignal.data(1) represents the spatial origin.
dsignal.label: a name for labeling purposes
dsignal.dimension_labels: cell array of labels for each dimension. E.g.,
{'time','amplitude'}
dsignal.version: version number to allow for expansion in the future
(perhaps this is a hidden field)
%-----
Aside: The definition of dsignal.origin above is the simplest and was
presented just for the argument's sake. An alternative way to achieve the
same results would be to designate the shift required to bring matrix
element (1,1,....) to the spatial origin. That is, view the data stored in
dsignal.data as x[n-n0] and specify the n0. Thus, an n0 of zero would imply
that dsignal.data(1) is at the spatial origin, i.e., there is no shift. The
value of n0 would be a vector for multi-d data.
This one problem I have with the signal approach is how to easily allow a
dsignal object to pass its data to a function that hasn't been overloaded.
If such a method existed, then one could combine new code that handles the
dsignal object directly with old code that nobody really wants to take the
time to rewrite to handle the new object. Here are the sparks of a few
ideas:
1) One simple solution to this problem is to create a subscript reference
that returns the values in the dsignal.data field. For example, if one had
some old code that implemented a spectrum estimater, then one could call it
with something like (assume x is a dsignal object):
a) y = my_estimate(x.data);
b) y = my_esimate(getdata(x));
c) y = my_estimate(x(:))
d) y = my_estimate(x(:,:));
e) y = my_estimate(x~);
f) y = my_estimate(~x);
% These last two are not actually possible
% for the current implementation of subscript references,
% but I think it would solve many problems if something
% like this were possible.
2) Create another field of the object called dsignal.reference_mode. It can
take two values ('object' or 'array') defined as follows:
a) 'object': any reference to the dsignal object will return the object.
Thus, a call to my_estimate(d) passes the object d to the function
my_estimate and x = d assigns the object d to x. This is no different from
the way objects currently work.
b) 'array': any reference to the dsignal object will return a reference to
dsignal.data. Thus, a call to my_estimate(d) passes the value of
dsignal.data to the function my_estimate and the assignment x = d assigns x
to dsignal.data. The only exception to this rule is when referencing the
reference_mode field itself. Thus, d.reference_mode will operate on the
object d not the data d.data. This allows one to change the mode back to
'object' without causing errors.
With this new field, one can call older functions with:
d.reference_mode = 'array';
y = my_estimate(d);
d.reference_mode = 'object';
Now, to address the questions that you brought up:
> What does "find" return? How do I write a matrix library
> if I don't know where the subscripts start? How do I refer to the leading
> element of a string? In fact, is it the "first" element, or the
> "zeroth" element?
With a signal object, the questions concerning the matrix-library and
strings are irrevelant --- they don't apply. As for the "find" function:
it should be overloaded to return a value that matches whatever the
subscript assign does for the object. If the user really wants find to
return values based directly on the data, they can use one of the above
methods to make sure the right data gets passed to the find function.
I have many other ideas for what a signal object could do, but I'll stop
here to keep the discussion more focused.
Thanks for listening,
Jordan
Bob Cain wrote:
>
> robert bristow-johnson wrote:
> >
> > In a later post, I will present (again, since I did this a year ago) a
> > perfectly backward compatible solution that, for some odd reason, MathWorks
> > apparently refuses to consider or to debate.
>
> Ah, the old zero/one origin indexing argument. I look forward to your
> proposal. The last time this discussion went around that I was in
> attendance the compatibility question nailed the coffin shut on the
> argument and there wasn't a whole lot of argument even presented for the
> wisdom of the original decision.
>
> As an APL user for more than 30 years I am very annoyed by the 1-origin
> only architecture of Matlab but APL was designed from the ground up to
> deal with dual origins so that function authors knew they had to be
> aware that their functions could be called from either context and that
> those considerations extended to the data itself.
Bob,
In APL, a new library function just calls quad IO to get the workspace
wide index origin and saves it. Then sets it to what it needs to run its
algo, and then resets it to the old value when done. I 1st used APL
back in 1972! - a really cool language and I learned to do this then.
Grant would like APL since it allows one to write very opaque code ;-)
But seriously, I think R-B-J has a viable soln to the indexing problem.
Clay
I liked the way APL handled it but then I liked APL all the way
around. One of the first things that attracted me to MATLAB was its
similarity to APL. I wish I had the time to invest in laerning J, the
'new' APL.
Having the ability to zero-index would be great. I get this question
all the time when I use MATLAB in a course and in my work I run into
this problem from time to time myself. I'd like to see the possibility
of zero-indexing.
How bad would it be to make it a new type for the nonce (shades of
APL's "NONCE ERROR")? There are so many different data types in MATLAB
nowadays --Viswanath hits on this in his recent review of Higham and
Higham--that a new type, the zero-indexed array, could be introduced
as a separate type with concersion facilities. Structures, function
handles, and zero-based arrays won't work in MATLAB 4.0 or earlier.
Merge the two types of arrays in the distant future when noone cares
about backward compatibility with MATLAB 2.0 any more. In any event,
many interesting ideas have been proposed--I hope the MathWorks can do
something about this.
(After we get back flops, at least in the Student Version.)
>the context of attempting to be a generalized tool. As a DSP and
>communications person, I -really-, _really_, *really* don't care why
>MATLAB was implemented with one-based indexing.
You should care. This is a long, posting, if you read it from the
start I will shortly explain how to use zero based indexing in Matlab
at the end.
The reason why DSP folks use 0-based indexing is that the C or assemby
language is used to program DSPs. Here 0-based indexing is natural
becauseit imples the following:
*(array + offset) == array[offset]
when array is a pointer to the first element, &array[0]. This is great
for programming hardware (which implies direct manipulation of
memory adresses) but no so great for numerical math. In linear
algebra, 1-based indexing is natural because it simplifies the
algorithms. Matlab uses 1-based indexing for two reasons. It is common
among mathematicans, and Fortran uses the same. Matlab was originally
written in Fortran to give a simple interface to the EISPACK and BLAS
routines.
What is the solution? In C++, the [] operator can be overloaded for
any class except the native datatypes such as double and int. If a C++
programmer needs an 1-based array, the solution is to wrap the array
in a class and overload the [] operator. An other solution, inspired
by the book "Numerical Receips in C", is to use
*(array + offset) = *( (array-base)+(offset+base))
= (array-base)[offset+base]
which actually invokes undefined behaviour and should not be used.
Now for the solution to the indexing problem in Matlab:
Base 1 indexing is already implented in Matlab. Yes it is, and here is
how you do it:
Like C++, Matlab implements classes and objects of classes.
But in stead of overloading the () operator for a class, you can
overload the behaviour of the indexing variable (actually the indexing
object). If a member function "subsindex" is defined for a class, this
function is called when an object of the class is used to subscript an
array. The return values from "subsindex" is used to subscript the
array as if it was zero-based, i.e
X(A) calles X( subsindex(A)+1 )
when A is an object of an user defined class. And that is all there
is to it! In other words, all you need to do is to use an user
defined class for your subscripting instead of using the elementary
types "double", "int32" or whatever. Although subsindex(A) default to
zero based indexing, it will of course allow you to select any base.
To make things simple, you can let the constructor of the indexing
object set the base.
So what are you actually complaining about?
The User Manual perhaps?
Sturla Molden
>Its funny, but one of the reasons I remember APL could
Being one of the folks on the DSP side that sorely needs zero- or
variable based indexing, I wholeheartedly agree with you. Clearly the
tool serves a lot of people well with 1-based indexing, and even us
DSP folks have a ton of legacy code developed under the 1-based
indexing rules. It would be a very bad move to abandon what is
apparently the bulk of the customer base in order to accomodate the
needs of a portion of the market. I think any reasonable developer
would recognize that the best solution would broaden the customer
base, not shrink it.
i am no MATLAB expert at all and have never tried to code with or use
MATLAB OOP. anyway, could this new object or class be defined so that
normal old vanilla MATLAB operators like + and - and * and .*, etc.
could be used?
this doesn't look like it:
> d.reference_mode = 'array';
> y = my_estimate(d);
> d.reference_mode = 'object';
if i can't do regular MATLAB programming (with just a couple of extra
calls to change the base or origin of an array from 1) with this
class, then MATLAB still fails to deliver on the promises: "familiar
mathematical notation" and "just as
they are written mathematically".
Lars Gregersen did make a stab at creating such a class and sent me
code, but (and i'm no expert) it appeared to both of us that the
syntax in using such a class would look more like LISP than MATLAB. i
want the syntax to be mostly unchanged, just extended so that i can
create/define arrays (or "matrices" if that's the word you want) that
can have non-positive indices.
> Thanks for listening,
yer welcome. and same for me.
i will apologize a bit for my previous tone, but, at the time i was
convinced that i (and "we" - i'm not alone) were not being listened to
by anyone that really mattered about this and i felt that i may as
well vent my spleen.
> Jordan
r b-j
i wouldn't know (since i never used APL), but for me when i created C
arrays that were not 0-based (using an offset pointer as a sorta
pseudonym to the array), it was clear to any other functions using
such an array that they couldn't do anything with that array name
except access its elements, and the range of indices was explicitly
defined.
> Some of us actually like 1 based indexing. A lot of us even.
and in many cases, that is the most appropriate notation since math
(linear algebra) texts and lit define matrices to have (1,1) in the
upper left corner. when dealing with those kind of matrices/arrays,
*I* would want to have 1-based indexing. it still does not speak to
the need to extend MATLAB so that other bases can be covered to be
consistent with other mathematical/engineering fields that may have
arrays that are not 1-based.
> So TMW, if you do decide to fix their problems, please do
> it so it does not blow the rest of us out of the water.
i think, John, that you can be assured that Cleve & Company won't let
that happen (the fear i have is they will use that as a false pretext
to do nothing). also, i want to repeat, for the record, that *I*
don't want that to happen. as Eric Jacobsen said, we DSPers *also*
have legacy MATLAB code (where we likely spent hours fudging indices
by 1 to get the damn thing to work) that we don't want to break
either.
PLEASE LISTEN: (as Matthew Donadio said) *none* of us want to "break
backward compatibility", we just want "an extension to have the
ability to specify the starting base of our arrays." i'm gratified
that other frustrated comp.dspers are piping up about this. i still
find it amazing that when Tom Krauss brought this to TMW's attention
when the Signal Processing Toolbox was being developed, that they
lacked the insight or DSP experience to understand how important this
is in the context of MATLAB's promise to code formulae with "familiar
mathematical notation" and "just as they are written mathematically".
that fundamental MATLAB promise is not delivered in the context of
DSP, and what makes that so much worse, is that i believe the largest
application market of MATLAB *is* DSP. it's sadly ironic.
there is no reason in the world that this extension cannot be done,
and if an idiot like me can see exactly how it can be done, i cannot
see how the priests at TMW can't figger it out. especially when i
spell it out. i invite you or anyone else on c.s-s.m (or comp.dsp,
for that matter) to examine that and point out a flaw in the argument,
but i feel pretty safe for the same reason i feel pretty safe that
adding or subtracting 0 to an index will not change it.
> John D'Errico
>
> --
>On Sat, 15 Dec 2001 07:26:17 GMT, eric.j...@ieee.org (Eric
>Jacobsen) wrote:
>
>>the context of attempting to be a generalized tool. As a DSP and
>>communications person, I -really-, _really_, *really* don't care why
>>MATLAB was implemented with one-based indexing.
>
>You should care.
Why? Do you care whether your computer uses a Wallace-Tree multiplier
algorithm? Do you care whether your cell phone uses a synchronous
scrambler or not? Do you care whether your car uses a table based or
rule based ignition firing strategy? Do you care whether the
electricity provided at your wall socket was produced from a direct
drive or geared generator?
Most folks don't. Most folks just care that their computers produce
accurate results, their cell phone calls go through with decent
quality, and their car starts when they turn the key.
I care that MATLAB is able to make me productive in getting useful
simulation results. When I write a line of MATLAB code, I really
*really* don't care how they get the result, just that the result is
computed quickly and the results are correct for the documented
purpose of the operation.
>This is a long, posting, if you read it from the
>start I will shortly explain how to use zero based indexing in Matlab
>at the end.
>
>The reason why DSP folks use 0-based indexing is that the C or assemby
>language is used to program DSPs.
You have cause and effect backwards. The reasons we want zero- or
variable- based indexing is because algorithms written with standard
mathematical notation for DSP and communications seldom start indices
at 1, and this is largely because the indices will often have physical
meaning and significance. Some easy examples are the indices in the
Discrete Fourier Transform, which indicate the number of cycles per
aperture correlated to each output bin (so index zero is DC), and the
lag in a correlator, where index zero indicates zero delay. The
disconnect between the physical significance of the indices and how
MATLAB code has to be written causes an unbelievable amount of
frustration in both coding, debugging, and interpretation.
> Here 0-based indexing is natural
>becauseit imples the following:
>
> *(array + offset) == array[offset]
>
>when array is a pointer to the first element, &array[0]. This is great
>for programming hardware (which implies direct manipulation of
>memory adresses) but no so great for numerical math.
In some applications, but in DSP it is absolutely horrible for
discrete mathematics. See the above simple two examples. There are
thousands of others, and if you can understand that, then perhaps you
will understand the source of our complaint.
Since most folks use technical papers or notes written in standard
mathematical notation to capture analysis and often results of
algorithm developments, and many tools (e.g., MathCAD, SPW, C,
SystemView, etc., etc.) allow zero or variable based indexing, and, as
you point out, many software implementations (as well as hardware) are
developed consistent with the mathematical notation, simulations in
other tools, and the underlying physical meaning of the indices, in
that context MATLAB is the deviating exception. So when those of us
who work in the above context, which is essentially the DSP and
communications industries, et al, try to simulate systems in MATLAB we
are always confronted with this forced disconnect between the
analysis, work done in other tools consistent with the analysis, and
implementations done consistent with the analysis, other tools, and
the physical meanings of the underlying indices. Do you begin to see
the problem?
> In linear
>algebra, 1-based indexing is natural because it simplifies the
>algorithms.
Absolutely. Believe me, we get this. Many of us just don't see that
as an excuse to not be able to provide a more generalized tool. Other
folks do, and some of us who like a lot of the other features of
MATLAB are just trying to not be forced to go use those other tools,
although that seems to be what TMW may prefer.
> Matlab uses 1-based indexing for two reasons. It is common
>among mathematicans,
You may want to correct that to "a subset of mathematicians". You
shouldn't have to look very far at all to realize that folks dealing
with remote sensing, biological metrics, optics, communications, etc.,
etc., etc., deal with physical quantities that often wind up being
indices into arrays. Does it not make sense that a tool that forces a
disconnect between the analysis, the physical significance, and a
straightforward implementation is less useful than one that doesn't?
> and Fortran uses the same.
...which is a great comfort to both people still using FORTRAN.
> Matlab was originally
>written in Fortran to give a simple interface to the EISPACK and BLAS
>routines.
Again, most people care more about what the tool can do for them than
how it is implemented or why.
>What is the solution?
...discussion of a reasonable start for a solution that has been
discussed before...
I think many people, perhaps many within TMW, have the same lack of
understanding of the point of view of many within the DSP community
toward using their tool.
Here are a few sentences from Robert's posting that relate to
our concern for backwards compatability.
>Now just as there are dimension compatibility rules that exist now for
>MATLAB operations, there would be a few natural rules that would be added so
>that "rebased" MATLAB arrays could have operations applied to them in a
>sensible way.
...
>The proposed system would, of course, continue this constraint and add a new
>constraint in that index offsets for each dimension (the offset[] vector)
>would have to be equal for two arrays to be added. The resulting array
>would have the same shape and offset[] vector as the input arrays.
...
>MATRIX DIVISION ('/' and '\'):
>
>I have to think about that a little, but I'm pretty sure a backward
>compatible extension to the operations can be figgered out. If not, it
>would be an illegal operation unless the bases were 1.
...
>I think that this is easy enough to define. The only hard part is to
>identify all MATLAB functions that search through an array and modify them
>to start and end at indices that might be different than 1 and size[dim] as
>are the search bounds today. It would instead search from 1-offset[dim]
>(the base) to size[dim]-offset[dim] which would default to the current
>operation if the offset equals zero.
...
>FOR ALL OTHER MATLAB OPERATIONS, until a reasonable extended definition for
>non 1-based arrays is thunk up, MATLAB could either bomb out with an illegal
>operation error if the base is not 1 or could, perhaps, ignore the base.
>Either way it's still backwards compatible.
...
> ... I might also suggest
>that "straw men" counter examples to backwards compatibility requirement be
>avoided in your response and that you folks stick closely to the strict
>definition of "backwards compatibility". That strict definition means that
>any code previously written in MATLAB will run with this extension exactly
>as it had before the extension (except a minute amount slower due to the
>offset addition).
I agree that it would be possible to add zero-based indexing to MATLAB,
using either new notation or an object that overloaded all of our
existing indexing notation. It could be backwards compatible in the
sense that old code that didn't know about or use the new stuff would still
work.
But here is the key point: none of our existing code would work with
the new object. You couldn't plot it; you couldn't print it; you
certainly couldn't do any matrix operations with it. Yes, over time,
everything could be rewritten to handle the extension, but that
is what we want to avoid. It would be like AM and FM radio -- two
systems living in the same box, but with separate and independent
underlying technology.
-- Cleve Moler
mo...@mathworks.com
>I agree that it would be possible to add zero-based indexing to MATLAB,
>using either new notation or an object that overloaded all of our
>existing indexing notation. It could be backwards compatible in the
>sense that old code that didn't know about or use the new stuff would still
>work.
>
>But here is the key point: none of our existing code would work with
>the new object. You couldn't plot it; you couldn't print it; you
>certainly couldn't do any matrix operations with it. Yes, over time,
>everything could be rewritten to handle the extension, but that
>is what we want to avoid. It would be like AM and FM radio -- two
>systems living in the same box, but with separate and independent
>underlying technology.
>
> -- Cleve Moler
> mo...@mathworks.com
I think your last paragraph is a pretty good synopsis of what I have
suspected to be the case for some time. The bottom line is that you
don't want to do the change because it would entail a lot of effort.
This is understandable, but I wish representatives of your company
hadn't been telling me for years that a fix was imminent, because I
could have saved an awful lot of time dealing with this issue by just
moving on to other toolsets. Instead, I've put up with this highly
frustrating problem in the hope, based on what I was told, that I'd
soon be able to not only use the parts of MATLAB that I already like,
but that this problem would be fixed.
Now that you've indicated that a fix is probably never going to
happen, it will be much easier for me to plan migrations to other
tools, and avoid starting new projects in MATLAB. Thanks for this,
because ultimately it will save me and the people I work with a ton of
time.
I don't know your code. (If you told me, you'd have to kill me!) So I go
out on a limb when disputing your point, but nobody can fire me.
It doesn't matter if your existing code would work with the new objects
you have in mind. Backward compatibility requires only that the _new_
print and plot and (...) work with _both_ new and old objects. That
seems self evident to me; what's the hitch?
Jerry
--
"I view the progress of science as ... the slow erosion of the
tendency to dichotomize." Barbara Smuts, U. Mich.
---------------------------------------------------------------------
I whole-heartedly support this. Because of this problem, I've been
using Octave http://www.octave.org for quite a while. It is more or
less a matlab clone and it's getting better all the time. As to other
paid tools, I am looking for suggestions on what can replace Matlab. I
already have and use Mathcad and like it a lot too.
Muzaffer Kal
http://www.dspia.com
DSP algorithm implementations for FPGA systems
> Yes, over time,
> everything could be rewritten to handle the extension, but that
> is what we want to avoid. It would be like AM and FM radio -- two
> systems living in the same box, but with separate and independent
> underlying technology.
Isn't that what we have now as function handles replace inline functions?
Jordan
robert bristow-johnson wrote:
>
> It be below.
>
[definition snipped]
My first blush is that I don't like it because, following good OOP
philosophy, it attaches the symantics to each array individually rather
than to the global nature of index related operations. This will force
me to pay way more attention to the issue than I would like. Not
knowing how operands (indices) will apply to an operator by just knowing
the local execution environment to me is truly obscure even though it
has become the modern way of thinking. I don't want to have to seek out
an array's definition to figure that out or have conditional code to
cover the possibilities.
I'm unable with the time at hand to assess whether the conditions of
backward compatibility are truly met by the proposal. I don't like the
appearance that to use old code I must use backwards compatible
_variables_.
To me the simplest solution is to provide a system variable which
defines the index origin, make it local to functions and initialize it
to 1 at every function call. All Matlab operators that take or return
indices would depend on that system variable. Also define a system
variable that is initialized at the function call to the value of the
callers origin which could be used to condition results that are
returned. Existing defined functions that take or return indicies would
behave as before and those of us that work in 0-origin would just need
to remember that. I don't think any existing code would break and I
could set the index origin to be whatever I want at the start of a
function I write. There is probably something obviously wrong with such
a simple minded approach which I don't see and I invite criticism.
"Clay S. Turner" wrote:
>
> In APL, a new library function just calls quad IO to get the workspace
> wide index origin and saves it. Then sets it to what it needs to run its
> algo, and then resets it to the old value when done.
It's been a long time but didn't they just make quad IO (the system
variable that determines the current index origin) implicitly local to a
function so that calling a function that changed it had no effect on the
calling function?
> I 1st used APL
> back in 1972! - a really cool language and I learned to do this then.
> Grant would like APL since it allows one to write very opaque code ;-)
I got it in 1967. Nothing I ever learned had such a profound effect on
my productivity. That was the thing I hated most to lose when I left
IBM in '77. As far as opaque code that was always very equivocal. My
experience was that the simplicity of the syntax and the power of the
operators made it far easier for me to scope out what an APL program did
than an equivalent program written in any other language which was
inevitably an order of magnitude more verbose. You were looking at the
cake rather than the recipe.
> This is a followup to article <_CxS7.2384$BK....@nwrddc02.gnilink.net>,
> from robert bristow-johnson <rob...@wavemechanics.com>
>
> Here are a few sentences from Robert's posting that relate to
> our concern for backwards compatability.
i think that we agree on the strict definition of "backwards compatibility".
however, whether we do or not might shake out below:
>>Now just as there are dimension compatibility rules that exist now for
>>MATLAB operations, there would be a few natural rules that would be added so
>>that "rebased" MATLAB arrays could have operations applied to them in a
>>sensible way.
>
does not affect backwards compatibility.
> ...
>
>>The proposed system would, of course, continue this constraint and add a new
>>constraint in that index offsets for each dimension (the offset[] vector)
>>would have to be equal for two arrays to be added. The resulting array
>>would have the same shape and offset[] vector as the input arrays.
>
does not affect backwards compatibility.
> ...
>
>>MATRIX DIVISION ('/' and '\'):
>>
>>I have to think about that a little, but I'm pretty sure a backward
>>compatible extension to the operations can be figgered out. If not, it
>>would be an illegal operation unless the bases were 1.
even though i think we both agree that this needs to be worked on in
advance, and a sensible extension to the current definition that degenerates
to the current definition when the base = 1 in all dimensions, i concede
that, but it still does not affect backwards compatibility. it can be
figgered out a little bit later (like next week).
> ...
>
>>I think that this is easy enough to define. The only hard part is to
>>identify all MATLAB functions that search through an array and modify them
>>to start and end at indices that might be different than 1 and size[dim] as
>>are the search bounds today. It would instead search from 1-offset[dim]
>>(the base) to size[dim]-offset[dim] which would default to the current
>>operation if the offset equals zero.
>
i only mean core MATLAB functions that return indices (that oughta be a
finite number and i have to believe that you guys *must* know which of your
"built-in" functions return indices). code that users run in .m files would
still work without any modification. even if they are using arrays based on
something other than 1, find() would still return the correct indices.
> ...
>
>>FOR ALL OTHER MATLAB OPERATIONS, until a reasonable extended definition for
>>non 1-based arrays is thunk up, MATLAB could either bomb out with an illegal
>>operation error if the base is not 1 or could, perhaps, ignore the base.
>>Either way it's still backwards compatible.
>
do you you dispute that this is backwards compatible? it just means that
there may be MATLAB operations that may not make sense for arrays unless
they are 1-based (ya know, the regular old MATLAB matrices for which these
operations *do* make sense). why is that such a conern? even if a sensible
extension to the operation to make it work with non-1-based arrays is thunk
up in the future, that extension could be implemented in the future and
nobody is hurt.
> ...
>
>> ... I might also suggest
>>that "straw men" counter examples to backwards compatibility requirement be
>>avoided in your response and that you folks stick closely to the strict
>>definition of "backwards compatibility". That strict definition means that
>>any code previously written in MATLAB will run with this extension exactly
>>as it had before the extension (except a minute amount slower due to the
>>offset addition).
>
> I agree that it would be possible to add zero-based indexing to MATLAB,
it should be more general than just adding 0-based indexing. even though i
used causal FIR convolution and DFT of examples of the egregious error in
notation we must accept with MATLAB presently, if this extension is done, it
should be more general. we should be able to model acausal filters in
MATLAB.
> using either new notation
it wouldn't be worth as much if it required any significant new notation
save for functions (like the DFT) that would take advantage of the extended
indexing base.
> or an object that overloaded all of our existing indexing notation.
it overloads a few specific places where subscripts or subscript ranges are
used before accessing MATLAB variable data. only places where your code
makes a reference to the matrix shape is where the offset would be have to
be added. only core functions that return indices would have to be modifed
to subtract the offset before the index is returned.
> It could be backwards compatible in the sense that old code that didn't know
> about or use the new stuff would still work.
yes. isn't that the definition? it would be backward compatible in that
sense, ergo it wouldn't be blowing John D'Errico and "the rest of us out of
the water."
> But here is the key point: none of our existing code would work with
> the new object.
that seems to be the conclusion Lars Gregersen and i came to. that's why
this problem appears that it cannot be decently fixed using the existing
MATLAB OOP facility (overloading subsasgn() and subsref() ). this is why i
am convinced that the solution to this problem lies ONLY with this *small*,
backward compatible, change to the core structure of the MATLAB variable.
we are not proposing a new notation (save for the new "base()", and
"rebase()" core functions that wouldn't be seen in legacy .m file code) nor
an object written within the current MATLAB OOP facility. (those are straw
men, i believe.) we are proposing a small and backwards compatible
extension to the internal parameters of the MATLAB variable so that each
index can be *potentially* adjusted before data is accessed so that the
MATLAB user can *choose* to change the default base index from its default
of 1 to some other finite integer.
> You couldn't plot it;
sure you can. why not? the worst that happens is the plot(X) facility
write 1 to size() at the bottom because it ignores the base. it, and a few
other MATLAB core functions should be modified to plot the correct indices
when there is only one argument. still sounds quite doable and finite to
me.
> you couldn't print it;
again, why not?
> you certainly couldn't do any matrix operations with it.
again, i have never proposed creating a new MATLAB object in the context of
the existing MATLAB language. i am proposing a *small*, backward
compatible, change to the core structure of the MATLAB variable and some of
the core functions that return (or use) indices.
> Yes, over time, everything could be rewritten to handle the extension, but
> that is what we want to avoid.
EVERYTHING? surely you do not mean the millions of lines of MATLAB code out
there. there would be a *finite* number of changes to the MATLAB
application code and those places can be readily found because the *only*
changes would be to the application code where references are made to the
array shape of the matlab variable or *core* functions that return indices.
THAT IS FINITE.
> It would be like AM and FM radio -- two systems living in the same box, but
> with separate and independent underlying technology.
it's an apt comparison Cleve (but not entirely accurate). even if it was
accurate, does that imply that you'ld rather have MATLAB to stay in the "AM"
era? why??? "FM" is better than "AM"!! we don't wanna be stuck with "AM"
when for a small increase in price (the finite development effort that gets
amortized over the entire remaining life of MATLAB), you can get the higher
quality of "FM" and get that inperpetuity? think of all the man-hours
you'ld be saving!
now the reason that it is not entirely an accurate comparison is that the
vast, vast majority of the MATLAB application code would be shared between
these "two" technologies. (it's really just one technology that got a
backward compatible extension.) the code that would be changed has only a
finite number of locations. the changes would happen only at those
locations where the matrix shape (the numbers returned by the size()
functions) are referenced and MATLAB core functions that return indices. if
that is done, it's backward compatible but not yet particularly useful. to
make it useful, the MATLAB variables that are created as a consequence of
some of the core functions described in my previous post would have to take
on the base (or offset[] vector) of the MATLAB variables that were arguments
of the function. i well defined that for '+', '-', '.*', './', '.^',
elemfun(), and even '*' (matrix multiply). i forgot to mean transpose ('),
but that one would be an easy extension also. '/' and '\' will come soon
enough.
Cleve, this is doable. *very* doable. and will be quite useful and
appreciated by potentially a large number of people in the future (which is
why fixing this sooner increases it's value). it really is doable and your
apparent skepticism and resistance is curious (i think there were a few
straw men in your argument). what do you expect the future life of MATLAB
to be? at least 10 years? if so, it's still not too late to fix this.
please, let's keep the conversation going on this.
r b-j
> I agree that it would be possible to add zero-based indexing to MATLAB,
> using either new notation or an object
> that overloaded all of our existing indexing notation.
> It could be backwards compatible in the sense that
> old code that didn't know about or use the new stuff would still work.
>
> But here is the key point:
> none of our existing code would work with the new object.
> You couldn't plot it; you couldn't print it;
> you certainly couldn't do any matrix operations with it.
> Yes, over time, everything could be rewritten to handle the extension
> but that is what we want to avoid. It would be like AM and FM radio --
> two systems living in the same box
> but with separate and independent underlying technology.
Nonsense!
Balderdash!
Poppycock!
The index base is irrelevant to a MATLAB matrix object.
If you derive a new type from matrix that supports zero-based indexing,
you can pass it to your existing software as a matrix
and it will all work as expected.
You don't need to change anything.
I have written both C and Fortran 77 routines that I call from MATLAB.
I use 0-based indexing in the C routines and
I use 1-based indexing in the Fortran 77 routines.
But I don't think that adding 0-based indexing is necessary.
In fact, MATLAB might be better off with one less indexing operator.
The whole purpose of supporting vector and matrix arithmetic
in a language like MATLAB is to eliminate loops over an indexed array.
Loops are inherently serial and introduce too many opportunities
for the programmer to make mistakes
which are very difficult to detect and debug.
If you read a lot of MATLAB code which implements DSP algorithms
and find a lot of loops over indexed arrays,
you can be pretty sure that the programmer
doesn't know what he or she is doing.
There are a few exceptions such as recursive definition of array elements
x(i+1) = b(i)*x(i) + a(i)
for example.
>
>
> robert bristow-johnson wrote:
>>
>> It be below.
>>
>
> [definition snipped]
>
> My first blush is that I don't like it because, following good OOP
> philosophy, it attaches the symantics to each array individually rather
> than to the global nature of index related operations.
no, i think i'm proposing an extension to "the global nature of index
related operations". that extension allows arrays to begin with and end
with any index bounds you like (providing you have enough memory). so
rather than just have the end bounds attached to each array (and have the
start bounds hardwired to 1) as it is presently the case, both the start
bounds and end bounds (defined indirectly by the offset[] and size[] fields
of that hypothetical MATLAB variable structure) would have to be attached to
each array. that's the only way you can have some arrays that start with 1,
other arrays that start with 0, and still other arrays that start with some
other possibly non-positive integer.
> This will force me to pay way more attention to the issue than I would like.
no. you don't have to pay attention to it at all (because it will be
backwards compatible) *unless* you might want to take advantage of this new
base redefinition feature. if you don't like it, fine. don't use it. how
does that put you out?
> Not
> knowing how operands (indices) will apply to an operator by just knowing
> the local execution environment to me is truly obscure even though it
> has become the modern way of thinking.
sorry, Bob. i can't decode this. i have absolutely no idea what you mean.
can you elaborate?
> I don't want to have to seek out
> an array's definition to figure that out or have conditional code to
> cover the possibilities.
if you don't choose to use this, you do not have to seek out the array's
definition to figure that out or have conditional code to cover the
possibilities". is your worry that someone else will pass to one of your
routines a non-1-based array and your routine won't know it and won't check?
if that is your worry, just stick with 1-based arrays and only warrant your
routine for proper operation if people supply you with the default 1-based
arrays for function arguments.
> I'm unable with the time at hand to assess whether the conditions of
> backward compatibility are truly met by the proposal. I don't like the
> appearance that to use old code I must use backwards compatible
> _variables_.
but your old code would only generate 1-based variables. how are you
deprived of anything if your old code that was meant to work with 1-based
arrays still only works right with 1-based arrays? how does that make your
life harder?
> To me the simplest solution is to provide a system variable which
> defines the index origin, make it local to functions and initialize it
> to 1 at every function call.
making a global system index origin sounds like a bad idea to me. just
because i have some 0-based or -128 based arrays for my DSP operations, i
still want to be able to use al