> Seems like every Camera manufacturer has one or more of his own versions
> of a RAW image format. And many (most?) versions are proprietary.
> Nikon even encrypts some of its RAW files to discourage 3rd parties
> from developing "converters" to change the raw file to a more
> conventional and useful format like tiff, jpeg, etc.
nikon encrypted the white balance data (not critical) and it's not just
nikon who encrypts. and if you are working on a raw converter, you can
get an sdk. it's a non-issue.
<http://www.dpreview.com/news/0504/05042701davecoffininterview.asp>
Phase One, Sony, Foveon, and Canon all apply some form of encryption
to their RAW files. Dcraw decodes them all -- you can easily find
decryption code by searching for the ^ operator.
> Considering that Adobe and many other 3rd parties can successfully
> convert the RAW files, why all the secrecy and annoyance of having so
> many different incompatible RAW formats.
> The difference in final results can't be all that significant....Can it?
it can be very different. just like with various types of film, there
are endless debates over which raw converter works best.
> What's the chances of a single, industry-standard RAW-Type format?
dng is one such attempt.
> Suppose every manufacturer used a non-standard type of jpeg format.....
the problem is that every sensor is just a little bit different (or a
lot different in some cases) and the corresponding raw files will be
different. there is not likely to be a single format that supports
every possible sensor. dng tries to encapsulate the differences, but
it does not work well for nonstandard sensors such as foveon. however,
for bayer sensors, it's a good attempt.
Chances that if either Nikon or Canon adopts the current DNG format, a
published and open standard, the other big company would, and the rest
would bump their shins getting in line.
Will this happen? Bets down at Window 5. My guess is 3:1 against; it
just makes too much sense.
--
john mcwilliams
RAW, by its very definition, is not an "image format" but simply the raw
data straight off the camera sensor. Thus any change in the sensor -
including going to a higher resolution - results in a different RAW output.
Absolutely agree. Adobe already submitted the DNG format for consideration
as a vendor independent standard to the International Standard's
Organization (ISO) in May last year. Hopefully it's only a matter of time.
I don't know of anyone except certain camera manufacturers who would object
to being able to shoot DNG RAW files straight out of the camera.
I expect it has to do with the way the components are assembled - most
likely the path of least resistance is often taken.
RAW is the sensor data file. It is not an image format.
>Nikon even encrypts some of its RAW files to discourage 3rd parties
>from developing "converters" to change the raw file to a more
>conventional and useful format like tiff, jpeg, etc.
This is not true. You can get SDK's . Nikon, Fuji (and I assume the
others do similar) supply free RAW converters
>Considering that Adobe and many other 3rd parties can successfully
>convert the RAW files,
How does this square with you paragraph above that "to discourage 3rd
parties from developing "converters""
> why all the secrecy and annoyance of having so many different
>incompatible RAW formats.
The formats are different because they all use different sensors
>The difference in final results can't be all that significant....Can it?
Yes.
>What's the chances of a single, industry-standard RAW-Type format?
Zero. There are many different types of sensors and they all output
different data. Do you think a 2MP DX camera will have the same
information as a 25MP FX camera?
There are many RAW convertors out there. I use DXO which works with
several RAW formats. There are several others Most Camera makers have
their own free converter SW
>Suppose every manufacturer used a non-standard type of jpeg format.....
>Where would we be? ......Your Thoughts.
>Bob Williams
RAW is the sensor data, JPG is an Image format. Not the same thing. You
can use RAW processors to convert to an image file.
However if you are that concerned shoot in JPEG not RAW
--
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
\/\/\/\/\ Chris Hills Staffs England /\/\/\/\/
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
I think the encryption is an attempt to "discourage" casual 3rd parties
from developing converters....Otherwise why encrypt?
Adobe, dcraw and others with a high degree of expertise in this area
probably consider encryption an annoyance but certainly not a deterrent
to developing their own converters.
>
>> why all the secrecy and annoyance of having so many different
>> incompatible RAW formats.
>
> The formats are different because they all use different sensors
>
>> The difference in final results can't be all that significant....Can it?
>
> Yes.
Do you think that anyone could really tell the difference between three
8x10 prints of the same subject, taken under the same conditions.....One
each taken with say a 10 MP Canon, Nikon or Sony DSLR?
Actually, RAW files DO receive some in-camera "processing, albeit not a
whole lot.
I do not disagree with with your saying that RAW is not actually a
format, but for lack of a better term it is frequently referred to as a
format.
See for instance, dpreview's description of the "IMAGE FORMATS" used by
the Canon XSi Camera.
http://www.dpreview.com/reviews/canoneos450d/page2.asp
Bob
The key term is "image format".
>> but simply the raw data straight off the camera
>> sensor. Thus any change in the sensor -
>> including going to a higher resolution - results in a different RAW output.
>
>Actually, RAW files DO receive some in-camera "processing, albeit not a
>whole lot.
>I do not disagree with with your saying that RAW is not actually a
>format, but for lack of a better term it is frequently referred to as a
>format.
Of course raw data files are a "file format". But they
are not an *image format*.
>See for instance, dpreview's description of the "IMAGE FORMATS" used by
>the Canon XSi Camera.
> http://www.dpreview.com/reviews/canoneos450d/page2.asp
Which of course does *not* suggest that the "RAW" format
is an image format.
--
Floyd L. Davidson <http://www.apaflo.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl...@apaflo.com
> I think the encryption is an attempt to "discourage" casual 3rd parties
> from developing converters....Otherwise why encrypt?
what benefit is there for 'casual 3rd parties' to develop a raw
converter? having a bunch of poorly written raw converters is not
really a good thing.
if someone is serious about it, they'll get the sdk and do it right or
they'll figure out the encryption. and there's only one camera of
which i am aware that heavily encrypts the data and has not been
cracked, and that's the sigma dp1.
> >> The difference in final results can't be all that significant....Can it?
> >
> > Yes.
>
> Do you think that anyone could really tell the difference between three
> 8x10 prints of the same subject, taken under the same conditions.....One
> each taken with say a 10 MP Canon, Nikon or Sony DSLR?
that's comparing cameras, not just raw converters. the question is can
someone tell the difference when the *same* raw image is processed by
various raw converters, and yes, there is a difference in both output
quality and speed of conversion. whether it's significant or not is
the subject of countless threads on dpreview and elsewhere.
raw data files, as any other files, have a format.
But it is not an "image format".
> ray <r...@zianet.com> wrote:
>> On Mon, 09 Feb 2009 07:30:15 -0800, Matt Ion wrote:
>>
>>> Bob Williams wrote:
>>>> Seems like every Camera manufacturer has one or more of his own
>>>> versions of a RAW image format. And many (most?) versions are
>>>> proprietary.
>>>
>>> RAW, by its very definition, is not an "image format" but simply the raw
>>> data straight off the camera sensor. Thus any change in the sensor -
>>> including going to a higher resolution - results in a different RAW
>>> output.
>>
>> raw data files, as any other files, have a format.
>
> But it is not an "image format".
Sure it is. It stores an image. There is no intrinsic difference
between a RAW file and any other image format -- it is all 1s and 0s
storing an image. In fact, Nikon calls the NEF files an "image format"
and compares them to JPG and TIFF.
--
Waddling Eagle
World Famous Flight Instructor
Nikon Electronic Format is *not* a raw image format.
Nikon often refers to "NEF raw image data", but near as
I can tell they never call NEF a "raw image format".
That is true because, if for no other reason, NEF is
used for many very different types of raw data.
And there *is* an intrinsic difference between a data
file that contains raw sensor data and an image file
which contains the data for one specific image. The raw
data file *must* be interpolated to produce an image,
and there is no one single way to do that, which means a
raw data file's data can be interpolated many different
ways. It is not data for one image, but rather data
from which an infinite number of very different images
can be made.
Regardless, to be pedantic, NEF is a TIFF image file...
but the RAW data is not the TIFF image! (Read that with
care.)
Each NEF has what is usually thought of as an "embedded"
thumbnail, but that is in fact *the* TIFF image. It
also has special tags that make it possible to use TIFF
headers to carry along the raw sensor data from which
the thumbnail was made. Which is to say that actually
the raw data is embedded, not the thumbnail.
I suspect, like others do, that the path of most $ return for newer camera
model is taken. Change the format periodically and have others try to
reverse engineer it. In the meantime sell the update software that handles
it.
--
SneakyP
To reply: newsgroup only, what's posted in ng stays in ng.
Some choose to swim in the potty bowl of nan-ae rather than flush it
down :0)
How about a 2mp DX camera and a 25 MP FX camera with different
resolutions using different sensors and processing tecnology ? This is
why the formats differ.
The final picture will depend on what you do with the data in RAW
processing and photoshop. However much of it will depend on the amount
of data you have to play with at the start.
The final outcome of a 10MP Canon, Sony and Nikon may well be similar
but the sensors and process in the camera will be different hence the
different RAW formats.
As they all supply FREE RAW convertors to users and SDK's to the
industry I can't see the problem.
By your definition, there is no such thing as an image file. Even
jpegs will look very different on each monitor they're displayed on or
on each printer they're printed on. That's because processing,
including interpolation among other processing, *must* be done to a
jpeg in order to produce an image you can see on any specific medium
and there is no single way to do that, which means a jpeg file's data
can be interpolated many different ways.
In fact, a raw file is even more an image file format than a jpeg file
because it is an actual photograph, i.e., a recording of photons that
struck a specific area over a specific time. Once you start messing
with that via whatever processing you choose, you're getting further
and further away from a photograph although it's still an image.
Also by your definition, a film negative is not an image because it
must be printed to create an image and there's an infinite number of
ways any single negative can be printed. But I contend that a film
negative *is* an image format, just like a raw file.
Steve
It's not important, but how I understand it is that a RAW file is different
to a TIFF, JPEG, etc. There can be some confusion though because Nikon
offer *.NEF format for scanned images from their scanners too, but a *.NEF
from a film scanner is not the same as a *.NEF from a digital camera.
*.NEF files from a scanner are more like TIFF's. You can also convert
TIFF's and JPEG's to DNG RAW format (which, although controversial, I do
myself for all originals), but they will never truly be RAW images because
they have already gone through the demosiacing process.
That has nothing to do with it. A JPEG format image
defines just one image. It might well display
differently, and it can also be edited... but it is
still just one image. There is no single image defined
by the raw data from a DSLR sensor. It is a set of data
that defines many images, not just one. And there are
many ways to get an image from the data too.
>on each printer they're printed on. That's because processing,
>including interpolation among other processing, *must* be done to a
>jpeg in order to produce an image you can see on any specific medium
There is no interpolation necessary with a JPEG image.
And the processing to convert a JPEG image to some other
format is well defined to produce a *single specific*
image. Granted that it might no always actually produce
exactly the same image, but the difference is an *error*!
>and there is no single way to do that, which means a jpeg file's data
>can be interpolated many different ways.
False. JPEG data is not interpolated.
Each pixel in a JPEG is encoded into a single set of RGB
color values. That data is not used for multiple pixels
except within blocks of identical pixels.
Raw sensor data is not a one to one relationship with
image pixels. The value eventually calculated for any
single pixel location is *interpolated* from multiple
sensor locations (and multiple pixels, each of which
might be different, use data from any single sensor
location).
>In fact, a raw file is even more an image file format than a jpeg file
>because it is an actual photograph, i.e., a recording of photons that
>struck a specific area over a specific time. Once you start messing
>with that via whatever processing you choose, you're getting further
>and further away from a photograph although it's still an image.
That is silliness.
>Also by your definition, a film negative is not an image because it
>must be printed to create an image and there's an infinite number of
>ways any single negative can be printed. But I contend that a film
>negative *is* an image format, just like a raw file.
Correct, a film negative is an image.
<snip>
> Also by your definition, a film negative is not an image because it
> must be printed to create an image and there's an infinite number of
> ways any single negative can be printed.
Lemme guess--you're a digital guy who has never actually seen a
negative.
The negative most assuredly is an image. It's a negative image but
it's an image--you can look at it and easily identify the object being
photographed.
--
--
--John
to email, dial "usenet" and validate
(was jclarke at eye bee em dot net)
There are numerous RAW formats to ensure that, in the long run, your
images will become obsolete and increasing difficult to process. The
same is probably true for JPG but JPG will likely stand the test of
time because there are just so many freaking images out there. So in
a few years, you will be able to use a 25 year old JPG image but you
might find it difficult to use a 25 year old RAW image because it
won't be supported. If they introduce 5 new RAW formats a year for
the next 25 years, there will be 125 new formats that are used
relatively infrequently. There will only be 1 JPG format that will be
used all the time.
Of course the flip side of it is that in 25 years people will be
cursing the "antique" JPG format because it is so limiting -- it won't
be doing 3-D or holograms or whatever the new thing will be.
Funny how technology keeps marching on.
I must take exception to several statements here. A jpeg or any other
graphic file must be interpolated as well. You can't simply have it
'typed' to a terminal window. Those images can also be manipulated to
produce different images. Since there are only a finite number of
settings involved, and each has only a finite number of settings - only a
finite number of images can be produced from a RAW file - although it may
be a relatively large finite number.
>
> Regardless, to be pedantic, NEF is a TIFF image file... but the RAW data
> is not the TIFF image! (Read that with care.)
>
> Each NEF has what is usually thought of as an "embedded" thumbnail, but
> that is in fact *the* TIFF image. It also has special tags that make it
> possible to use TIFF headers to carry along the raw sensor data from
> which the thumbnail was made. Which is to say that actually the raw
> data is embedded, not the thumbnail.
So you're telling us that TIFF is not an image file format either. By
your same reasoning, nothing is a graphic file format since one can
always embed other data in the file.
> ray <r...@zianet.com> wrote in news:6vb2b2F...@mid.individual.net:
>
>> On Mon, 09 Feb 2009 00:18:43 -0800, Bob Williams wrote:
>>
>>> Seems like every Camera manufacturer has one or more of his own
>>> versions of a RAW image format. And many (most?) versions are
>>> proprietary. Nikon even encrypts some of its RAW files to discourage
>>> 3rd parties from developing "converters" to change the raw file to a
>>> more conventional and useful format like tiff, jpeg, etc. Considering
>>> that Adobe and many other 3rd parties can successfully convert the RAW
>>> files, why all the secrecy and annoyance of having so many different
>>> incompatible RAW formats. The difference in final results can't be all
>>> that significant....Can it? What's the chances of a single,
>>> industry-standard RAW-Type format? Suppose every manufacturer used a
>>> non-standard type of jpeg format..... Where would we be? ......Your
>>> Thoughts. Bob Williams
>>
>> I expect it has to do with the way the components are assembled - most
>> likely the path of least resistance is often taken.
>
> I suspect, like others do, that the path of most $ return for newer
> camera model is taken. Change the format periodically and have others
> try to reverse engineer it. In the meantime sell the update software
> that handles it.
If that's the reason - it does not work - at least for me and for many
others as well. For one fundamental reason it will never work for me and
many others: they don't supply Linux executables.
It defines one image just as much as a jpeg file defines one image. Of
course, neither one defines just one image. For instance, there is no
definition in a jpeg file for what values at each pixel equate to what
luminosity. So a single jpeg file defines an infinite number of
images
>>on each printer they're printed on. That's because processing,
>>including interpolation among other processing, *must* be done to a
>>jpeg in order to produce an image you can see on any specific medium
>
>There is no interpolation necessary with a JPEG image.
>And the processing to convert a JPEG image to some other
>format is well defined to produce a *single specific*
>image. Granted that it might no always actually produce
>exactly the same image, but the difference is an *error*!
No it's not an error when you get differences, because there is no
single defined way to do a conversion of a jpeg file to an image.
>>and there is no single way to do that, which means a jpeg file's data
>>can be interpolated many different ways.
>
>False. JPEG data is not interpolated.
It is if you don't print or display it at it's native resolution. But
it's not just interpolated. It's also converted to light at a certain
amplitude and wavelengh at each pixel. And there's no single way
defined to do that.
>Each pixel in a JPEG is encoded into a single set of RGB
>color values. That data is not used for multiple pixels
>except within blocks of identical pixels.
And how do those RGB color values get display as a image? Undefined.
If you turn up or down the brightness of your monitor, you're looking
at a different image from the same jpeg.
>Raw sensor data is not a one to one relationship with
>image pixels. The value eventually calculated for any
>single pixel location is *interpolated* from multiple
>sensor locations (and multiple pixels, each of which
>might be different, use data from any single sensor
>location).
Raw sensor data is a recording of an image. There's many different
ways to interpolate it to form a different image. But it's still an
image. It just wouldn't look like the final result you see printed or
display. Just like a film negative is an image that doesn't look like
the final result printed.
>>In fact, a raw file is even more an image file format than a jpeg file
>>because it is an actual photograph, i.e., a recording of photons that
>>struck a specific area over a specific time. Once you start messing
>>with that via whatever processing you choose, you're getting further
>>and further away from a photograph although it's still an image.
>
>That is silliness.
Silly, but true.
>>Also by your definition, a film negative is not an image because it
>>must be printed to create an image and there's an infinite number of
>>ways any single negative can be printed. But I contend that a film
>>negative *is* an image format, just like a raw file.
>
>Correct, a film negative is an image.
By admitting that a film negative is an image, you're being internally
inconsistent with your ascerting that a raw file is not an image file.
Steve
>TheRealSteve wrote:
>
><snip>
>
>> Also by your definition, a film negative is not an image because it
>> must be printed to create an image and there's an infinite number of
>> ways any single negative can be printed.
>
>Lemme guess--you're a digital guy who has never actually seen a
>negative.
>
>The negative most assuredly is an image. It's a negative image but
>it's an image--you can look at it and easily identify the object being
>photographed.
I have boxes full of negatives. And to me, a film negative is just as
much an image container as a raw file.
Steve
Are scanner NEFs, raw files? Hmmm... I'm not even sure, does it do three
passes with color filters so those are stacked and no bayer pattern is
involved?
> *.NEF files from a scanner are more like TIFF's. You can also convert
> TIFF's and JPEG's to DNG RAW format (which, although controversial, I do
> myself for all originals), but they will never truly be RAW images
> because they have already gone through the demosiacing process.
You do this to take advantage of the compression?
--
Paul Furman
www.edgehill.net
www.baynatives.com
all google groups messages filtered due to spam
Look up the definition of "interpolation". You want the
"math" definition. JPEG has the RGB values needed, and
all that is done is conversion to any given format, not
interpolation of values.
Raw sensor data however does not have RGB values for
each pixel. The RGB data for any give pixel is spread
out in the data from at least 9 different sensor
locations. Interpolation is the process of generating
one pixel from the data in many sensor locations.
>'typed' to a terminal window. Those images can also be manipulated to
>produce different images.
That amounts to "editing", or *changing* the data for an
image. Interpolation of raw sensor data does not change
the data. It just selects which data is used and which
is not, and selects a method for combining the used data
to generate the required results.
>Since there are only a finite number of
>settings involved, and each has only a finite number of settings - only a
>finite number of images can be produced from a RAW file - although it may
>be a relatively large finite number.
The above is a non sequitor, but I would point out that
in fact there are (because it eventually is analog at
some point) an infinite number of possible changes that
can be made.
>> Regardless, to be pedantic, NEF is a TIFF image file... but the RAW data
>> is not the TIFF image! (Read that with care.)
>>
>> Each NEF has what is usually thought of as an "embedded" thumbnail, but
>> that is in fact *the* TIFF image. It also has special tags that make it
>> possible to use TIFF headers to carry along the raw sensor data from
>> which the thumbnail was made. Which is to say that actually the raw
>> data is embedded, not the thumbnail.
>
>So you're telling us that TIFF is not an image file format either. By
Stop being absurd.
>your same reasoning, nothing is a graphic file format since one can
>always embed other data in the file.
TIFF defines an image format. It also defines a data
file format. NEF uses the TIFF file format to hold a
thumbnail image. It uses the facilities (tags and
headers) to embed other data in the file too. It
embeds, for example, the Exif data, it embeds the Maker
Notes... and it also embeds sensor data.
But the "image" in an NEF file is a thumbnail, not the
sensor data.
That is not true.
>>>on each printer they're printed on. That's because processing,
>>>including interpolation among other processing, *must* be done to a
>>>jpeg in order to produce an image you can see on any specific medium
>>
>>There is no interpolation necessary with a JPEG image.
>>And the processing to convert a JPEG image to some other
>>format is well defined to produce a *single specific*
>>image. Granted that it might no always actually produce
>>exactly the same image, but the difference is an *error*!
>
>No it's not an error when you get differences, because there is no
>single defined way to do a conversion of a jpeg file to an image.
There are multiple ways, but each is *supposed* to
regenerate the original image. The difference between
the original and what is regenerated is "error". For
example, that is one reason that JPEG is called a
"lossy" format.
>>>and there is no single way to do that, which means a jpeg file's data
>>>can be interpolated many different ways.
>>
>>False. JPEG data is not interpolated.
>
>It is if you don't print or display it at it's native resolution. But
>it's not just interpolated. It's also converted to light at a certain
>amplitude and wavelengh at each pixel. And there's no single way
>defined to do that.
If you resize it, the data is interpolated. But of
course that is also making an entirely *different* image
that the JPEG data described. (Differences in pixel
values are the error discussed above.)
>>Each pixel in a JPEG is encoded into a single set of RGB
>>color values. That data is not used for multiple pixels
>>except within blocks of identical pixels.
>
>And how do those RGB color values get display as a image? Undefined.
>If you turn up or down the brightness of your monitor, you're looking
>at a different image from the same jpeg.
See "error".
>>Raw sensor data is not a one to one relationship with
>>image pixels. The value eventually calculated for any
>>single pixel location is *interpolated* from multiple
>>sensor locations (and multiple pixels, each of which
>>might be different, use data from any single sensor
>>location).
>
>Raw sensor data is a recording of an image.
No, it is recording sensor data.
>There's many different
>ways to interpolate it to form a different image.
At which point yes you do have an image. And those are
save to image formats such as TIFF, PPM, JPEG, etc.
>But it's still an
>image. It just wouldn't look like the final result you see printed or
>display. Just like a film negative is an image that doesn't look like
>the final result printed.
But a film negative is just one image. The same as JPEG
is just one image.
But raw sensor data is not just one image. It is data
from which many different images could be generated.
>>>In fact, a raw file is even more an image file format than a jpeg file
>>>because it is an actual photograph, i.e., a recording of photons that
>>>struck a specific area over a specific time. Once you start messing
>>>with that via whatever processing you choose, you're getting further
>>>and further away from a photograph although it's still an image.
>>
>>That is silliness.
>
>Silly, but true.
If you believe that then there is no point in discussing
technical details of digital imaging with you.
>>>Also by your definition, a film negative is not an image because it
>>>must be printed to create an image and there's an infinite number of
>>>ways any single negative can be printed. But I contend that a film
>>>negative *is* an image format, just like a raw file.
>>
>>Correct, a film negative is an image.
>
>By admitting that a film negative is an image, you're being internally
>inconsistent with your ascerting that a raw file is not an image file.
Not even close.
You evidently have a very simplistic concept of what is involved with
decoding a jpeg file.
>
> Raw sensor data however does not have RGB values for each pixel. The
> RGB data for any give pixel is spread out in the data from at least 9
> different sensor locations. Interpolation is the process of generating
> one pixel from the data in many sensor locations.
Actually, mathematically, interpolation involves estimating the value at
missing points - not quite the same thing.
>
>>'typed' to a terminal window. Those images can also be manipulated to
>>produce different images.
>
> That amounts to "editing", or *changing* the data for an image.
> Interpolation of raw sensor data does not change the data. It just
> selects which data is used and which is not, and selects a method for
> combining the used data to generate the required results.
??????????? You have a rather sketchy concept of what occurs - not really
very accurate.
>
>>Since there are only a finite number of settings involved, and each has
>>only a finite number of settings - only a finite number of images can be
>>produced from a RAW file - although it may be a relatively large finite
>>number.
>
> The above is a non sequitor, but I would point out that in fact there
> are (because it eventually is analog at some point) an infinite number
> of possible changes that can be made.
>
>>> Regardless, to be pedantic, NEF is a TIFF image file... but the RAW
>>> data is not the TIFF image! (Read that with care.)
>>>
>>> Each NEF has what is usually thought of as an "embedded" thumbnail,
>>> but that is in fact *the* TIFF image. It also has special tags that
>>> make it possible to use TIFF headers to carry along the raw sensor
>>> data from which the thumbnail was made. Which is to say that actually
>>> the raw data is embedded, not the thumbnail.
>>
>>So you're telling us that TIFF is not an image file format either. By
>
> Stop being absurd.
>
>>your same reasoning, nothing is a graphic file format since one can
>>always embed other data in the file.
>
> TIFF defines an image format. It also defines a data file format. NEF
> uses the TIFF file format to hold a thumbnail image. It uses the
> facilities (tags and headers) to embed other data in the file too. It
> embeds, for example, the Exif data, it embeds the Maker Notes... and it
> also embeds sensor data.
No, actually, TIFF defines a container. As a matter of fact, the RAW data
for my Kodak P850 is in a TIFF format file. In this case, it IS the
sensor data that is included in the file. You would do well to look up
the definition of TIFF.
I'm not even talking about the "lossiness" of the jpeg format. So
take a TIFF or a GIF or a BMP, none of which are lossy formats and all
of which define an infinite number of images from the same image file.
That's because the file does not define well enough what the final
image will look like. It will look different on each monitor and each
printer, none of which may be an "error" because the final result is
just not specified in the file. It will look different even on the
same monitor with different settings and again, it's not an error and
it's not lossy. It's just not defined in the file how bright the
monitor should be set to properly view it.
>>>>and there is no single way to do that, which means a jpeg file's data
>>>>can be interpolated many different ways.
>>>
>>>False. JPEG data is not interpolated.
>>
>>It is if you don't print or display it at it's native resolution. But
>>it's not just interpolated. It's also converted to light at a certain
>>amplitude and wavelengh at each pixel. And there's no single way
>>defined to do that.
>
>If you resize it, the data is interpolated. But of
>course that is also making an entirely *different* image
>that the JPEG data described. (Differences in pixel
>values are the error discussed above.)
That's why I'm not just talking about resizing it. I'm talking about
the infinite ways of display the *same* jpeg, gif, or tiff file...
each looking different but none of wich are an *error* because the
file doesn't specify enough about how the image should actually look
to a viewer. How much ink of what type goes where? How bright should
a pixel be lit for a given RGB value? It's just not defined. So
there are infinite ways of displaying a given jpeg, gif, bmp, tiff
etc. image file. Just like raw.
>>>Each pixel in a JPEG is encoded into a single set of RGB
>>>color values. That data is not used for multiple pixels
>>>except within blocks of identical pixels.
>>
>>And how do those RGB color values get display as a image? Undefined.
>>If you turn up or down the brightness of your monitor, you're looking
>>at a different image from the same jpeg.
>
>See "error".
It's not an "error" if it's not definied. If you like viewing your
monitor set to 75% brightness and your coworker likes viewing it at
74% brightness, which one of you is in "error"? Which setting is the
*correct* setting for viewing a particular jpeg file? How does the
file specify that? Answer: it doesn't.
>>>Raw sensor data is not a one to one relationship with
>>>image pixels. The value eventually calculated for any
>>>single pixel location is *interpolated* from multiple
>>>sensor locations (and multiple pixels, each of which
>>>might be different, use data from any single sensor
>>>location).
>>
>>Raw sensor data is a recording of an image.
>
>No, it is recording sensor data.
Exactly, which is a way of specifying image data. The sensor data is
the data that the sensor output when it captured a certain image. If
you had a monitor with the same pattern of colored dots as the sensor
that captured the image, you wouldn't have to interpolate the sensor
data into RGB pixels. When you convert a raw sensor file to a jpeg,
all you're doing is converting one image file to another image file
because your monitor/printer can't handle the image format contained
in the raw image file directly. That doesn't mean it's not possible
to design and build one that can. It just wouldn't be very cost
effective.
>>There's many different
>>ways to interpolate it to form a different image.
>
>At which point yes you do have an image. And those are
>save to image formats such as TIFF, PPM, JPEG, etc.
You have an image before also. Just like when you interpolate a jpeg
image to form another image of a different size. Just because you can
interpolate one image to form another image doesn't mean the original
wasn't an image.
>>But it's still an
>>image. It just wouldn't look like the final result you see printed or
>>display. Just like a film negative is an image that doesn't look like
>>the final result printed.
>
>But a film negative is just one image. The same as JPEG
>is just one image.
>
>But raw sensor data is not just one image. It is data
>from which many different images could be generated.
A film negative is just like sensor data in that both are an image
from which many different images could be generated. I take it you've
never worked in a darkroom printing your own negatives and played
around with different enlargements, exposures, etc.
>>>>In fact, a raw file is even more an image file format than a jpeg file
>>>>because it is an actual photograph, i.e., a recording of photons that
>>>>struck a specific area over a specific time. Once you start messing
>>>>with that via whatever processing you choose, you're getting further
>>>>and further away from a photograph although it's still an image.
>>>
>>>That is silliness.
>>
>>Silly, but true.
>
>If you believe that then there is no point in discussing
>technical details of digital imaging with you.
If you don't know what a photograph is, i.e., a graphical
representation of the photons hitting a certain area over a certain
time, then there's no point in discussing technical details of digital
imaging with you. And for the purposes of this discussion, a file
that contains the details of a photograph (i.e., a raw sensor file)
also contains an image. Of course that photograph/image contained in
the raw sensor file can be manipulated to form an infinite number of
other images. Just like any other image file can.
>>>>Also by your definition, a film negative is not an image because it
>>>>must be printed to create an image and there's an infinite number of
>>>>ways any single negative can be printed. But I contend that a film
>>>>negative *is* an image format, just like a raw file.
>>>
>>>Correct, a film negative is an image.
>>
>>By admitting that a film negative is an image, you're being internally
>>inconsistent with your ascerting that a raw file is not an image file.
>
>Not even close.
Closer than you realize.
Steve
Actually, I'm just thinking about how many negatives I've gone
through. For a few years back in the 80's, I was the darkroom manager
for my university newspaper and yearbook. I've developed, printed and
halftoned so many pictures I think I can still smell the chemicals.
I did just about every picture that went into the yearbooks except for
the posed student portraits, and every picture for several years of a
weekly newspaper. Those were the days.
Steve
I think I see what your problem is. You don't realize that just
because the data isn't in the format of RGB pixel locations doesn't
mean that it's not an image. Yes, for many raw sensor formats you are
interpolating a bayer sensor pattern to form RGB pixels. But that in
no way means that the sensor data for the bayer pattern in the raw
image file does not define an image. It does.
Steve
The define a single image.
>That's because the file does not define well enough what the final
>image will look like.
If it is decoded the resulting image should be exactly the same
as the original. If it is re-encoded, the exact same encoded image
file will result.
>It will look different on each monitor and each
>printer, none of which may be an "error" because the final result is
>just not specified in the file.
No, they will look different because each printer or monitor is
different, not because the image is different.
...
>>>>>In fact, a raw file is even more an image file format than a jpeg file
>>>>>because it is an actual photograph, i.e., a recording of photons that
>>>>>struck a specific area over a specific time. Once you start messing
>>>>>with that via whatever processing you choose, you're getting further
>>>>>and further away from a photograph although it's still an image.
>>>>
>>>>That is silliness.
>>>
>>>Silly, but true.
>>
>>If you believe that then there is no point in discussing
>>technical details of digital imaging with you.
>
>If you don't know what a photograph is, i.e., a graphical
>representation of the photons hitting a certain area over a certain
>time, then there's no point in discussing technical details of digital
>imaging with you. And for the purposes of this discussion, a file
>that contains the details of a photograph (i.e., a raw sensor file)
>also contains an image. Of course that photograph/image contained in
>the raw sensor file can be manipulated to form an infinite number of
>other images. Just like any other image file can.
Sigh. Your own definitions make discussion fun, but not productive.
Plonk.
Or maybe I actually do understand it, and therefore
don't see it as complex beyond comprehension.
>> Raw sensor data however does not have RGB values for each pixel. The
>> RGB data for any give pixel is spread out in the data from at least 9
>> different sensor locations. Interpolation is the process of generating
>> one pixel from the data in many sensor locations.
>
>Actually, mathematically, interpolation involves estimating the value at
>missing points - not quite the same thing.
Raw data is not an image until the data is interpolated
to provide an image. Weasel all you like with words, it
is still a fact. It isn't exactly an unknown either,
and is the reason that Nikon (despite the quoted claim
above) does not call NEF's an image.
>>>'typed' to a terminal window. Those images can also be manipulated to
>>>produce different images.
>>
>> That amounts to "editing", or *changing* the data for an image.
>> Interpolation of raw sensor data does not change the data. It just
>> selects which data is used and which is not, and selects a method for
>> combining the used data to generate the required results.
>
>??????????? You have a rather sketchy concept of what occurs - not really
>very accurate.
It is precisely accurate. The problem is that no matter
how I describe it using words, you can find a way to
take the words to mean something other than what I'm
trying to say. If you really don't know, then I need to
find different ways to say it until all put together you
see what I'm saying. But it appears in this case that
you are purposely avoiding the intended meaning...
With a JPEG you start with an image, and encode it to
the JPEG image format. When it is unencoded the intent
is to produce exactly the same image. To the degree
that it can do that using whatever algorithm you choose,
you get the same image that was originally encoded. Of
course what you actually get is different, and the
difference is error. That said, one can generate
different images by intentionally introducing changes.
That is "editing". The point still is that if you find
a way to produce an error free image and do not edit it
purposely, it will be exactly the same image that was
originally encoded.
That characteristic is not true of raw data files.
There are *many* changes that can be made that do *not*
reflect any difference from an original or any single
reference image. None of them are "correct", they are
all acceptable. There is no specific image in the raw
data!
>>>Since there are only a finite number of settings involved, and each has
>>>only a finite number of settings - only a finite number of images can be
>>>produced from a RAW file - although it may be a relatively large finite
>>>number.
>>
>> The above is a non sequitor, but I would point out that in fact there
>> are (because it eventually is analog at some point) an infinite number
>> of possible changes that can be made.
>>
>>>> Regardless, to be pedantic, NEF is a TIFF image file... but the RAW
>>>> data is not the TIFF image! (Read that with care.)
>>>>
>>>> Each NEF has what is usually thought of as an "embedded" thumbnail,
>>>> but that is in fact *the* TIFF image. It also has special tags that
>>>> make it possible to use TIFF headers to carry along the raw sensor
>>>> data from which the thumbnail was made. Which is to say that actually
>>>> the raw data is embedded, not the thumbnail.
>>>
>>>So you're telling us that TIFF is not an image file format either. By
>>
>> Stop being absurd.
>>
>>>your same reasoning, nothing is a graphic file format since one can
>>>always embed other data in the file.
>>
>> TIFF defines an image format. It also defines a data file format. NEF
>> uses the TIFF file format to hold a thumbnail image. It uses the
>> facilities (tags and headers) to embed other data in the file too. It
>> embeds, for example, the Exif data, it embeds the Maker Notes... and it
>> also embeds sensor data.
>
>No, actually, TIFF defines a container.
TIFF defines both an image format and a file structure
to hold TIFF formatted images. NEF uses the TIFF file
structure, and has a thumbnail image formatted in the
TIFF image format, but the sensor data is embedded in
the file using other mechanisms of the file structure.
A lot of data other than sensor data (Exif data is one
example) is also embedded in the TIFF file structure.
None of that is an "image", other than the thumbnail.
>As a matter of fact, the RAW data
>for my Kodak P850 is in a TIFF format file.
TIFF does not define a 12 bit data format. Can you
explain how Kodak does that?
Actually Kodak uses a TIFF file structure, but I can't
find anything that says it uses the TIFF specification
for data. (They do have data for a JPEG image, just
like Nikon, and the same is true of Canon. Canon and
Nikon do not use TIFF image formating for the data.)
One thing that might be interesting... Can you post the
output of "tiffinfo" when run on a kdc file?
Here are a few lines from the output generated reading
an NEF file, before it seg faults:
TIFFReadDirectory: Warning, d3f_1629.nef: unknown field with tag 36867 (0x9003) encountered.
TIFFReadDirectory: Warning, d3f_1629.nef: unknown field with tag 37398 (0x9216) encountered.
TIFF Directory at offset 0x8 (8)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 160 Image Length: 120
Resolution: 300, 300 pixels/inch
Bits/Sample: 8
Compression Scheme: None
Photometric Interpretation: RGB color
Orientation: row 0 lhs, col 0 bottom
Samples/Pixel: 3
>In this case, it IS the
>sensor data that is included in the file. You would do well to look up
>the definition of TIFF.
Nobody is claiming the sensor data is not included in
the file. What are you talking about? What I'm talking
about is the data at location IFD 0.
That still would not necessarily make it an image! But
even if the sensor data is shoehorned into IFD 0 by some
non standardized method, it still doesn't define an
image!
Regardless, it isn't the TIFF specification that I would
need to look at, but a specification for the Kodak P850
raw file.
>> But the "image" in an NEF file is a thumbnail, not the sensor data.
--
>TheRealSteve <st...@example.com> wrote:
>>I'm not even talking about the "lossiness" of the jpeg format. So
>>take a TIFF or a GIF or a BMP, none of which are lossy formats and all
>>of which define an infinite number of images from the same image file.
>
>The define a single image.
I can see we're going round and round on this but the fact is that
they do no define a single image. If they did, you would see the same
image each time you looked at it no matter where and how the image is
viewed. The fact that you don't means they do not define a single
image.
>>That's because the file does not define well enough what the final
>>image will look like.
>
>If it is decoded the resulting image should be exactly the same
>as the original. If it is re-encoded, the exact same encoded image
>file will result.
Well hell, if I use any process and it's inverse, I can decode and
recode a raw file and produce the same resultant encoded image file
also. So if that's your standard then a raw file is an image file.
>>It will look different on each monitor and each
>>printer, none of which may be an "error" because the final result is
>>just not specified in the file.
>
>No, they will look different because each printer or monitor is
>different, not because the image is different.
I'm sorry, but if you can't understand that if an image looks
different then it's a different image, there's no hope for you.
>>>>>>In fact, a raw file is even more an image file format than a jpeg file
>>>>>>because it is an actual photograph, i.e., a recording of photons that
>>>>>>struck a specific area over a specific time. Once you start messing
>>>>>>with that via whatever processing you choose, you're getting further
>>>>>>and further away from a photograph although it's still an image.
>>>>>
>>>>>That is silliness.
>>>>
>>>>Silly, but true.
>>>
>>>If you believe that then there is no point in discussing
>>>technical details of digital imaging with you.
>>
>>If you don't know what a photograph is, i.e., a graphical
>>representation of the photons hitting a certain area over a certain
>>time, then there's no point in discussing technical details of digital
>>imaging with you. And for the purposes of this discussion, a file
>>that contains the details of a photograph (i.e., a raw sensor file)
>>also contains an image. Of course that photograph/image contained in
>>the raw sensor file can be manipulated to form an infinite number of
>>other images. Just like any other image file can.
>
>Sigh. Your own definitions make discussion fun, but not productive.
It's actually not my definition. I didn't make it up. It's one of
the definitions of what the word means, albeit a technical one.
Nevertheless, it doesn't matter what definitions one uses. The
discussion can not be productive if you're too hardheaded to even
conceed the fact that if two images look different then they are
different. It's a simple factual discussion point that you can't
admit to because I think you're smart enough to realize it's a fact
that is inconsistent with your assertion.
>Plonk.
Thanks for at least admitting you're incapable of a productive
discussion.
Steve
You are nuts. It is the SAME image it is just your perception of it that
has changed NOT the Image.
The RAW data is the sensor data not an image. You process the RAW data
and get an image. You can get several different images from the data.
A JPEG is a single image. You can not change the information in a JPEG
in the way you can in a RAW data file
>> *.NEF files from a scanner are more like TIFF's. You can also convert
>> TIFF's and JPEG's to DNG RAW format (which, although controversial, I do
>> myself for all originals), but they will never truly be RAW images
>> because they have already gone through the demosiacing process.
>
> You do this to take advantage of the compression?
Although I use a Nikon scanner, I always scan to TIFF. With TIFF scans,
reduced file size one advantage. Obviously, with JPEGS the file size is
increased, although not much of a problem as the file size is relatively
small anyway. Other advantages are consistency so that all originals are
DNG, and also it's too easy to accidently overwrite original TIFF's and
JPEG's during editing, but you can't accidently overwrite a DNG file.
> There are multiple ways, but each is *supposed* to
> regenerate the original image. The difference between
> the original and what is regenerated is "error". For
> example, that is one reason that JPEG is called a
> "lossy" format.
I'm a little confused here. Are you saying that the conversion of a
JPEG file
to a bitmap representation is not deterministic - ie that a given JPEG
file might
legitimately be converted to one of several different bitmaps ?
Thanks,
Richard
>In message <emh5p45pahvoa52qs...@4ax.com>, TheRealSteve
><st...@example.com> writes
>>>>It will look different on each monitor and each
>>>>printer, none of which may be an "error" because the final result is
>>>>just not specified in the file.
>>>
>>>No, they will look different because each printer or monitor is
>>>different, not because the image is different.
>>
>>I'm sorry, but if you can't understand that if an image looks
>>different then it's a different image, there's no hope for you.
>
>You are nuts. It is the SAME image it is just your perception of it that
>has changed NOT the Image.
Absolutely wrong. My perception of it has nothing to do with it. You
can put an instrumented measurement device that measures lumens and it
will read different values. It's a different image.
>The RAW data is the sensor data not an image. You process the RAW data
>and get an image. You can get several different images from the data.
Absolutely wrong. The RAW data *is* data that represents an image.
The image is just not made up of RGB pixels. It's made up of data
frim a Bayer pattern sensor, or whatever other pattern the sensor
physically is. The ascertion that only RGB pixel image files are
represent an image is just plain silly.
What do you think the sensor captured? It's just intuitively obvious
to anyone that the sensor captured an image. Just like film captures
an image. And when you develop film, you chemically convert it to a
format that's more suitable for long term storage, i.e., a developed
negative. Just like when the camera reads the data from the sensor
and stores it into a RAW file, which is a format that's more suitable
for long term storage than just charge on a sensor.
If you try and say that the RAW data is not an image, then you're
saying that the image the sensor captured is not an image. Again,
that's just plain silly.
Of course you have to manipulate the RAW sensor data file to get it
into RGB pixels and there are many ways to do that conversion. But
it's still image data before you do any kind of conversion to RGB
pixels. To say that the image data that camera captured from the
sensor and stored into the RAW file does not represent an image is
again just plain silly. That's the same thing as saying that film
does not represent an image because it's not RGB pixels. Silly.
>A JPEG is a single image. You can not change the information in a JPEG
>in the way you can in a RAW data file
Just because you can change the image data in a RAW file to make
another image file in a different format doesn't mean that the image
data in the RAW file does not define an image. It does. It defines
the image that the sensor captured.
It really amazes me that the simple concept of a lens focusing an
image onto a digital sensor, the sensor converting the photons
striking it into a charge at defined pixel locations in a defined
pattern such as a bayer pattern, and the camera reading the charge
from those pixels off the sensor and storing the data into a RAW file
without doing further processing on it (such as converting it to a
jpeg) seems so confusing to so many people.
Once again for the slow: just because there are many different ways of
interpolating the image captured by a bayer pattern's pixels into RGB
pixels in no way means that the image captured by the bayer pattern's
pixels and stored in the RAW file is not an image.
Steve
What lossy means is that if you convert a bitmap into a jpeg and then
back to a bitmap, you won't get the same bitmap you started with. The
difference between the bitmap you started with and the one after
conversion to jpeg and back is the error.
There are a multitude of ways of converting bitmaps into jpegs, each
with different amounts of error in the converted back bitmap. Usually
(but not always) the less error you want, the less compression you get
and/or the more complex and processor time consuming algorithm you
need to do the conversion.
Steve
> What lossy means is that if you convert a bitmap into a jpeg and then
> back to a bitmap, you won't get the same bitmap you started with. The
Sure, but the question was about the other direction. Given a JPEG,
can it legitimately be converted back into more than one bitmap
representation, or is the conversion completely defined by the info in
the JPEG file ?
Richard
>TheRealSteve wrote:
That's a big "it depends". It depends on what kind of bitmap file
you're talking about. If you're converting the YCbCr pixels from the
JPEG file directly to RGB pixels in a bitmap file that has the same
RGB pixel locations as the YCbCr pixels, then yes. If your bitmap
file does not have the same RGB pixel locations as the YCbCr pixels of
the JPEG file, then no.
Here's the formula for a pixel to pixel translation from JPEG to an
RGB pixel bitmap with the same pixel locatitons:
R = Y + 1.402 (Cr-128)
G = Y - 0.34414 (Cb-128) - 0.71414 (Cr-128)
B = Y + 1.772 (Cb-128)
A RAW file is just a bitmap file that does not have the same pixel
locations as a JPEG file.
One other thing to note is that while a JPEG file has pixel centers
that are compatible with some industry standards you might want to
convert to, such as postscript for printing, it's NOT compatible with
others.
For example, it's not compatible with digital video formats that
conform to CCIR 601-1 even though they are both YCbCr. So while a
JPEG file may fully describe a postscript image of the same
resolution, it does NOT fully describe a digital video image of the
same resolution. Depending on how you pre-process the chrominance
components, you can get different results when going from jpeg to DV.
So now is someone going to try and convince me that digital video
images are not images because a JPEG file has an infinite number of
ways it can correctly be converted to DV? Or that JPEG image files
are not really an image file because you have to process it to make a
DV image just like you have to process a RAW file to make a JPEG?
That's just as silly as saying a RAW image file is not an image file
because there are multitudes of ways of converting it to a JPEG. Silly
Silly.
Steve
>>> *.NEF files from a scanner are more like TIFF's. You can also convert
>>> TIFF's and JPEG's to DNG RAW format (which, although controversial, I do
>>> myself for all originals), but they will never truly be RAW images
>>> because they have already gone through the demosiacing process.
>>
>> You do this to take advantage of the compression?
>
> Although I use a Nikon scanner, I always scan to TIFF. With TIFF scans,
> reduced file size one advantage. Obviously, with JPEGS the file size is
> increased, although not much of a problem as the file size is relatively
> small anyway. Other advantages are consistency so that all originals are
> DNG, and also it's too easy to accidently overwrite original TIFF's and
> JPEG's during editing, but you can't accidently overwrite a DNG file.
Oh, also forgot to mention that DNG's are much faster as showing a preview
than large TIFF's too. With a folder full of TIFF scans it takes forever to
show a preview, whereas a folder full of DNG's of the same files is pretty
much instant.