Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
PhotoKB Home
Discussion Groups
Digital Photography
Digital PhotoDSLR CamerasZLR CamerasPoint & Shoot Cameras
Film Photography
35 mmLarge FormatMedium formatDarkroomFilm and LabsOther Equipment
Photo Technique
Nature PhotographyPeople PhotographyTechnique General
General Photo Topics
General TopicsAustralian PhotographyUK Photography
DirectoryPhoto Clubs

Photo Forum / Digital Photography / DSLR Cameras / December 2006

Tip: Looking for answers? Try searching our database.

Archiving images

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
David Azose - 13 Dec 2006 02:30 GMT
I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via
a card reader built into my computer. So now the RAW files are saved.
But when I work on the files, I use adjustment layers and save as a
tiff. Should I also be saving a flattened version? That would mean
saving 3 different versions of the same image:

    1, Original RAW
    2, Corrected, edited with layers Tiff
    3, Final flattened Tiff.
I also archive the RAW files first to a CD or DVD before any editing.

These are all saved in 16 bit mode. Should the final, flattened Tiff be
changed to 8 bit?

Is that the way it should be done? Seems like a lot of storage space and
 also lots of images to be named, sorted, rated, named and cataloged.
Any comments, suggestions would be appreciated.

David A.
jeremy - 13 Dec 2006 04:17 GMT
>I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via a
>card reader built into my computer. So now the RAW files are saved. But
[quoted text clipped - 15 lines]
>
> David A.

There has been much speculation that RAW formats are inappropriate for
long-term archiving because it is unlikely that those proprietary formats
will be supported over the long term.
Ken Lucke - 13 Dec 2006 04:34 GMT
> >I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via a
> >card reader built into my computer. So now the RAW files are saved. But
[quoted text clipped - 19 lines]
> long-term archiving because it is unlikely that those proprietary formats
> will be supported over the long term.

What?  You think that software writers are going to take the code OUT
of their programs when it hurts nothing to leave it in?  Or that the
current programs that handle those files just fine will suddenly stop
working on the current images at some point in the future if they do
change the format and the software publishers do remove the code?  Or
that by the time that someone decides to have one single RAW file
standard that everyone (except MicroSloth, of course) will adhere to,
that someone esle won't have a converter to change the old RAW
format(s) to the new one?

I think that's just nuts.

RAW, in other words the actual data that comes out of the camera, will
probably ALWAYS be the best way to store the data - it can always be
changed from there, but once changed, you never have the same base data
to work with again.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

Roger N. Clark (change username to rnclark) - 13 Dec 2006 06:05 GMT
>>There has been much speculation that RAW formats are inappropriate for
>>long-term archiving because it is unlikely that those proprietary formats
[quoted text clipped - 9 lines]
> that someone esle won't have a converter to change the old RAW
> format(s) to the new one?

I wrote my first book in Multimate on a *186 machine.  For a while
wordperfect and word could read those files.  Not any more.
Good thing I made *.prn ascii output!

> I think that's just nuts.
>
> RAW, in other words the actual data that comes out of the camera, will
> probably ALWAYS be the best way to store the data - it can always be
> changed from there, but once changed, you never have the same base data
> to work with again.

Not a guarantee.

Roger
jeremy - 13 Dec 2006 15:13 GMT
"Roger N. Clark (change username to rnclark)" <username@qwest.net> wrote in
message >>
>> RAW, in other words the actual data that comes out of the camera, will
>> probably ALWAYS be the best way to store the data - it can always be
>> changed from there, but once changed, you never have the same base data
>> to work with again.
>
> Not a guarantee.

There is a relatively new branch of IT called "Data Archeology."  Already
there are problems decoding old formats--and we are looking at only a
25-year time horizon since PCs came into use.

Who seriously believes that our current image formats will remain in use in,
say, 100 years?

My transparencies have a better chance of long-term survival than my scanned
images of those same slides.  I think that format, media failure and media
obsolescence may ultimately be much bigger threats than the loss of film
negs and slides.  Even though the introduction of digital imaging has mad it
possible for people to create many millions of images that might not have
been taken with film, we just cannot predict how many of those images will
survive another hundred years, even when people are willing to go to some
lengths to maintain them.  Most people do not give much thought to this.
J. Clarke - 13 Dec 2006 16:00 GMT
> "Roger N. Clark (change username to rnclark)" <username@qwest.net> wrote in
> message >>
[quoted text clipped - 11 lines]
> Who seriously believes that our current image formats will remain in use in,
> say, 100 years?

We should all be so lucky as to give a damn about them 100 years from
now.

> My transparencies have a better chance of long-term survival than my
> scanned images of those same slides.

Neither has any chance of survival to that time frame unless somebody
besides you gives a damn about preserving them.

> I think that format, media failure
> and media obsolescence may ultimately be much bigger threats than the
[quoted text clipped - 4 lines]
> people are willing to go to some lengths to maintain them.  Most people
> do not give much thought to this.

If anybody cares about them then they will find a way to preserve them, if
nobody cares about them then who cares?

Signature

--John

to email, dial "usenet" and validate

(was jclarke at eye bee em dot net)

John McWilliams - 13 Dec 2006 16:50 GMT
>> I think that format, media failure
>> and media obsolescence may ultimately be much bigger threats than the
[quoted text clipped - 7 lines]
> If anybody cares about them then they will find a way to preserve them, if
> nobody cares about them then who cares?

I very much like this summation. May we all rest in peace!

Signature

John McWilliams

josh@phred.org - 13 Dec 2006 18:43 GMT

> > My transparencies have a better chance of long-term survival than my
> > scanned images of those same slides.

> Neither has any chance of survival to that time frame unless somebody
> besides you gives a damn about preserving them.

I'd disagree, but only because I've found prints and transparencies
nobody cared about for generations -- things left behind when people
moved, for example.  

I suspect that the web, and web archiving, will create a similar form of
immortality for many very undistinguished images, not because anyone
cares to save them, but because memory is so cheap that it costs less to
store everything than to sort out what to throw away.

I recently realized something similar on my home network.  I had lots of
CDs of old images, and was getting ready to burn new CDs of them before
the old CDs became unreadable. (I don't trust them more than 4-5 years.)  

I did burn new CDs, but after loading all my old CDs, I noticed what a
small part of my home RAID they took up.  

From now on, while I'll still make CDs for off-site backup, I'm also
keeping all my images on my file server.  Whenever I upgrade drive sizes
or technologies, I always start by copying across everything off the old
system.  

If the formats I store in start to become uncommon, I have no doubt
there will be a batch-conversion-tool available for one of the new
formats.  Maybe it will take a while to actually run on all those
images, but it won't take much work, and it will be easier to to it to
all of them than to pick which ones to convert.  Not like when I had to
pick which 8-inch floppies to stick in the drive one at a time by
hand....

Signature

josh@phred.org is Joshua Putnam
<http://www.phred.org/~josh/>
Braze your own bicycle frames.  See
<http://www.phred.org/~josh/build/build.html>

Paul Furman - 14 Dec 2006 02:19 GMT
> Jclarke.usenet@cox.net says...
>
[quoted text clipped - 5 lines]
> cares to save them, but because memory is so cheap that it costs less to
> store everything than to sort out what to throw away.

I think web images are more accessible according to how popular they are
in search engines so the ones that the web public cares about remain
high on the lists of search engines and caches.
JC Dill - 14 Dec 2006 20:08 GMT
>I'd disagree, but only because I've found prints and transparencies
>nobody cared about for generations -- things left behind when people
>moved, for example.  

In 30 more years, old transparencies and negatives will be very hard
to "read" as the film industry dies.

jc

Signature

"The nice thing about a mare is you get to ride a lot
of different horses without having to own that many."  
    ~ Eileen Morgan of The Mare's Nest, PA

Scott W - 14 Dec 2006 21:44 GMT
> >I'd disagree, but only because I've found prints and transparencies
> >nobody cared about for generations -- things left behind when people
> >moved, for example.
>
> In 30 more years, old transparencies and negatives will be very hard
> to "read" as the film industry dies.
I don't think it is going to take 30 years.  How many people now have
the equipment needed to view slides, not just hold them up to the light
but really view them?

I have a bunch of my fathers old negatives, the 35mm stuff I can scan
well enough but the 120 film is much harder, and if I wanted a print I
would have to mail the negatives off to get it done.  

Scott
THO - 16 Dec 2006 06:18 GMT
> In 30 more years, old transparencies and negatives will be very hard
> to "read" as the film industry dies.

Do you expect everyone to be blind in 30 years? People are reading 150
year old film today.
Scott W - 16 Dec 2006 11:56 GMT
> > In 30 more years, old transparencies and negatives will be very hard
> > to "read" as the film industry dies.
>
> Do you expect everyone to be blind in 30 years? People are reading 150
> year old film today.
You need equipment to read the film.  Sure you can look at a slide but
without some kind
of viewer why bother.  Negatives you need to some how either make a
print or scan, in
30 years both of these are likely to be not very common at all.

Scott
THO - 17 Dec 2006 17:39 GMT
> > > In 30 more years, old transparencies and negatives will be very hard
> > > to "read" as the film industry dies.
[quoted text clipped - 8 lines]
>
> Scott

Think of how many years worth of photographic images exist  -- 180 years
worth (since 1826). Whether it's a museum, grandkids, government, or
industry, someone will still be digitizing old film in 30 years and the
equipment will still be available. The original comment was more about
spreading FUD than anything else.
Ken Lucke - 13 Dec 2006 15:41 GMT
> >>There has been much speculation that RAW formats are inappropriate for
> >>long-term archiving because it is unlikely that those proprietary formats
[quoted text clipped - 13 lines]
> wordperfect and word could read those files.  Not any more.
> Good thing I made *.prn ascii output!

IOW, you converted upwards either through foresight or when technology
required you to, using what you perceived to be the best available
method at the time.  Thank you for reiterating my point. :^)

> > I think that's just nuts.
> >
[quoted text clipped - 4 lines]
>
> Not a guarantee.

Oh, yes.  I'm not referring to the actual file format (stucture) used -
yes, it may change.  Eventually, at future time.  

But as long as you have the uncorrupted raw data available (through
whatever file or media storage method is available), you can still
recreate (or improve) your results.  If you convert to something like a
TIFF, you don't have the same baseline, direct-from-from-the-camera
data with which to start over again.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

POHB - 13 Dec 2006 13:22 GMT
> What?  You think that software writers are going to take the code OUT
> of their programs when it hurts nothing to leave it in?  Or that the
[quoted text clipped - 7 lines]
>
> I think that's just nuts.

What's more likely is that in 30 years time you won't be using the same
programs, and the new ones won't ever have had support for the current
RAW formats.  Your old programs will no longer work on your new
operating system, if indeed you can even read the install disks.
Your DVD archives will have degraded, or you won't be able to find a
machine with a DVD reader any more.
Sure, someone will write a converter.  But you'd better make sure you
use it during the time window that it is useable.
The only way to maintain an archive of digital material is to keep it
live and moving forward as file and media formats change over time.
Aad - 13 Dec 2006 14:04 GMT
>> What?  You think that software writers are going to take the code OUT
>> of their programs when it hurts nothing to leave it in?  Or that the
[quoted text clipped - 18 lines]
> The only way to maintain an archive of digital material is to keep it
> live and moving forward as file and media formats change over time.

In 30 years from now you will convert 20.000 pictures in about 5 secs.
(maybe faster)
br
Aad
J. Clarke - 13 Dec 2006 15:30 GMT
>> What?  You think that software writers are going to take the code OUT
>> of their programs when it hurts nothing to leave it in?  Or that the
[quoted text clipped - 12 lines]
> RAW formats.  Your old programs will no longer work on your new
> operating system, if indeed you can even read the install disks.

Reading the installation media might be an issue.  Working on the new
operating system however is unlikely to be a problem--soft emulation of
machines is a well known technology--Bochs will run just about any Intel
OS on any machine for which there is gcc port.  Pearpc will do the same
for OS/X.  Further, 30 years from now that emulated machine will be as fast
as or faster than the fastest desktop machines available now.

> Your DVD archives will have degraded, or you won't be able to find a
> machine with a DVD reader any more.

Degrading is debatable.  There is one school of thought that says that
DVDs last about five minutes and another that says that they last about a
hundred years and the fact is that nobody really knows.

As for a DVD reader being available, why would one not be available?  This
notion that DVDs are just going to disappear one night is silly.  There
are far, far too many of them out there for the ability to read them to be
discarded.

Now, you're probably going to trot out diskettes.  Well they aren't
comparable--diskettes were always a niche product, DVDs are mass-market.
You never saw prerecorded diskettes on a stand next to the checkout at the
supermarket, but you do see DVDs and CDs there.

> Sure, someone will write a converter. But you'd better make sure you use
> it during the time window that it is useable. The only way to maintain
> an archive of digital material is to keep it live and moving forward as
> file and media formats change over time.

Signature

--John
to email, dial "usenet" and validate
(was jclarke at eye bee em dot net)

Ken Lucke - 13 Dec 2006 15:34 GMT
> > What?  You think that software writers are going to take the code OUT
> > of their programs when it hurts nothing to leave it in?  Or that the
[quoted text clipped - 10 lines]
> What's more likely is that in 30 years time you won't be using the same
> programs,

Without doubt that's true.

> and the new ones won't ever have had support for the current
> RAW formats.

Why not?  I can still open Apple ][ (1980's technology) files as-is,
without even an emulator, on my current Mac OS X computer.  What makes
it likely that backwards compbatibility will be so quick to be dropped?
Especially withh the huge usage of those file formats currently?  Oh,
yeah, all the OS programmers will just suddenly drop support for all
those people using those pesky cammeras that save in RAW format without
warning.

> Your old programs will no longer work on your new
> operating system, if indeed you can even read the install disks.

I can still emulate Apple ][, //e, //c, & //gs computers on my Mac
easily, and hundreds of times faster than the originals.  I can run
Windoze on my Mac via Virtual PC (on the PowerPC ) or directly (on the
Intel Mac).  I can emulate Atari, Amiga, Commodore 64, and virtually
every other computer made - so how does that reconcile with not being
able to use old programs on new systems?  

A large percentage of computer programmers are a strange combination of
forward-looking and nostalgic people - they want to push the envelope,
but they always look fondly back on the "Old Times" and try to keep
those days alive somehow.  I still remember fondly being able to shave
one or two BYTES of code off of program I wote back in the 70's & 80's,
because back then you might have the LUXURY of having 16KB (that's
Kilobytes.  Not Megabytes, not Gigabytes) of memory to work with,
although you had to a.s/u/me that the enduser only had 8.  I also push
every overly bloated, non-optimized program I currently own right to
the limits and sometimes beyond (which is why I often beta test, so I
can add more bloated features To some of these programs)

You'll probably be able to emulate todaty's computers for centuries,
simply for the valid-to-the-programmer's reason "because we can".

> Your DVD archives will have degraded, or you won't be able to find a
> machine with a DVD reader any more.

Yeah, right.  DVD degradation is a valid point.  At which time those
who are aware of it simply recopy the files to a new DVD before it
degrades.  Rocket science, huh?  What about disk drives spinning at
7200+ RPM on a constant basis?  No danger of problems there, of course,
right?  And as for DVD degradation, I have no doubt that there will be
available recovery systems for gleaning that data back off of them
should it become necessary, just as there are systems now for
recovering data from damaged hard drives.  Hell, they can already get
data back off of drives that have been written back over several times.
Ain't forensic science fun?

But not being able to find a DVD reader?  C'mon.  They still sell 8"
floppy drive controllers if you want to look for them, for gosh sakes.
I've still seen huge tape spool machines interfaced with modern
computers.  No, the average Joe isn't gong to do that.  It's an
extremme example shoiwng that blanket statements like "you won't be
able to" are not true.  If your data is important enough to you, you'll
find a way, should all the rest of your dire predictioons come true.

And if by that time it might possibly be true, I have no doubt that
there will be a conversion process somewhere.

You notice that they still sell video tape players/recorders, although
a lot of humanity has switched to DVDs?  Cassettes are still out there
in the stores for those who prefer them over CDs.  Hell, they still
even make *record* *players* - you know, that ANCIENT technology thhat
uses vinyl and a physical (::shudder!::) stylus?  Huh.  You can still
buy parallel printer interfaces even though USB has virtually taken
over.  Why?  Because people still have and use parallel printers.  You
can still buy SCSI interface cards, despite the fact that ATAPI/IDE has
virtually taken over.  Why (besides the fact that they are faster)?
Because people still have and use SCSI interface devices.  The point is
thhat as long as there is any sort of reasonable demand for something,
someone,somewhere, will keep providing it.  Thers's A LOT of demand
(and actually rapidly growing at this point) for RAW file software
tools.  Why, in the near or conceivable future, would support for that
be dropped?

By your same logic, no one should be storing anything on DVDs, or even
buy DVD videos or players or such because SOMEday(!) they might become
obsolete.   Well, yeah.  Almost everything does.  Does that mean you
don't use the BEST available means at the time, in order to put off the
obsolescence further?  

> Sure, someone will write a converter.  But you'd better make sure you
> use it during the time window that it is useable.

"Window that it is useable"?  And what short time period do you think
that that might be?  Minutes?  Hours?  Days?  Months?  Years?  Decades?
Boy, I can see that window opening and closing so fast that I'd never
be able to convert.  Yah, shüre, youbetcha.

> The only way to maintain an archive of digital material is to keep it
> live and moving forward as file and media formats change over time.

And I don't argue with that point.  But your argument that it is a
detriment to using RAW files now... how?  Use the best available method
at the time, and by the time something better comes up, someone will
have a converter, so you can progress.

All this ALSO _blatantly_ begs the question:  Why would RAW file format
be any more likely to change than TIFF/JPEG/etc. has been?  And so how
is using a TIFF file (or any other format, for that matter) going to
make it immune to your same arguments, a.s/u/ming that they are valid?

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

jeremy - 13 Dec 2006 15:59 GMT
>> > What?  You think that software writers are going to take the code OUT
>> > of their programs when it hurts nothing to leave it in?  Or that the
[quoted text clipped - 116 lines]
> is using a TIFF file (or any other format, for that matter) going to
> make it immune to your same arguments, a.s/u/ming that they are valid?

You miss the point entirely.  Over a long time horizon, all these formats
and media will pass from the scene.  Your comparing that with a 25-year time
frame is inappropriate.   Innovation is taking place so rapidly that in 100
years it will be virtually impossible to read an image file from a CD
("What's a CD?")
John McWilliams - 13 Dec 2006 16:26 GMT
> "Ken Lucke" <ken@glass-stones.com> wrote in message
>>
[quoted text clipped - 8 lines]
> years it will be virtually impossible to read an image file from a CD
> ("What's a CD?")

No it won't, except through physical degradation of the medium. Same as
any other physical manifestation of art, or documents, buildings, etc.
Now, CDs will doubtless have a shorter life than "La Jocande", the Dead
Sea Scrolls, or any other oil painting or document on linen, paper or
papyrus.

Signature

John McWilliams

Ken Lucke - 13 Dec 2006 16:43 GMT
> You miss the point entirely.

No, I don't.

> Over a long time horizon, all these formats
> and media will pass from the scene.

Yes.  Long after my lifetime, probably.  At which time it wil become an
SEP.

> Your comparing that with a 25-year time
> frame is inappropriate.   Innovation is taking place so rapidly that in 100
> years it will be virtually impossible to read an image file from a CD
> ("What's a CD?")

I agree to the possibility thaht that might be the case.  At that point
all the data you ever own will probably fit on a chip the size of your
thumbnail, be totally impervious to EMP, and will do you laundry, too.
But that's in the future.  Are you predictiing that there will be no
interim upgrade steps?

However, YOU miss the point entirely that I am trying to make.  It
doesn't matter a whit what might be in thhe future.  Not knowing, or
even if you were psychic and DID know, what has that got to do with
using best current practices if you can't implement [unknowable] future
standards today?  

As I keep repeating over and over and over, use the best current
methods, and upgrade as time moves on.  You seem to imply that those
changes will occur so rapidly that there will be no upgrade path
available during the interim, and quote some silly bit about one or two
propietary hardware manufactures' demises that have caused problems as
rationale.  What relationship does that have to millions of varying
types of personal computers all over the world running software
commercially available thhat handles the data just fine at this point?

Do you figure that all the PCs/Macs/Unix boxes/Whatevers that can
currently read those file formats are going to give up the ghost
simultaneously overnight all over the world, with an also simultaneous
total upgrade in hardware technology that will make the data storage
mechanism itself obsolete, thus leave us all in the lurch because we
can't read the data format any more?  That's silly.

I will maintain that the best current practice is to store the file
exactly as it comes out of the camera.  If at some future time, there
is a sustainable demand (and response) for a one-standard raw file
format, there will be an upgrade/conversion path for those who have
been storing them in that fashion, at which time the best current
practice THEn will be to upgrad/convert to the new standard.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

Roger N. Clark (change username to rnclark) - 14 Dec 2006 01:04 GMT
> You miss the point entirely.  Over a long time horizon, all these formats
> and media will pass from the scene.  Your comparing that with a 25-year time
> frame is inappropriate.   Innovation is taking place so rapidly that in 100
> years it will be virtually impossible to read an image file from a CD
> ("What's a CD?")

I agree.  The problem today with growing data sizes is that
people will not have time to upgrade.  How many people even
know to back up their data?  I struggle to keep 3 rotating
backups of my ~700 GBytes of image data.

Data archiving bit NASA.  Old planetary data was becoming lost
and when someone tried to read the 7-channel magnetic tapes, the
oxide layer flaked off.  They had only one chance to get it.

Like I said in an earlier post, my first book was in
multimate format.  The original data are on Tandy 760 KByte
5 1/4 inch (or whatever that was) disks.  IBM's standard was
720 kbytes.  Does anyone know of a 5-inch floppy reader
that plugs into usb port and will read both 720 and 760
kbyte floppies?  I do have some data I would like to recover.
I can do it another way with an old Tandy I still have, but
it would be easier in one step, rather than 760 to 360 kbyte
5-inch disk, then fire up an old 486 and convert 5-inch to
3.5 inch floppy, then 3.5 floppy to new server hard drive
and CD backup.

As for camera raw data, I recommend everyone download the source code
to DCRAW, and keep downloading every version.  That way you
should be able to read you raw data files into the future,
assuming compilers will exist (I do).

Now days, NASA requires source code and instructions be archived
with all spacecraft data.  It is good practice and everyone
should do that too.  At least with public domain source code,
you maximize the probability of reading the raw files far into the
future.

Roger
jeremy - 14 Dec 2006 04:00 GMT
> As for camera raw data, I recommend everyone download the source code
> to DCRAW, and keep downloading every version.  That way you
> should be able to read you raw data files into the future,
> assuming compilers will exist (I do).

A dedicated photo enthusiast might be willing to keep up with that, but I'd
bet that 99% of family photos will eventually fade into image format
oblivion.

I never got into digital in a big way.  I continue to shoot 35mm and scan my
negs and transparencies.  Low-tech film may outlast digital data, if only
because it can be easily read in the future.

I recently found and scanned a roll of negs that I shot 32years ago, of my
hometown, taken from the top of a 10-story building.  It was such a pleasure
to have a look at what things were like back then.  I believe that, had I
had a digital camera in 1974, those image files would have been lost
forever, because I'd have not bothered to migrate them to newer formats and
media.  But my negative required no maintenance at all.  There is something
to be said for that.
Jeff R. - 14 Dec 2006 05:02 GMT
>> As for camera raw data, I recommend everyone download the source code
>> to DCRAW, and keep downloading every version.  That way you
[quoted text clipped - 16 lines]
> formats and media.  But my negative required no maintenance at all.  There
> is something to be said for that.

More than just "something". An awful lot.  Forget sensor size, megapixels,
crop factors, gelatin content, etc. etc. blah blah.

Archival and retrieval of digital images is going to be *the* issue of the
decade(s) to come.

Just to drop in another anecdote, my beloved uncle passed away a few years
back, and when the family went through his stuff we found some old tin boxes
full of glass plate negs that Granddad had taken before the first War.
Apart from some minor issues around the edges of some of them, they were in
perfect condition, ready to yield huge (if necessary) b/w prints of the era.

I doubt my Mavica floppies will fare as well.

There's going to be a whole generation who curse the digital medium, in
retrosopect, because of the loss of all these family records.  It won't be
the photographers - they'll be gone - it'll be their grandchildren.

I look forward to the technological breakthrough which will solve this
problem.  Not the cobbled-together, recopy-every-ten-years *half* solutions
I see bandied about - I mean the genuine, reliable, accessible century-plus
archival device.  Ain't seen one I'd trust yet.

...though I like the idea of hex scratched into clay tablets - cuniform
style.  That'd cut down on frivolous snapping away.

--
Jeff R.
jeremy - 14 Dec 2006 13:39 GMT
> I look forward to the technological breakthrough which will solve this
> problem.  Not the cobbled-together, recopy-every-ten-years *half*
> solutions I see bandied about - I mean the genuine, reliable, accessible
> century-plus archival device.  Ain't seen one I'd trust yet.

It will have to be an entity that spans generations and has a clear mandate
to preserve the images.  I am thinking that it might end up as a
governmental department.  We fund all sorts of libraries, archives, research
projects, historical projects, etc.  We, as a nation, might have an interest
in preserving our past.  It would not be so difficult to set up large scale
servers to store the images, and they could also take steps to ensure that
they could be read in the future, by creating emulator software.

Long-term preservation (i.e., 100+ years) is simply beyond the capabilities
and resources of the majority of families.

For the cost of a few cruise missiles, America could preserve all this stuff
for many generations to come.  Imagine the treasure trove for historians 200
years from now if they could get their hands on all the gazillions of
digital images that we currently have.
Scott W - 14 Dec 2006 14:28 GMT
> >> As for camera raw data, I recommend everyone download the source code
> >> to DCRAW, and keep downloading every version.  That way you
[quoted text clipped - 30 lines]
>
> I doubt my Mavica floppies will fare as well.
I would hope you have long ago transfered that images off those
floppies.

> There's going to be a whole generation who curse the digital medium, in
> retrosopect, because of the loss of all these family records.  It won't be
> the photographers - they'll be gone - it'll be their grandchildren.
And what people can do here is grab copies of the photos before their
parents/grandparents past away.   I Have a much better set of my
parents photos since they went digital then before they went digital.
My niece and nephew are not yet old settled enough in their lives to
really be bother about their grandparents photo collection, but when my
parents are gone they are both very likely to want copies and this will
be very easy to get to them.  What will be much harder to get to them
are copies of the film photos, I have my fathers negatives, but my
parents only kept the BW negatives my dad developed himself, they did
not keep or did not know when the years of color negatives went.  Their
slide collection is going to be a pain to deal with, it was bad enough
scanning through my own slide collection.  How many people will be able
to do more then hold their parents slides up to the light and look at
the tiny image?

> I look forward to the technological breakthrough which will solve this
> problem.  Not the cobbled-together, recopy-every-ten-years *half* solutions
> I see bandied about - I mean the genuine, reliable, accessible century-plus
> archival device.  Ain't seen one I'd trust yet.
Yup a good long term solution would be pretty nice, for now grab the
photos and can from the older generation while they are still around.

Scott
jeremy - 13 Dec 2006 15:07 GMT
> What?  You think that software writers are going to take the code OUT
> of their programs when it hurts nothing to leave it in?

Yes, that is exactly what I think.  Remember, we are discussing long-term
archiving, not short-to-medium term storage of 5 years or less.  And it is a
virtual certainty that many of these proprietary formats will fade from the
scene over the long term.

There are already numerous problems opening up compressed TIF files from
their first version, when Aldus owned the format.  Lotus 1-2-3 version 1
files cannot be opened by current versions of Lotus (Spreadsheet files).
The original developers' notes disappeared when IBM bought Lotus, years ago.
FlashPix is a dead format.  After HP, Kodak and LivePix making a big splash
over its debut (the format was related to, and was similar to Kodak's PCD
format for Photo CD) the consortium broke up and they literally abandoned
the format a few years ago.

There are numerous word processor formats from the late 80s that have gone
under (prior to the shakeout where Microsoft Word became the de facto
industry standard).  I remember using "Multi-Mate" from Ashton-Tate, which
was a software emulation of the Wang Word Processor (a big piece of
hardware).  Ashton-Tate discontinued operations, orphaning Multi-Mate users.
Try and find any conversion tool that will print those files to look the way
they were originally formatted.  You'll not find anything.

MS Word has changed their word processor file format numerous times--many of
us suspect that the reason is to force users to keep upgrading whenever a
new version is introduced.  In 50 years, will anyone be able to conveniently
open a MS Word for DOS Version 1 file???

> Or that the current programs that handle those files just fine will
> suddenly stop
> working on the current images at some point in the future if they do
> change the format and the software publishers do remove the code?

YEP!  Adobe PS has not supported FlashPix in the last two versions of PS.
You can still open and edit FPX files in Paint Shop Pro, but it is only a
matter of time before they stop supporting the format.  About 10 years ago,
I archived a few hundred digital images in FPX format, and I later had to
migrate those files over to TIF.

The problem with migration is there is the risk that the newer format may
not display the image exactly as it appeared in its original format.

There is a debate in the archivist community about whether to migrate file
formats, or to create emulator software, to create virtual computers that
can run the old software in their native operating systems.

> Or  that by the time that someone decides to have one single RAW file
> standard that everyone (except MicroSloth, of course) will adhere to,
> that someone esle won't have a converter to change the old RAW
> format(s) to the new one?

YES.  I not only believe it, I know it to already be a fact.

> I think that's just nuts.

Perhaps you should educate yourself more on these matters before you come
out swinging.

Do you know that part of the 1970 US Census data cannot be read by the US
Government because the media on which it was stored no longer has any
hardware that supports that media type?

Do you know that one of NASA's missions to Mars resulted in data that was
stored on proprietary media where the vendor went out of business, and the
existing reader hardware failed, leaving the data unreadable?

Do you know that the National Archives elected to PRINT AND THEN MICROFILM
millions of emails from Bill Clinton and Al Gore, when they left office,
because they were concerned that they would disappear if stored
electronically?

If the US Government, with all its resources, is concerned about LONG TERM
VIABILITY OF DATA AND STORAGE MEDIA, how can any of us be cavalier about it?
Ken Lucke - 13 Dec 2006 16:21 GMT
> > What?  You think that software writers are going to take the code OUT
> > of their programs when it hurts nothing to leave it in?
[quoted text clipped - 12 lines]
> format for Photo CD) the consortium broke up and they literally abandoned
> the format a few years ago.

But the original programs still exist, and can still open those files.
They mmay not be in common (or even uncommon) use, but they still
exist.  On two of my hard drives, I have archives of every single
program I ever had, clear back to the Apple II.

> MS Word has changed their word processor file format numerous times--many of
> us suspect that the reason is to force users to keep upgrading whenever a
> new version is introduced.

Well, that's a standard MS tactic in everything they do, so how is that
a surprise.  :^)

> In 50 years, will anyone be able to conveniently
> open a MS Word for DOS Version 1 file???

Probably.  If you notice, even though they've mysteriously upgraded
their file formats, their "new, latest, greatest" program still opens
the previous one.  When that happens, you open it, and resave it as the
new format, until the next week  when MS changes stuff yet again.  It's
called conversion and upgrade, and it's exactly what I said may have to
be done IF the various raw formats are discontinued in favor of new
ones or a single standard.

> > Or that the current programs that handle those files just fine will
> > suddenly stop
[quoted text clipped - 6 lines]
> I archived a few hundred digital images in FPX format, and I later had to
> migrate those files over to TIF.

I said current programs.   A program which currently reads the file.
When you upgrade to a new program, it may not - but the old program
(which WAS current at the time, hence the designation) will.  And
usually be able to export it in a fashion that the new program can read
and convert. If not, you'll still be able to deal with it in the same
way you always did with that earlier (then"currennt") program.

> The problem with migration is there is the risk that the newer format may
> not display the image exactly as it appeared in its original format.
[quoted text clipped - 9 lines]
>
> YES.  I not only believe it, I know it to already be a fact.

Ohh?  Care to lay out the "fact" that there is now one single raw
format which there is no converter to?  Because the first part of that
statement is not even currently true (thhat the time has come that
someone has ome up with a raw file format that everyone will adhere
to), the second part cannot be, and thus the whole statement cannot be
shown to be true.

> > I think that's just nuts.
>
[quoted text clipped - 4 lines]
> Government because the media on which it was stored no longer has any
> hardware that supports that media type?

And if it was important enough to them, they could recreate the
hardware easily enough.  They just don't want to, apparently.  Not that
it couldn't be done.

> Do you know that one of NASA's missions to Mars resulted in data that was
> stored on proprietary media where the vendor went out of business, and the
> existing reader hardware failed, leaving the data unreadable?

Again, there is no data storage method that someone, given the
resources, cannot duplicate. Besides, you're talking hardware.  I'm
talking data.  How many of these formats, while proprietary to the
companies, are secret enough that you cannot find out about them?
Hell, this is the internet age.  All you have to do is Google for the
data format, and you can find it.  Any decent programmer could thhen
write a conversionprogram from one format to another.  A physical,
proprietary piece of hardware may be hard to duplicate (although not
impossible), but a software program to access a known set of data
structures is not.

> Do you know that the National Archives elected to PRINT AND THEN MICROFILM
> millions of emails from Bill Clinton and Al Gore, when they left office,
> because they were concerned that they would disappear if stored
> electronically?

So, we should store all our raw data in 1's & 0's on paper, then
microfilm them?  What about degradation of microfilm, if it's not kept
in relatively controlled condition?  No storage method is perfect.  As
I keep saying, you use the best method available at the time, and
upgrade when the circumstances make it necessary.

> If the US Government, with all its resources, is concerned about LONG TERM
> VIABILITY OF DATA AND STORAGE MEDIA, how can any of us be cavalier about it?

Who's being cavalier?  I merely stated that the best current available
method for storing the original data from the camera is to use the
original raw file from the camera (unless you can somehow access the
sensor data directly on an every-photo basis), and if the time comes
that the storage format is obsolescent, you convert it and upgrade.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

G.T. - 13 Dec 2006 17:58 GMT
> Who's being cavalier?  I merely stated that the best current available
> method for storing the original data from the camera is to use the
> original raw file from the camera (unless you can somehow access the
> sensor data directly on an every-photo basis), and if the time comes
> that the storage format is obsolescent, you convert it and upgrade.

And you're really going to remember all those raw files you stashed on
DVD 20 years ago?  Unless one consciously sets up a workflow to check
one's archives on a routine basis for readability (something I doubt
many are doing) people are going to lose files for one reason or
another.  DVD rot (and recovering optical media is not as trivial as
recovering magnetic media), abandoned formats, whatever.

Greg
Signature

"All my time I spent in heaven
Revelries of dance and wine
Waking to the sound of laughter
Up I'd rise and kiss the sky" - The Mekons

Ken Lucke - 13 Dec 2006 18:16 GMT
> > Who's being cavalier?  I merely stated that the best current available
> > method for storing the original data from the camera is to use the
[quoted text clipped - 4 lines]
> And you're really going to remember all those raw files you stashed on
> DVD 20 years ago?

Uh, yeah.  All my archive DVDs, photo or otherwise, are clearly
labelled with contents and date, and stored and indexed physically.  I
guess that's part of a "consciously set[ting] up a workflow".  Yours
aren't?

So I choose some point in the future somewhere about the MTBF date (if
anyone ever determines what that is), I spend a day and copy them, if I
haven't already upgraded them to the mythical one-raw-format standard
before that time.

> Unless one consciously sets up a workflow to check
> one's archives on a routine basis for readability (something I doubt
> many are doing) people are going to lose files for one reason or
> another.  DVD rot

People are going to crash their cars for one reason or another, too.
That doesn't mean that by careful use I can't decrease the chances.
"People" is a mass term.  "People" operate their computers without
backups of any kind.  Mine backs up automatically every single night at
0200, because I choose to be more careful than the average "people" out
there.

Individuals can choose to override the mass norms for themselves if
they so desire.  I have no interest in what "people" on the whole are
going to experience, if I can influence the my experience in a positive
way by my actions.  It's not MY responsibility to make sure that
"people" are all safe and warm and comfortable and that their photos
are all secure for whatever time frame they feel is desireable.  It IS
my choice, and therefore my responsibility, to do so for myself.

> (and recovering optical media is not as trivial as
> recovering magnetic media)

How do you know?  Really.  How _do_ you know?  Someone in some dark
room right now could be discovering exactly how to do that (or maybe
already has), just like they discovered how to read hard disk data that
had been overwritten many many times.  No, they won't do it with a
standard drive, but then the forensic data recovery experts don't do it
with a standard hard drive mechanism and off-the-shelf software,
either.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

Scott W - 13 Dec 2006 16:37 GMT
<snip> many examples of code going obsolete.

I would also do not trust raw files to be readable in the long term,
there are too many versions of them and they are under the control of
individual companies.

There are other file formats that will likely be readable way into the
future, just as ASCII is now. I would expect there to be support for
jpeg files for a very long time, this is not to say that jpeg will not
be replaced at sometime in the future but the is will continue to be
supported.  We see many examples of this even now, graphical file
formats that have not been used for years and yet it is easy to find
programs to open them and to do batch conversions on them.

With most file formats I can use a programs like IrfanView to do batch
conversions from one format to another.  But with Raw this is harder,
in large part because raw converters need input as to how you want to
converter the raw image.  I know I would not be happy to simply have
all my raw images converter using the camera setting.

So there are a few file formats that I trust to be around and I always
keep one of these formats in my backup as well as the raw files.
Mostly I use jpeg, It is small and for high quality jpeg there is no
practical difference to a tiff file.

As for CD and DVDs going obsolete, I think we will have a good long
warning before this happens.  Nine years ago I had people telling be
that within 5 years the CD format would be obsolete and we wold no
longer be able to read them, I still have backups from nine years ago
on CD that I can read just fine.  Sure at some point the nearest
computers might not be able to read CDs, but it should be easy to
migrate to the new media.  In the same way that I can buy a external
floppy drive now I will be able to buy an external CD drive if at
sometime I end up with a computer that is lacking one.

Scott
Ken Lucke - 13 Dec 2006 17:16 GMT
> <snip> many examples of code going obsolete.
>
> I would also do not trust raw files to be readable in the long term,
> there are too many versions of them and they are under the control of
> individual companies.

So are many of the other file formats you use on your computer on a
daily basis.  How many versions of Word documents, or Excel
spreadsheets, or other programs' proprietary (note, proprietary does
NOT imply "secret") formats have their been - but the new version
always allows you to read at least the immediately previous format and
resave it in the current one.

> There are other file formats that will likely be readable way into the
> future, just as ASCII is now.  I would expect there to be support for
> jpeg files for a very long time, this is not to say that jpeg will not
> be replaced at sometime in the future but the is will continue to be
> supported.  

Howevever, JPEG is a horrible way to store pure, untouched camera image
information.  It is lossy, and loses more every time it is resaved.  It
is only 8 bits, where 24 bits in a raw, or even 16 bits in a TIFF,
comes closer to approximating the currently unachievable gamut of the
human eye.

> We see many examples of this even now, graphical file
> formats that have not been used for years and yet it is easy to find
> programs to open them and to do batch conversions on them.

Indeed.

> With most file formats I can use a programs like IrfanView to do batch
> conversions from one format to another.  But with Raw this is harder,

Of course it's harder.  Generally speaking, anything that is going to
give better results, or that you want done a certain way as opposed to
the way of the masses, is probably going to be harder.  It's frequently
harder to use an SLR than it is a P & S, but the SLR will usually give
better results to the user who has taken the time to learn to use it
properly.   It's harder to adjust a raw image to what you want than to
just accept a JPEG version that the camera throws at you, but the
results can be much better if you take the time.  It's hard to learn to
use all the tricks and techiques that Photoshop allows, but the results
from a skilled user are often far better than a JPEG out of a camera.

> in large part because raw converters need input as to how you want to
> converter the raw image.

Which is exacly why raw is important to someone who considers the final
image important enough to justify using it - so YOU control how the
image turns out, not some generic-profiled piece of software.

> I know I would not be happy to simply have
> all my raw images converter using the camera setting.

But you're happy to allow it to do so with JPEGS?

> So there are a few file formats that I trust to be around and I always
> keep one of these formats in my backup as well as the raw files.
> Mostly I use jpeg, It is small and for high quality jpeg there is no
> practical difference to a tiff file.

Perhaps not, until you get to a point of printing very large and very
wide gamut images on a printer capable of handling them.  JPEG is only
8 bits.  TIFF can be 8 or 16.  Much wider gamut on each individual
pixel.

> As for CD and DVDs going obsolete, I think we will have a good long
> warning before this happens.  Nine years ago I had people telling be
[quoted text clipped - 5 lines]
> floppy drive now I will be able to buy an external CD drive if at
> sometime I end up with a computer that is lacking one.

Exactly.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

Morris Jones - 13 Dec 2006 17:57 GMT
> There has been much speculation that RAW formats are inappropriate for
> long-term archiving because it is unlikely that those proprietary formats
> will be supported over the long term.

If you were referring to a physical storage format as opposed to a data
format, you might be right.

Data formats have a lot more longevity, but you'll probably have to keep
moving the data to a physical storage that has a working reader on your
system.

Mojo
Signature

Morris Jones
Monrovia, CA
http://www.whiteoaks.com
Old Town Astronomers http://www.otastro.org

Ken Lucke - 13 Dec 2006 04:25 GMT
> I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via
> a card reader built into my computer. So now the RAW files are saved.
[quoted text clipped - 13 lines]
>   also lots of images to be named, sorted, rated, named and cataloged.
> Any comments, suggestions would be appreciated.

Oh, oh... Whooo, boy... you got me started....

This is where a program like Apple Aperture
(<http://www.apple.com/aperture/>) or Adobe Lightroom
(<http://labs.adobe.com/technologies/lightroom/>) shines.  Not only do
they provide file management, but they also use non-destructive editng
- they save their changes via XML or Sidecar files, which are merely
text data (and thus very tiny in comparison to an entire image) which
tell the program which changes to make to the original for display or
output - they never change the original file.  Plus you don't have to
store all those multiple versions, only exporting your final one when
you need it at precisely whatever size and resolution you need it,
directly from the RAW data + the sidecar changes.

Either program will allow you to store the files in your own location
and "reference" them, or have the program move or copy (your choice)
your files into its own "managed" file.  Or you can use a combination
of both methods - "managed" for some and "referenced" for others - and
move them from one to the other freely (at least you can move them
freely in Aperture, not sure about Lightroom, as I only used it
temporarily while Aperture was being upgraded to handle the newer .CR2
files form the 400D, and for those files only, which I never moved from
their location that was also referenced by Aperture).  I usually opt
for full management rather than referencing, siomply because it's
tideier on disk and more bullet proof to eye-dee-ten-tee errors.

Both will help you manage your files, use keywords for searching, do
sophisticated RAW adjustments, etc.

I like Aperture, for several reasons:

Moved "referenced" files can be reconnected to their sidecar data very
easily.  "Managed" files are EXTREMELY hard to screw up - to delete
them, you literally have to tell the program to do so AND dismiss TWO
warning dialogs that pop up in your face to warn you.

If you do need to actually edit on the pixel level (as opposed to
overall things like exposure, levels, white balance, saturation, etc.,
etc., which either program handles fine), with Aperture you can export
it with a single key combination to your chosen editing program, do
your edits, and the file is returned as a copy right back to Aperture
to the same "stack"* as the original - all without ever editing the
original file at all - Aperture even opens the editing app for you.

You can also create named projects, folders within the projects, albums
(which can hold images even from several differing projects without
duplicating the actual image file), do searches and create albums or
web pages from the searches, and lots of other "management" thingies.
Lightroom has some similar features, but IMO not as flexible.  Entire
projects, including the metadata and structure can be exported if
desired, and can be imported into another Aperture as is.  Imagine
being ablle to send an editor an entirely prepared project, including
light table mockups (Aperture has them) and everything, in a single
file.

Aperture also lets you assign metadata of almost any kind uponimport,
so an entire batch of imports can be metedata'ed (did I just make up a
word?) at one time as they are importing, rather than spending time on
it later.  You can also "stack" similar images while they are
importing.  

Aperture can also import and store (although not edit) .pdf files
(first page only).  I use this to attach a copy of any release forms or
other information that I might need to a photo so it never becomes
lost.

Aperture will also will automatically back up your data to any number
of other "vaults", even networked or on optical media, saving all your
changes, including addition or deletion of pictures, changes in sidecar
data, projects, folders, etc.  Files "deleted" from the vaults in the
process can either be automatically trashed, or saved in a separate
folder for inspection prior to manually trashing them if you're worried
that you might have done something you regret,and then backed it up
that way as well. :^)  You can also have more than one ".aplibrary" for
use in Aperture, which means you can have completely separate libraries
for different users or different reasons (if separating by folders or
projects isn't enough).  With Lightroom, you can't do that (AFAIK).

Lightroom, on the other hand, while there are a couple of features in
the "develop" panel that are improvements over the same thing in
Aperture (in useage only - the same things can be achieved in Aperture
in other ways), and while it allows similar seeming file management, it
doesn't do so in as sophisticated a way as Aperture does, IMO.  One
advantage of LR is that currently, it's a free Beta version (eventually
it will expire and/or be replaced by a final version, at which point
you'll have to buy it, though)

Aperture, unfortunately for you (at least from the headers of your
message, which indicate you are on a Windoze box), is Apple only.
Lightroom has both Mac & Win versions.  Apple has a free 30 day trial
version.  

LR beta is free.  While beta, it IS a "public" beta, meaning it's
pretty stable and they are trying to get user reactions and feedback
prior to finalizing it.  In my experience, it was as stable as any
final release program, or at least it was on the Mac.

* Stacks are... well, you just have to see and work with them to
appreciate the genius behind them.  They are one of Aperture's finest
points, and one that Lightroom doesn't even have a close comparison to
- they allow you to group images (mmanually, or automatically based
upon time between shots, from 1 second to 1 minute apart can
automatically become a stack, as might be when you are doing fast,
multiple shots and then need to later pick the best one), compare them
together side by side, change their priorities, choose the best, and
then click the little tab and they all disappear under the one you
picked as the best which then is the only one that shows up in the
display - until you click the little tab again and they all reappear
for more play.  You can manipluate a whole stack of images (no limit as
to the number within a stack) as if it were a single image, moving it
around the whole stack at a time from one project or album to another,
and when you use it in things like a web gallery, only your "pick" or
the top image will be shown unless you tell it specifically to ignore
the stack groupings.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

David Azose - 13 Dec 2006 07:42 GMT
>> I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via
>> a card reader built into my computer. So now the RAW files are saved.
[quoted text clipped - 128 lines]
> the top image will be shown unless you tell it specifically to ignore
> the stack groupings.

WOW!

Thanks for the comprehensive reply. You are correct, I use a windows
computer so Aperture, which sounds wonderful, is out of the question for
me. But I will download Lightroom and give it a try. My needs are
perhaps not as critical as some in this newsgroup. I'm not a
professional photographer but I consider myself an advanced amateur.

Thanks again for the information.

David A.
John McWilliams - 13 Dec 2006 16:37 GMT
> Thanks for the comprehensive reply. You are correct, I use a windows
> computer so Aperture, which sounds wonderful, is out of the question for
[quoted text clipped - 3 lines]
>
> Thanks again for the information.

I'd be interested to hear how you get along with LR. It's been from
fabulous to frustrating since I started using it six months ago, and I
keep having to remind myself that it's a Beta. My posting name is
Mcphotoman if you have any questions on the LF forum.

Signature

John McWilliams

Ken Lucke - 13 Dec 2006 17:38 GMT
> > Thanks for the comprehensive reply. You are correct, I use a windows
> > computer so Aperture, which sounds wonderful, is out of the question for
[quoted text clipped - 5 lines]
>
> I'd be interested to hear how you get along with LR.

If I hadn't already been spoiled by Aperture, which I've owned since
V1.0, I might have gotten used to LR.  It's capable, and fairly
intuitive, unlike much of Adobe's stuff, a lot of which I find is often
counter-intuitive.  I do (as you've seen) recommend it to those who,
not being Mac-based, have no choice to use Aperture.  Betweenthe two,
except for a couple of minor things in the "Develop" panel whcih could
have been implemented better in Aperture, Aperture wins hands down,
IMO.

> It's been from
> fabulous to frustrating since I started using it six months ago, and I
> keep having to remind myself that it's a Beta. My posting name is
> Mcphotoman if you have any questions on the LF forum.

Not on the LR forum, sorry.

To be honest, the only reason I started using it (other than a quick
look to see what it was when I first heard about it) was that it
immediately supported the 400D's version of the .CR2 files, where
Aperture didn't do so until two days ago.  I used it in a rather
unorthodox and rigamarolish-fashion:

I imported the ima.cr2 files into Aperture, using "move files" rather
than "reference in place".  I then used "relocate master" to move them
back out to the world outside its Aperture's own file, using a
YYYY/MM/DD hierarchical format based on EXIF image date.

I then imported them into LR in that same hierarchical format, created
.tif versions of them, and could do manipulations with them and export
results if desired for interim printing.

I also imported into Aperture the files that I had exported from LR as
untouched .tif, creating stacks with the original, unrecognized images
that matched.  Now I had visual representations in Aperture that
matched the .cr2 files which showed up a "unrecognized image format".
I could further manipulate the .tif files using Aperture.  Why didn't I
do that in Lightroom?  Because I knew at some point that Aperture would
start supporting those files again, so I could simply copy and paste
the manipulations I had done on the .tifs to the .cr2s and have a
pretty good first approximation of what I wanted, already done.

I could then, when desired, do a "consolidate master files" in
Aperture, which would move them all (.cr2 & .tifs from LR) into the
Aperture library itself, so I could back them up to both of my backup
vaults.  After backing up, I "relocated masters" back out to the same
previous location, so that LR could still access them.  The moves were
so easy using Aperture, and only took 15 or 20 seconds, that I could do
this as needed without suffering much.

What a colossal waste of time.  :^(

Now of course, I just use Aperture for it all, like I did before
switching cameras.  I've copied all the adjustments from the .tifs to
the .cr2s, deleted all the extraneous .tif files, moved everything back
into the main library permanently, and my workflow has improved 1000%.
Thank God.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

G.T. - 13 Dec 2006 18:02 GMT
>>> Thanks for the comprehensive reply. You are correct, I use a windows
>>> computer so Aperture, which sounds wonderful, is out of the question for
[quoted text clipped - 61 lines]
> into the main library permanently, and my workflow has improved 1000%.
> Thank God.

This is a great testimonial for a standard raw format like DNG.

Greg

Signature

"All my time I spent in heaven
Revelries of dance and wine
Waking to the sound of laughter
Up I'd rise and kiss the sky" - The Mekons

Ken Lucke - 13 Dec 2006 18:18 GMT
> This is a great testimonial for a standard raw format like DNG.

DNG is not a standard format, yet.  And it might never be.  This is a
fairly balanced argumennt on both sides of the issue:

http://www.openraw.org/node/1482/531

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

John Francis - 13 Dec 2006 18:21 GMT
>> This is a great testimonial for a standard raw format like DNG.
>
>DNG is not a standard format, yet.  And it might never be.  This is a
>fairly balanced argumennt on both sides of the issue:
>
>http://www.openraw.org/node/1482/531

I would not by any stretch of the imagination call openraw.org's
argument "fair and balanced".  They are agenda driven, with the
agenda being "DNG is wrong".
Ken Lucke - 13 Dec 2006 18:24 GMT
> >> This is a great testimonial for a standard raw format like DNG.
> >
[quoted text clipped - 6 lines]
> argument "fair and balanced".  They are agenda driven, with the
> agenda being "DNG is wrong".

It's far more balanced (in that it gives both the pros and the cons)
than does any exposition by Adobe itself, who hails it as the Second
Coming for photographers.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

Barry Pearson - 14 Dec 2006 14:10 GMT
[snip]
> > >http://www.openraw.org/node/1482/531
> >
> > I would not by any stretch of the imagination call openraw.org's
> > argument "fair and balanced".  They are agenda driven, with the
> > agenda being "DNG is wrong".

Correct. See:
http://www.barry.pearson.name/articles/dng/openraw.htm

I believe this arises from a fear that DNG will detract from their
primary objective, the documentation of proprietary raw formats.
OpenRAW appears to have a "block", and can't see that DNG is part of
the solution, and here to stay.

(ps: I became an OpenRAW supporter very early - number 11. But I have
always met a frustrating opposition on all sorts of spurious grounds).

> It's far more balanced (in that it gives both the pros and the cons)
> than does any exposition by Adobe itself, who hails it as the Second
> Coming for photographers.

Incorrect. See:
http://www.barry.pearson.name/articles/dng/commentary1.htm

Much of his article defies logic, as I show in that commentary.

Signature

Barry Pearson
http://www.barry.pearson.name/photography/

John McWilliams - 13 Dec 2006 19:32 GMT
>>> Thanks for the comprehensive reply. You are correct, I use a windows
>>> computer so Aperture, which sounds wonderful, is out of the question for
[quoted text clipped - 61 lines]
> into the main library permanently, and my workflow has improved 1000%.
> Thank God.

Good heavens! I got into LR seriously when I got my 5D and all of a
sudden had to use it to convert its RAW files, having only CS I. I've
played a bit with Aperture, but haven't had the time to try to do
serious work with it. I like LR in part because it's X-Platform,
although I have no plans to switch.

Signature

John McWilliams

Ken Lucke - 13 Dec 2006 19:48 GMT
> Good heavens!

yeah - I SAID it was rigamarolish <g>.  Unfortunately, in the
beginning, I lost a dozen or so of the original .cr2 files (a couple of
which I had planned on being saleable, one being an absolutely gorgeous
shot of the inside of a 1st Order Fresnel lighthouse lens in
operation).  I was able to salvage the .tif files, though, and may
still be able to whip them into shape.  

After I got a set routine down for the Rube Goldberg model workflow
<g>, I never lost anything again.  But I'm sure glad I'm back to a
one-step process of import/storage.  :^) :^)

> I got into LR seriously when I got my 5D and all of a
> sudden had to use it to convert its RAW files, having only CS I.

The initial raw file conversion is slightly different in each -
Aperture's seems to be just a bit flatter than Lightroom's.  Just due
to the difference in raw conversion routines, I'm sure.  Both are
different yet from the Canon DPP software that comes from the camera,
which I would suppose would probably be the truest conversion, because
it's Canon's own. Take an unmodified TIFF export from the same raw
image from all three programs and line them up side-by-side and there
is a noticable difference, but it's just minor tweaks in areas that I
probably would be tweaking anyway.

> I've
> played a bit with Aperture, but haven't had the time to try to do
> serious work with it. I like LR in part because it's X-Platform,
> although I have no plans to switch.

If nothing else, the stacks and the file management options alone are a
good reason to re-eval it.  Plus the light table feature, which LR
doesn't even approximate, either.  I like the GUI better, too - much
more flexible and configurable.

Maybe I should get Apple to pay me to be a commercial  :^)  I sound
like it; but it's one of the few programs of late that I can really get
enthused about, as IMO it stands head and shoulders above the
competition.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

John McWilliams - 13 Dec 2006 21:44 GMT
> Maybe I should get Apple to pay me to be a commercial  :^)  I sound
> like it; but it's one of the few programs of late that I can really get
> enthused about, as IMO it stands head and shoulders above the
> competition.

And, I say, why not? You maybe should send a nice email to marketing....

And, yes, it's a good thing to get enthused about new software that does
good things. Both programs will further processing for all of us,
whether we adjust most images, or simply do a conversion at default
settings.

Signature

John McWilliams

JC Dill - 14 Dec 2006 21:32 GMT
>I like Aperture, for several reasons:
>
>Moved "referenced" files can be reconnected to their sidecar data very
>easily.  "Managed" files are EXTREMELY hard to screw up - to delete
>them, you literally have to tell the program to do so AND dismiss TWO
>warning dialogs that pop up in your face to warn you.

I made a deposit at the bank last month, and accidentally typed an
extra 0 in the amount.  I saw my mistake just as I clicked the "are
you sure" button.  This "are you sure" button is useless - I'm so used
to entering the amount, then clicking the enter button then the "are
you sure" button *every* time I make a deposit.  So, the "are you
sure" button simply becomes an extra and *unnecessary* step in the
routine process, and it interferes with the routine transaction while
providing nothing of value when the transaction is unusual.  (When I
don't really want to delete the files, or when the deposit was keyed
in error.)

>If you do need to actually edit on the pixel level (as opposed to
>overall things like exposure, levels, white balance, saturation, etc.,
[quoted text clipped - 3 lines]
>to the same "stack"* as the original - all without ever editing the
>original file at all - Aperture even opens the editing app for you.

Unless you accidentally do a "save as" in your editing program (such
as in Photoshop).  At which point you can't save the image back where
Aperture expects it, and the edited version of the image can't be
found in the Aperture library.

> You can also have more than one ".aplibrary" for
>use in Aperture, which means you can have completely separate libraries
>for different users or different reasons (if separating by folders or
>projects isn't enough).

Such as when your image library is greater than the number of images
Aperture can index in a single library, eh?

These are some of the reasons I am yet to be convinced that Aperture
is "all that".

jc

Signature

"The nice thing about a mare is you get to ride a lot
of different horses without having to own that many."  
    ~ Eileen Morgan of The Mare's Nest, PA

Ken Lucke - 17 Dec 2006 06:40 GMT
> >I like Aperture, for several reasons:
> >
[quoted text clipped - 13 lines]
> don't really want to delete the files, or when the deposit was keyed
> in error.)

Well, you can't programout human error totally.  If you respond to all
dialogs automatically, you might as well turn them off.  If youu want a
computer program totally immune to YOU screwing up (and I have, too, so
I'm not pointing fingers), then shut the computer off and play cards.
Stuff happens.

> >If you do need to actually edit on the pixel level (as opposed to
> >overall things like exposure, levels, white balance, saturation, etc.,
[quoted text clipped - 8 lines]
> Aperture expects it, and the edited version of the image can't be
> found in the Aperture library.

So, you import it in and stack it right back with the others, and
delete the one you "exported" to photoshop.  Geeze, it's not life or
death, nor rocket science.

> > You can also have more than one ".aplibrary" for
> >use in Aperture, which means you can have completely separate libraries
[quoted text clipped - 3 lines]
> Such as when your image library is greater than the number of images
> Aperture can index in a single library, eh?

So far, I have yet to hit a limit, and haven't seen any published
limit.  Have you?

> These are some of the reasons I am yet to be convinced that Aperture
> is "all that".
>
> jc

::shrug::  I'm not trying to convert everyone.  If you want to continue
to deal with photos piecemeal and individually, thhat's youur
perogative, and I won't try to argue you out of it.  I prefer a more
managed approach.

Signature

You need only reflect that one of the best ways to get yourself a
reputation as a dangerous citizen these days is to go about repeating
the very phrases which our founding fathers used in the struggle for
independence.
       -- Charles A. Beard

Paul Furman - 13 Dec 2006 05:22 GMT
> I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via
> a card reader built into my computer. So now the RAW files are saved.
[quoted text clipped - 13 lines]
>  also lots of images to be named, sorted, rated, named and cataloged.
> Any comments, suggestions would be appreciated.

Here's what I keep:

1) Raw file as DNG (makes it a bit smaller & more compatible)
2) JPEG saved at 9/10 quality (just fine if you don't edit again)
3) original jpegs (for reference and cause it's got all the original
metadata that might get lost in a DNG conversion)

The edits... who knows, I can probably redo them better in the future
from raw, PSDs are just too big.
jeremy - 13 Dec 2006 15:19 GMT
>> I use a Nikon D70 in RAW mode. I transfer my images to my hard drive via
>> a card reader built into my computer. So now the RAW files are saved. But
[quoted text clipped - 23 lines]
> The edits... who knows, I can probably redo them better in the future from
> raw, PSDs are just too big.

You probably have some protection if you keep your image files in
uncompressed TIF, because that appears to be the current de facto standard,
and there is protection in numbers.

JPG format may give you less size, but it does come at a price.  You can
never get back the discarded image information once JPG compresses it (JPG
2000 has lossless compression, making it a safer choice, but there is not a
lot of support for it.  Same with PNG.)

I'd bet that today's RAW formats will be virtually unreadable in 20 years
without specialized conversion emulator software, and there is no guarantee
that anyone will develop such an application.  It may not have a lot of
potential to be profitable.
achilleaslazarides@yahoo.co.uk - 13 Dec 2006 15:34 GMT
> I'd bet that today's RAW formats will be virtually unreadable in 20 years
> without specialized conversion emulator software, and there is no guarantee
> that anyone will develop such an application.  It may not have a lot of
> potential to be profitable.

I don't know, unless something dramatic happens, I'd bet that people
will still be using Unix machines and that gcc will run on them. So if
you have a copy of dcraw, you can compile and use that. Of course, who
knows if dcraw still can be found somewhere then: it seems to be a
single-person effort, and the code seems too messy to be easily
maintainable by others. But I'm not a programmer, so what do I know?.
jeremy - 13 Dec 2006 15:56 GMT
>> I'd bet that today's RAW formats will be virtually unreadable in 20 years
>> without specialized conversion emulator software, and there is no
[quoted text clipped - 8 lines]
> single-person effort, and the code seems too messy to be easily
> maintainable by others. But I'm not a programmer, so what do I know?.

The prospect is, not that it will be IMPOSSIBLE to read old media and
formats, but that it will be so inconvenient to do so that many of the
stored images will be discarded rather than sent to a university or
specialty data archeologist for decoding.

It is somewhat similar to being given an Edison cylinder.  Sure you can find
someone that has equipment to play it, but will you bother?
achilleaslazarides@yahoo.co.uk - 13 Dec 2006 16:17 GMT
> >> I'd bet that today's RAW formats will be virtually unreadable in 20 years
> >> without specialized conversion emulator software, and there is no
[quoted text clipped - 13 lines]
> stored images will be discarded rather than sent to a university or
> specialty data archeologist for decoding.

Yes, I understand what you are saying. Maybe in 100 years. But, to
repeat myself, I think that in 20 years, variants of Unix will be as
commonplace as today. And gcc will also be available. You may disagree
with these two beliefs, of course, but this is what I think. And if
this is true, and you have dcraw.c somewhere, you'll be able to compile
it (perhaps with "a little" work, and if you have the necessary
libraries!). I agree that this may not be as convenient as just using
ACR (or the equivalent), but if I personally found my raw files
unreadable, I'm 100% sure I'd sit down and do battle with the code
until it does compile in then current compilers. And I don't really
think that the then current raw converters will not read today's raw
formats (but maybe they won't, I don't know, obviously).

Well, if CDs and DVDs are unreadable in 20 years, or if the portable
HDs on which your files are archived either aren't compatible with the
current technology or stop working, then you have indeed a problem. But
I have a 16 year old hard disk that still does work.

Now, if we're talking about 100 years, then yes, I'm sure there will be
much more serious problems than this. Just not in 20.

> It is somewhat similar to being given an Edison cylinder.  Sure you can find
> someone that has equipment to play it, but will you bother?

OK, if your point is that it may be inconvenient, then I agree. But
we're talking about longer timescales than 20 years.

I think, though, that the point is that the only way to be reasonably
sure that everything stays readable etc is to keep everything moving,
update it with current technology (and maybe to current formats) and so
on.
Paul Furman - 13 Dec 2006 16:37 GMT
>>Here's what I keep:
>>
[quoted text clipped - 14 lines]
> 2000 has lossless compression, making it a safer choice, but there is not a
> lot of support for it.