Card formatting

**UncleVanya** · 09-24-2015, 08:55 PM

Originally posted by K-Three

I think "Computer Standards" is an oxymoron

Wasn't RAW supposed to be RAW on all devices?
Yet we need updated software to make the Adobe products (who wrote the "standard") with each new camera, even from the same camera manufacturer.
There are lots of other examples

Actually RAW is quite different. That is an extensible standard. File formats - drive formats have to be very strictly enforced or they don't offer portability. Otherwise when you format in the computer you can't read or write in the camera or vice versa. The file allocation table, the way the bits are laid down has to be the same or nothing but the device it is originally formatted in can read or write it.

That said, perhaps some cameras had glitchy implementations that were just close enough that the computers could read the cards but only the cameras could format them or the camera's couldn't read them. I know I saw odd issues with my older point and shoot cameras where sharing a card between multiple cameras could be odd.

stevebrot · 09-24-2015, 09:47 PM

Originally posted by UncleVanya

Any idea why? FAT32 should be FAT32 should be FAT32.

With SDXC cards, it ain't FAT32. Instead, it is a Microsoft proprietary format called exFAT.

That is part of the problem. The other that affects SD and SDHD as well is one of residual filesystem structures (particularly with OS X) that may persist after an unclean unmount. Those may still be present even after a reformat using Windows. That is where the utility from the SD Association comes in. It does a nice job of making a clean sweep of a corrupted card. Oh...and one more thing. There are aspects of the format process that are specific to SD. Those are covered briefly on the SD Assoc Web site.

https://www.sdcard.org/downloads/formatter_4/index.html

https://en.wikipedia.org/wiki/ExFAT

Steve

stevebrot · 09-24-2015, 09:49 PM

Originally posted by K-Three

Wasn't RAW supposed to be RAW on all devices?

No...RAW is raw as in carrots, broccoli and hamburger before they are cooked. All may be raw, but in very different ways. There is no RAW standard (no not even DNG*) and never has been. The RAW files from the different makers have always been proprietary.

Steve

* DNG is not a standard either. Rather, it is an Adobe-controlled format that has a published specification.

**UncleVanya** · 09-24-2015, 10:03 PM

Originally posted by stevebrot

With SDXC cards, it ain't FAT32. Instead, it is a Microsoft proprietary format called exFAT.

That is part of the problem. The other that affects SD and SDHD as well is one of residual filesystem structures (particularly with OS X) that may persist after an unclean unmount. Those may still be present even after a reformat using Windows. That is where the utility from the SD Association comes in. It does a nice job of making a clean sweep of a corrupted card.

https://www.sdcard.org/downloads/formatter_4/index.html

https://en.wikipedia.org/wiki/ExFAT

Steve

Thanks. That's helpful. I know about the unclear dismount on Macs been there.

Didn't know SDXC used exFAT - interesting.

**bkpix** · 09-25-2015, 06:14 PM

I used to use Windows7 (happily switched last summer to Linux), and it always gave me a message saying the SD card plugged in the reader needed to be formatted. I just ignored the message, downloaded the card, put it back in the camera for formatting, and everything worked fine.

One nice thing about Linux, let me add, is the much lower rate of irritating messages like that.

**K-Three** · 09-25-2015, 07:27 PM

Originally posted by stevebrot

There is no RAW standard (no not even DNG*) and never has been

I did mean DNG, not RAW , my bad, each camera and sensor has its own RAW format off the chip

Here is that specification, Sure sounds like a standard to me, if the intent is to have these files readable in perpetuity, by multiple softwares (Adobe makes several references to archive needs in the spec), then I have to think this is a standard that those who claim to be writing/reading DNG files should follow. If the standard (spec) is not followed by the camera makers and the software writers then the archival nature of the DNG file is useless.

stevebrot · 09-26-2015, 02:26 PM

Originally posted by K-Three

Sure sounds like a standard to me

Thanks for the link. I am familiar with the DNG spec. Standards and specifications are two different things. The difference is subtle, but real. A specification is a set of rules, attributes, and descriptions. A standard is a specification that has been codified and is maintained by a standards body. In my line of work (software development) there are many specifications and some standards. Examples of the former would be FAT32 filesystem, TIFF image format, and the DNG specification. Examples of the later would include HTML 5 (W3C), film speeds (ISO), and ANSI C programming language. Out in far left field are working metrics that we use like standards, but are neither standards nor specifications. A good example would ISO-equivalent sensor sensitivity settings on digital cameras where standards exist, but industry practice is sketchy.

In the case of DNG, while the specification is published, it is entirely controlled by Adobe and serves as a point of integration for Adobe products. The pace and nature of change of the DNG specification is determined by the functional requirements of Adobe products, which in turn is driven by their market analysis and business plan. Broad adaptation of DNG is good for Adobe, but of no real benefit to adapters beyond a generalized better chance of compatibility with Adobe products*. There are no performance or compliance specs for DNG implementations nor are there reference implementations or specific guidelines for DNG readers.

Steve

* Adobe products should always be able to read a valid DNG, though support for various features may be missing. This issue has cropped up several times in the past (starting with the K10D) and is generally fixed with release of an updated version of Adobe Camera Raw (ACR). DNG support outside Adobe is spotty and unpredictable. Poor cross-compatibility is a weak point in the specification. A good example might be Pentax cameras and DCU software. Both support Pentax-generated DNG, but neither are able to ready Adobe-generated DNG, even those converted from Pentax PEF. In both cases, the DNGs are fully compliant, but the Adobe-generated files make Pentax software barf. I am not sure about other vendors. Interestingly, both are readable, after a sort, by TIFF/EP readers. Go figure...

Last edited by stevebrot; 09-26-2015 at 03:18 PM.

09-24-2015, 09:47 PM	#17
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by UncleVanya Any idea why? FAT32 should be FAT32 should be FAT32. With SDXC cards, it ain't FAT32. Instead, it is a Microsoft proprietary format called exFAT. That is part of the problem. The other that affects SD and SDHD as well is one of residual filesystem structures (particularly with OS X) that may persist after an unclean unmount. Those may still be present even after a reformat using Windows. That is where the utility from the SD Association comes in. It does a nice job of making a clean sweep of a corrupted card. Oh...and one more thing. There are aspects of the format process that are specific to SD. Those are covered briefly on the SD Assoc Web site. https://www.sdcard.org/downloads/formatter_4/index.html https://en.wikipedia.org/wiki/ExFAT Steve Last edited by stevebrot; 09-24-2015 at 10:05 PM.

09-24-2015, 09:49 PM	#18
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by K-Three Wasn't RAW supposed to be RAW on all devices? No...RAW is raw as in carrots, broccoli and hamburger before they are cooked. All may be raw, but in very different ways. There is no RAW standard (no not even DNG) and never has been. The RAW files from the different makers have always been proprietary. Steve DNG is not a standard either. Rather, it is an Adobe-controlled format that has a published specification. Last edited by stevebrot; 09-24-2015 at 09:56 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
One more K-3 FW bug - 128GB card, file system & formatting.	Prakticant	Pentax K-3 & K-3 II	9	06-11-2015 08:07 AM
Pentax k-x SD Card Formatting Help	RealPotato	Troubleshooting and Beginner Help	6	02-10-2015 09:05 PM
Camera memory card data is lost for formatting. Need your help!	Jona373	Troubleshooting and Beginner Help	2	03-20-2014 07:54 AM
SD Card formatting?	Brownie4u	Digital Processing, Software, and Printing	5	05-23-2013 02:23 PM
SD card formatting	fredok1	Pentax DSLR Discussion	29	04-01-2012 06:02 AM

09-24-2015, 08:55 PM	#16
UncleVanya Loyal Site Supporter Join Date: Jul 2014 Photos: Gallery \| Albums Posts: 28,451	Originally posted by K-Three I think "Computer Standards" is an oxymoron Wasn't RAW supposed to be RAW on all devices? Yet we need updated software to make the Adobe products (who wrote the "standard") with each new camera, even from the same camera manufacturer. There are lots of other examples Actually RAW is quite different. That is an extensible standard. File formats - drive formats have to be very strictly enforced or they don't offer portability. Otherwise when you format in the computer you can't read or write in the camera or vice versa. The file allocation table, the way the bits are laid down has to be the same or nothing but the device it is originally formatted in can read or write it. That said, perhaps some cameras had glitchy implementations that were just close enough that the computers could read the cards but only the cameras could format them or the camera's couldn't read them. I know I saw odd issues with my older point and shoot cameras where sharing a card between multiple cameras could be odd.

09-24-2015, 10:03 PM	#19
UncleVanya Loyal Site Supporter Join Date: Jul 2014 Photos: Gallery \| Albums Posts: 28,451	Originally posted by stevebrot With SDXC cards, it ain't FAT32. Instead, it is a Microsoft proprietary format called exFAT. That is part of the problem. The other that affects SD and SDHD as well is one of residual filesystem structures (particularly with OS X) that may persist after an unclean unmount. Those may still be present even after a reformat using Windows. That is where the utility from the SD Association comes in. It does a nice job of making a clean sweep of a corrupted card. https://www.sdcard.org/downloads/formatter_4/index.html https://en.wikipedia.org/wiki/ExFAT Steve Thanks. That's helpful. I know about the unclear dismount on Macs been there. Didn't know SDXC used exFAT - interesting.

09-25-2015, 06:14 PM	#20
bkpix Site Supporter Join Date: Feb 2010 Location: Creswell, Oregon Photos: Albums Posts: 570	I used to use Windows7 (happily switched last summer to Linux), and it always gave me a message saying the SD card plugged in the reader needed to be formatted. I just ignored the message, downloaded the card, put it back in the camera for formatting, and everything worked fine. One nice thing about Linux, let me add, is the much lower rate of irritating messages like that.

09-25-2015, 07:27 PM	#21
K-Three Loyal Site Supporter Join Date: May 2014 Location: Pugetopolis, WA Photos: Gallery Posts: 938	Originally posted by stevebrot There is no RAW standard (no not even DNG*) and never has been I did mean DNG, not RAW , my bad, each camera and sensor has its own RAW format off the chip Here is that specification, Sure sounds like a standard to me, if the intent is to have these files readable in perpetuity, by multiple softwares (Adobe makes several references to archive needs in the spec), then I have to think this is a standard that those who claim to be writing/reading DNG files should follow. If the standard (spec) is not followed by the camera makers and the software writers then the archival nature of the DNG file is useless.

09-26-2015, 02:26 PM	#22
stevebrot Otis Memorial Pentaxian Join Date: Mar 2007 Location: Vancouver (USA) Photos: Gallery \| Albums Posts: 42,007	Originally posted by K-Three Sure sounds like a standard to me Thanks for the link. I am familiar with the DNG spec. Standards and specifications are two different things. The difference is subtle, but real. A specification is a set of rules, attributes, and descriptions. A standard is a specification that has been codified and is maintained by a standards body. In my line of work (software development) there are many specifications and some standards. Examples of the former would be FAT32 filesystem, TIFF image format, and the DNG specification. Examples of the later would include HTML 5 (W3C), film speeds (ISO), and ANSI C programming language. Out in far left field are working metrics that we use like standards, but are neither standards nor specifications. A good example would ISO-equivalent sensor sensitivity settings on digital cameras where standards exist, but industry practice is sketchy. In the case of DNG, while the specification is published, it is entirely controlled by Adobe and serves as a point of integration for Adobe products. The pace and nature of change of the DNG specification is determined by the functional requirements of Adobe products, which in turn is driven by their market analysis and business plan. Broad adaptation of DNG is good for Adobe, but of no real benefit to adapters beyond a generalized better chance of compatibility with Adobe products. There are no performance or compliance specs for DNG implementations nor are there reference implementations or specific guidelines for DNG readers. Steve Adobe products should always be able to read a valid DNG, though support for various features may be missing. This issue has cropped up several times in the past (starting with the K10D) and is generally fixed with release of an updated version of Adobe Camera Raw (ACR). DNG support outside Adobe is spotty and unpredictable. Poor cross-compatibility is a weak point in the specification. A good example might be Pentax cameras and DCU software. Both support Pentax-generated DNG, but neither are able to ready Adobe-generated DNG, even those converted from Pentax PEF. In both cases, the DNGs are fully compliant, but the Adobe-generated files make Pentax software barf. I am not sure about other vendors. Interestingly, both are readable, after a sort, by TIFF/EP readers. Go figure... Last edited by stevebrot; 09-26-2015 at 03:18 PM.