[hydroclops]

I did some research and THINK I know more about my K-x motion-jpeg files than is in any documents. After trying to analyse the files with various brands of movie inspector software, I tried JPEGsnoop. This works on jpegs and AVI motion-jpegs. JPEGsnoop is free.
It gives up a lot of info, most of which I don't understand. BUT I've surmised that the K-x uses 4:2:0 chroma subsampling.

There appear to be 256 levels of pixel brightness (not 16-235 for example)....

After learning about the chroma, My next concern would be preserving all the brightness levels when transcoding.

I'm on a PC with Windows XP pro, running Adobe Production Premium CS4, in particular Premier, After Effects, and Photoshop.

I have the Morgan Mutimedia MJPG codec installed. Clips play well in Premiere, but not so great in AE. I'm considering using Premiere for OFFline editing and then using the original files for any effects, grading, and for mastering in AE. Then rendering to TIFF sequence for a master file that can be used to render down to DVD or web video.

So, my questions are:

1) Why do so many video makers on the interweb forums immediately transcode their original camera files? Presumably it's to enable their NLEs to be able to play and make edits and apply effects easily. But do people know what they're throwing away when they do this?

2)When you transcode, don't you need to know what is in your camera original?

3) Isn't there an offline/online workflow in digital video post production?




I only sort of know what I'm doing. any comments would be greatly appreciated.

---

Last edited by hydroclops; 01-27-2010 at 08:56 AM. Reason: I removed false info about jpegsnoop

[kmlo1984]

I did some research and THINK I know more about my K-x motion-jpeg files than is in any documents. After trying to analyse the files with various brands of movie inspector software, I tried JPEGsnoop. This works on jpegs and AVI motion-jpegs. JPEGsnoop is free but installs adware/spyware. It seems OK and looks like it can be uninstalled easily.


It gives up a lot of info, most of which I don't understand. BUT I've surmised that the K-x uses 4:2:0 chroma subsampling.

There appear to be 256 levels of pixel brightness (not 16-235 for example)....

After learning about the chroma, My next concern would be preserving all the brightness levels when transcoding.



I'm on a PC with Windows XP pro, running Adobe Production Premium CS4, in particular Premier, After Effects, and Photoshop.

I have the Morgan Mutimedia MJPG codec installed. Clips play well in Premiere, but not so great in AE. I'm considering using Premiere for OFFline editing and then using the original files for any effects, grading, and for mastering in AE. Then rendering to TIFF sequence for a master file that can be used to render down to DVD or web video.

So, my questions are:

1) Why do so many video makers on the interweb forums immediately transcode their original camera files? Presumably it's to enable their NLEs to be able to play and make edits and apply effects easily. But do people know what they're throwing away when they do this?

2)When you transcode, don't you need to know what is in your camera original?

3) Isn't there an offline/online workflow in digital video post production?




I only sort of know what I'm doing. any comments would be greatly appreciated.

Hi thanks for the info, i m also looking for softwarem, particular on ISO.
I use virtualdub with morgan codec, chroma smoothen to 4:4:4 and temporal smoothen a bit and output to uncompress TGA or BMP

i think it;'s 8bit, but if u want to be better in post u may upres to 10bit.
it;s better when u correct it for less banding effect.
I think u can be sure change nothing if u r not doing levels or grading same format like Jpg, and read the both highlight and shadow in terms of SRGB.
in mac --digit color meter is handy.
most p[ost hse use DPX with LUT, so they can have multiple color reference test without saving again. but TIFF, BMP or TGA works too just bigger.

KM

[TrueIndigo]

Another good choice for sequence frames is PNG. RGB images can be 8-bit or 16-bit depth and both types can have an alpha channel. It uses a fully lossless patent-free compression system (re-saving will not degrade the image), which is among the best that can be had without losing data. Individual frame sizes vary according to image content; from my experience an 8-bit 1920 x 1080 frame could be anything from 1.08 MB to 2.57 MB. Another personal example of space saving with 1080p video: an uncompressed avi clip (duration: 1 minute, 2 seconds, 22 frames) was 9.11 GB, while the PNG sequence version (as 1,572 frames) was only 2.51 GB.

[adelena]

1) You've answered your own question. I am a video editor and I can tell you that some do and some don't "know what they're throwing away." Honestly, very few video editors work with footage from DSLRs. It is becoming more common though. As a professional, being able to edit in a native codec is where it's at for many, many different reasons that I will not go into here.

2) Not necessarily.

3) You've just posted an online/offline workflow. I'm not aware of any others but yours seems like a good pass at it. Let us know how it goes.

In the final analysis it's more about what the finished product looks like and less about how many levels of brightness are preserved in transcoding. Naturally all editors want to work with the best footage possible but reality is quite another thing.

I've been transcoding Pentax K-x avi's to 720p24 ProRes using Compressor via FCP. I can tell you that the resulting clips look great even on the HD monitor in my studio. Personally I feel that offlining H264 or, in our case, MJPEG AVIs is pointless and a waste of time. The essence of the file is not changed when transcoding the wrapper.

Andre

[hydroclops]

Thanks to everyone for the responses. The info is very helpful to my continuing understanding of all this.

My concern is based on two things: my previous experience with motion picture post production and the Digital Rebel Guide by Stu Mashwitz.

In both of the above, there is concern that any change to the camera original is a compromise that is best deferred to the final stage of post. In the case of digital files, when transcoding to even a lossless file, you can lose data, for example a lossless file may store luminence differently than the original, tossing data to fit the new file. Also when transcoding you may miss an opportunity to "upscale" something like chrominance, an opportunity that is best taken advantage of at this stage rather than later.

My plan is to shoot staged scenes with rehearsed camera moves and to use lights and to gel windows etc, in order to get good exposure on a "digital negative", then to do color correction in After Effects.

Yes, my concerns may be based on misundertandings and ignorance...

Thanks again

[impulse]

hydroclops wrote:

I did some research and THINK I know more about my K-x motion-jpeg files than is in any documents. After trying to analyse the files with various brands of movie inspector software, I tried JPEGsnoop. This works on jpegs and AVI motion-jpegs. JPEGsnoop is free but installs adware/spyware. It seems OK and looks like it can be uninstalled easily."

As the author of JPEGsnoop, I absolutely must point out that JPEGsnoop does NOT contain adware/spyware! This has been independently verified by numerous software directories, including CNET. If you have any concerns or questions, please contact me directly.

The program is installer-free (ie. portable) and therefore doesn't require installation or uninstall.

[hydroclops]

Yikes!

Sorry, I must have confused it with another program.

I edited my original post to remove my mistake.

I'll reiterate that JPEGsnoop is a good utility for working with motion jpeg files.

[osv]

In both of the above, there is concern that any change to the camera original is a compromise that is best deferred to the final stage of post.

that is not correct, i would suggest that you find other sources of information.

there are situations where a quality intermediate codec is preferable, including multi-layered rendering... in the case of fcp, it still can't edit native avchd, if there was a quality loss problem there because of the use of apple prores, you'd have heard about it a long time ago.

you should be a lot more worried about the lossy bloated mjpeg codec the camera is recording with, than the use of a quality intermediate.

[osv]

Personally I feel that offlining H264 or, in our case, MJPEG AVIs is pointless and a waste of time. The essence of the file is not changed when transcoding the wrapper.

there is no such thing as "transcoding the wrapper"... when you re-encode to an intermediate codec, it totally changes the entire video... the mjpeg codec is not a wrapper format, you can't put another codec inside of an mjpeg file.

here is some further info that will help you understand what a wrapper really is:
"A container or wrapper format is a meta-file format whose specification describes how data and meta-data is stored (not coded)."
http://en.wikipedia.org/wiki/Container_format_(digital)

[hydroclops]

...i would suggest that you find other sources of information.

Would you be my source?

you should be a lot more worried about the lossy bloated mjpeg codec the camera is recording with, than the use of a quality intermediate.

This IS my concern, that I have to start with a less than ideal camera original. What I want to do is transcode at the best place in the workflow, making sure I maintain and maximize whatever quality I have (am stuck with).

I could believe it might be to transcode to a TIFF image sequence as the very first step. Or maybe later, say after editing but before color correction or effects. Or maybe some other file format. I wish I knew.

Someone out there knows a good workflow that starts with mjeg files.

[Voytech]

My understanding was that if you transcode into a more robust codec, you won't loose information. Correct me if I'm wrong here.

[distudio]

there is no such thing as "transcoding the wrapper"... when you re-encode to an intermediate codec, it totally changes the entire video... the mjpeg codec is not a wrapper format, you can't put another codec inside of an mjpeg file.

here is some further info that will help you understand what a wrapper really is:
"A container or wrapper format is a meta-file format whose specification describes how data and meta-data is stored (not coded)."
http://en.wikipedia.org/wiki/Container_format_(digital)

Thanks guys, great info, this is all new territory for me (and I expect a lot of other primarily still photographers).

Cheers,

[adelena]

No, MJPEG is certainly not a container format but AVI is (per the article you linked to by the way.) In the Kx's case it contains MJPEG. There are innumerable formats that come as Quicktimes. I use IMX, ProRes and DVCPro50 footage every day and they are all Quicktimes. The Wikipedia article on Quicktime calls it a "proprietary multimedia framework" which to me is sort of like a container format (QuickTime - Wikipedia, the free encyclopedia) in that a Quicktime can contain many different formats or essences of information.

I did not mean to suggest that changing the wrapper or file format would leave the essence intact or untouched but that sometimes, in the real world, we have to make hard choices about "preserving levels of brightness" or moving to a format that is easily editable.

Hydro, Stu's book must have some good information on this, no? My first thought would be to move to ProRes (if you're editing in FCP) as soon as possible. This will allow you to send to Color or Motion with less rendering and just facilitate the overall FCP workflow. I've read, though I have no practical experience, that Premiere works with AVIs much better. Finally, if you're that concernced about the camera original, get another camera.

[osv]

No, MJPEG is certainly not a container format but AVI is (per the article you linked to by the way.) In the Kx's case it contains MJPEG. There are innumerable formats that come as Quicktimes.

i see your point about the kx avi container format, with an mjpeg codec file inside of it... transcoding would still totally change the file, even if the output was to avi again.

I did not mean to suggest that changing the wrapper or file format would leave the essence intact or untouched but that sometimes, in the real world, we have to make hard choices about "preserving levels of brightness" or moving to a format that is easily editable.

the "level of brightness" was a serious problem with quicktime only, to the best of my knowledge, it was never a problem with the avi container, or any other container format that i know of:

"As we see the problem at the moment:
- 5D Mark II writes signal in QuickTime H.264 MOV-file in its full quality.
- Apple QuickTime Player can adequately display these MOV-files.
- But most NLE,s (Edius, Vegas, Premiere, After Affects, ...) perceive these MOV-files in a distorted form (see attached image):
--- Shadows and Highlights is lost (red zone);
--- The subtle gradations is lost (gaps in the histogram).

It is pretty much a known problem 'quicktime gamma shift' (search Google). For example: DV - Columns - Brighter Whites; Richer Colors, Part 1

Either we are waiting for improvements in the NLE. Or need a utility that correctly converts MOV-files in a format suitable for NLE."
Problem with Canon 5D Mk. II video? at DVinfo.net



since we are still photographers who understand histograms, we know how serious of an issue that was... basically, you couldn't bring quicktime files into a pc editing environment... i think that apple finally fixed that huge bug in a recent and totally unheralded quicktime update.

cr*p like that is why i don't ever use quicktime or the mac platform, but if you are using it, you should be just fine with transcoding to the prores intermediate codec before editing... with imovie, i believe that you don't get prores, you'd have to use aic instead, which is decidedly inferior to prores... on the pc, i used to use the canopus intermediate codec with avchd, before i stepped up to the overclocked i7 920 cpu.

[Coby]

This discussion is kinda confused ... the OPs concern is certainly valid. For example, HDV and AVCHD footage both contain 'superwhites" - i.e., data above 100IRE. It's very easy to lose that data if the codec, NLE, or filters being used don't recognize them properly. (Not sure if this is the case in Pentax mjpeg...). This behaviour can be variable even among similar filters within a particular NLE.

Quote: 1) Why do so many video makers on the interweb forums immediately transcode their original camera files?

One important reason, in the context of this discussion, is that the transcoding can be done in such a way as to ensure all the info in the raw footage - the 'tails" in the histogram osv posted - is retained during editing. If the raw footage is edited directly, those tails might be lost ... depending on the choice of software.