EXIF metadata stores random gibberish for dates & times

I hadn’t ’til yesterday realised that EXIF metadata doesn’t actually store dates & times correctly.  Whoever came up with the spec all those decades ago clearly didn’t know how to work with dates & times correctly.  This is immensely frustrating since now we have countless images taken with timestamps that are collectively gibberish.

The problem is that the standard doesn’t specify time zones in almost all cases (the sole exception being for GPS timestamps, which are in UTC).  Which means if you see the date & time “2016-02-03T10:36:33.40” in your photo, that could be any actual time give or take ~25 hours to either side of that.

I realise now, in hindsight, that programs like Aperture & Lightroom manage this by implicitly associating a time zone with photos as they’re imported (and both have controls of varying degrees for ‘correcting’ the time of the photos, in cases where the camera’s clock is set wrong – including being set to the wrong time zone).  They leave it to the user to ensure the time zone that’s set for import matches what was on the camera at the time the photos were recorded.

However, if you’re processing images at scale and don’t have that explicit information from the user(s), you’re SOL.

Additionally, I don’t know anyone with a DSLR who hasn’t at least occasionally forgotten to change the date & time on their camera to account for changes in daylight savings time, or movement to a new time zone.  If the time zone were recorded, this wouldn’t really matter since you could reliable change it later.  But since it’s not, it’s impossible to tell programatically when and where the time zone changes, in a given series of photos.

Now, you might think that since the GPS timestamp is actually recorded as a real, definitive time, that you could just use that to determine the time zone of other dates & times in the metadata (by simply looking at the difference between them).  Unfortunately, in this case, the GPS timestamp is defined as the time at which the GPS data was recorded, not when the photo was created (or edited, or any of the other types of timestamps recorded in EXIF metadata).  Which means that in practice the GPS timestamp can be an unspecified & unpredictable amount of time older than the other timestamps1.

If it were just a matter of a few minutes difference then this wouldn’t be an issue, since the vast majority of the world only acknowledges half hour increments in time zone steps2 and thus you could just round and get things right most of the time.  Unfortunately, at least some notable GPS implementations in popular cameras have potentially huge deltas (hours or more) – e.g. all of Nikon’s SnapBridge cameras, including the D500, D5600, & D3400.

  1. And that’s assuming the camera’s clock isn’t set wrong anyway – it’s possible to include GPS data in your photos but not sync the camera’s clock, in at least some popular cameras like Nikon’s.
  2. Wikipedia reports that there are a couple of small regions of Australia & New Zealand which use 15 minute offsets, and the entirety of Nepal does too, but those are the only exceptions.  And only a small minority use half hour offsets, as opposed to hour offsets, to begin with.

Lightroom’s flaws & limitations

I’m attempting to switch to Lightroom, from Aperture, given lack of better alternatives. I’ve attempted this switch a couple of times before, without success.

I’m finding that it’s still kind of annoying.  Here’s the start of my laundry list against it:

  • It’s very slow.  I am already so very tired of seeing that blasted “Loading…” floater.
    • I’m running it off of a pair of SATA-connected SSDs – one for the library, one for the originals.  It has enough usable bandwidth to read in dozens of photos a second.  But it still takes three to five seconds to show just one.
  • It’s also a bit slow to actually show adjustments as you make them, a lot of the time.  Aperture, despite not being updated for many years to take advantage of all sorts of new technologies, is still much more responsive and fluid, and consequently faster to use.
    • I did notice that Capture One is also a bit sluggish sometimes, in this respect.  Sad panda – everyone’s screwing this up, it seems.
  • When you scroll around in the grid view, it’s a bit janky – see prior point on Lightroom just generally being sluggish – but even worse, it doesn’t show the photo’s metadata reliably.  A lot of it is invisible for a while, ‘loading in’ some time later.  It’s really annoying to have to frequently wait for Lightroom to get around to actually showing me the grid view properly.  And it’s also very distracting to see the metadata popping in at random intervals for random photos.
  • Its search & filtering tools are kinda piss-weak.  e.g. you can do a text search of metadata for only one thing at a time.  And you can’t even specify what metadata you want to search, with very few exceptions.
  • As I’d shown previously, its RAW renderer is not very good.  Image quality out of the box is very poor – mainly that even bright, ISO 100 shots have unacceptably high noise visible and require strong noise reduction.  This is unnecessary in every other RAW renderer I’ve ever used (Aperture wasn’t great either, granted, but at least it erred on the side of not noisy-as-heck by default; it’s much easier to add sharpening as necessary than to have to noise reduce every time).
    • And its noise reduction tool isn’t great either – much like Aperture’s, it’s ham-fisted and often requires painstaking brush-work to selectively apply it to each photo.
  • Its sharpening tool is not very good either.  It introduces grittiness and nasty edge haloes really quickly, relative to how much actual sharpness & detail it’s revealing.  Now I understand why so many Lightroom users seem to use 3rd-party sharpening (and noise reduction) plug-ins.
    • On the upside, vs Aperture, it does let you set a detail-sensitive ‘mask’, which helps reduce brushwork somewhat.  Unfortunately it lacks the corresponding slider for its noise reduction tool, which reintroduces the need to do painstaking brushwork.
  • Its system for managing brush masks is surprisingly awkward.  Aperture was much more straightforward (though not without its own foibles, like not letting you share masks between adjustment tools).  e.g. to see an overlay of where the mask actually is, you have to however the mouse over a tiny little dot – and there may be several on the photo, if you’ve added several different masks, with no way to distinguish them other than their arbitrary locations.  And worse, you have to wait a couple of seconds, every time, for the mask overlay to actually appear.  And then as soon as you move off the dot – e.g. to actually modify the mask – it disappears!  Ridiculous!
    • Capture One has the best implementation of this that I’ve seen, so far.  There’s a couple of things about Capture One’s that I find a little awkward, compared to Aperture’s, but overall it’s still the best.
  • It’s possible to have filters applied but the filter bar hidden.  Very confusing – it took me quite some time to figure out why a bunch of photos simply weren’t showing up.  It’d make much more sense, IMHO, to have the ‘hide the filter bar’ option only apply when there aren’t any filters actually applied.  Though I do recognise the counter-opinion.
  • Deleting photos is unnecessarily difficult.  If you’re viewing a collection, there’s no way to do it directly.  The least sucky way I’ve found is to mark target photos as ‘Rejected’, and then go to the “All Photographs” view and hit command-delete.  Lots of unnecessary steps and distracting from the workflow.
  • Its built-in Flickr export forces all photos to sRGB.  Sad, blunt fail.
    • There is a popular third-party plug-in, Photo Upload, which I’m using for now, but I keep finding it rubs me a bit the wrong way.  Hopefully just a matter of getting used to it – FlickrExport for Aperture, which I’d relied on previously, wasn’t perfect either.  And on the upside, the author seems very responsive and courteous, which is refreshing and encouraging.
  • Import from SD cards is a little sluggish.  Not terrible – I’m talking ~60 MB/s on a card & reader combo capable of ≥80 MB/s – but it’s still a bit disappointing.  Aperture is faster.
  • Its metadata editing interface is certainly better than Capture One’s horrible & buggy one, but still a far cry from Aperture’s.  I miss being able to customise the metadata view, but beyond that, even basic things are a bit awkward, like the ‘Title’ and ‘Caption’ fields being tiny and thrown arbitrarily down towards the end of the list.
    • Though it does have one handy thing – clicking on the field name shows a pop-up menu listing recently used values, which is actually a thoughtful touch.  Auto-completion is also available, but it’s nice to avoid the keyboard <-> mouse shift.
  • I still don’t like its modality.  I find I have to shift between ‘Library’ and ‘Develop’ modes a lot, which’d be tedious enough even without the fact that a whole bunch of keyboard shortcuts are different between the two.  Super annoying.  I keep hitting keys expecting some predictable result, and instead just get NSBeeps, or totally undesired effects, or worse – just a flash of a menubar item with no indication of what the #%@! just happened, or how to undo it.  Gah!

And I could go on.  And probably will in future – I need somewhere to vent and mind-bogglingly Adobe offer no way to actually file bug reports & feature requests with them directly.

I really get the impression that Adobe haven’t paid due attention to their [former] competition, Aperture.  There’s just so many things that Aperture has done better for nearly a decade now.  If they were holding off ‘copying’ it out of some sense of honour, that’s respectable, but the time for it has long passed, given Aperture’s demise.

Raw editor comparison – Shadows

This is a follow-up to my previous Raw converter comparison – you can see that post for info on things like my motivation, basic processing methods, etc.

The goal in this case is just to test how well each raw editor – Aperture 3.6, Capture One 9 Pro, DxO Optics 10, and Lightroom CC 2015.3 – can lift an underexposed photo.  For pedants, note that I’m specifically applying an overall exposure adjustment, not just shadow recovery, though I expect that fundamentally any ‘Shadow’ sliders in these editors are just a strict subset of their ‘Exposure’ sliders.

Image format

If you cannot see the images – they’re missing or appear as “broken links” – then try changing image formats:

JPEG2000 (best – least bandwidth used & shortest load times)
PNG (second best)
TIFF (worst)
Preload all images (warning: may be many tens of megabytes!)

First, here’s the test image as it first appears when viewed in each of the contenders:

Some differences in how each raw converter operates by default, unsurprisingly. But clearly the photo is massively underexposed. In case you’re wondering, this was just setting up for a macro shoot, and for whatever reason the flashes didn’t fire in this exposure.

And below is the edited version. My goal here was to bring the photo up to a broadly ‘normal’ or ‘correct’ exposure. At first I assumed this would just mean some ‘Exposure’ adjustment applied identically between the four contenders, but it quickly became apparent that their adjustment tools just aren’t equivalent like that.

For reference, the adjustments made for each were:

  • Aperture 3.6 – +6 exposure
  • Capture One 9 Pro – +4 exposure, +35 brightness
  • DxO Optics 10 – +3 exposure
  • Lightroom CC 2015.3 – +5 exposure

The additional use of the ‘Brightness’ slider in Capture One was necessary because its ‘Exposure’ slider is hard-limited to ±4, obnoxiously.

Clearly one of these things is not like the other. Or, put another way:

What-The-Fuck-Are-You-Doing-(Aperture)

Here’s a zoomed in view of a representative part of the image:

So, Aperture… enough said.

Amongst the other three, there’s lots of room for opinion. DxO Optics has applied much more noise reduction, to the point where it’s quite obvious and (in my opinion) a bit over-done. Capture One and Lightroom have done a similar job, though I think Capture One has done genuinely better at suppressing the banding, and the noise in certain areas – particularly the midtones.

Now, this section of the image is out of focus. A necessary question – given it appears the main difference is simply in the noise reduction being applied – is how well genuine detail is retained. So here’s a second 100% clipping from the image:

DxO Optics happens to have retained the highlights a bit better, which is fine but remember that I didn’t bother recovering highlights at all, which I’m sure all of them would be able to do just fine (since ‘recovery’ in this sense means merely not so massively lifting them out of shadow).

There’s not much else to say here, though. Aperture is incredibly horrible. And the other three behave much the same as we’ve seen before:

  • DxO Optics has gone for very strong noise reduction by default, which has actually rendered the small in-focus area not too badly. But really softened everything else, in a slightly blotchy "I’ve been noise reduced" way.
  • Capture One has a more subtle look, that’s actually a little crisper than DxO Optics’, but has significantly more noise and banding visible.
  • Lightroom goes even further, with a very noisy and very banded rendition that can perhaps give an illusion of extra detail, though there actually isn’t anything there that Capture One & DxO Optics don’t also reveal.

Again, it looks like the difference is primarily in noise reduction. I’m not going to try to ‘equalize’ their noise reduction settings – from experience that’s highly subjective – but I suspect you could ultimately get similar results, to suit your own taste, from any of Capture One, DxO Optics, or Lightroom.

But you’ll never get anything usable out of Aperture, from this kind of scenario. That one’s dead, Jim.

An update on Aperture

“noirdesir” on DPReview’s forums (and Jim M here in the comments) identified Aperture’s problem: that it’s got a fixed blackpoint which is preventing it from actually bring up any real shadow detail.

Unfortunately I wasn’t expecting to add to this post, so I didn’t save a few key tidbits like the crop coordinates for the two 100% views. But FWIW here’s broadly what you can get out of Aperture if you also adjust the ‘Black Point’ alongside ‘Exposure’:

Aperture with black point adjustment

Aperture’s rendition still isn’t as good as any of the other three – it’s much noisier, even than Lightroom’s, and the deepest shadows are still somewhat clipped. The latter may be an imperfect ‘Black point’ setting, though I did spend quite some time playing with it and this was the best that I could seem to get from it, w.r.t. both overall image quality and actually getting roughly the desired exposure.

So after all these years and many images given-up on, it turns out Aperture can actually recover underexposed images to a reasonable degree. It’s just far more fiddly than any of the other raw editors – you have to adjust the ‘Exposure’ slider a little, then the ‘Black point’, and then repeat numerous times to narrow in on the right complementary settings. But even then, the point remains that it still doesn’t do as good a job of it as Capture One, DxO Optics, or Lightroom.