Handbrake’s H.265 ‘Preset’ setting affects ‘constant’ quality

I always consternate over what the ‘Preset’ setting should be when doing H.264 encodes with Handbrake.  It’s always tempting to slide right on over to ‘placebo’ to, in theory, ensure you’ve got the best possible encoding.  And in my experience that at least roughly works – file size decreases (marginally) as you use more time-consuming encoding Presets, for equivalent quality.

Now that I’m transitioning to H.265 instead, I thought I’d do a ‘quick’ experiment to see how it behaves in comparison.  The result is baffling.

Handbrake H.265 file size vs Preset

PresetFile size (MiB)Encode time (Hours)
Placebo32916.65
Veryslow2183.70
Slower2192.75
Slow1830.93
Medium1850.53
Fast1560.36
Faster1550.31
Veryfast1550.31
Superfast1290.23
Ultrafast830.22
Encoding from ProRes 422 of a 4000x3000 timelapse, at constant quality 22. A 270° rotation was performed in the process. No audio.

 

The outputs should all be visually essentially identical, by virtue of constant-quality encoding with the same quality factor.  However, they are not – there’s a small but quite visible difference in quality, with the slower-encoded versions having progressively more detail retained, while at the other end, the faster encodings, it looks like a strong noise reduction pass has been applied (which isn’t literally what’s happened, I assume, but rather the consequence of a lower-quality encoding).

Regardless of the specifics, this shouldn’t be happening.  ‘Constant quality’ seems quite self-explanatory.

Note also how the encoding times go up exponentially – ‘placebo’ was really gruelling, as the equivalent H.264 encode would have been an order of magnitude shorter, and that’s what I’d budgeted my time from.

So I guess the moral of the story is… leave ‘Preset’ on its default value for now, until it stops misbehaving?  Choose it based primarily on your encode time constraints, or file size concerns – whichever is highest priority?  Frustrating.

Note: I used the Handbrake Nightly build as of mid-November 2017 (version 20171113130119-17a4bb7-master), as the latest released version (1.0.7) produces H.265 files that macOS High Sierra refuses to play.  Apparently it’s using a different codec tag – ‘hev1’ instead of ‘hvc1’ – from what Apple’s expecting (see for example this thread).  I have no idea which is correct, or maybe both are in some contexts… either way it’s a concern for ongoing H.265 device compatibility.

Lightroom “Classic” doesn’t play well with others

So far the new “Classic” Lightroom looks & feels mostly identical to the prior version(s), which isn’t really a compliment, but could be worse.  There’s no apparent performance improvements, that’s for sure, so as expected Adobe’s promises to suddenly learn how to write efficient & performant software, well… at least their marketing department gave it the college try.

One thing I have very quickly discovered, however, is that Lightroom “Classic” deliberately chooses not to perform some functions if it is le tired.  Or it thinks your computer is le tired.  By which I mean, if there is pretty much anything else running and consuming CPU time (and/or RAM?), it refuses to even attempt some operations.  HDR merges is the first one I hit.  I was a bit flummoxed by it just happily queuing up a number of HDR merge operations, and them just sitting there in its queue, with no indication of error – just never executing.

Only after I quit or disabled a bunch of other processes – any and all that were using any measurable CPU time – did it finally, about ten seconds later, decide that it was now willing to consider my ‘requests’.

#%@!ing fussy little turd.

It’s worth noting that it’s not the only popular app, on macOS, that does this same bullshit.  Time Machine is another big one.  At least in Time Machine’s case I can see a more plausible line of reasoning behind it, even if it is misguided – the user’s probably not explicitly waiting for a Time Machine backup to complete.  As in, not all the time.  Sometimes they are. And they certainly expect backups to happen at all, which on a consistently busy machine simply doesn’t happen.  So Time Machine’s reluctance to function on a working machine is still stupid overall.  But Lightroom refusing to complete a user initiated, user-interactive, and user-blocking operation, is just patently stupid by its very notion.

Update:  Worse, now it doesn’t work at all.  And a quick web search shows many other people having the same problem, and Adobe as usual doing nothing about it.

Incidentally, I tried to log in to Adobe’s forums in order to ‘Me too’ those issues, only it won’t let me log in anymore, falsely claiming my password is invalid.  Good job, Adobe, good job.

Your system has run out of application memory HUR HUR HUR

I hate this dialog with the fire of a thousand suns.

When this appears, it basically means one (or both) of two things:

  1. Some application went nuts and chewed through all your memory and/or disk space.
  2. macOS got itself into a darkly comical & embarrassing deadlock.

Quitting any of the listed applications is rarely the correct move.  It’s often enough the case that none of them are the root cause, and you can kill all of them if you want, but it won’t fix the problem.

One important thing to clarify first, though, is that this dialog does not necessarily use the term ‘memory’ in the conventional sense – i.e. RAM.  It can also refer to disk space.  Unfortunately it doesn’t bother to distinguish between the two, which is particularly stupid of it since any possible resolution of the issue is highly dependent on which of the two cases it in fact is.

Thank goodness for iStatMenus, though, which in the most recent incident showed that I had ~20 GiB of RAM completely free (not even inactive, actually outright free).  So immediately that rules out what the daft bloody dialog’s actually saying.

The worst thing about all this is when it’s #2 the occurs.  For example, I had Lightroom do a 63-image panorama merge.  As Lightroom is a gross memory pig when doing panorama merging, it consumed something like 40 GiB of memory.  Which caused a bunch of stuff to page to disk.  Which consumed all the disk space.  Which led to that obnoxious dialog.  Which further led to macOS in its infinite fucking wisdom ‘pausing’ (SIGSTOPing) almost all running programs, including evidently whatever daemon actually handles paging.  Thus when Lightroom actually completed the panorama merge and released all that memory, I now had 20 GiB of free memory and the system refused to use any of it to page back in all that memory it’d paged out.  Because it was out of disk space.

The only solution – short of hard rebooting and hoping it resolves itself – was to delete a bunch of files I actually do still want, but which will now have to be recovered from a backup.  Great job macOS, thanks for all your help.

Of course, even once you do that and recover the system from the derpeche mode it put itself into, it won’t actually unpause any of the shit it broke.  You have to do that manually.  It pretends you can do that via that dialog that started the whole thing – assuming you left it open the entire time, blocking your view as you actually help the situation – but that only shows user-visible applications, not all the other system & background processes that it also rudely halted.

So, simple tip for resuming everything:

sudo killall -CONT -m '.'

Elegant, after a fashion.  Though every time, it reminds me that whomever named it ‘killall’ was either not very friendly or not very wise.

Note that the system will probably still be a bit broken in places, as despite what macOS thinks, you can’t just blindly pause random system tasks and not have things get really, really confused.  A reboot is always wise after seeing this dialog, to properly undo its fuckery.