iCloud ‘Optimize Mac Storage’ breaks the Mojave installer

Yet another example of a really bizarre macOS bug that’s pretty inexcusable as a test escape, given it occurs with the default installation settings on a completely clean OS install.

In short, the Mojave update installer does not work (on High Sierra at least) if you have ‘Optimize Mac Storage’ enabled for iCloud Drive (System Preferences > iCloud pane > iCloud Drive Options… button > Documents tab > Optimize Mac Storage checkbox).

Specifically, the installer reports:

Installation requires downloading important content. That content can’t be downloaded at this time. Try again later.

…and indeed fails to download the actual Mojave update files (the installer app as ‘installed’ via the App Store is merely a 22 MB bootstrapping app, that downloads the actual image only after you run it & start the installation).

Even more obnoxiously, if you use the dosdude1 Mojave Patcher Tool to force-download the entire installer, as soon as it completes the 6.5 GB download and produces the ‘Install macOS Mojave’ app in /Applications, the system deletes the downloaded installation files out from under that app, rendering it just as broken as the official App Store version. Infuriating.

Aside: to be clear, turning off ‘Optimize Mac Storage’ enabled me to produce – and keep – a working installer as downloaded by dosdude1’s tool. I did not verify that it also fixes the regular installer as downloaded via the App Store.

I also ran into the “The recovery server could not be contacted” error message even before all the above, but thankfully that was fixable via the means normally prescribed online – running “sudo ntpdate -u time.apple.com”.

tmutil is broken by SIP in Mojave

A diskutil bug unceremoniously erased an entire hard drive of mine a few weeks back.  While I was able to successfully (AFAICT) restore the drive’s contents to it from various backups, the erasure gave the drive a new identity (UUID, specifically).  The next time Time Machine ran, it compounded the diskutil bug by also unceremoniously deleting all my old backups (bar one, the latest), because it didn’t recognise the new drive with identical contents to the old drive as being the same drive, and tried to back it up again, requiring way more space, causing all existing backups to be purged, etc.


It turns out there’s actually a nominally supported way to address exactly this scenario – tmutil associatedisk (kudos to Simon Heimlicher for documenting this).  From the man page:

     associatedisk [-a] mount_point snapshot_volume

             Bind a snapshot volume directory to the specified local disk, thereby reconfigur-

             ing the backup history. Requires root privileges.

             In Mac OS X, HFS+ volumes have a persistent UUID that is assigned when the file

             system is created. Time Machine uses this identifier to make an association

             between a source volume and a snapshot volume. Erasing the source volume creates

             a new file system on the disk, and the previous UUID is not retained. The new

             UUID causes the source volume -> snapshot volume association to be broken. If one

             were just erasing the volume and starting over, it would likely be of no real

             consequence, and the new UUID would not be a concern; when erasing a volume in

             order to clone another volume to it, recreating the association may be desired.

             A concrete example of when and how you would use associatedisk:

             After having problems with a volume, you decide to erase it and manually restore

             its contents from a Time Machine backup or copy of another nature. (I.e., not via

             Time Machine System Restore or Migration Assistant.) On your next incremental

             backup, the data will be copied anew, as though none of it had been backed up

             before. Technically, it is true that the data has not been backed up, given the

             new UUID. However, this is probably not what you want Time Machine to do. You

             would then use associatedisk to reconfigure the backup so it appears that this

             volume has been backed up previously:

             thermopylae:~ thoth$ sudo tmutil associatedisk [-a] “/Volumes/MyNewStuffDisk”


             The result of the above command would associate the snapshot volume MyStuff in

             the specified snapshot with the source volume MyNewStuffDisk. The snapshot volume

             would also be renamed to match. The -a option tells associatedisk to find all

             snapshot volumes in the same machine directory that match the identity of

             MyStuff, and then perform the association on all of them.

Perfect – and I particularly like the subtext of the prose, which seems to be a subtle acknowledgment that this is a thing that happens frequently, and that macOS’s default behaviour is stupid… “recreating the association may be desired”.  No shit.

Unfortunately, that command doesn’t work in Mojave.  I’m apparently not the first person to notice.

It appears the tightened security, and in particular expansion of SIP to cover many more parts of the system including Time Machine backups, are to blame.  Even granting tmutil Full Disk Access etc in the system security settings is of no use (contrary to the stated purpose of Full Disk Access).

So you have to disable SIP first – which requires a reboot, obnoxiously – and only then does tmutil work again.  You’ll want to enable SIP again once you’re done, most likely, as the protections it provides are useful – it appears tmutil nve eeds to be updated to account for the new protections.

Apple Mail crashes on launch if connection logging is enabled

This was a fun one.  Mail started crashing on launch for absolutely no apparent reason – nothing had changed to its config or similar in a long time.  The crash logs were all fingering an identical culprit – -[IMAPTaskManager secondaryIdleMailboxName] called on the wrong GCD queue:

Process: Mail [19884]
Path: /Applications/Mail.app/Contents/MacOS/Mail
Identifier: com.apple.mail
Version: 11.3 (3445.6.18)
Build Info: Mail-3445006018000000~4
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: Mail [19884]
User ID: …

Date/Time: 2018-04-19 16:44:45.717 -0700
OS Version: Mac OS X 10.13.4 (17E199)
Report Version: 12
Anonymous UUID: …

Sleep/Wake UUID: …

Time Awake Since Boot: 94000 seconds
Time Since Wake: 530 seconds

System Integrity Protection: enabled

Crashed Thread: 13 Dispatch queue: Task Manager Serialization Queue (QOS: UNSPECIFIED)

Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000

Application Specific Information:
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This method should only be called on the serialization queue'
terminating with uncaught exception of type NSException
abort() called

Application Specific Backtrace 1:
0 CoreFoundation 0x00007fff55da96bb __exceptionPreprocess + 171
1 libobjc.A.dylib 0x00007fff7d4c1942 objc_exception_throw + 48
2 CoreFoundation 0x00007fff55daf2a2 +[NSException raise:format:arguments:] + 98
3 Foundation 0x00007fff57ee7340 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 193
4 IMAP 0x00007fff6fe88959 -[IMAPTaskManager secondaryIdleMailboxName] + 216
5 IMAP 0x00007fff6fe880cb -[IMAPTask mailboxIsUserVisibleUsingDataSource:] + 180
6 IMAP 0x00007fff6fe6ab68 -[IMAPMailboxSyncTask _nextNetworkPriorityAndOperation:reservedNetworkPriority:] + 164
7 IMAP 0x00007fff6fe6c4ba -[IMAPMailboxSyncTask recalculatePriorities] + 398
8 IMAP 0x00007fff6fe67dd9 -[IMAPMailboxSyncTask initWithDataSource:taskManager:imapMailbox:fromStatus:forceFullSync:] + 766
9 IMAP 0x00007fff6fe931f7 -[IMAPTaskManager _syncMailboxWithDataSource:withIMAPMailbox:fromStatus:forceFullSync:userInitiated:] + 370
10 IMAP 0x00007fff6fe92e68 -[IMAPTaskManager syncMailboxWithDataSource:withIMAPMailbox:fromStatus:forceFullSync:userInitiated:] + 240
11 IMAP 0x00007fff6fe9631a -[IMAPTaskManager didAddMessagesWithUnknownUID:toDataSource:] + 872
12 Foundation 0x00007fff57e4a5df __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 7
13 Foundation 0x00007fff57e4a441 -[NSBlockOperation main] + 68
14 Foundation 0x00007fff57e488ee -[__NSOperationInternal _start:] + 778
15 Foundation 0x00007fff57e44917 __NSOQSchedule_f + 369
16 libdispatch.dylib 0x00007fff7e09ee08 _dispatch_client_callout + 8
17 libdispatch.dylib 0x00007fff7e0b1ed1 _dispatch_continuation_pop + 472
18 libdispatch.dylib 0x00007fff7e0a9783 _dispatch_async_redirect_invoke + 703
19 libdispatch.dylib 0x00007fff7e0a09f9 _dispatch_root_queue_drain + 515
20 libdispatch.dylib 0x00007fff7e0a07a5 _dispatch_worker_thread3 + 101
21 libsystem_pthread.dylib 0x00007fff7e3f0169 _pthread_wqthread + 1387
22 libsystem_pthread.dylib 0x00007fff7e3efbe9 start_wqthread + 13

Long story short, the issue turns out to be having connection logging enabled.  That’d been turned on many months before in order to debug a different stupid Mail bug, and had been simply left on (deliberately IIRC, since Mail tends to bug-out quite often, so why not have logs already available when it comes time to debug it yet again?).

Connection logging is enabled or disabled by opening the “Connection Doctor” (Window menu > Connection Doctor) and using the checkbox titled “Log Connection Activity”.

So how do you get to that checkbox when Mail crashes on launch?  Well, in this specific instance I was able to disable all mail accounts via System Preference’s Accounts pane, launch Mail, disable logging, quit Mail, re-enable all mail accounts via System Preferences, and then relaunch Mail to have it finally actually work.

From even just brief web searching, it’s clear that this issue has been present and well-known in Mail for a really long time.  Sigh.  Apple’s protestations that they care about software quality, or the Mac, are relentlessly undermined by their actual actions.