This post is really a note-to-self for when I next have to remember how to deal with missing photo and QuickTime movie metadata. Nevertheless, since it took me a little while to work out, maybe someone else searching for the same issue will benefit from it. If you’re short on time the ExifTool command that helped me is right at the end – skip down there and copy and paste.
For anyone else who can’t sleep, it might be useful, but I won’t be offended if you skip reading this.
The photo management problem
Like many people, my photo libraries have grown to many gigabytes over the years, encrusted with cruft from various photo management apps. Multiple versions of iPhoto and Lightroom, not to mention a few corrupt libraries and recoveries have left their scars. Since I had an iPhone, everything got worse. Do I use iCloud, Dropbox or iPhoto? I used to religiously use Lightroom, but it was always a drag importing photos from my iPhone into Lightroom and sorting them. Because it was a drag, I didn’t do it all that often and I lost track of which photos were where.
Of all things that would be hard to replace if all my devices caught fire and all my backups failed, the photos would be the one I care about most. Almost everything else is either replaceable, recoverable or possible to take a Zen-like attitude to letting go of. But I would be very sad to lose the photos of my daughter when she was tiny.
I tried various automatic services, such as Dropbox’s automated Camera Upload feature and I also tried Everpix before they shut down and then moved to Loom, but then were acquired by Dropbox. Back then, the whole point of using another service was because I didn’t have enough space on Dropbox for over 72GB of images and videos. Since Dropbox upgraded their plans, I know have the opposite problem, which is that my 1.1TB of cloud space is larger than my laptop’s hard drive.
Having tried all these services on and off, I could no longer remember which photos I had backed up from my iPhone and where. My iPhone was getting too full and I was nervous to delete older images in case that was the only copy.
I decided to follow most of Federico Viticci’s photo workflow and go for near Camera Roll Zero. I use Camera Sync on the iPhone to upload all my photos to Dropbox and use Hazel to rename and sort them based on their metadata. Once uploaded I keep a handful of photos of my family on my iPhone, but otherwise delete all the others.
Similar to Federico, my Hazel rules delete screenshots, since I generally don’t want to keep them, but I also have Camera Sync set up not to upload screenshots anyway:
Then I have a rule to sort the photos into date based folders, by year and then simply date. The files are also renamed with date and time, such as “2015-01-02 3.08.29.jpg”:
Videos are renamed and sorted into a video sub-folder of each year:
Deadly exciting, I know, but I’ve wasted so much time searching for photos and so much drive space with duplicates that the time spent getting this to work has been fantastic for me. I usually have a pretty good memory of roughly when important events happened in life and there are plenty of ways to view folders of images as thumbnails (see Federico’s article for a round-up of these). Duplicates are now easily spotted, because they are in the same folder with the same filename (a number is appended if filenames are identical).
The stripped EXIF data problem
This approach has one major drawback, enforced upon it by other apps. It relies on accurate file creation dates and EXIF data – the metadata stored in all image files. Sometimes this data is missing or incorrect.
My Hazel rules sort files based on their creation dates. One of the previous photo storage cloud services – either Loom or Everpix (the culprit, I think) – re-stamped the file creation dates. This meant I had a bunch of different images all with the same creation dates and, thus, all the same filename if I ran my Hazel rules on them.
I thought I could probably recover the proper dates by using the EXIF data. There are apps to view and extract this and in fact Hazel can examine some EXIF data in its “other” settings:
But I needed something more powerful and Phil Harvey’s excellent ExifTool came to the rescue. With it, recreating the correct file-creation date based on the EXIF data is trivial.
exiftool -a /Users/andypolaine/Dropbox/Camera\ Uploads/2015/2015-01-27/2015-01-27\ 9.38.57.jpg
gives you this enormous output:
If you pop that GPS data into Google Maps, you’ll see I took this photo
when I was on the train in Switzerland:
(Actually the GPS data seems slightly off, because I was already about 200m past that position when I took the photo. I can only assume a laggy GPS location, since the signal is pretty bad and the train is moving fast. You can also see why snooping governments’ claims of “we only want to look at the metadata” is such a load of nonsense.)
You will also notice there are several tags that have a date stamp. There is a Date/Time Original or the GPS Date/Time (they’re different because of Daylight Savings time) plus the File Modification Date/Time stamps. (My favourite EXIF tag is “Circle Of Confusion” – sounds like my life.)
I ran ExifTool on my problem images and restamped all the File Creation Dates using the EXIF data.
WhatsApp with your EXIF data?
This was all good until I found a whole load of photos all time-stamped with 1999-11-30 12.00.00 and heading into a November 1999 folder. These turned out to be mostly images saved from people’s tweets or from WhatsApp. I’m not sure about Twitter, but WhatsApp deliberately strips the EXIF data from images presumably as a privacy measure, unless it’s just a bug/feature in the app or iOS. iOS 8’s new photo editing extension that lets third-party apps edit photos directly also strips the EXIF data.
An image sent via WhatsApp yields only this:
(This is post-Hazel renaming, so the filename and File Modification and Inode dates are based on when I uploaded the files to Dropbox, not the real ones of when the image was originally created.)
The lack of this EXIF data is an enormous pain, since a common practice when getting together with friends is to set up a WhatsApp group and share all the photos with each other that way. Of course my friends look at me blankly if I ask them to send the photos another way “because I need the EXIF data.” So, unfortunately all those image end up in 1999 and I have to manually sort them out of there again.
The only salvation would be to upload them to Dropbox on the day of the event and use that date, but that defeats the point of the whole exercise. If anyone knows a way around this, please let me know.
The video metadata problem
The other problem I had was with a bunch of QuickTime movies I had shot with my iPhone. I don’t know which app (Everpix?) screwed these file creation dates up. This time the Creation Date was 1st January 1970 on all of them.
I often used QuickTime Player 7 for transcribing because it has better controls (jog shuttle, playback speed, etc.) that Apple removed from later versions. I have an old QuickTime Pro licence too, which allows you to see individual track data and manipulate them:
I noticed there was a Track Creation Date annotation in there, which hadn’t been stripped out and had the correct capture date. Fortunately, ExifTool can handle videos too and extracting the EXIF data on the video gave me these tags (among many others):
I could use one of these to set the rest of the metadata of the file. The following command extracts the Track Creation Date and writes it as the File Creation Date and also renames the file to match (and appends a lowercase file extension):
exiftool '-FileModifyDate<TrackCreateDate' '-FileName<TrackCreateDate' -d %Y-%m-%d_%H.%M.%S.%%e
This left me with a movie file that Hazel could correctly sort. I think “2011-02-27 1.11.11.mov” might possibly have the wrong time (or miscalculate the GMT offset), but the date is certainly correct. It’s good enough for me.
You can run ExifTool commands on individual files (essential when testing), but also on directories of files and it all happens pretty quick.
Now if only I could do the same thing to my brain.