iCloud Photos Downloader
github.com385 points by reconnecting 11 hours ago
385 points by reconnecting 11 hours ago
HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
Project page is here if it’s useful: https://photosbackup.app/
Happy to explain details if anyone’s curious — there are a lot of sharp edges in Photos once you go beyond “export originals”.
My current process for offloading photos off the iPhone is to copy them in subsequent batches of '0-9999' from the 'Image Capture' app.
This is because I usually have far more than 10K photos and apple starts renaming the files after 9999 as 00001(1) for the rest. This is pretty undesirable.
Is there a way for me to export unmodified raw/jpeg/live/videos off the iphone to an external drive without a macbook with a large enough ssd, and wanting to use icloud as an intermediate bottleneck?
I use libimobiledevice on linux
plug iphone into usb. lsusb should show it.
I backup my photos with:
sudo ifuse -o allow_other /mnt
rsync -a /mnt/DCIM <photos-dir>
sudo umount /mnt
sudo idevicebackup2 backup <backup-dir>
Do you take into account the iPhone not holding the original images of every photo? It will offload originals and just keep thumbnails if the library is too large.
Mine is approaching 1.5TB, I’ve got no hope of keeping that all on an iPhone, and also no guarantee that any given photo is fully available locally.
Does the PhotoSync app permit that? I use it to copy files to my NAS but it has some USB-related options I never explored. I used to use Image Capture but heard of PhotoSync and have never looked back.
That looks like it might do the trick. I feel like this should be something possible only using first party apps but I'll take it! Thanks.
I did it on Linux once I extracted them all as-is in the strange storage way that iOS stores them but I dont recall steps to make it mount the drive.
That would be perfect, I might chase down this path again. It's been a while since I've tried to directly mount the iphone as a drive on linux.
I have taken time to slowly extract photos from old androids and its such a nightmare, and if you cant get a meaningful interface to load you have to resort to tooling that scrapes the whole drive and hope it grabs everything.
My current workflow for this is to install the 'simpleSSHD' app on the android phone and rsync the files off at full network speed.
The sshd running on the phone also supports key based auth , so it's pretty simple to automate.
Thanks for commenting, do you support S3 compatible targets? Backblaze B2, for example.
Not directly, no. If you can mount the S3 target as a drive, it can be used as a backup destination.
have you looked at parachute backup? they also boast ability to backup the more mercurial types of iPhotos data.
Backing up “mercurial” Photos data is only half the problem. The tricky part is restoring it in a way Photos actually recognizes as equivalent to the original library state. Photos Backup Anywhere restore works by re-importing items while explicitly reapplying Photos-level attributes: paired assets for Live Photos, burst membership and picks, slo-mo metadata, edits, locations, adjusted capture dates, and then reconstructing albums after the items exist again in the library.
In other words, the filesystem copy isn’t treated as the source of truth. The restore verifies items against what was backed up and only then rebuilds higher-level structure like albums. That’s the piece I didn’t see addressed elsewhere, and what originally motivated me to build it.
This is awesome! This might be a great replacement to attempting to get the Windows app to work. Has anyone had luck with the iCloud app on windows?
Similar to some other folks in this thread I have ~2TB of iCloud data, a Macbook with far less than 2TB of space, an external hard drive somewhere with the external Photo Library that I need to plug in if I want to look at photos on the Macbook, and a Windows desktop with 10TB+ of rusty disks.
I was excited when they added the iCloud app + iCloud photos to Windows, but it never seems to catch up or finish what it is doing. It appears to be almost constantly download at 50MB/s, stressing both disk & internet, and yet navigating to the folder reveals that they are all 'available when online'.
It seems like there is not an option in Windows to actually grab everything in full quality (actually now that I look at it - its gotten to 944GB on disk / 1.91TB total, so it is getting there.)
I guess a real question - with these photos finally on a Windows desktop - is there a better photo browser than Microsoft photos that can show the HEIC and the Live Photo?
Surprisingly, there is no official way to download all (400 Gb) photos from iCloud. Here is an open-source command-line tool to download all your iCloud photos.
That’s not true. On any Mac or iPhone you can choose the iCloud Photo Library storage option to download all instead of letting the system optimize the storage. And if you turn off iCloud Photo Library, it will also try to download it all. I know this because I stopped using iCloud Photo Library and that was how I got all my photos downloaded.
+1 to this method. After optimise storage is disabled on the Mac, wait for all photos to download. Then, open the photos library bundle and you'll see every photo there, full res. Copy them wherever you like.
Also, if you leave optimise storage disabled and continue to use Photos, every photo will be cloned in any local or cloud backups of your machine. This strategy creates additional photo redundancy separate from iCloud while still benefiting from library syncing.
Or use the great osxphotos tool that works with Apple Photo’s SQLite database to let you manage all the photos in your library.
Demo .gif sold me
(Been meaning to make a software demo gif gallery, best way to understand many categories of apps)
This was my strategy too, but with a disgusting script which quit photos.app, rsync the photo library to a network share, then reopened photos.app so that it kept downloading from iCloud.
Not sure if the open/close is required, but I didn’t want to find out.
I don’t fully trust iCloud Drive / Photos therefore I use FSViewer to download all photos from my iOS device du jour (making sure to keep the HEIF formats), this way I get the Edited (slo-mo, live, portrait, usw) and pristine versions as Jobs intended. All kidding aside, after the gray area gate of 2017-2021 I had to find a more reliable backup workflow. As of today I only use iCloud Drive / Photos to extract some RAW photos that for some reason some picky apps don’t save to the photo album (looking at you ProCam 8.0). I made several tests including hash comparisons and imagemagick diffs and I am quite pleased.
Someone gave me a new iPhone (120GB) and a new MacBook Pro and asked me to download all their photos from iCloud. Long story short, after 120GB of photos were synchronised to the iPhone, the MacBook Pro refused to copy them, and now there's no storage left on the iPhone.
Also, Photos on Mac doesn't have an option to download photos directly, so the only valid option Apple offers is to download them through the web interface (max 1,000 at a time).
There is no official way to download iCloud library that is over phone capacity. Period.
> Photos on Mac doesn't have an option to download photos directly
Yes it does. It's called Download Originals to this Mac.
https://support.apple.com/guide/photos/use-icloud-photos-pht...
You keep asserting to the contrary, but I've been syncing my entire photos library to my Mac for years, since it was iPhoto even.
Obviously if you have a larger photos library than storage space on a particular device, you cannot synchronize the entire library to that specific device. e.g. my photos library vastly exceeds my iPhone 13 mini storage, so on my iPhone, I don't sync everything. But my Mac has 2 TB of storage, and Photos is setup to sync all my photos, and does so, reliably, and has been, again, for years now.
Additionally, unlike with this open source tool, I can keep advanced data protection enabled.
This is from the iCloud manual:
> Any new photos and videos you add to Photos appear on all your devices that have iCloud Photos turned on.
You have your photos because they are new. If they had been taken before, they would not have synchronised automatically with Photos on MacOS.
Please stop repeating your incorrect points that are contradicted by everyone else’s real experiences.
Yes, new ones will be uploaded. That doesn’t mean old ones won’t also be downloaded.
I have tried 3 different Macs with different versions of macOS prior to looking for a workaround, and everywhere the result is the same: old photos are not downloaded automatically from iCloud, and there is no button to start this process - for this exact reason.
Want to prove me wrong? Create a new macOS user and open Photos with your iCloud. It will be empty until you start copying photos from your phone. It will take much less time than arguing here.
You're arguing with a lot of people who have personally seen this work. You can listen to other people. You can also go to an Apple Store and let them show you what's going wrong here.