feat(synology): add Synology Photos adapter for server-to-server migration#1330
Open
ywwzwb wants to merge 36 commits into
Open
feat(synology): add Synology Photos adapter for server-to-server migration#1330ywwzwb wants to merge 36 commits into
ywwzwb wants to merge 36 commits into
Conversation
Add support for migrating photos from Synology Photos to Immich: - New client for Synology Photos API (SYNO.Foto.* endpoints) - Support importing photos, videos, albums, and tags - Face recognition data converted to "Person: Name" tags - GPS coordinates and descriptions preserved - Filter by albums, tags, or people - CLI command: upload from-synology Includes: - adapters/synology/client.go - API client with auth and retry - adapters/synology/models.go - Data models for Synology API - adapters/synology/synology.go - Adapter implementation - adapters/synology/cmd.go - CLI command registration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Synology Photos to the list of supported sources: - Key Features section - Quick Start example - Upload commands overview - Popular Use Cases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update API version from 3 to 6 for better DSM compatibility - Add session and syno_token parameters for proper auth - Add detailed error messages for common error codes - Include response body in parsing errors for debugging - Add helpful tips for common authentication failure causes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Based on acme.sh implementation: 1. Query SYNO.API.Info first to get correct API path and version 2. Use GET request instead of POST for login 3. Add format=sid and enable_syno_token=yes parameters 4. Store and reuse actual api path from DSM discovery This should fix error code 101 by using the correct API endpoint that the specific DSM version supports. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously when API returned success=false, the code would still try to parse the response into the result struct and return nil. Now properly check for success=false and return an error with the error code and message before attempting to parse the result. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Simplify the doRequest retry logic to properly return errors instead of tracking lastErr which could be nil. - Return errors immediately on final retry instead of using lastErr - Remove debug output clutter - Better error messages for API failures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The saveTags function uses tag.Value when calling UpsertTags, but synology adapter was only setting Name field, leaving Value empty. This caused 400 Bad Request errors when creating tags. Now set both fields: - Name: for display/logging - Value: for API calls (required by UpsertTags) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Problem: Direct streaming from Synology caused "closed pipe" errors because network reader was too slow for upload, or connection timed out. Solution: Download files to local temp files before upload: 1. Open() creates a temp file 2. Downloads entire content to temp file 3. Seeks to beginning 4. Returns temp file handle 5. Close() removes temp file This ensures: - Stable upload without network timeout - Proper fs.File interface implementation - Automatic temp file cleanup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Live photos in Synology have type="live" and contain both an image and a video file with the same basename (e.g., IMG_9948.HEIC and IMG_9948.MOV). Changes: 1. Add IsLivePhoto() and LivePhotoVideoFilename() methods to Item 2. Detect live photos and process both image and video parts 3. Create separate Assets for image and video with correct filenames 4. Use thumbnail's unit_id for video download when different from main item This ensures Immich can properly recognize and merge live photos. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Problem: When user didn't specify --from-albums, but SYNO.Foto.Browse.Album API returned albums, code only processed items within those albums. Photos not belonging to any album were skipped. Solution: Changed logic to: - If --from-albums specified: process only those album items - Otherwise: process ALL items via processAllItems (timeline API) This ensures all photos are imported, not just those in albums. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synology Photos provides a special API to download live photos as a ZIP bundle containing both the HEIC image and MOV video files. This is more reliable than downloading them separately. Changes: - Add DownloadLivePhotoZip() to download live photos as ZIP - Add synologyLivePhotoFS to handle ZIP extraction - Update processLivePhoto() to use shared ZIP bundle - Files are extracted on first access and cleaned up after use Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The shared synologyLivePhotoFS was being cleaned up when the first file (image) was closed, causing the second file (video) to fail with "no such file or directory". Fix: Each live photo asset now has its own FS instance, avoiding lifecycle conflicts. Each FS downloads and extracts the ZIP independently and cleans up when its file is closed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add mutex protection and reference counting to live photo handling: - Add sync.Mutex to synologyLivePhotoFS to protect download - Add refCount to track open files using shared FS - Add verifyZipFile() to validate ZIP content before extraction - Add detailed logging for download and extraction - Cleanup only happens when last file is closed (refCount=0) - Shared FS between image and video avoids duplicate downloads This fixes "not a valid zip file" and "io: read/write on closed pipe" errors caused by concurrent downloads and premature cleanup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Content-Type logging to help diagnose live photo download problems: - DownloadLivePhotoZip now returns content-type for debugging - Log complete curl command when ZIP validation fails - This helps users manually reproduce and investigate issues Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synology's live photo API may return either: - A ZIP file containing both image and video (Content-Disposition: download.zip) - A single image file (Content-Disposition: *.jpg/*.heic) Previously the code assumed all responses were ZIP files, causing failures when Synology returned a single file. Changes: - Check Content-Disposition header to determine response type - Handle ZIP responses: extract both image and video files - Handle single file responses: save the image directly - Add isZip flag to track response type Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When Synology returns a single file (not ZIP), the video file doesn't exist. Previously the code would still try to access the video file, causing 'file not found' errors. Changes: - Add hasVideo() method to check if live photo contains video - Only create video asset for ZIP responses (isZip=true) - Skip video processing for single file responses Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The issue was that hasVideo() was called before Open() completed the download, causing it to return false and skip video asset creation. Later when Open() completed and found a ZIP, the video file was missing because the asset was never created. Changes: - Add preDownload() method to download before creating assets - Call preDownload() in processLivePhoto before hasVideo() check - Open() now uses pre-downloaded files instead of downloading again Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synology sometimes stores live photos with mismatched filenames in ZIP, e.g., IMG_4009.MOV paired with IMG_4009_1.HEIC. The video and image have different base names. Changes: - When exact video filename not found, search for any .mov/.mp4 file - Log which alternative video file is used - Add getAvailableFiles() for better error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add closed flag to synologyFile and synologyLivePhotoFile to prevent potential issues with double-close operations that could cause "io: read/write on closed pipe" errors during upload. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The 'io: read/write on closed pipe' errors were caused by temporary files being deleted while still being read for upload. Changes: - Add mutex protection to synologyFile and synologyLivePhotoFile - Add Read() method that checks if file is closed before reading - Delay temp file deletion by 5 seconds after Close() to ensure all upload readers have completed - Delay cleanup for live photo FS to prevent race conditions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synology Photos stores Unix timestamps in local timezone, not UTC. Previously the code used time.Unix() which returns UTC time, causing an 8-hour offset for users in UTC+8 timezone (e.g., China). Changes: - Convert capture time to local timezone using .In(time.Local) - This preserves the original capture time as displayed in Synology Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Synology stores capture time as local timezone seconds since epoch, not UTC. For example, a photo taken at 11:33 Beijing time has timestamp 1771932799, which is 11:33 UTC, not 11:33 CST. Previously the code preserved local time, but Immich interprets the time as UTC and adds timezone offset, causing 8-hour difference. Changes: - Interpret Synology timestamp as local time components - Convert to UTC before sending to Immich - This ensures Immich displays the correct local time Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…logy API Synology's time metadata has timezone issues, and GPS coordinates may have format differences. Immich can read these directly from EXIF more accurately. Changes: - Don't set CaptureDate - let Immich read from EXIF - Don't set Latitude/Longitude - let Immich read from EXIF - Only set Description and FileDate (index time) from Synology Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tadata Since we now let Immich read time and GPS directly from EXIF, the CaptureTime function and related metadata fields are no longer needed. Changes: - Remove CaptureTime() function from models.go - Remove DateTaken, Latitude, Longitude from FromApplication metadata - Keep FileName and Description which are still used Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix nil pointer dereference in Asset.Close(): condition was inverted, causing panic when cacheReader is nil (the common case before OpenFile) - Fix nil reader dereference in preDownload: DownloadItem may return nil reader on error; removed the erroneous reader.Close() call - Fix errors.Join in upload run.go: was discarding accumulated errGroup, now correctly joins with previous errors - Restore album filter in processItem: --albums flag had no filtering effect after refactor; added matchesAlbumFilter using albumImageCache - Fix temp file leak in processSingleItem: clean up downloaded temp file when mapToAsset returns nil (unsupported file type) - Fix misleading error message: "pre-download live photo" -> "pre-download item" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Asset.Close: verify no panic when cacheReader is nil, all CloseOptions are invoked, and early exit on first error - matchesAlbumFilter: cover no-filter passthrough, item-in-album, item-not-in-album, and multi-album scenarios - correctTimestamp: verify UTC no-op and fixed-offset (+8h) correction Also make matchesAlbumFilter self-contained by returning true early when no album filter is active (previously relied solely on the call-site guard). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… dir Live Photo assets share a temp directory. When multiple assets from the same directory are closed concurrently, one may delete the directory before another attempts to list it, causing a spurious "no such file or directory" error log. Skip cleanup if the directory is already gone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a new
upload from-synologycommand that migrates photos from Synology Photos to Immich.adapters/synology/) with full API client forSYNO.Foto.*endpoints, supporting authentication with SynoTokenPerson: <name>tags--albums,--tags,--peopleflags to limit scope of importAsset.Close()(nil pointer),errors.Joinaccumulation, and temp file cleanupAsset.Close, album filter, and timestamp correction logicUsage
Tested with
Test plan
go test ./...)go build ./...)🤖 Generated with Claude Code