diff --git a/FLAMENCO_BUG_REPORT.md b/FLAMENCO_BUG_REPORT.md deleted file mode 100644 index d245b42..0000000 --- a/FLAMENCO_BUG_REPORT.md +++ /dev/null @@ -1,99 +0,0 @@ -**System Information** -Operating System(s): Win11 25h2 - -**Flamenco Version** -Is Broken: 3.8 -Worked OK: unknown - -When Flamenco is ran from a Windows machine, with the shared-storage set to an SMB share, the symlinks created by Shaman appear to be either completely useless at worst, context dependent at best. - -1. Download and configure Flamenco -2. Configure the security policy as instructed [in the documentation](https://flamenco.blender.org/usage/shared-storage/shaman/#windows) -3. Set `shaman: enabled: true`. -4. Set `shared_storage_path`. I was originally hosting the files on a TrueNAS SMB share (mapped to F:\). Testing it with a local drive (T:\Flamenco) results with the intended behavior. -5. Submit job - -The symlinks created within `jobs` will not be universally accessible by Windows. All Windows workers (I have only tested Windows) will fail: - -``` -2026-01-08T15:51:58-07:00 Task assigned to worker i9kf (3690a968-f7ee-49d5-adb7-76154761d435) -2026-01-08T15:51:58-07:00 task changed status queued -> active -going to run: - - 'F:\software\Blender\blender.exe' -b -y --gpu-backend opengl 'F:\jobs/SSD_2B/SSD_2B.flamenco.blend' --python-expr ' -import bpy - -cycles_prefs = bpy.context.preferences.addons['"'"'cycles'"'"'].preferences -cycles_prefs.compute_device_type = '"'"'OPTIX'"'"' -for dev in cycles_prefs.get_devices_for_type('"'"'OPTIX'"'"'): - dev.use = (dev.type != '"'"'CPU'"'"') -' --python-expr 'import bpy; bpy.context.scene.cycles.device = '"'"'GPU'"'"'' --render-output 'F:/renders/SSD_2B/2026-01-08_151057/######' --render-format PNG --render-frame 0..15 - -pid=32124 > 00:10.047 reports | ERROR File format is not supported in file "F:\jobs\SSD_2B\SSD_2B.flamenco.blend" -pid=32124 > Blender 5.0.1 (hash a3db93c5b259 built 2025-12-16 01:32:30) -pid=32124 > Cannot read ARP prefs -pid=32124 > Update armature presets... -pid=32124 > animation_layers_updater addon: Verbose is enabled -pid=32124 > animation_layers_updater addon: Read in JSON settings from file -pid=32124 > -pid=32124 > Blender quit -Failed: command exited abnormally with code 1 -2026-01-08T15:52:11-07:00 Task failed by 1 worker, Manager will mark it as soft failure. 2 more failures will cause hard failure. -2026-01-08T15:52:11-07:00 task changed status active -> soft-failed -``` - -Examining the jobfiles reveals the issue. -image.png -The file is detected as `.symlink` type. -image.png -However, the symlink points to the proper Shaman Blob. - -If you right click and edit the symlink in notepad, it opens the blob: -image.png -A proper blendfile with gzip compression. - -When opened in VSCode, it opens blank. -image.png - -When copied to another folder, it copies a blank file. -image.png - -If you open the symlink in blender, the error is replicated: -File format is not supported in file "F:\jobs\SSD_2B\SSD_2B.flamenco.blend" - -Opening the actual blob in blender works, but obviously it's still pointing to files that cannot be found from the blob's directory. I only did this to verify that the blobs were written correctly, and they clearly are. - -``` -18:03.360 blend | Read blend: "F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\1253809.blob" -18:03.391 blend.readfile | Read library: 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Scenes\LivingRoom_v2.blend', '//_outside_project/A/1 Amazon_Active_Projects/1 BlenderAssets/Amazon/Scenes/LivingRoom_v2.blend', parent '' -18:03.391 blend.readfile | Cannot find lib 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Scenes\LivingRoom_v2.blend' -18:03.391 blend.readfile | Read library: 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Char\Customers\Mom.blend', '//_outside_project/A/1 Amazon_Active_Projects/1 BlenderAssets/Amazon/Char/Customers/Mom.blend', parent '' -18:03.391 blend.readfile | Cannot find lib 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Char\Customers\Mom.blend' -18:03.422 blend.readfile | LIB: Collection: 'Living Room' missing from 'F:\file-store\stored\14\0c6efa48408ce1ee0a7296e057e798b1a162d7bc60d542b03d8938f33776c8\_outside_project\A\1 Amazon_Active_Projects\1 BlenderAssets\Amazon\Scenes\LivingRoom_v2.blend', parent '' -18:03.422 -``` - -### Root Cause - -Windows creates **Windows reparse points** (not Unix symlinks) when creating symlinks on SMB shares. The behavior differs based on how the file is accessed: - -1. **Windows reparse points on SMB shares:** - - Windows stores the target path in reparse point metadata - - Samba stores these as empty regular files on the server (doesn't understand Windows reparse points) - - The target path is correctly stored and can be read via `Get-Item` `.Target` property - -2. **Why some applications work:** - - **Windows Explorer/Notepad:** Use file system filters that detect reparse points and automatically resolve them client-side by reading the `Target` property and opening the target file - - This works because resolution happens on the Windows client, not the SMB server - -3. **Why Blender/other applications fail:** - - **Blender/VSCode/programmatic I/O:** Use direct file I/O operations (`ReadAllBytes()`, `FileStream`, etc.) - - These operations don't use reparse point filters - - SMB client doesn't automatically follow reparse points - - Returns the empty file from the server (0 bytes) - -## Workarounds - -1. **Use local storage:** Set `shared_storage_path` to a local NTFS drive (symlinks work perfectly) -2. **Run Flamenco Manager on Linux/WSL:** Use `mfsymlinks` mount option for SMB shares -3. Disable Shaman. \ No newline at end of file diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..8829579 --- /dev/null +++ b/TODO.md @@ -0,0 +1,6 @@ +# todo + +* [ ] When setting the Windows Flamenco Manager's shared storage to a local drive, the reparse points work locally. If I share that on the network over SMB, will it allow other Windows machines to read the reparse points properly? +* [ ] Test how difficult it is to allow wide links on shares hosted by: + * [ ] Unraid + * [ ] Synology diff --git a/plan.md b/plan.md deleted file mode 100644 index cbb0dca..0000000 --- a/plan.md +++ /dev/null @@ -1,6 +0,0 @@ -1. put windows and linux flamenco manager and worker on the shared storage (on linux machine that hosts the shared storage) -2. set up two way variable for the shared storage path on both windows (F:\) and linux (/mnt/whatever/Flamenco) -3. create batch to launch the manager from windows using ssh to the linux machine -4. run batch to replace running the manager from windows - -i still need to submit a bug report that highlights that the symlinks are not working on windows when the shared storage is on an SMB share, which is the entire point... sybren does it on his demo video, so idk how it broke. \ No newline at end of file