Compare commits
15 Commits
57f95fc74d
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
 Raincloud
|
3d64eb3094 | ||
|
Raincloud
|
d1221f83b3 | ||
|
Raincloud
|
1ad613429a | ||
|
Raincloud
|
427abd6870 | ||
|
Raincloud
|
9c8cc0faca | ||
|
Raincloud
|
6588cb7957 | ||
|
Raincloud
|
baf17ec369 | ||
|
Raincloud
|
02d901e57e | ||
|
Raincloud
|
da21cafe1c | ||
|
Raincloud
|
0201d3e8bd | ||
|
Raincloud
|
7b48b709ce | ||
|
Raincloud
|
70684f5644 | ||
|
Raincloud
|
dc7b224005 | ||
|
Raincloud
|
a76bb1e1e0 | ||
|
Raincloud
|
0f29c0c457 |
@@ -7,6 +7,7 @@ logs/*
|
||||
# Encoded files
|
||||
input/*
|
||||
output/*
|
||||
tmp/*
|
||||
# Video files
|
||||
*.mp4
|
||||
*.mkv
|
||||
@@ -19,3 +20,11 @@ output/*
|
||||
*.ts
|
||||
*.m2ts
|
||||
*.mts
|
||||
|
||||
# Python cache files
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# IDE files
|
||||
.specstory/
|
||||
.vscode/
|
||||
@@ -1,4 +0,0 @@
|
||||
# SpecStory explanation file
|
||||
/.what-is-this.md
|
||||
# SpecStory project identity file
|
||||
/.project.json
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,470 +0,0 @@
|
||||
<!-- Generated by SpecStory, Markdown v2.1.0 -->
|
||||
|
||||
<!-- cursor Session c905dc1e-3b07-499f-8906-d3b9d49bb984 (2025-06-26 04:00Z) -->
|
||||
|
||||
# Test encode for one minute (2025-06-26 04:00Z)
|
||||
|
||||
_**User (2025-07-03 15:00Z)**_
|
||||
|
||||
can we do a test encode for 1 min?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
Yes, you can perform a test encode for just 1 minute of each file by modifying the FFmpeg command to only process the first 60 seconds. In FFmpeg, you can use the -t 60 option to limit the output duration to 60 seconds.
|
||||
|
||||
Here's how you can do it concisely:
|
||||
|
||||
- Add '-t', '60' to the FFmpeg command list in the encode_dvr function, right after the input file argument.
|
||||
|
||||
Would you like this as a permanent option (e.g., via a command-line flag or prompt), or just a quick hardcoded test for now? If you want it as a toggle, let me know your preferred method (prompt, argument, etc.). Otherwise, I can make the quick change for a 1-minute encode.
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-03 15:00Z)**_
|
||||
|
||||
the output got stuck on this, but it shows my download rate @ ~300-400 mbps (it's encoding on a network share) do I just wait?
|
||||
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: frame= 898 fps=3.1 q=0.0 size= 176384KiB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A
|
||||
FFmpeg: [out#0/mp4 @ 00000287de38b840] All streams finished
|
||||
FFmpeg: [out#0/mp4 @ 00000287de38b840] Terminating thread with return code 0 (success)
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
Yes, you should wait! The output shows that FFmpeg is still processing and hasn't actually gotten stuck. Here's what's happening:
|
||||
|
||||
1. **Frame 898** - FFmpeg is still encoding frames (you're at frame 898)
|
||||
2. **fps=3.1** - It's processing about 3.1 frames per second, which is normal for lossless VP9 encoding
|
||||
3. **size=176384KiB** - The output file is growing (about 172MB so far)
|
||||
4. **time=N/A** - This is normal during encoding, especially with lossless encoding
|
||||
5. **speed=N/A** - Also normal during lossless encoding
|
||||
|
||||
The key indicators that it's working:
|
||||
- Frame count is increasing (898 frames processed)
|
||||
- File size is growing (176384KiB)
|
||||
- No error messages
|
||||
- The final lines show "All streams finished" and "Terminating thread with return code 0 (success)"
|
||||
|
||||
**Lossless VP9 encoding is very slow** - 3.1 fps is actually quite good for lossless encoding. A 15-second clip might take several minutes to encode losslessly.
|
||||
|
||||
The 300-400 Mbps network activity you're seeing is likely the input file being read from the network share, which is normal.
|
||||
|
||||
**Recommendation**: Let it finish. The process appears to be working correctly, just slowly (which is expected for lossless encoding).
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-03 15:00Z)**_
|
||||
|
||||
what does this mean then? keep in mind my cpu is no longer being utilized
|
||||
FFmpeg: [out#0/mp4 @ 00000287de38b840] All streams finished
|
||||
FFmpeg: [out#0/mp4 @ 00000287de38b840] Terminating thread with return code 0 (success)
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
That output means **the encoding is complete and successful!**
|
||||
|
||||
The messages you're seeing:
|
||||
- `All streams finished` - FFmpeg has finished processing all video and audio streams
|
||||
- `Terminating thread with return code 0 (success)` - The process completed successfully with no errors
|
||||
|
||||
The fact that your CPU is no longer being utilized confirms this - FFmpeg has finished the encoding job and is just cleaning up.
|
||||
|
||||
**The process should exit automatically** and return you to the command prompt. If it's still showing those messages repeatedly without exiting, that might indicate the script is stuck in a loop reading the output.
|
||||
|
||||
You can:
|
||||
1. **Wait a few more seconds** - it might just be finishing up
|
||||
2. **Press Ctrl+C** if it doesn't exit - the encoding is already done
|
||||
3. **Check your output folder** - the encoded file should be there
|
||||
|
||||
The 15-second test encode is complete! You should find the output file in your `output/` directory.
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-03 15:00Z)**_
|
||||
|
||||
why would ffmpeg.exe still be accessing files at this point?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
FFmpeg might still be accessing files for several reasons even after the encoding is "complete":
|
||||
|
||||
1. **Finalizing the output file** - Writing the final metadata, headers, and closing the file properly
|
||||
2. **Cleanup operations** - Releasing file handles, flushing buffers to disk
|
||||
3. **Network share overhead** - If writing to a network location, there might be additional I/O for file locking, permissions, or network protocol cleanup
|
||||
4. **File system operations** - Updating timestamps, file attributes, or directory entries
|
||||
5. **Buffer flushing** - Ensuring all data is actually written to disk before closing
|
||||
|
||||
The "All streams finished" message indicates the encoding work is done, but FFmpeg still needs to:
|
||||
- Write the final file headers
|
||||
- Close the output file properly
|
||||
- Release system resources
|
||||
|
||||
This is normal behavior, especially when working with network shares where file operations can take longer due to network latency and protocol overhead. The process should exit once all file operations are complete.
|
||||
|
||||
If it's taking an unusually long time (>30 seconds), you could check if there are any file system issues or network connectivity problems.
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-03 15:00Z)**_
|
||||
|
||||
the resultant file is broken.
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
The broken output file could be due to several issues. Let me help you troubleshoot:
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="read" data-tool-name="read_file">
|
||||
<details><summary>Tool use: **read_file** • Read file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
The issue is likely that the process was interrupted or didn't complete properly. Here are the most common causes and solutions:
|
||||
|
||||
**Potential Issues:**
|
||||
|
||||
1. **Process interruption** - The script might have been stopped before FFmpeg could properly finalize the file
|
||||
2. **Network share issues** - Writing to network locations can cause file corruption if interrupted
|
||||
3. **Incomplete encoding** - The 15-second limit might have caused issues with the VP9 encoder
|
||||
|
||||
**Quick fixes to try:**
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="edit_file">
|
||||
|
||||
<details><summary>Tool use: **edit_file** • Edit file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
I'll modify the FFmpeg command to be more robust by adding proper output format specification and ensuring clean termination
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 2, lines removed: undefined
|
||||
|
||||
```diff
|
||||
@@ -117,4 +117,6 @@
|
||||
'-c:a', 'copy',
|
||||
'-map', '0',
|
||||
+ '-f', 'mp4', # Explicitly specify output format
|
||||
+ '-movflags', '+faststart', # Optimize for streaming
|
||||
]
|
||||
# Add metadata for each audio stream if label exists
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
**Also, let's add a simple test to verify the output file:**
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="edit_file">
|
||||
|
||||
<details><summary>Tool use: **edit_file** • Edit file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
I'll add a function to verify the output file integrity after encoding
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 9, lines removed: undefined
|
||||
|
||||
```diff
|
||||
@@ -68,4 +68,13 @@
|
||||
return f"{size_bytes:.2f} TB"
|
||||
|
||||
+ def verify_output_file(output_path):
|
||||
+ """Verify that the output file is valid and playable"""
|
||||
+ try:
|
||||
+ cmd = ['ffprobe', '-v', 'error', str(output_path)]
|
||||
+ result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
|
||||
+ return result.returncode == 0
|
||||
+ except (subprocess.TimeoutExpired, subprocess.CalledProcessError):
|
||||
+ return False
|
||||
+
|
||||
def encode_dvr(input_file, output_dir, gpu):
|
||||
input_path = Path(input_file)
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="edit_file">
|
||||
|
||||
<details><summary>Tool use: **edit_file** • Edit file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
I'll add file verification after successful encoding
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 8, lines removed: undefined
|
||||
|
||||
```diff
|
||||
@@ -159,9 +159,17 @@
|
||||
compression_ratio = input_size / output_size if output_size > 0 else 0
|
||||
|
||||
+ # Verify output file integrity
|
||||
+ if verify_output_file(output_path):
|
||||
logging.info(f"Successfully encoded: {output_path}")
|
||||
logging.info(f"Output size: {format_size(output_size)}")
|
||||
logging.info(f"Compression ratio: {compression_ratio:.2f}x")
|
||||
print(f"{Colors.GREEN}Successfully encoded: {output_path}{Colors.ENDC}")
|
||||
print(f"Compression ratio: {compression_ratio:.2f}x")
|
||||
+ else:
|
||||
+ logging.error(f"Output file verification failed: {output_path}")
|
||||
+ print(f"{Colors.RED}Output file verification failed: {output_path}{Colors.ENDC}")
|
||||
+ # Remove corrupted file
|
||||
+ if output_path.exists():
|
||||
+ output_path.unlink()
|
||||
else:
|
||||
logging.error(f"FFmpeg process failed with return code {process.returncode}")
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
**Now try the test again with these improvements:**
|
||||
|
||||
1. **Explicit format specification** - Added `-f mp4` to ensure proper container format
|
||||
2. **Fast start optimization** - Added `-movflags +faststart` for better file structure
|
||||
3. **File verification** - The script now checks if the output file is valid and removes corrupted files
|
||||
|
||||
**Additional troubleshooting steps:**
|
||||
- Delete the broken output file before running again
|
||||
- Try encoding to a local drive first (not network share) to rule out network issues
|
||||
- Check if your input file is valid: `ffprobe -v error your_input_file.mp4`
|
||||
|
||||
The updated script should be more robust and will automatically detect and remove corrupted output files.
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-09 00:17Z)**_
|
||||
|
||||
results are working, but I'm noticing massive sizes. is there a cq 0 option?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
Yes! The issue is that you're using lossless VP9 encoding, which creates very large files. VP9 doesn't have a traditional "CQ 0" option like H.264/H.265, but you can use **CRF (Constant Rate Factor)** which is similar.
|
||||
|
||||
For VP9, the CRF range is **0-63**, where:
|
||||
- **0** = lossless (what you're currently using)
|
||||
- **10-20** = visually lossless (very high quality)
|
||||
- **20-30** = high quality
|
||||
- **30+** = lower quality
|
||||
|
||||
Let me modify the script to use CRF instead of lossless:
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="search_replace">
|
||||
|
||||
<details><summary>Tool use: **search_replace** • Edit file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 1, lines removed: 1
|
||||
|
||||
```diff
|
||||
@@ -104,5 +104,5 @@
|
||||
logging.info(f"Audio labels: {audio_labels}")
|
||||
|
||||
- # FFmpeg command for lossless VP9 encoding with 60s timeout
|
||||
+ # FFmpeg command for high-quality VP9 encoding with 60s timeout
|
||||
cmd = [
|
||||
'ffmpeg',
|
||||
```
|
||||
|
||||
**Chunk 2**
|
||||
Lines added: 3, lines removed: 2
|
||||
|
||||
```diff
|
||||
@@ -111,6 +111,7 @@
|
||||
'-t', '1',
|
||||
'-c:v', 'libvpx-vp9',
|
||||
- '-lossless', '1',
|
||||
- '-cpu-used', '0',
|
||||
+ '-crf', '15', # High quality (0=lossless, 10-20=visually lossless, 20-30=high quality)
|
||||
+ '-b:v', '0', # Let CRF control bitrate
|
||||
+ '-cpu-used', '2', # Faster encoding (0=slowest, 5=fastest)
|
||||
'-row-mt', '1',
|
||||
'-threads', '0',
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
**Key changes:**
|
||||
- **Removed `-lossless 1`** - This was causing the massive file sizes
|
||||
- **Added `-crf 15`** - High quality, visually lossless setting
|
||||
- **Added `-b:v 0`** - Let CRF control the bitrate automatically
|
||||
- **Changed `-cpu-used` from 0 to 2** - Faster encoding while maintaining quality
|
||||
|
||||
**CRF values you can try:**
|
||||
- **10** - Very high quality (larger files)
|
||||
- **15** - High quality (good balance)
|
||||
- **20** - Still high quality (smaller files)
|
||||
- **25** - Good quality (much smaller files)
|
||||
|
||||
The current setting of CRF 15 should give you excellent quality with much smaller file sizes than lossless. You can adjust the CRF value based on your quality vs. size preferences.
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-09 00:17Z)**_
|
||||
|
||||
these look good, can I test the full video lengths now?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
Yes! To test the full video lengths, you just need to remove the `-t 1` parameter from the FFmpeg command. This will encode the entire duration of each video.
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="search_replace">
|
||||
|
||||
<details><summary>Tool use: **search_replace** • Edit file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 1, lines removed: 2
|
||||
|
||||
```diff
|
||||
@@ -104,10 +104,9 @@
|
||||
logging.info(f"Audio labels: {audio_labels}")
|
||||
|
||||
- # FFmpeg command for high-quality VP9 encoding with 60s timeout
|
||||
+ # FFmpeg command for high-quality VP9 encoding
|
||||
cmd = [
|
||||
'ffmpeg',
|
||||
'-v', 'verbose',
|
||||
'-i', str(input_path),
|
||||
- '-t', '1',
|
||||
'-c:v', 'libvpx-vp9',
|
||||
'-crf', '15', # High quality (0=lossless, 10-20=visually lossless, 20-30=high quality)
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
Perfect! Now the script will encode the full duration of each video. With your current settings:
|
||||
|
||||
- **CRF 15** - High quality, visually lossless
|
||||
- **Max bitrate 10 Mbps** - Good for 1080p content
|
||||
- **CPU-used 2** - Reasonable encoding speed
|
||||
|
||||
The encoding will take much longer now (probably 10-30 minutes per video depending on your hardware), but you'll get the full videos encoded with good quality and reasonable file sizes.
|
||||
|
||||
You can run the script now and it will process all 14 files in your input directory with their full durations.
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-07-09 00:17Z)**_
|
||||
|
||||
other than increasing the rate limit, how can I make this higher quality?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
For higher quality VP9 encoding, here are the key parameters you can adjust:
|
||||
|
||||
**1. CRF (Constant Rate Factor) - Most Important:**
|
||||
- **0** = Lossless (what you're using now)
|
||||
- **5-10** = Extremely high quality
|
||||
- **10-15** = Very high quality
|
||||
|
||||
**2. CPU-used (Encoding Speed vs Quality):**
|
||||
- **0** = Slowest, highest quality (what you're using)
|
||||
- **1** = Slightly faster, still very high quality
|
||||
- **2** = Good balance
|
||||
|
||||
**3. Additional Quality Parameters:**
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="search_replace">
|
||||
|
||||
<details><summary>Tool use: **search_replace** • Edit file: encode_dvr_lossless_vp9.py</summary>
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 5, lines removed: 2
|
||||
|
||||
```diff
|
||||
@@ -111,10 +111,13 @@
|
||||
'-t', '1', # Render only 1 second
|
||||
'-c:v', 'libvpx-vp9',
|
||||
- '-crf', '0',
|
||||
+ '-crf', '5', # Extremely high quality (0=lossless, 5-10=extremely high)
|
||||
'-b:v', '10000k',
|
||||
'-maxrate', '10000k',
|
||||
- '-cpu-used', '0',
|
||||
+ '-cpu-used', '0', # Slowest encoding for highest quality
|
||||
'-row-mt', '1',
|
||||
'-threads', '0',
|
||||
+ '-tile-columns', '2', # Better quality for 1080p
|
||||
+ '-frame-parallel', '1', # Parallel frame processing
|
||||
+ '-lag-in-frames', '25', # More frames for better compression
|
||||
'-c:a', 'copy',
|
||||
'-map', '0',
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
**Quality improvements added:**
|
||||
|
||||
1. **CRF 5** - Extremely high quality (vs 0 which is lossless but much larger)
|
||||
2. **tile-columns 2** - Optimizes encoding for 1080p content
|
||||
3. **frame-parallel 1** - Enables parallel frame processing
|
||||
4. **lag-in-frames 25** - Uses more frames for better compression analysis
|
||||
|
||||
**Other quality options you could try:**
|
||||
|
||||
- **Increase bitrate limit** to 15-20 Mbps for even higher quality
|
||||
- **Use CRF 0** for true lossless (but much larger files)
|
||||
- **Add `-deadline best`** for maximum quality (very slow)
|
||||
|
||||
The current settings should give you excellent quality while being more reasonable in file size than lossless encoding.
|
||||
|
||||
---
|
||||
|
||||
@@ -1,74 +0,0 @@
|
||||
<!-- Generated by SpecStory, Markdown v2.1.0 -->
|
||||
|
||||
<!-- cursor Session fd1849e6-e572-4051-8efe-3106af3956ce (2025-10-01 20:10Z) -->
|
||||
|
||||
# Delete specific files from output (2025-10-01 20:10Z)
|
||||
|
||||
_**User (2025-10-01 20:08Z)**_
|
||||
|
||||
could you delete these 4 files from output? don't touch input, and don't touch any other files in output.
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
Deleting the 4 problematic files from the output directory.
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="delete_file">
|
||||
<details><summary>Tool use: **delete_file**</summary>
|
||||
|
||||
Explanation: undefined
|
||||
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="delete_file">
|
||||
<details><summary>Tool use: **delete_file**</summary>
|
||||
|
||||
Explanation: undefined
|
||||
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="delete_file">
|
||||
<details><summary>Tool use: **delete_file**</summary>
|
||||
|
||||
Explanation: undefined
|
||||
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="delete_file">
|
||||
<details><summary>Tool use: **delete_file**</summary>
|
||||
|
||||
Explanation: undefined
|
||||
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
Deleted the 4 files from the output directory:
|
||||
|
||||
- `2024-08-08 smithereens.ts.mp4` (audio stream count mismatch)
|
||||
- `2024-08-28 dejection.ts.mp4` (duration mismatch)
|
||||
- `2024-09-04 chaos.ts.mp4` (audio stream count mismatch)
|
||||
- `2024-09-05 resolute.ts.mp4` (duration mismatch)
|
||||
|
||||
No other files were modified.
|
||||
|
||||
---
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -1,496 +0,0 @@
|
||||
<!-- Generated by SpecStory, Markdown v2.1.0 -->
|
||||
|
||||
<!-- cursor Session 63dc8d99-2f1d-42e1-8f92-f9e14be8f352 (2025-10-08 02:35Z) -->
|
||||
|
||||
# Can damaged HEVC mp4s be repaired? (2025-10-08 02:35Z)
|
||||
|
||||
_**User (2025-10-08 02:35Z)**_
|
||||
|
||||
multiple damaged HEVC mp4s in input. can they be fixed?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model claude-4.5-sonnet-thinking, mode Agent)**_
|
||||
|
||||
<think><details><summary>Thought Process</summary>
|
||||
The user is asking about damaged HEVC mp4 files in the input directory. They want to know if they can be fixed. Let me first check what files are in the input directory to understand what we're working with.
|
||||
|
||||
I should:
|
||||
1. List the input directory to see what files are there
|
||||
2. Check if there are any tools or scripts already in place for fixing files
|
||||
3. Potentially suggest or implement a solution for fixing damaged MP4 files
|
||||
|
||||
Let me start by exploring the input directory.</details></think>
|
||||
|
||||
---
|
||||
|
||||
I'll help you check the damaged HEVC MP4 files in your input directory and see what can be done to fix them.
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="generic" data-tool-name="list_dir">
|
||||
<details>
|
||||
<summary>Tool use: **list_directory** • Listed directory undefined • **0** results</summary>
|
||||
|
||||
No results found
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<think><details><summary>Thought Process</summary>
|
||||
I can see there are many MP4 files in the input directory. The user says they are damaged HEVC MP4 files. Let me check one of the existing scripts to understand the structure, and then I can create a tool to analyze and potentially fix these files.
|
||||
|
||||
For damaged MP4 files, common fixes include:
|
||||
1. Using ffmpeg to remux (copy streams without re-encoding) which can fix container issues
|
||||
2. Using ffmpeg to recover what's possible from the damaged file
|
||||
3. Using specialized tools like untrunc, recover_mp4, or ffmpeg's error concealment
|
||||
|
||||
Let me check one of the existing encode scripts to see the pattern they use.</details></think>
|
||||
|
||||
---
|
||||
|
||||
I can see you have many DVR MP4 files. Let me check what kind of damage we're dealing with and create a tool to fix them. First, let me look at one of your existing scripts to understand your setup:
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="read" data-tool-name="read_file">
|
||||
<details><summary>Tool use: **read_file** • Read file: encode_dvr.py</summary>
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<think><details><summary>Thought Process</summary>
|
||||
Good, I can see the structure. Now let me create a repair tool for damaged MP4 files. The best approaches for fixing damaged HEVC MP4s are:
|
||||
|
||||
1. Try to remux with ffmpeg (copy streams to a new container)
|
||||
2. Use ffmpeg with error concealment to recover what's possible
|
||||
3. Try to rebuild the MP4 structure
|
||||
|
||||
Let me create a Python script that follows their existing pattern and attempts to fix damaged MP4s using ffmpeg.</details></think>
|
||||
|
||||
---
|
||||
|
||||
Now let me create a repair tool to analyze and fix the damaged MP4 files. I'll create a script that attempts several repair strategies:
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="write">
|
||||
|
||||
<details><summary>Tool use: **code_edit** • Edit file: repair_mp4.py</summary>
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 237, lines removed: 1
|
||||
|
||||
```diff
|
||||
@@ -1,1 +1,237 @@
|
||||
-
|
||||
+ import os
|
||||
+ import subprocess
|
||||
+ from pathlib import Path
|
||||
+ import json
|
||||
+ import logging
|
||||
+ from datetime import datetime
|
||||
+
|
||||
+ # ANSI color codes
|
||||
+ class Colors:
|
||||
+ PURPLE = '\033[95m'
|
||||
+ BLUE = '\033[94m'
|
||||
+ GREEN = '\033[92m'
|
||||
+ YELLOW = '\033[93m'
|
||||
+ RED = '\033[91m'
|
||||
+ ENDC = '\033[0m'
|
||||
+
|
||||
+ # Set up logging
|
||||
+ log_dir = "logs"
|
||||
+ os.makedirs(log_dir, exist_ok=True)
|
||||
+ log_file = os.path.join(log_dir, f"repair_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log")
|
||||
+ logging.basicConfig(filename=log_file, level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
|
||||
+
|
||||
+ def check_file_health(input_file):
|
||||
+ """Check if MP4 file has issues using ffmpeg"""
|
||||
+ print(f"\n{Colors.BLUE}Analyzing: {input_file}{Colors.ENDC}")
|
||||
+ logging.info(f"Analyzing file: {input_file}")
|
||||
+
|
||||
+ cmd = [
|
||||
+ 'ffmpeg',
|
||||
+ '-v', 'error',
|
||||
+ '-i', str(input_file),
|
||||
+ '-f', 'null',
|
||||
+ '-'
|
||||
+ ]
|
||||
+
|
||||
+ result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
+
|
||||
+ if result.stderr:
|
||||
+ print(f"{Colors.RED}Issues found:{Colors.ENDC}")
|
||||
+ for line in result.stderr.split('\n')[:10]: # Show first 10 errors
|
||||
+ if line.strip():
|
||||
+ print(f" {Colors.YELLOW}{line}{Colors.ENDC}")
|
||||
+ logging.warning(f"File has errors: {result.stderr[:500]}")
|
||||
+ return False
|
||||
+ else:
|
||||
+ print(f"{Colors.GREEN}No errors detected{Colors.ENDC}")
|
||||
+ logging.info("File appears healthy")
|
||||
+ return True
|
||||
+
|
||||
+ def get_file_info(input_file):
|
||||
+ """Get basic file info"""
|
||||
+ cmd = [
|
||||
+ 'ffprobe',
|
||||
+ '-v', 'error',
|
||||
+ '-show_entries', 'format=duration,size:stream=codec_name,width,height',
|
||||
+ '-of', 'json',
|
||||
+ input_file
|
||||
+ ]
|
||||
+ result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
+ try:
|
||||
+ return json.loads(result.stdout)
|
||||
+ except:
|
||||
+ return None
|
||||
+
|
||||
+ def repair_method_1_remux(input_file, output_file):
|
||||
+ """Method 1: Simple remux (copy streams to new container)"""
|
||||
+ print(f"\n{Colors.BLUE}Trying Method 1: Remux (copy streams){Colors.ENDC}")
|
||||
+ logging.info("Attempting repair method 1: Remux")
|
||||
+
|
||||
+ cmd = [
|
||||
+ 'ffmpeg',
|
||||
+ '-v', 'warning',
|
||||
+ '-i', str(input_file),
|
||||
+ '-c', 'copy',
|
||||
+ '-map', '0',
|
||||
+ '-y',
|
||||
+ str(output_file)
|
||||
+ ]
|
||||
+
|
||||
+ try:
|
||||
+ result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
|
||||
+ if result.returncode == 0 and Path(output_file).exists():
|
||||
+ print(f"{Colors.GREEN}Method 1 successful!{Colors.ENDC}")
|
||||
+ logging.info("Method 1 successful")
|
||||
+ return True
|
||||
+ else:
|
||||
+ print(f"{Colors.RED}Method 1 failed{Colors.ENDC}")
|
||||
+ logging.warning(f"Method 1 failed: {result.stderr[:200]}")
|
||||
+ return False
|
||||
+ except Exception as e:
|
||||
+ print(f"{Colors.RED}Method 1 error: {e}{Colors.ENDC}")
|
||||
+ logging.error(f"Method 1 error: {e}")
|
||||
+ return False
|
||||
+
|
||||
+ def repair_method_2_error_concealment(input_file, output_file):
|
||||
+ """Method 2: Remux with error concealment"""
|
||||
+ print(f"\n{Colors.BLUE}Trying Method 2: Error concealment{Colors.ENDC}")
|
||||
+ logging.info("Attempting repair method 2: Error concealment")
|
||||
+
|
||||
+ cmd = [
|
||||
+ 'ffmpeg',
|
||||
+ '-v', 'warning',
|
||||
+ '-err_detect', 'ignore_err',
|
||||
+ '-i', str(input_file),
|
||||
+ '-c', 'copy',
|
||||
+ '-map', '0',
|
||||
+ '-fflags', '+genpts+igndts',
|
||||
+ '-y',
|
||||
+ str(output_file)
|
||||
+ ]
|
||||
+
|
||||
+ try:
|
||||
+ result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
|
||||
+ if result.returncode == 0 and Path(output_file).exists():
|
||||
+ print(f"{Colors.GREEN}Method 2 successful!{Colors.ENDC}")
|
||||
+ logging.info("Method 2 successful")
|
||||
+ return True
|
||||
+ else:
|
||||
+ print(f"{Colors.RED}Method 2 failed{Colors.ENDC}")
|
||||
+ logging.warning(f"Method 2 failed: {result.stderr[:200]}")
|
||||
+ return False
|
||||
+ except Exception as e:
|
||||
+ print(f"{Colors.RED}Method 2 error: {e}{Colors.ENDC}")
|
||||
+ logging.error(f"Method 2 error: {e}")
|
||||
+ return False
|
||||
+
|
||||
+ def repair_method_3_force_keyframes(input_file, output_file):
|
||||
+ """Method 3: Force keyframe processing"""
|
||||
+ print(f"\n{Colors.BLUE}Trying Method 3: Force keyframe processing{Colors.ENDC}")
|
||||
+ logging.info("Attempting repair method 3: Force keyframe processing")
|
||||
+
|
||||
+ cmd = [
|
||||
+ 'ffmpeg',
|
||||
+ '-v', 'warning',
|
||||
+ '-fflags', '+discardcorrupt+genpts',
|
||||
+ '-i', str(input_file),
|
||||
+ '-c', 'copy',
|
||||
+ '-map', '0',
|
||||
+ '-avoid_negative_ts', 'make_zero',
|
||||
+ '-y',
|
||||
+ str(output_file)
|
||||
+ ]
|
||||
+
|
||||
+ try:
|
||||
+ result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
|
||||
+ if result.returncode == 0 and Path(output_file).exists():
|
||||
+ print(f"{Colors.GREEN}Method 3 successful!{Colors.ENDC}")
|
||||
+ logging.info("Method 3 successful")
|
||||
+ return True
|
||||
+ else:
|
||||
+ print(f"{Colors.RED}Method 3 failed{Colors.ENDC}")
|
||||
+ logging.warning(f"Method 3 failed: {result.stderr[:200]}")
|
||||
+ return False
|
||||
+ except Exception as e:
|
||||
+ print(f"{Colors.RED}Method 3 error: {e}{Colors.ENDC}")
|
||||
+ logging.error(f"Method 3 error: {e}")
|
||||
+ return False
|
||||
+
|
||||
+ def repair_file(input_file, output_dir):
|
||||
+ """Attempt to repair a damaged MP4 file"""
|
||||
+ input_path = Path(input_file)
|
||||
+ output_path = Path(output_dir) / f"{input_path.stem}_repaired{input_path.suffix}"
|
||||
+
|
||||
+ print(f"\n{Colors.PURPLE}{'='*60}{Colors.ENDC}")
|
||||
+ print(f"{Colors.PURPLE}Processing: {input_path.name}{Colors.ENDC}")
|
||||
+ print(f"{Colors.PURPLE}{'='*60}{Colors.ENDC}")
|
||||
+
|
||||
+ # Skip if already repaired
|
||||
+ if output_path.exists():
|
||||
+ print(f"{Colors.YELLOW}Already repaired: {output_path.name}{Colors.ENDC}")
|
||||
+ logging.info(f"Skipping {input_path} - already repaired")
|
||||
+ return
|
||||
+
|
||||
+ # Check file health first
|
||||
+ is_healthy = check_file_health(str(input_path))
|
||||
+
|
||||
+ if is_healthy:
|
||||
+ print(f"{Colors.GREEN}File appears healthy, no repair needed{Colors.ENDC}")
|
||||
+ logging.info("File is healthy, skipping")
|
||||
+ return
|
||||
+
|
||||
+ # Try repair methods in sequence
|
||||
+ methods = [
|
||||
+ repair_method_1_remux,
|
||||
+ repair_method_2_error_concealment,
|
||||
+ repair_method_3_force_keyframes
|
||||
+ ]
|
||||
+
|
||||
+ for method in methods:
|
||||
+ if method(str(input_path), str(output_path)):
|
||||
+ # Verify the repaired file
|
||||
+ print(f"\n{Colors.BLUE}Verifying repaired file...{Colors.ENDC}")
|
||||
+ if check_file_health(str(output_path)):
|
||||
+ print(f"{Colors.GREEN}✓ Repair successful and verified!{Colors.ENDC}")
|
||||
+ logging.info(f"Successfully repaired: {output_path}")
|
||||
+
|
||||
+ # Show size comparison
|
||||
+ original_size = input_path.stat().st_size / (1024*1024)
|
||||
+ repaired_size = output_path.stat().st_size / (1024*1024)
|
||||
+ print(f"Original: {original_size:.2f} MB → Repaired: {repaired_size:.2f} MB")
|
||||
+ logging.info(f"Size: {original_size:.2f} MB → {repaired_size:.2f} MB")
|
||||
+ return
|
||||
+ else:
|
||||
+ print(f"{Colors.YELLOW}Repair completed but file still has issues{Colors.ENDC}")
|
||||
+ logging.warning("Repaired file still has issues")
|
||||
+ return
|
||||
+
|
||||
+ print(f"{Colors.RED}All repair methods failed{Colors.ENDC}")
|
||||
+ logging.error(f"Failed to repair: {input_path}")
|
||||
+
|
||||
+ if __name__ == "__main__":
|
||||
+ input_dir = "input"
|
||||
+ output_dir = "output"
|
||||
+ os.makedirs(output_dir, exist_ok=True)
|
||||
+
|
||||
+ # Get list of MP4 files
|
||||
+ files = [f for f in os.listdir(input_dir) if f.endswith(('.mp4', '.DVR.mp4'))]
|
||||
+ total_files = len(files)
|
||||
+
|
||||
+ if total_files == 0:
|
||||
+ print(f"{Colors.YELLOW}No MP4 files found in input directory{Colors.ENDC}")
|
||||
+ logging.info("No files to process")
|
||||
+ else:
|
||||
+ print(f"\n{Colors.BLUE}Found {total_files} MP4 files to check{Colors.ENDC}")
|
||||
+ logging.info(f"Found {total_files} files to process")
|
||||
+
|
||||
+ for i, file in enumerate(files, 1):
|
||||
+ input_file = os.path.join(input_dir, file)
|
||||
+ print(f"\n{Colors.BLUE}[{i}/{total_files}]{Colors.ENDC}")
|
||||
+ repair_file(input_file, output_dir)
|
||||
+
|
||||
+ print(f"\n{Colors.GREEN}{'='*60}{Colors.ENDC}")
|
||||
+ print(f"{Colors.GREEN}Repair process complete!{Colors.ENDC}")
|
||||
+ print(f"{Colors.GREEN}Check the output directory for repaired files{Colors.ENDC}")
|
||||
+ print(f"{Colors.GREEN}Log file: {log_file}{Colors.ENDC}")
|
||||
+ print(f"{Colors.GREEN}{'='*60}{Colors.ENDC}")
|
||||
+
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="write" data-tool-name="write">
|
||||
|
||||
<details><summary>Tool use: **code_edit** • Edit file: repair_mp4.bat</summary>
|
||||
|
||||
**Chunk 1**
|
||||
Lines added: 4, lines removed: 1
|
||||
|
||||
```diff
|
||||
@@ -1,1 +1,4 @@
|
||||
-
|
||||
+ @echo off
|
||||
+ python repair_mp4.py
|
||||
+ pause
|
||||
+
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<think><details><summary>Thought Process</summary>
|
||||
Great! I've created the repair script. Now let me explain to the user what I've created and how it works. The script will:
|
||||
|
||||
1. Check each MP4 file for errors using ffmpeg
|
||||
2. Try multiple repair methods if issues are found
|
||||
3. Verify the repaired files
|
||||
4. Log everything
|
||||
|
||||
Let me provide a comprehensive explanation to the user.</details></think>
|
||||
|
||||
---
|
||||
|
||||
I've created a repair tool for your damaged HEVC MP4 files! Here's what it does:
|
||||
|
||||
## **`repair_mp4.py`** - MP4 Repair Tool
|
||||
|
||||
The script will:
|
||||
|
||||
1. **Analyze each file** - Uses ffmpeg to detect errors and issues
|
||||
2. **Skip healthy files** - Only attempts repair on damaged files
|
||||
3. **Try 3 repair methods** in sequence:
|
||||
- **Method 1**: Simple remux (copy streams to new container)
|
||||
- **Method 2**: Error concealment mode (ignore errors and regenerate timestamps)
|
||||
- **Method 3**: Force keyframe processing with corruption discard
|
||||
4. **Verify repairs** - Checks if the repaired file is actually fixed
|
||||
5. **Log everything** - Creates detailed logs in the `logs` directory
|
||||
|
||||
### Repaired files will be saved as:
|
||||
- Input: `input/filename.mp4`
|
||||
- Output: `output/filename_repaired.mp4`
|
||||
|
||||
### To use it:
|
||||
|
||||
Just run `repair_mp4.bat` or `python repair_mp4.py`
|
||||
|
||||
The script will automatically:
|
||||
- Process all MP4 files in the `input` directory
|
||||
- Show colored output with progress and results
|
||||
- Skip files that are already healthy
|
||||
- Skip files already repaired
|
||||
- Create a timestamped log file
|
||||
|
||||
### What causes MP4 damage?
|
||||
- Interrupted recordings (DVR stopped abruptly)
|
||||
- Missing/corrupted moov atom (MP4 metadata)
|
||||
- Corrupted keyframes
|
||||
- Invalid timestamps
|
||||
|
||||
These repair methods work well for most common DVR recording issues. If files can't be repaired, the script will tell you which ones failed and you can check the log for details.
|
||||
|
||||
Would you like me to run it now to see which files are damaged and attempt repairs?
|
||||
|
||||
---
|
||||
|
||||
_**User (2025-10-08 05:55Z)**_
|
||||
|
||||
@repair_20251007_203713.log
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model gpt-5, mode Agent)**_
|
||||
|
||||
<think><details><summary>Thought Process</summary>
|
||||
**Evaluating code logic**
|
||||
|
||||
I'm seeing a problem in the log after method 1 succeeds—it's showing errors and saying the "Repaired file still has issues." In our code, when verification fails, it prints that "Repair completed but the file still has issues" and then returns. This means we aren't continuing to methods 2 and 3 if method 1 has an output but isn't verified as healthy. We should adjust the design to allow proceeding with subsequent methods, ensuring they’re all attempted if verification fails. I'll need to edit the repair_mp4.py file for this.
|
||||
**Improving video processing methods**
|
||||
|
||||
I'm considering adding a method to re-encode the video track for fixing DTS issues. This could involve decoding and re-encoding using flags like `-fflags +genpts`, `-vsync 1`, or `-bsf:v hevc_mp4toannexb`. For non-monotonic DTS issues, re-muxing with `-fflags +genpts` may help too. I think we should also allow the script to proceed to methods 2 and 3 if method 1 reveals errors. Overall, I need to ensure that all methods are tested to fully address the problems.
|
||||
**Addressing HEVC stream errors**
|
||||
|
||||
When dealing with HEVC stream corruption and POC errors, I think transcoding the video stream to create clean frames while copying the audio could be essential. I could propose method 4, which involves re-encoding the video with `hevc_nvenc` and resetting PTS using specific flags. Alternatively, I might consider re-encoding to H.264 for compatibility, but since the user is using HEVC, I should focus on the best HEVC commands. Overall, I want to ensure the solution addresses the monotonic DTS issues effectively.
|
||||
**Resolving DTS issues**
|
||||
|
||||
The "non-monotonically increasing DTS to muxer" error seems to happen when decode timestamps are out of order. To fix this, I think using `-vsync 1` can enforce CFR, or possibly `-copyts 1` to keep input timestamps. Our method 2 uses `-err_detect ignore_err` and `-fflags +genpts+igndts`, which might be helpful. For method 3, it employs `-fflags +discardcorrupt+genpts`. I should consider trying both methods. I could even explore using `-vsync 1` or `-vsync 2` to manage frame rates as part of this solution.
|
||||
**Analyzing HEVC MP4 Issues**
|
||||
|
||||
I've reviewed the user's question about fixing multiple damaged HEVC MP4 files, and the log indicates most have DTS monotonic issues. Simple remuxing didn't resolve the problems, and our script didn't move on to advanced steps due to control-flow issues. I propose modifying the script to continue to the next methods and add a re-encode option to regenerate timestamps. Since the user didn't request a new script, I'll recommend commands they can run instead. I can provide concise commands for PowerShell that could help fix the files. Let’s outline some clear options for them!
|
||||
**Preparing FFmpeg Commands**
|
||||
|
||||
I’ll provide a command for re-encoding videos with FFmpeg, specifically using HEVC settings. Here's a sample command that utilizes hardware acceleration:
|
||||
|
||||
```
|
||||
ffmpeg -v warning -hwaccel cuda -i "in.mp4" -map 0:v:0 -map 0:a? -c:v hevc_nvenc -preset p6 -tune hq -rc vbr -cq 28 -g 240 -fflags +genpts -vsync 1 -c:a copy -movflags +faststart -max_interleave_delta 0 -avoid_negative_ts make_zero "output\in_clean.mp4"
|
||||
```
|
||||
|
||||
For a PowerShell one-liner that processes files with errors:
|
||||
|
||||
```
|
||||
Get-ChildItem input -File -Include *.mp4, *.DVR.mp4 | ForEach-Object { $out = Join-Path output ($_.BaseName + "_clean" + $_.Extension); ffmpeg -v warning -hwaccel cuda -fflags +genpts -err_detect ignore_err -i $_.FullName -map 0 -c:v hevc_nvenc -preset p6 -tune hq -rc vbr -cq 28 -g 240 -c:a copy -movflags +faststart -max_interleave_delta 0 -avoid_negative_ts make_zero $out }
|
||||
```
|
||||
|
||||
If needed, I can update `repair_mp4.py` to continue processing methods after failures and add a re-encode option. Most files are experiencing 'non-monotonically increasing DTS' issues, persisting despite remuxing. Many files appear healthy, but the warnings remain.
|
||||
|
||||
I'm ready to assist with further actions!</details></think>
|
||||
|
||||
---
|
||||
|
||||
- I reviewed your `repair_20251007_203713.log`. The script only tried Method 1 (remux) and then stopped even when verification still showed errors. It never attempted Methods 2/3. Most failures are timestamp issues: “non monotonically increasing dts”; a few have actual HEVC decode corruption (POC/ref errors). Many files are fine.
|
||||
|
||||
### What this means
|
||||
- **Timestamp-only issues (DTS/PTS)**: Usually fixable by regenerating timestamps; remux alone often isn’t enough.
|
||||
- **Decode corruption (HEVC/H264 errors)**: Requires re-encoding video to make a clean stream; audio can usually be copied.
|
||||
|
||||
### Quick fixes you can run (no new scripts)
|
||||
- **Try “copy with regenerated PTS” (handles many DTS issues):**
|
||||
```powershell
|
||||
# One file example
|
||||
ffmpeg -v warning -err_detect ignore_err -fflags +genpts+igndts -i "input\FILE.mp4" -map 0 -c copy -max_interleave_delta 0 -movflags +faststart -avoid_negative_ts make_zero "output\FILE_ptsfix.mp4"
|
||||
```
|
||||
|
||||
- **If still broken, re-encode video, copy audio (fixes DTS + decode errors):**
|
||||
```powershell
|
||||
# NVIDIA (uses your GPU)
|
||||
ffmpeg -v warning -hwaccel cuda -fflags +genpts -err_detect ignore_err -i "input\FILE.mp4" -map 0:v:0 -map 0:a? -c:v hevc_nvenc -preset p6 -tune hq -rc vbr -cq 28 -g 240 -vsync 1 -c:a copy -movflags +faststart -max_interleave_delta 0 -avoid_negative_ts make_zero "output\FILE_clean.mp4"
|
||||
```
|
||||
|
||||
- **Batch across all MP4/DVR files (re-encode video, copy audio):**
|
||||
```powershell
|
||||
Get-ChildItem input -File -Include *.mp4,*.DVR.mp4 | ForEach-Object {
|
||||
$out = Join-Path output ($_.BaseName + "_clean" + $_.Extension)
|
||||
ffmpeg -v warning -hwaccel cuda -fflags +genpts -err_detect ignore_err -i $_.FullName `
|
||||
-map 0:v:0 -map 0:a? -c:v hevc_nvenc -preset p6 -tune hq -rc vbr -cq 28 -g 240 -vsync 1 `
|
||||
-c:a copy -movflags +faststart -max_interleave_delta 0 -avoid_negative_ts make_zero $out
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `-fflags +genpts`/`-vsync 1` rebuild timestamps; `-err_detect ignore_err` skips bad frames; `-g 240` adds periodic keyframes; audio is copied to avoid quality loss.
|
||||
- This will repair both the DTS warnings and most HEVC “POC/ref” decode errors by producing a clean stream.
|
||||
|
||||
### Optional: fix your tool
|
||||
If you want, I can update `repair_mp4.py` to:
|
||||
- Keep trying Method 2 and 3 when Method 1’s verification fails.
|
||||
- Add a Method 4 that re-encodes video as shown above (audio copy), only used if 1–3 fail.
|
||||
|
||||
Say “update the repair script” and I’ll apply the edits.
|
||||
|
||||
---
|
||||
|
||||
@@ -1,186 +0,0 @@
|
||||
<!-- Generated by SpecStory, Markdown v2.1.0 -->
|
||||
|
||||
<!-- cursor Session 3bab80f9-de45-4af3-9283-05d0a836948d (2025-11-07 07:55Z) -->
|
||||
|
||||
# OBS update issues and encoding speed (2025-11-07 07:55Z)
|
||||
|
||||
_**User (2025-11-07 07:55Z)**_
|
||||
|
||||
Kinda crazy. OBS updating murdered all of my preferences, so I forgot to change the default rescaled output and default framerate from 60. My @encode_VOD.py script recoded the 2hr+ file in 18 minutes. @1080p60, I would normally expect that to take at least an hour... That's the difference between ~2x speed and ~7x speed... @encode_20251107_003149.log
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reviewing the log to understand why encoding was faster than expected.
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="read" data-tool-name="read_file">
|
||||
<details><summary>Tool use: **read_file** • Read file: logs/encode_20251107_003149.log</summary>
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="read" data-tool-name="read_file">
|
||||
<details><summary>Tool use: **read_file** • Read file: encode_VOD.py</summary>
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
Checking key sections of the log for input file details and encoding speed:
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="read" data-tool-name="read_file">
|
||||
<details><summary>Tool use: **read_file** • Read file: logs/encode_20251107_003149.log</summary>
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="search" data-tool-name="grep">
|
||||
<details>
|
||||
<summary>Tool use: **grep** • Grep for "width|height|r_frame_rate|fps=|frame=" in "logs/encode_20251107_003149.log" • 49 matches</summary>
|
||||
|
||||
Output mode: content
|
||||
|
||||
|
||||
| File | Content | Line |
|
||||
|------|------|------|
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,958 - INFO - width: 1280` | L9 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,959 - INFO - height: 720` | L10 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,959 - INFO - r_frame_rate: 30/1` | L11 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,960 - INFO - r_frame_rate: 0/0` | L16 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,961 - INFO - r_frame_rate: 0/0` | L21 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:53,413 - INFO - Progress: frame= 211 fps=205 q=3.0 size= 3072KiB time=00:00:06.86 bitrate=3665.0kbits/s speed=6.69x elapsed=0:00:01.02` | L87 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:54,433 - INFO - Progress: frame= 478 fps=235 q=3.0 size= 3584KiB time=00:00:15.83 bitrate=1854.3kbits/s speed=7.78x elapsed=0:00:02.03` | L88 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:55,436 - INFO - Progress: frame= 783 fps=256 q=3.0 size= 4352KiB time=00:00:26.00 bitrate=1371.2kbits/s speed=8.51x elapsed=0:00:03.05` | L89 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:56,458 - INFO - Progress: frame= 1080 fps=266 q=3.0 size= 5376KiB time=00:00:35.90 bitrate=1226.8kbits/s speed=8.84x elapsed=0:00:04.05` | L90 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:57,475 - INFO - Progress: frame= 1382 fps=272 q=3.0 size= 6144KiB time=00:00:45.96 bitrate=1095.0kbits/s speed=9.05x elapsed=0:00:05.08` | L91 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:58,493 - INFO - Progress: frame= 1684 fps=276 q=3.0 size= 9984KiB time=00:00:56.03 bitrate=1459.7kbits/s speed=9.19x elapsed=0:00:06.09` | L92 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:59,511 - INFO - Progress: frame= 1982 fps=279 q=4.0 size= 14336KiB time=00:01:05.96 bitrate=1780.3kbits/s speed=9.27x elapsed=0:00:07.11` | L93 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:00,515 - INFO - Progress: frame= 2282 fps=281 q=4.0 size= 20224KiB time=00:01:15.96 bitrate=2180.9kbits/s speed=9.34x elapsed=0:00:08.13` | L94 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:01,534 - INFO - Progress: frame= 2552 fps=279 q=2.0 size= 23808KiB time=00:01:25.10 bitrate=2291.8kbits/s speed=9.31x elapsed=0:00:09.13` | L95 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:02,558 - INFO - Progress: frame= 2854 fps=281 q=2.0 size= 28160KiB time=00:01:35.03 bitrate=2427.4kbits/s speed=9.36x elapsed=0:00:10.15` | L96 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:03,564 - INFO - Progress: frame= 3156 fps=282 q=4.0 size= 32000KiB time=00:01:45.10 bitrate=2494.2kbits/s speed= 9.4x elapsed=0:00:11.18` | L97 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:04,583 - INFO - Progress: frame= 3454 fps=283 q=2.0 size= 34816KiB time=00:01:55.03 bitrate=2479.4kbits/s speed=9.44x elapsed=0:00:12.18` | L98 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:05,599 - INFO - Progress: frame= 3746 fps=284 q=3.0 size= 38912KiB time=00:02:04.76 bitrate=2554.9kbits/s speed=9.45x elapsed=0:00:13.20` | L99 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:06,620 - INFO - Progress: frame= 3910 fps=275 q=4.0 size= 40192KiB time=00:02:10.23 bitrate=2528.2kbits/s speed=9.16x elapsed=0:00:14.22` | L100 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:07,627 - INFO - Progress: frame= 4212 fps=276 q=4.0 size= 44544KiB time=00:02:20.30 bitrate=2600.9kbits/s speed= 9.2x elapsed=0:00:15.24` | L101 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:08,645 - INFO - Progress: frame= 4442 fps=273 q=4.0 size= 47104KiB time=00:02:27.96 bitrate=2607.9kbits/s speed=9.11x elapsed=0:00:16.25` | L102 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:09,664 - INFO - Progress: frame= 4744 fps=275 q=4.0 size= 50688KiB time=00:02:38.03 bitrate=2627.5kbits/s speed=9.15x elapsed=0:00:17.26` | L103 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:10,681 - INFO - Progress: frame= 5044 fps=276 q=4.0 size= 54784KiB time=00:02:48.03 bitrate=2670.8kbits/s speed=9.19x elapsed=0:00:18.28` | L104 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:11,688 - INFO - Progress: frame= 5342 fps=277 q=4.0 size= 58624KiB time=00:02:57.96 bitrate=2698.5kbits/s speed=9.22x elapsed=0:00:19.30` | L105 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:12,707 - INFO - Progress: frame= 5616 fps=276 q=4.0 size= 62464KiB time=00:03:07.10 bitrate=2734.9kbits/s speed=9.21x elapsed=0:00:20.31` | L106 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:13,727 - INFO - Progress: frame= 5918 fps=277 q=4.0 size= 66304KiB time=00:03:17.16 bitrate=2754.8kbits/s speed=9.24x elapsed=0:00:21.33` | L107 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:14,737 - INFO - Progress: frame= 6220 fps=278 q=4.0 size= 70656KiB time=00:03:27.23 bitrate=2793.1kbits/s speed=9.27x elapsed=0:00:22.35` | L108 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:15,755 - INFO - Progress: frame= 6516 fps=279 q=3.0 size= 74496KiB time=00:03:37.10 bitrate=2811.0kbits/s speed=9.29x elapsed=0:00:23.36` | L109 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:16,774 - INFO - Progress: frame= 6818 fps=280 q=12.0 size= 79616KiB time=00:03:47.16 bitrate=2871.1kbits/s speed=9.32x elapsed=0:00:24.37` | L110 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:17,794 - INFO - Progress: frame= 7116 fps=280 q=9.0 size= 92160KiB time=00:03:57.10 bitrate=3184.2kbits/s speed=9.34x elapsed=0:00:25.39` | L111 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:18,798 - INFO - Progress: frame= 7383 fps=279 q=9.0 size= 103424KiB time=00:04:06.00 bitrate=3444.1kbits/s speed=9.31x elapsed=0:00:26.41` | L112 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:19,818 - INFO - Progress: frame= 7594 fps=277 q=11.0 size= 112384KiB time=00:04:13.03 bitrate=3638.5kbits/s speed=9.23x elapsed=0:00:27.42` | L113 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:20,836 - INFO - Progress: frame= 7878 fps=277 q=14.0 size= 124160KiB time=00:04:22.50 bitrate=3874.7kbits/s speed=9.23x elapsed=0:00:28.44` | L114 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:21,843 - INFO - Progress: frame= 8165 fps=277 q=9.0 size= 136192KiB time=00:04:32.06 bitrate=4100.8kbits/s speed=9.24x elapsed=0:00:29.45` | L115 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:22,867 - INFO - Progress: frame= 8434 fps=277 q=17.0 size= 147712KiB time=00:04:41.03 bitrate=4305.7kbits/s speed=9.22x elapsed=0:00:30.46` | L116 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:23,874 - INFO - Progress: frame= 8708 fps=277 q=16.0 size= 158720KiB time=00:04:50.16 bitrate=4481.0kbits/s speed=9.21x elapsed=0:00:31.49` | L117 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:24,894 - INFO - Progress: frame= 8984 fps=276 q=18.0 size= 170496KiB time=00:04:59.40 bitrate=4665.0kbits/s speed=9.21x elapsed=0:00:32.49` | L118 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:25,904 - INFO - Progress: frame= 9279 fps=277 q=17.0 size= 182528KiB time=00:05:09.20 bitrate=4835.9kbits/s speed=9.23x elapsed=0:00:33.51` | L119 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:26,928 - INFO - Progress: frame= 9567 fps=277 q=10.0 size= 195072KiB time=00:05:18.80 bitrate=5012.6kbits/s speed=9.23x elapsed=0:00:34.52` | L120 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:27,949 - INFO - Progress: frame= 9863 fps=277 q=11.0 size= 207104KiB time=00:05:28.70 bitrate=5161.5kbits/s speed=9.25x elapsed=0:00:35.55` | L121 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:28,961 - INFO - Progress: frame=10121 fps=277 q=12.0 size= 217856KiB time=00:05:37.26 bitrate=5291.6kbits/s speed=9.22x elapsed=0:00:36.57` | L122 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:29,980 - INFO - Progress: frame=10398 fps=277 q=-1.0 size= 228352KiB time=00:05:46.50 bitrate=5398.7kbits/s speed=9.22x elapsed=0:00:37.58` | L123 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:30,991 - INFO - Progress: frame=10663 fps=276 q=4.0 size= 237568KiB time=00:05:55.33 bitrate=5477.0kbits/s speed= 9.2x elapsed=0:00:38.60` | L124 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:32,001 - INFO - Progress: frame=10933 fps=276 q=9.0 size= 248576KiB time=00:06:04.33 bitrate=5589.2kbits/s speed= 9.2x elapsed=0:00:39.61` | L125 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:33,022 - INFO - Progress: frame=11176 fps=275 q=8.0 size= 258560KiB time=00:06:12.43 bitrate=5687.3kbits/s speed=9.17x elapsed=0:00:40.62` | L126 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:34,039 - INFO - Progress: frame=11472 fps=275 q=8.0 size= 271104KiB time=00:06:22.30 bitrate=5809.3kbits/s speed=9.18x elapsed=0:00:41.64` | L127 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:35,058 - INFO - Progress: frame=11774 fps=276 q=11.0 size= 283392KiB time=00:06:32.36 bitrate=5916.8kbits/s speed= 9.2x elapsed=0:00:42.66` | L128 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:36,070 - INFO - Progress: frame=12049 fps=276 q=5.0 size= 294656KiB time=00:06:41.53 bitrate=6011.5kbits/s speed=9.19x elapsed=0:00:43.68` | L129 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:32:37,085 - INFO - Progress: frame=12274 fps=275 q=12.0 size= 304128KiB time=00:06:49.03 bitrate=6091.0kbits/s speed=9.15x elapsed=0:00:44.69` | L130 |
|
||||
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
Checking the end of the log for completion stats:
|
||||
|
||||
|
||||
---
|
||||
|
||||
<tool-use data-tool-type="search" data-tool-name="grep">
|
||||
<details>
|
||||
<summary>Tool use: **grep** • Grep for "Successfully encoded|Compression ratio|Processing file" in "logs/encode_20251107_003149.log" • 6 matches</summary>
|
||||
|
||||
Output mode: content
|
||||
|
||||
|
||||
| File | Content | Line |
|
||||
|------|------|------|
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,612 - INFO - Processing file 1/2: 2025-11-06 graveyard_gameplay.mp4` | L3 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:31:50,956 - INFO - Processing file: input\2025-11-06 graveyard_gameplay.mp4` | L4 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:50:34,966 - INFO - Successfully encoded: output\2025-11-06 graveyard_gameplay.mp4` | L1194 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:50:34,967 - INFO - Compression ratio: 2.78x` | L1196 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:50:34,968 - INFO - Processing file 2/2: 2025-11-06 graveyard.ts.mp4` | L1197 |
|
||||
| `logs/encode_20251107_003149.log` | `2025-11-07 00:50:35,767 - INFO - Processing file: input\2025-11-06 graveyard.ts.mp4` | L1198 |
|
||||
|
||||
|
||||
</details>
|
||||
</tool-use>
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
The log shows why it was faster than expected:
|
||||
|
||||
**Input file specs:**
|
||||
- Resolution: **1280x720** (not 1080p)
|
||||
- Frame rate: **30 fps** (not 60 fps)
|
||||
- Duration: 2h 14m 30s (8070 seconds)
|
||||
|
||||
**Encoding performance:**
|
||||
- Speed: consistently **~9.2x–9.4x**
|
||||
- Total time: ~18 minutes (00:31:50 → 00:50:34)
|
||||
|
||||
**Why it was faster:**
|
||||
|
||||
720p30 vs 1080p60:
|
||||
- Pixels: 921,600 vs 2,073,600 (2.25x fewer)
|
||||
- Frame rate: 30 vs 60 fps (2x fewer frames)
|
||||
- Combined: ~4.5x less work
|
||||
|
||||
So ~9x speed for 720p30 roughly matches ~2x speed for 1080p60, which aligns with your expectation.
|
||||
|
||||
The script is working as expected; the input was recorded at 720p30 due to OBS settings resetting. If you re-encode a true 1080p60 file, expect ~2x speed.
|
||||
|
||||
The encoding completed successfully with a **2.78x compression ratio** (19.10 GB → ~6.87 GB).
|
||||
|
||||
---
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,26 +0,0 @@
|
||||
<!-- Generated by SpecStory, Markdown v2.1.0 -->
|
||||
|
||||
<!-- cursor Session a72eb019-4b76-4b1a-aa9c-82a980c5b71d (2026-01-18 18:26Z) -->
|
||||
|
||||
# Video encoding time calculation (2026-01-18 18:26Z)
|
||||
|
||||
_**User**_
|
||||
|
||||
at 2.26x speed, how long to encode 4:03:33 video?
|
||||
|
||||
---
|
||||
|
||||
_**Agent (model default, mode Agent)**_
|
||||
|
||||
|
||||
At 2.26x speed, encoding a 4:03:33 video takes about 1:47:47.
|
||||
|
||||
**Calculation:**
|
||||
- Video duration: 4:03:33 = 14,613 seconds
|
||||
- Encoding time: 14,613 ÷ 2.26 ≈ 6,467 seconds
|
||||
- Converted: **1 hour, 47 minutes, 47 seconds** (1:47:47)
|
||||
|
||||
Roughly 1 hour 48 minutes.
|
||||
|
||||
---
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,33 @@
|
||||
20260118_My ex wife has hired a lawyer.mp4
|
||||
20260118_Wife left and took the kids.mp4
|
||||
20260119_My ex wife has hired a lawyer.mp4
|
||||
20260121_nothing ever happens.mp4
|
||||
20260124_nothing ever happens.mp4
|
||||
20260125_nothing ever happens.mp4
|
||||
20260126_nothing ever happens.mp4
|
||||
20260128_nothing ever happens.mp4
|
||||
20260129_nothing ever happens.mp4
|
||||
20260131_nothing ever happens.mp4
|
||||
20260203_1 nothing ever happens.mp4
|
||||
20260203_2 nothing ever happens.mp4
|
||||
20260203_3 nothing ever happens.mp4
|
||||
20260204_1 Recording Main Channel Video (Please make unfunny donos to get in the video).mp4
|
||||
20260204_2 Recording Main Channel Video (Please make unfunny donos to get in the video).mp4
|
||||
20260205_1 nothing ever happens.mp4
|
||||
20260205_2 I trapped my wife inside a Black Soul Gem to boost the FPS of my Nintendo Switch.mp4
|
||||
20260207_nothing ever happens.mp4
|
||||
20260209_Five Night's At Freddy's 2.mp4
|
||||
20260210_Five Night's At Freddy's 2.mp4
|
||||
20260212_Paying off my Wife's alimony #Sponsored.mp4
|
||||
20260221_POPPY PLAYTIME CHAPTER 5 (After react andy).mp4
|
||||
20260222_POPPY PLAYTIME CHAPTER 5.mp4
|
||||
20260223_1 Poppy Playtime Video Tonight.mp4
|
||||
20260223_2 Poppy Playtime Video Tonight.mp4
|
||||
20260224_HUGE REACTION ANDY.mp4
|
||||
20260225_HUGE REACTION ANDY.mp4
|
||||
20260303_RESIDENT EVIL： REQUIEM.mp4
|
||||
20260304_RESIDENT EVIL： REQUIEM.mp4
|
||||
20260305_RESIDENT EVIL： REQUIEM.mp4
|
||||
20260307_FIND MY PAGES.mp4
|
||||
20260309_1 NOTHING EVER HAPPENS.mp4
|
||||
20260309_2 NOTHING EVER HAPPENS.mp4
|
||||
+65
-44
@@ -31,7 +31,7 @@ def get_file_info(file_path):
|
||||
'-of', 'json',
|
||||
str(file_path)
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, encoding='utf-8', errors='replace')
|
||||
if result.returncode != 0:
|
||||
return None, f"ffprobe failed: {result.stderr}"
|
||||
info = json.loads(result.stdout)
|
||||
@@ -41,6 +41,15 @@ def get_file_info(file_path):
|
||||
except Exception as e:
|
||||
return None, f"Error running ffprobe: {str(e)}"
|
||||
|
||||
def expected_mp4_output_name(input_filename):
|
||||
return Path(input_filename).stem + '.mp4'
|
||||
|
||||
def is_input_video_file(name):
|
||||
return name.lower().endswith(('.mp4', '.mkv', '.mov'))
|
||||
|
||||
def is_output_mp4_file(name):
|
||||
return name.lower().endswith('.mp4')
|
||||
|
||||
def check_file(input_path, output_path):
|
||||
try:
|
||||
input_info, input_error = get_file_info(input_path)
|
||||
@@ -166,7 +175,7 @@ def main():
|
||||
# Get list of files that exist in both directories, verifying each file exists
|
||||
input_files = set()
|
||||
for f in os.listdir(input_dir):
|
||||
if f.endswith(('.mp4', '.DVR.mp4')):
|
||||
if is_input_video_file(f):
|
||||
full_path = os.path.join(input_dir, f)
|
||||
if os.path.isfile(full_path):
|
||||
input_files.add(f)
|
||||
@@ -176,7 +185,7 @@ def main():
|
||||
|
||||
output_files = set()
|
||||
for f in os.listdir(output_dir):
|
||||
if f.endswith(('.mp4', '.DVR.mp4')):
|
||||
if is_output_mp4_file(f):
|
||||
full_path = os.path.join(output_dir, f)
|
||||
if os.path.isfile(full_path):
|
||||
output_files.add(f)
|
||||
@@ -193,47 +202,59 @@ def main():
|
||||
for f in sorted(output_files):
|
||||
logging.info(f" {f}")
|
||||
|
||||
# Report missing files
|
||||
missing_in_output = input_files - output_files
|
||||
missing_in_input = output_files - input_files
|
||||
|
||||
if missing_in_output:
|
||||
input_stems = {Path(f).stem for f in input_files}
|
||||
|
||||
# Report missing files (output is always {stem}.mp4 for encoder workflow)
|
||||
missing_in_output_inputs = sorted(
|
||||
inf for inf in input_files if expected_mp4_output_name(inf) not in output_files
|
||||
)
|
||||
missing_in_input_outputs = sorted(
|
||||
outf for outf in output_files if Path(outf).stem not in input_stems
|
||||
)
|
||||
|
||||
if missing_in_output_inputs:
|
||||
print(f"\n{Colors.YELLOW}Files missing from output directory:{Colors.ENDC}")
|
||||
for f in sorted(missing_in_output):
|
||||
print(f" {f}")
|
||||
logging.warning(f"File missing from output: {f}")
|
||||
issues_found.append((f, "Missing from output"))
|
||||
for inf in missing_in_output_inputs:
|
||||
outp = expected_mp4_output_name(inf)
|
||||
print(f" {inf} → expected {outp}")
|
||||
logging.warning(f"File missing from output: {inf} (expected {outp})")
|
||||
issues_found.append((inf, outp, "Missing from output"))
|
||||
issue_counts["Missing from output"] += 1
|
||||
|
||||
if missing_in_input:
|
||||
|
||||
if missing_in_input_outputs:
|
||||
print(f"\n{Colors.YELLOW}Files missing from input directory:{Colors.ENDC}")
|
||||
for f in sorted(missing_in_input):
|
||||
print(f" {f}")
|
||||
logging.warning(f"File missing from input: {f}")
|
||||
issues_found.append((f, "Missing from input"))
|
||||
for outf in missing_in_input_outputs:
|
||||
print(f" {outf}")
|
||||
logging.warning(f"File missing from input: {outf}")
|
||||
issues_found.append((None, outf, "Missing from input"))
|
||||
issue_counts["Missing from input"] += 1
|
||||
|
||||
files_to_check = input_files.intersection(output_files)
|
||||
logging.info(f"Number of matching files found: {len(files_to_check)}")
|
||||
|
||||
if not files_to_check:
|
||||
msg = "No matching files found in both input and output directories"
|
||||
|
||||
pairs_to_check = []
|
||||
for inf in sorted(input_files):
|
||||
outp = expected_mp4_output_name(inf)
|
||||
if outp in output_files:
|
||||
pairs_to_check.append((inf, outp))
|
||||
|
||||
logging.info(f"Number of input/output pairs to verify: {len(pairs_to_check)}")
|
||||
|
||||
if not pairs_to_check and not missing_in_output_inputs and not missing_in_input_outputs:
|
||||
msg = "No input video files and no mismatches to report"
|
||||
print(f"\n{Colors.YELLOW}{msg}{Colors.ENDC}")
|
||||
logging.warning(msg)
|
||||
return
|
||||
|
||||
for input_file in sorted(files_to_check):
|
||||
|
||||
for input_file, output_file in pairs_to_check:
|
||||
input_path = os.path.join(input_dir, input_file)
|
||||
output_path = os.path.join(output_dir, input_file)
|
||||
|
||||
print(f"\nChecking: {input_file}")
|
||||
logging.info(f"Checking file: {input_file}")
|
||||
|
||||
output_path = os.path.join(output_dir, output_file)
|
||||
|
||||
print(f"\nChecking: {input_file} ↔ {output_file}")
|
||||
logging.info(f"Checking file: {input_file} vs output {output_file}")
|
||||
|
||||
issue = check_file(input_path, output_path)
|
||||
if issue:
|
||||
issues_found.append((input_file, issue))
|
||||
issues_found.append((input_file, output_file, issue))
|
||||
print(f"{Colors.RED}ISSUE FOUND: {issue}{Colors.ENDC}")
|
||||
logging.warning(f"Issue with {input_file}: {issue}")
|
||||
logging.warning(f"Issue with {input_file} → {output_file}: {issue}")
|
||||
|
||||
# Count the type of issue
|
||||
if "Duration mismatch" in issue:
|
||||
@@ -250,8 +271,8 @@ def main():
|
||||
issue_counts["Other errors"] += 1
|
||||
else:
|
||||
print(f"{Colors.GREEN}No issues found{Colors.ENDC}")
|
||||
logging.info(f"No issues found for {input_file}")
|
||||
|
||||
logging.info(f"No issues found for {input_file} → {output_file}")
|
||||
|
||||
# Print summary
|
||||
print("\nSummary of issues found:")
|
||||
if issues_found:
|
||||
@@ -261,8 +282,9 @@ def main():
|
||||
print(f"{issue_type}: {color}{count}{Colors.ENDC}")
|
||||
|
||||
print("\nDetailed issues:")
|
||||
for file, issue in issues_found:
|
||||
print(f"\n{file}:")
|
||||
for in_base, out_base, issue in issues_found:
|
||||
label = f"{in_base} → {out_base}" if in_base else str(out_base)
|
||||
print(f"\n{label}:")
|
||||
print(f"{Colors.RED} {issue}{Colors.ENDC}")
|
||||
|
||||
# Prompt to delete problem files
|
||||
@@ -273,23 +295,22 @@ def main():
|
||||
deleted_count = 0
|
||||
failed_count = 0
|
||||
|
||||
for file, issue in issues_found:
|
||||
# Skip "Missing from output" - nothing to delete
|
||||
for in_base, out_base, issue in issues_found:
|
||||
if issue == "Missing from output":
|
||||
continue
|
||||
|
||||
# Delete output file for all other issues
|
||||
output_path = os.path.join(output_dir, file)
|
||||
if not out_base:
|
||||
continue
|
||||
output_path = os.path.join(output_dir, out_base)
|
||||
if os.path.exists(output_path):
|
||||
try:
|
||||
os.remove(output_path)
|
||||
deleted_count += 1
|
||||
logging.info(f"Deleted problematic output file: {output_path} (Issue: {issue})")
|
||||
print(f"{Colors.GREEN}Deleted: {file}{Colors.ENDC}")
|
||||
print(f"{Colors.GREEN}Deleted: {out_base}{Colors.ENDC}")
|
||||
except Exception as e:
|
||||
failed_count += 1
|
||||
logging.error(f"Failed to delete {output_path}: {str(e)}")
|
||||
print(f"{Colors.RED}Failed to delete {file}: {str(e)}{Colors.ENDC}")
|
||||
print(f"{Colors.RED}Failed to delete {out_base}: {str(e)}{Colors.ENDC}")
|
||||
|
||||
print(f"\n{Colors.GREEN}Deleted {deleted_count} file(s){Colors.ENDC}")
|
||||
if failed_count > 0:
|
||||
|
||||
+6
-4
@@ -82,7 +82,7 @@ def get_file_info(input_file):
|
||||
'-of', 'json',
|
||||
input_file
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, encoding='utf-8', errors='replace')
|
||||
return json.loads(result.stdout)
|
||||
|
||||
def get_audio_labels(input_file):
|
||||
@@ -94,7 +94,7 @@ def get_audio_labels(input_file):
|
||||
'-of', 'json',
|
||||
input_file
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, encoding='utf-8', errors='replace')
|
||||
info = json.loads(result.stdout)
|
||||
labels = []
|
||||
for stream in info.get('streams', []):
|
||||
@@ -163,7 +163,7 @@ def safe_log_error(message, print_msg=None):
|
||||
|
||||
def encode_dvr(input_file, output_dir, gpu):
|
||||
input_path = Path(input_file)
|
||||
output_path = Path(output_dir) / f"{input_path.stem}{input_path.suffix}"
|
||||
output_path = Path(output_dir) / f"{input_path.stem}.mp4"
|
||||
|
||||
# Get file info for logging
|
||||
file_info = get_file_info(str(input_path))
|
||||
@@ -232,6 +232,8 @@ def encode_dvr(input_file, output_dir, gpu):
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
universal_newlines=True,
|
||||
encoding='utf-8',
|
||||
errors='replace',
|
||||
bufsize=1
|
||||
)
|
||||
|
||||
@@ -288,7 +290,7 @@ if __name__ == "__main__":
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
# Get list of files to process
|
||||
files = [f for f in os.listdir(input_dir) if f.endswith(('.mp4', '.DVR.mp4'))]
|
||||
files = [f for f in os.listdir(input_dir) if f.lower().endswith(('.mp4', '.mkv', '.mov'))]
|
||||
total_files = len(files)
|
||||
|
||||
if total_files == 0:
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
@echo off
|
||||
python encode_VOD_bitrate.py
|
||||
pause
|
||||
@@ -1,5 +1,4 @@
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
import json
|
||||
@@ -28,6 +27,19 @@ def get_gpu_selection():
|
||||
return gpu
|
||||
print(f"{Colors.RED}Invalid selection. Please try again.{Colors.ENDC}")
|
||||
|
||||
def get_bitrate():
|
||||
while True:
|
||||
print(f"\n{Colors.BLUE}Select video bitrate:{Colors.ENDC}")
|
||||
print("Common values: 5000k (5 Mbps), 8000k (8 Mbps), 10000k (10 Mbps), 15000k (15 Mbps)")
|
||||
bitrate = input(f"{Colors.YELLOW}Enter bitrate in kbps (e.g., 10000 for 10 Mbps):{Colors.ENDC} ").strip()
|
||||
try:
|
||||
bitrate_val = int(bitrate)
|
||||
if bitrate_val > 0:
|
||||
return bitrate_val
|
||||
print(f"{Colors.RED}Bitrate must be positive. Please try again.{Colors.ENDC}")
|
||||
except ValueError:
|
||||
print(f"{Colors.RED}Invalid input. Please enter a number.{Colors.ENDC}")
|
||||
|
||||
# Custom file handler that silently handles I/O errors (for network shares)
|
||||
class SafeFileHandler(logging.FileHandler):
|
||||
"""File handler that silently handles I/O errors during flush"""
|
||||
@@ -36,26 +48,21 @@ class SafeFileHandler(logging.FileHandler):
|
||||
try:
|
||||
super().flush()
|
||||
except (OSError, IOError):
|
||||
# Silently ignore I/O errors (network share issues)
|
||||
pass
|
||||
except Exception:
|
||||
# Silently ignore all other errors during flush
|
||||
pass
|
||||
|
||||
|
||||
def emit(self, record):
|
||||
"""Override emit to handle errors gracefully"""
|
||||
try:
|
||||
super().emit(record)
|
||||
except (OSError, IOError):
|
||||
# Silently ignore I/O errors - we'll fall back to console output
|
||||
self.handleError(record)
|
||||
except Exception:
|
||||
# Handle other errors
|
||||
self.handleError(record)
|
||||
|
||||
|
||||
def handleError(self, record):
|
||||
"""Override to prevent error messages from being printed"""
|
||||
# Don't print "--- Logging error ---" messages
|
||||
pass
|
||||
|
||||
# Set up logging
|
||||
@@ -63,7 +70,6 @@ log_dir = "logs"
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
log_file = os.path.join(log_dir, f"encode_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log")
|
||||
|
||||
# Configure logging with custom handler that handles network share errors
|
||||
handler = SafeFileHandler(log_file, mode='w', encoding='utf-8')
|
||||
handler.setLevel(logging.INFO)
|
||||
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
|
||||
@@ -72,18 +78,17 @@ handler.setFormatter(formatter)
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.INFO)
|
||||
logger.addHandler(handler)
|
||||
# Remove default handlers to avoid duplicate output
|
||||
logger.handlers = [handler]
|
||||
|
||||
def get_file_info(input_file):
|
||||
cmd = [
|
||||
'ffprobe',
|
||||
'-v', 'error',
|
||||
'-show_entries', 'format=duration,size:stream=codec_type,codec_name,width,height,r_frame_rate,channels,channel_layout',
|
||||
'-show_entries', 'format=duration,size:stream=codec_name,width,height,r_frame_rate,channels,channel_layout',
|
||||
'-of', 'json',
|
||||
input_file
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, encoding='utf-8', errors='replace')
|
||||
return json.loads(result.stdout)
|
||||
|
||||
def get_audio_labels(input_file):
|
||||
@@ -95,7 +100,7 @@ def get_audio_labels(input_file):
|
||||
'-of', 'json',
|
||||
input_file
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, encoding='utf-8', errors='replace')
|
||||
info = json.loads(result.stdout)
|
||||
labels = []
|
||||
for stream in info.get('streams', []):
|
||||
@@ -103,18 +108,6 @@ def get_audio_labels(input_file):
|
||||
labels.append(title)
|
||||
return labels
|
||||
|
||||
def parse_fps(r_frame_rate):
|
||||
"""Parse ffprobe r_frame_rate e.g. '60/1' or '30000/1001' to float."""
|
||||
if not r_frame_rate:
|
||||
return None
|
||||
parts = str(r_frame_rate).strip().split('/')
|
||||
if len(parts) == 2 and float(parts[1]) != 0:
|
||||
return float(parts[0]) / float(parts[1])
|
||||
try:
|
||||
return float(parts[0])
|
||||
except (ValueError, IndexError):
|
||||
return None
|
||||
|
||||
def format_size(size_bytes):
|
||||
for unit in ['B', 'KB', 'MB', 'GB']:
|
||||
if size_bytes < 1024:
|
||||
@@ -123,11 +116,9 @@ def format_size(size_bytes):
|
||||
return f"{size_bytes:.2f} TB"
|
||||
|
||||
def safe_log_info(message, print_msg=None):
|
||||
"""Safely log info message, ensuring console output even if logging fails"""
|
||||
try:
|
||||
logging.info(message)
|
||||
except (OSError, IOError) as e:
|
||||
# Logging failed (likely network share issue) - print to console
|
||||
if print_msg is None:
|
||||
print(f"{Colors.YELLOW}[Logging failed: {e}] {message}{Colors.ENDC}")
|
||||
else:
|
||||
@@ -135,24 +126,20 @@ def safe_log_info(message, print_msg=None):
|
||||
print(f"{Colors.YELLOW}[Logging failed: {e}]{Colors.ENDC}")
|
||||
return
|
||||
except Exception as e:
|
||||
# Other logging errors
|
||||
if print_msg is None:
|
||||
print(f"{Colors.YELLOW}[Logging error: {e}] {message}{Colors.ENDC}")
|
||||
else:
|
||||
print(print_msg)
|
||||
print(f"{Colors.YELLOW}[Logging error: {e}]{Colors.ENDC}")
|
||||
return
|
||||
|
||||
# Always print to console if message provided
|
||||
|
||||
if print_msg is not None:
|
||||
print(print_msg)
|
||||
|
||||
def safe_log_error(message, print_msg=None):
|
||||
"""Safely log error message, ensuring console output even if logging fails"""
|
||||
try:
|
||||
logging.error(message)
|
||||
except (OSError, IOError) as e:
|
||||
# Logging failed (likely network share issue) - print to console
|
||||
if print_msg is None:
|
||||
print(f"{Colors.RED}[Logging failed: {e}] {message}{Colors.ENDC}")
|
||||
else:
|
||||
@@ -160,45 +147,46 @@ def safe_log_error(message, print_msg=None):
|
||||
print(f"{Colors.RED}[Logging failed: {e}]{Colors.ENDC}")
|
||||
return
|
||||
except Exception as e:
|
||||
# Other logging errors
|
||||
if print_msg is None:
|
||||
print(f"{Colors.RED}[Logging error: {e}] {message}{Colors.ENDC}")
|
||||
else:
|
||||
print(print_msg)
|
||||
print(f"{Colors.RED}[Logging error: {e}]{Colors.ENDC}")
|
||||
return
|
||||
|
||||
# Always print to console if message provided
|
||||
|
||||
if print_msg is not None:
|
||||
print(print_msg)
|
||||
else:
|
||||
print(f"{Colors.RED}{message}{Colors.ENDC}")
|
||||
|
||||
def encode_dvr(input_file, output_dir, gpu):
|
||||
def encode_dvr(input_file, output_dir, gpu, bitrate):
|
||||
input_path = Path(input_file)
|
||||
output_path = Path(output_dir) / f"{input_path.stem}{input_path.suffix}"
|
||||
output_path = Path(output_dir) / f"{input_path.stem}.mp4"
|
||||
|
||||
b_v = f"{bitrate}k"
|
||||
maxrate = f"{bitrate}k"
|
||||
bufsize = f"{bitrate * 2}k"
|
||||
|
||||
# Get file info for logging
|
||||
file_info = get_file_info(str(input_path))
|
||||
input_size = int(file_info['format']['size'])
|
||||
duration = float(file_info['format']['duration'])
|
||||
|
||||
|
||||
safe_log_info(f"Processing file: {input_path}")
|
||||
safe_log_info(f"Input size: {format_size(input_size)}")
|
||||
safe_log_info(f"Duration: {duration:.2f} seconds")
|
||||
|
||||
safe_log_info(f"Bitrate settings - b:v: {b_v}, maxrate: {maxrate}, bufsize: {bufsize}")
|
||||
|
||||
print(f"\n{Colors.BLUE}Processing file: {input_path}{Colors.ENDC}")
|
||||
print(f"Input size: {format_size(input_size)}")
|
||||
print(f"Duration: {duration:.2f} seconds")
|
||||
|
||||
# Log stream information
|
||||
print(f"Bitrate settings - b:v: {b_v}, maxrate: {maxrate}, bufsize: {bufsize}")
|
||||
|
||||
for i, stream in enumerate(file_info.get('streams', [])):
|
||||
stream_type = 'Video' if stream.get('codec_name', '').startswith('h') else 'Audio'
|
||||
safe_log_info(f"Stream {i} ({stream_type}):")
|
||||
for key, value in stream.items():
|
||||
safe_log_info(f" {key}: {value}")
|
||||
|
||||
# Skip if output file already exists
|
||||
if output_path.exists():
|
||||
output_size = output_path.stat().st_size
|
||||
safe_log_info(f"Skipping {input_path} - output already exists: {output_path}")
|
||||
@@ -206,32 +194,14 @@ def encode_dvr(input_file, output_dir, gpu):
|
||||
print(f"{Colors.YELLOW}Skipping {input_path} - output already exists{Colors.ENDC}")
|
||||
return
|
||||
|
||||
# Get audio labels and input FPS (for speed display when stderr is piped)
|
||||
audio_labels = get_audio_labels(str(input_path))
|
||||
safe_log_info(f"Audio labels: {audio_labels}")
|
||||
input_fps = None
|
||||
for s in file_info.get('streams', []):
|
||||
if s.get('codec_type') == 'video':
|
||||
input_fps = parse_fps(s.get('r_frame_rate'))
|
||||
break
|
||||
if input_fps is None:
|
||||
# Fallback: first stream with r_frame_rate (e.g. codec_type not in probe)
|
||||
for s in file_info.get('streams', []):
|
||||
if 'width' in s or s.get('codec_name', '').startswith(('h', 'm')):
|
||||
input_fps = parse_fps(s.get('r_frame_rate'))
|
||||
break
|
||||
if input_fps is None:
|
||||
for s in file_info.get('streams', []):
|
||||
input_fps = parse_fps(s.get('r_frame_rate'))
|
||||
if input_fps and input_fps > 0:
|
||||
break
|
||||
|
||||
# FFmpeg command with NVIDIA HEVC encoder and maximum quality
|
||||
cmd = [
|
||||
'ffmpeg',
|
||||
'-v', 'info', # Lower verbosity to reduce noise
|
||||
'-stats', # Emit periodic stats
|
||||
'-stats_period', '1.0', # Update stats every 1s (more stable)
|
||||
'-v', 'info',
|
||||
'-stats',
|
||||
'-stats_period', '1.0',
|
||||
'-i', str(input_path),
|
||||
'-c:v', 'hevc_nvenc',
|
||||
'-gpu', gpu,
|
||||
@@ -242,118 +212,87 @@ def encode_dvr(input_file, output_dir, gpu):
|
||||
'-spatial-aq', '1',
|
||||
'-aq-strength', '15',
|
||||
'-cq', '0',
|
||||
'-b:v', '9000k',
|
||||
'-maxrate', '9000k',
|
||||
'-bufsize', '18000k',
|
||||
'-b:v', b_v,
|
||||
'-maxrate', maxrate,
|
||||
'-bufsize', bufsize,
|
||||
'-c:a', 'copy',
|
||||
'-map', '0',
|
||||
]
|
||||
# Add metadata for each audio stream if label exists
|
||||
for idx, label in enumerate(audio_labels):
|
||||
if label:
|
||||
cmd += [f'-metadata:s:a:{idx}', f'title={label}']
|
||||
cmd.append(str(output_path))
|
||||
|
||||
try:
|
||||
# Run FFmpeg and capture combined output (avoid dual-pipe deadlocks on Windows)
|
||||
process = subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
universal_newlines=True,
|
||||
encoding='utf-8',
|
||||
errors='replace',
|
||||
bufsize=1
|
||||
)
|
||||
|
||||
# Stream output line-by-line; compute speed when stderr is piped (FFmpeg shows speed=N/A)
|
||||
for line in iter(process.stdout.readline, ''):
|
||||
if not line:
|
||||
break
|
||||
text = line.strip().strip('\r')
|
||||
text = line.strip()
|
||||
try:
|
||||
if text.startswith('frame=') or ' fps=' in text:
|
||||
display = text
|
||||
m_frame = re.search(r'frame=\s*(\d+)', text)
|
||||
m_elapsed = re.search(r'elapsed=(\d+):(\d+):(\d+\.?\d*)', text)
|
||||
m_size = re.search(r'size=\s*(\d+)\s*KiB', text)
|
||||
if m_frame and m_elapsed:
|
||||
frames = int(m_frame.group(1))
|
||||
h, m, s = float(m_elapsed.group(1)), float(m_elapsed.group(2)), float(m_elapsed.group(3))
|
||||
elapsed_sec = h * 3600 + m * 60 + s
|
||||
if elapsed_sec > 0:
|
||||
if input_fps and input_fps > 0:
|
||||
speed_x = (frames / input_fps) / elapsed_sec
|
||||
display = re.sub(r'speed=N/A', f'speed={speed_x:.2f}x', text)
|
||||
# time = output position (HH:MM:SS.ms)
|
||||
video_sec = frames / input_fps
|
||||
t_h = int(video_sec // 3600)
|
||||
t_m = int((video_sec % 3600) // 60)
|
||||
t_s = video_sec % 60
|
||||
time_str = f'{t_h}:{t_m:02d}:{t_s:06.3f}' if t_h else f'0:{t_m:02d}:{t_s:06.3f}'
|
||||
display = re.sub(r'time=N/A', f'time={time_str}', display)
|
||||
# bitrate when muxer has written data (stays N/A until size > 0)
|
||||
size_kib = int(m_size.group(1)) if m_size else 0
|
||||
if size_kib > 0 and elapsed_sec > 0:
|
||||
bitrate_kbps = (size_kib * 8192) / (elapsed_sec * 1000)
|
||||
display = re.sub(r'bitrate=N/A', f'bitrate={bitrate_kbps:.0f}kbits/s', display)
|
||||
else:
|
||||
enc_fps = frames / elapsed_sec
|
||||
display = re.sub(r'speed=N/A', f'speed={enc_fps:.0f}fps', text)
|
||||
safe_log_info(f"Progress: {text}", f"{Colors.PURPLE}Progress: {display}{Colors.ENDC}")
|
||||
safe_log_info(f"Progress: {text}", f"{Colors.PURPLE}Progress: {text}{Colors.ENDC}")
|
||||
else:
|
||||
safe_log_info(f"FFmpeg: {text}", f"{Colors.GREEN}FFmpeg: {text}{Colors.ENDC}")
|
||||
except (OSError, IOError) as e:
|
||||
# I/O error reading from pipe - log it
|
||||
safe_log_error(f"I/O error reading FFmpeg output: {e}")
|
||||
break
|
||||
except Exception as e:
|
||||
# Unexpected error
|
||||
safe_log_error(f"Unexpected error processing FFmpeg output: {e}")
|
||||
|
||||
process.wait()
|
||||
|
||||
if process.returncode == 0:
|
||||
# Get output file info
|
||||
output_info = get_file_info(str(output_path))
|
||||
output_size = int(output_info['format']['size'])
|
||||
compression_ratio = input_size / output_size if output_size > 0 else 0
|
||||
|
||||
|
||||
safe_log_info(f"Successfully encoded: {output_path}", f"{Colors.GREEN}Successfully encoded: {output_path}{Colors.ENDC}")
|
||||
safe_log_info(f"Output size: {format_size(output_size)}")
|
||||
safe_log_info(f"Compression ratio: {compression_ratio:.2f}x", f"Compression ratio: {compression_ratio:.2f}x")
|
||||
else:
|
||||
# Convert Windows error code to signed integer if needed
|
||||
return_code = process.returncode
|
||||
if return_code > 2147483647: # If it's a large unsigned int, convert to signed
|
||||
if return_code > 2147483647:
|
||||
return_code = return_code - 4294967296
|
||||
safe_log_error(f"FFmpeg process failed with return code {return_code}",
|
||||
safe_log_error(f"FFmpeg process failed with return code {return_code}",
|
||||
f"{Colors.RED}FFmpeg process failed with return code {return_code}{Colors.ENDC}")
|
||||
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
safe_log_error(f"Error encoding {input_path}: {e}", f"{Colors.RED}Error encoding {input_path}: {e}{Colors.ENDC}")
|
||||
except Exception as e:
|
||||
safe_log_error(f"Unexpected error encoding {input_path}: {type(e).__name__}: {e}",
|
||||
safe_log_error(f"Unexpected error encoding {input_path}: {type(e).__name__}: {e}",
|
||||
f"{Colors.RED}Unexpected error encoding {input_path}: {e}{Colors.ENDC}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Get GPU selection
|
||||
gpu = get_gpu_selection()
|
||||
safe_log_info(f"Selected GPU: {gpu}")
|
||||
|
||||
|
||||
bitrate = get_bitrate()
|
||||
safe_log_info(f"Selected bitrate: {bitrate}k")
|
||||
|
||||
input_dir = "input"
|
||||
output_dir = "output"
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
# Get list of files to process
|
||||
files = [f for f in os.listdir(input_dir) if f.endswith(('.mp4', '.DVR.mp4'))]
|
||||
|
||||
files = [f for f in os.listdir(input_dir) if f.lower().endswith(('.mp4', '.mkv', '.mov'))]
|
||||
total_files = len(files)
|
||||
|
||||
|
||||
if total_files == 0:
|
||||
safe_log_info("No files to process in input directory", f"{Colors.YELLOW}No files to process in input directory{Colors.ENDC}")
|
||||
else:
|
||||
safe_log_info(f"Found {total_files} files to process", f"{Colors.BLUE}Found {total_files} files to process{Colors.ENDC}")
|
||||
|
||||
|
||||
for i, file in enumerate(files, 1):
|
||||
input_file = os.path.join(input_dir, file)
|
||||
safe_log_info(f"Processing file {i}/{total_files}: {file}")
|
||||
print(f"\n{Colors.BLUE}Processing file {i}/{total_files}: {file}{Colors.ENDC}")
|
||||
encode_dvr(input_file, output_dir, gpu)
|
||||
encode_dvr(input_file, output_dir, gpu, bitrate)
|
||||
@@ -1,3 +0,0 @@
|
||||
@echo off
|
||||
python encode_VOD_pyro.py
|
||||
pause
|
||||
+2
-2
@@ -162,7 +162,7 @@ def safe_log_error(message, print_msg=None):
|
||||
|
||||
def encode_dvr(input_file, output_dir, gpu):
|
||||
input_path = Path(input_file)
|
||||
output_path = Path(output_dir) / f"{input_path.stem}{input_path.suffix}"
|
||||
output_path = Path(output_dir) / f"{input_path.stem}.mp4"
|
||||
|
||||
# Get file info for logging
|
||||
file_info = get_file_info(str(input_path))
|
||||
@@ -282,7 +282,7 @@ if __name__ == "__main__":
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
# Get list of files to process
|
||||
files = [f for f in os.listdir(input_dir) if f.endswith(('.mp4', '.DVR.mp4'))]
|
||||
files = [f for f in os.listdir(input_dir) if f.lower().endswith(('.mp4', '.mkv', '.mov'))]
|
||||
total_files = len(files)
|
||||
|
||||
if total_files == 0:
|
||||
|
||||
@@ -1,263 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
from queue import SimpleQueue, Empty
|
||||
from subprocess import Popen, DEVNULL, PIPE, TimeoutExpired
|
||||
import select
|
||||
from shutil import rmtree
|
||||
from glob import glob
|
||||
from os import mkdir, unlink, listdir, environ
|
||||
from os.path import basename, abspath, expanduser, isfile, isdir, getsize
|
||||
from threading import Thread
|
||||
from time import sleep
|
||||
from typing import List, NamedTuple, Callable, Union
|
||||
from signal import signal, SIGINT
|
||||
from sys import exit, stderr
|
||||
from shlex import split, join
|
||||
from hashlib import md5
|
||||
from time import strptime
|
||||
from tqdm import tqdm
|
||||
import re
|
||||
|
||||
DEBUG = 'DEBUG' in environ
|
||||
|
||||
def dprint(*args, **kwargs):
|
||||
if DEBUG:
|
||||
print(*args, *kwargs)
|
||||
|
||||
def _popen(args, **kwargs):
|
||||
dprint(f'calling subprocess: {args}')
|
||||
kwargs['stderr'] = PIPE
|
||||
kwargs['stdout'] = DEVNULL
|
||||
kwargs['universal_newlines'] = True
|
||||
return Popen(args, **kwargs)
|
||||
|
||||
class Task(NamedTuple):
|
||||
input_file: str
|
||||
output_file: str
|
||||
ffmpeg_args: List[str] = []
|
||||
|
||||
class FFMPEGProc:
|
||||
_duration_re = re.compile(r'.*Duration:\s*-?(?P<time_h>[0-9]+):(?P<time_m>[0-9]+):(?P<time_s>[0-9.]+),')
|
||||
_progress_re = re.compile(r'frame=\s*(?P<frame>[0-9]+)\s+fps=\s*(?P<fps>[0-9]+).*time=-?(?P<time_h>[0-9]+):(?P<time_m>[0-9]+):(?P<time_s>[0-9,.]+)\s+.*speed=(?P<speed>[0-9\.]+)x')
|
||||
|
||||
@staticmethod
|
||||
def _match_to_sec(match):
|
||||
return int(match.group('time_h'))*3600+int(match.group('time_m'))*60+float(match.group('time_s'))
|
||||
|
||||
def __init__(self, cmd: Union[list, str], shell=False, stdin=DEVNULL, stdout=DEVNULL, update_callback: Callable[[int,int,float,float,float], None] = None):
|
||||
self._cmd = cmd
|
||||
self._update_callback = update_callback
|
||||
self._should_stop = False
|
||||
self._shell = shell
|
||||
self._duration = None
|
||||
self._stdin = stdin
|
||||
self._stdout = stdout
|
||||
self.stderr = ''
|
||||
|
||||
def stop(self):
|
||||
self._should_stop = True
|
||||
|
||||
def run(self):
|
||||
self._proc = Popen(self._cmd, shell=self._shell, stderr=PIPE, stdin=self._stdin, stdout=self._stdout, universal_newlines=True)
|
||||
poll = select.poll()
|
||||
poll.register(self._proc.stderr)
|
||||
while self._proc.poll() is None and not self._should_stop:
|
||||
if not poll.poll(1):
|
||||
sleep(0.1)
|
||||
continue
|
||||
sleep(0.001)
|
||||
line = self._proc.stderr.readline()
|
||||
match = self._progress_re.match(line)
|
||||
if not match:
|
||||
self.stderr += line
|
||||
|
||||
if match and self._update_callback:
|
||||
self._update_callback(
|
||||
int(match.group('frame')),
|
||||
int(match.group('fps')),
|
||||
self._match_to_sec(match),
|
||||
self._duration,
|
||||
float(match.group('speed'))
|
||||
)
|
||||
elif self._duration is None:
|
||||
match = self._duration_re.match(line)
|
||||
if match:
|
||||
self._duration = self._match_to_sec(match)
|
||||
|
||||
try:
|
||||
out, err = self._proc.communicate(timeout=1)
|
||||
self.stderr += err
|
||||
except TimeoutExpired as ex:
|
||||
pass
|
||||
return self._proc.returncode
|
||||
|
||||
class TqdmAbsolute(tqdm):
|
||||
def __init__(self, *args, **kwargs):
|
||||
kwargs['bar_format'] = '{l_bar}{bar}|{n:.1f}/{total:.1f} [{elapsed}<{remaining}]'
|
||||
kwargs['dynamic_ncols'] = True
|
||||
if not 'total' in kwargs:
|
||||
kwargs['total'] = 99999999
|
||||
if not 'leave' in kwargs:
|
||||
kwargs['leave'] = False
|
||||
super().__init__(*args, **kwargs)
|
||||
def update(self, to):
|
||||
super().update(to - self.n) # will also set self.n = b * bsize
|
||||
|
||||
class TaskThread(Thread):
|
||||
def __init__(self, host: str, task_queue: SimpleQueue, bar_pos):
|
||||
super().__init__()
|
||||
self._should_stop = False
|
||||
self._host = host
|
||||
self._task_queue = task_queue
|
||||
self._ffmpeg = None
|
||||
self._bar = TqdmAbsolute(desc=host, position=bar_pos)
|
||||
self._current_file = None
|
||||
|
||||
def stop(self):
|
||||
self._should_stop = True
|
||||
if self._ffmpeg:
|
||||
self._ffmpeg.stop()
|
||||
|
||||
def run(self):
|
||||
def upd(frames, fps, time, duration, speed):
|
||||
self._bar.total = duration or 999
|
||||
self._bar.desc = self._host + ': ' + self._current_file
|
||||
self._bar.update(time)
|
||||
try:
|
||||
while not self._should_stop:
|
||||
task = self._task_queue.get(False)
|
||||
|
||||
self._current_file = basename(task.input_file)
|
||||
with open(task.input_file, 'r') as infile, open(task.output_file, 'w') as outfile:
|
||||
ffmpeg_cmd = [
|
||||
'nice', '-n10', 'ionice', '-c3',
|
||||
'ffmpeg', '-f', 'matroska', '-i', 'pipe:',
|
||||
*task.ffmpeg_args,
|
||||
'-f', 'matroska', 'pipe:'
|
||||
]
|
||||
if self._host != 'localhost':
|
||||
ffmpeg_cmd = ['ssh', self._host, join(ffmpeg_cmd)]
|
||||
self._ffmpeg = FFMPEGProc(ffmpeg_cmd, stdin=infile, stdout=outfile, update_callback=upd)
|
||||
|
||||
ret = self._ffmpeg.run()
|
||||
if ret != 0:
|
||||
tqdm.write(f'task for {self._current_file} failed on host {self._host}', file=stderr)
|
||||
tqdm.write(self._ffmpeg.stderr, file=stderr)
|
||||
self._task_queue.put(task)
|
||||
except Empty:
|
||||
pass
|
||||
self._bar.close()
|
||||
|
||||
def encode(hosts: List[str], input_file: str, output_file: str, segment_seconds: float = 60, remote_args: str = '', concat_args: str = '', tmp_dir: str = None, keep_tmp=False, resume=False, copy_input=False):
|
||||
input_file = abspath(expanduser(input_file))
|
||||
output_file = abspath(expanduser(output_file))
|
||||
tmp_dir = tmp_dir or 'ffmpeg_segments_'+md5(input_file.encode()).hexdigest()
|
||||
tmp_in = f'{tmp_dir}/in'
|
||||
tmp_out = f'{tmp_dir}/out'
|
||||
try:
|
||||
mkdir(tmp_dir)
|
||||
mkdir(tmp_in)
|
||||
mkdir(tmp_out)
|
||||
except FileExistsError:
|
||||
if not resume:
|
||||
raise
|
||||
|
||||
# skip splitting on resume
|
||||
if len(listdir(tmp_in)) == 0 or not resume:
|
||||
cv = ['copy'] if copy_input else ['libx264', '-crf', '0', '-preset', 'ultrafast', '-bf', '0']
|
||||
with TqdmAbsolute(desc="splitting input file") as bar:
|
||||
def upd(frames, fps, time, duration, speed):
|
||||
bar.total = duration
|
||||
bar.update(time)
|
||||
ffmpeg = FFMPEGProc([
|
||||
'ffmpeg', '-i', expanduser(input_file),
|
||||
'-an', '-sn',
|
||||
'-c:v', *cv,
|
||||
'-f', 'segment', '-reset_timestamps', '1', '-segment_time', str(segment_seconds) + 's',
|
||||
tmp_in + '/%08d.mkv'
|
||||
],
|
||||
update_callback=upd
|
||||
)
|
||||
ret = ffmpeg.run()
|
||||
if ret != 0:
|
||||
tqdm.write(ffmpeg.stderr, file=stderr)
|
||||
return
|
||||
|
||||
task_queue = SimpleQueue()
|
||||
for f in sorted(glob(tmp_in+'/*')):
|
||||
output_segment = tmp_out+f'/{basename(f)}'
|
||||
# skip already encoded segments
|
||||
if not isfile(output_segment):
|
||||
task_queue.put(Task(f, output_segment, split(remote_args)))
|
||||
|
||||
threads = [TaskThread(host, task_queue, pos) for pos,host in enumerate(hosts,0)]
|
||||
|
||||
def sigint(sig, stack):
|
||||
print('Got SIGINT, stopping...')
|
||||
for thread in threads:
|
||||
thread.stop()
|
||||
for thread in threads:
|
||||
thread.join()
|
||||
exit(1)
|
||||
|
||||
signal(SIGINT, sigint)
|
||||
|
||||
for thread in threads:
|
||||
thread.start()
|
||||
for thread in threads:
|
||||
thread.join()
|
||||
|
||||
with open('output_segments.txt', 'w') as f:
|
||||
f.write('\n'.join([f"file '{file}'" for file in sorted(glob(tmp_out+'/*'))]))
|
||||
|
||||
with TqdmAbsolute(desc='concatenating output segments') as bar:
|
||||
def upd(frames, fps, time, duration, speed):
|
||||
bar.total = duration
|
||||
bar.update(time)
|
||||
ffmpeg = FFMPEGProc([
|
||||
'ffmpeg', '-i', input_file,
|
||||
'-f', 'concat', '-safe', '0', '-i', 'output_segments.txt',
|
||||
'-map_metadata', '0:g',
|
||||
'-map', '1:v',
|
||||
'-map', '0:a?',
|
||||
'-map', '0:s?',
|
||||
'-c:v', 'copy',
|
||||
'-c:s', 'copy',
|
||||
*split(concat_args),
|
||||
'-y', output_file
|
||||
],
|
||||
update_callback=upd
|
||||
)
|
||||
if ffmpeg.run() != 0:
|
||||
tqdm.write(ffmpeg.stderr, file=stderr)
|
||||
return
|
||||
unlink('output_segments.txt')
|
||||
|
||||
if not keep_tmp:
|
||||
rmtree(tmp_dir)
|
||||
|
||||
if __name__ == '__main__':
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description='Splits a file into segments and processes them on multiple hosts in parallel using ffmpeg over SSH.')
|
||||
parser.add_argument('input_file', help='File to encode.')
|
||||
parser.add_argument('output_file', help='Path to encoded output file.')
|
||||
parser.add_argument('remote_args', help='Arguments to pass to the remote ffmpeg instances. For example: "-c:v libx264 -crf 23 -preset fast"')
|
||||
parser.add_argument('concat_args', default='', help='Arguments to pass to the local ffmpeg concatenating the processed video segments and muxing it with the original audio/subs/metadata. Mainly useful for audio encoding options, or "-an" to get rid of it.')
|
||||
parser.add_argument('-s', '--segment-length', type=float, default=10, help='Segment length in seconds.')
|
||||
parser.add_argument('-H', '--host', action='append', help='SSH hostname(s) to encode on. Use "localhost" to include the machine you\'re running this from. Can include username.', required=True)
|
||||
parser.add_argument('-k', '--keep-tmp', action='store_true', help='Keep temporary segment files instead of deleting them on successful exit.')
|
||||
parser.add_argument('-r', '--resume', action='store_true', help='Don\'t split the input file again, keep existing segments and only process the missing ones.')
|
||||
parser.add_argument('-t', '--tmp-dir', default=None, help='Directory to use for temporary files. Should not already exist and will be deleted afterwards.')
|
||||
parser.add_argument('-c', '--copy-input', action='store_true', help='Don\'t (losslessly) re-encode input while segmenting. Only use this if your input segments frame-perfectly with "-c:v copy" (i.e. it has no B-frames)')
|
||||
args = parser.parse_args()
|
||||
encode(
|
||||
args.host,
|
||||
args.input_file,
|
||||
args.output_file,
|
||||
segment_seconds=args.segment_length,
|
||||
remote_args=args.remote_args,
|
||||
concat_args=args.concat_args,
|
||||
tmp_dir=args.tmp_dir,
|
||||
keep_tmp=args.keep_tmp,
|
||||
resume=args.resume,
|
||||
copy_input=args.copy_input
|
||||
)
|
||||
Reference in New Issue
Block a user