⏱️ 3 min read

The Ultimate Guide to Automated Library Pruning

Table of Contents

As your recording library grows, storage management becomes a critical part of your workflow. An unmanaged library isn’t just a waste of disk space; it leads to slower metadata indexing, longer search times, and increased wear on your RAID arrays.

This guide covers how to implement Automated Library Pruning—the process of identifying and removing files that no longer serve your business goals, based on objective technical metadata.

The Library Performance Tax

Every terabyte of stale data carries a performance tax. When you have 50,000+ clips, standard file explorers and recording managers start to lag. By pruning your library, you ensure that your active datasets (the ones you actually use for VODs or training) remain fast and accessible.

mastering find for Metadata

The foundation of pruning is the find command. It allows you to target files precisely based on age, size, and location.

Basic Age-Based Pruning

To find (and optionally delete) all .mp4 files older than 90 days:

# Dry run: list files first!
find /path/to/archives -name "*.mp4" -mtime +90

# Destructive: delete files
# find /path/to/archives -name "*.mp4" -mtime +90 -delete

Safety First: The Dry Run

Never run a deletion command without a dry run. Always pipe the output to a log file or review it in the terminal before adding the -delete flag.

Deleting Low-Bitrate Clips (Python)

Sometimes age isn’t the best metric. You might want to keep a 5-year-old 4K high-bitrate masterpiece but delete a 1-week-old 720p clip that suffered from network congestion.

Since CaptureGem saves a metadata.json alongside every recording, we can use Python to parse that metadata and delete files that fall below a specific quality threshold.

import os
import json

ARCHIVE_DIR = "/home/user/recordings"
MIN_BITRATE_KBPS = 2500  # 2.5 Mbps

def prune_low_quality():
    for root, dirs, files in os.walk(ARCHIVE_DIR):
        for file in files:
            if file.endswith("metadata.json"):
                meta_path = os.path.join(root, file)
                try:
                    with open(meta_path, 'r') as f:
                        data = json.load(f)
                        bitrate = data.get("stream", {}).get("bitrate", 0)
                        
                        if bitrate < MIN_BITRATE_KBPS:
                            # Identify the video file
                            video_file = meta_path.replace(".metadata.json", ".mp4")
                            if os.path.exists(video_file):
                                print(f"DRY RUN: Deleting low-bitrate clip: {video_file} ({bitrate} kbps)")
                                # os.remove(video_file)
                                # os.remove(meta_path)
                except Exception as e:
                    print(f"Error parsing {meta_path}: {e}")

if __name__ == "__main__":
    prune_low_quality()

Automation with cron

Once you have a pruning script or command that works, you should automate it. On Linux, cron is the standard tool for scheduled tasks.

To run your pruning script every Sunday at 3:00 AM, add this to your crontab -e:

0 3 * * 0 /usr/bin/python3 /home/user/scripts/prune_recordings.py >> /home/user/logs/prune.log 2>&1

Summary Checklist

  1. Define Goals: Are you pruning by age, bitrate, or specific model tags?
  2. Write the Script: Use find for simple tasks, Python for metadata-heavy logic.
  3. Dry Run: Always verify the list of files to be deleted.
  4. Schedule: Use cron to keep your library clean without manual intervention.
  5. Monitor: Check your logs occasionally to ensure the automation is working as expected.

Related guides

Rate this guide

Loading ratings...

Was this guide helpful?

Comments