Manage garbage collection

This guide explains how to configure and manage garbage collection in the Numspot Container Registry.

---

Overview

Garbage collection (GC) is the process of removing unused blobs, artifacts and manifests from the Harbor registry to reclaim storage space. Harbor garbage collection removes:

• Unreferenced blobs: image layers not associated with a manifest
• Orphaned artifacts: manifests without tags or references
• Deleted artifacts: artifacts marked for deletion but not removed from storage

Why garbage collection matters

• Storage optimization: reclaim disk space occupied by unused data
• Cost reduction: lower object storage costs
• Performance: improve registry performance by removing unnecessary data
• Compliance: comply with data retention policies

---

How garbage collection works

Harbor storage architecture

┌────────────────────────────────────────────────────────┐
│                   Harbor Registry                       │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐ │
│  │   Metadata   │  │   Manifests  │  │    Blobs    │ │
│  │  (PostgreSQL)│  │   (JSON)     │  │  (Layers)   │ │
│  └──────────────┘  └──────────────┘  └─────────────┘ │
│                            │                           │
│                            ▼                           │
│                  ┌─────────────────────┐              │
│                  │   Object Storage    │              │
│                  │   (S3/MinIO)        │              │
│                  └─────────────────────┘              │
└────────────────────────────────────────────────────────┘

GC process

1. Identify unreferenced blobs: find the blobs not referenced by a manifest
2. Mark for deletion: mark artifacts and blobs for deletion in the metadata
3. Remove from storage: remove the marked blobs from object storage
4. Clean up metadata: remove the artifact records from the database

GC modes

Harbor supports two garbage collection modes:

Mode	Description	Impact
Online GC	Runs while the registry is accessible	Minimal impact, slower
Offline GC	The registry is in read-only mode during GC	Faster, but registry in read-only mode

---

Prerequisites

• Administrator access: required to configure and run GC
• Registry instance: a running Numspot Container Registry
• Storage quota visibility: know your current storage usage
• Maintenance window: for offline GC, schedule during low-traffic periods

---

Step 1: Check current storage usage

Through the Harbor interface

1. Log in to the Harbor interface
2. Go to Administration → System Settings
3. Check the Storage section to view:
   • total storage used
   • available free space
   • storage quota

Through the API

curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/systeminfo" \
  -u "admin:{password}" | jq '.storage'

Response:

{
  "total": 107374182400,
  "free": 53687091200,
  "used": 53687091200
}

---

Step 2: Configure garbage collection

Through the Harbor interface

1. Go to Administration → Garbage Collection
2. Configure the GC settings:

Setting	Value
GC type	Online or Offline
Schedule	Cron expression or manual
Delete untagged artifacts	Enable to remove untagged manifests
Include untagged artifacts	Include untagged artifacts in GC

3. Click Save

Schedule examples

Schedule	Cron expression	Use case
Daily at 2 a.m.	0 2 * * *	Regular cleanup
Weekly on Sunday	0 2 * * 0	Weekly maintenance
Every 6 hours	0 */6 * * *	High-churn environments
Manual	N/A	On-demand cleanup

Through the API

curl -X PUT "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/system/gc/schedule" \
  -u "admin:{password}" \
  -H "Content-Type: application/json" \
  -d '{
    "schedule": {
      "type": "Scheduled",
      "cron": "0 2 * * *"
    },
    "delete_untagged": true
  }'

---

Step 3: Run garbage collection

Manual run

Through the Harbor interface

1. Go to Administration → Garbage Collection
2. Click GC Now
3. Select the GC type:
   • Online GC: the registry remains accessible
   • Offline GC: the registry switches to read-only mode
4. Select the options:
   • delete untagged artifacts
5. Click Start GC
6. Track progress in the GC history

Through the API

curl -X POST "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/system/gc/schedule" \
  -u "admin:{password}" \
  -H "Content-Type: application/json" \
  -d '{
    "schedule": {
      "type": "Manual"
    },
    "delete_untagged": true,
    "gc_executed_now": true
  }'

---

Step 4: Monitor garbage collection

Through the Harbor interface

1. Go to Administration → Garbage Collection
2. Check the GC History tab
3. Click a GC run to view:
   • status ("In progress", "Success", "Failed")
   • start time and duration
   • number of artifacts deleted
   • space reclaimed
   • logs

Through the API

# List GC runs
curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/system/gc" \
  -u "admin:{password}"

# Get a specific GC run
curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/system/gc/{gc_id}" \
  -u "admin:{password}"

# Get the GC log
curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/system/gc/{gc_id}/log" \
  -u "admin:{password}"

GC statuses

Status	Description
Pending	GC task queued
Running	GC in progress
Success	GC completed successfully
Failed	GC encountered errors
Stopped	GC stopped manually

---

Best practices

Scheduling

1. Schedule during off-peak hours: run GC when registry usage is low
2. Frequency based on churn: high-churn registries require more frequent GC
3. Start with online GC: test with online GC before using offline GC
4. Monitor the duration: track GC duration to estimate maintenance windows

Configuration

1. Enable untagged artifact deletion: remove artifacts without tags
2. Use offline GC for large cleanups: faster for significant reclamation
3. Set project quotas: prevent unlimited storage growth
4. Configure retention policies: use artifact retention to automatically remove old images

Testing

1. Test in non-production: validate the GC configuration in staging first
2. Monitor after GC: check that storage is reclaimed and the registry works
3. Verify image availability: ensure critical images remain accessible after GC

---

Garbage collection and image retention

Combined strategy

Implement a complete cleanup strategy:

1. Artifact retention policies: automatic deletion of old artifacts based on time or count
2. Project quotas: limit storage per project
3. Tag retention rules: keep only the necessary tags
4. Scheduled garbage collection: regular cleanup of unreferenced data

Configure artifact retention

1. Go to Administration → Retentions
2. Create a retention rule:

Field	Value
Scope	Global or a specific project
Repository pattern	** or a specific pattern
Criteria	Pulled within the last 30 days, or the 10 most recently pushed
Action	Delete

Through the API

curl -X POST "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/retentions" \
  -u "admin:{password}" \
  -H "Content-Type: application/json" \
  -d '{
    "algorithm": "or",
    "rules": [
      {
        "action": "delete",
        "scope_selectors": {
          "repository": [
            {
              "kind": "doublestar",
              "decoration": "repoMatches",
              "pattern": "**"
            }
          ]
        },
        "tag_selectors": [
          {
            "kind": "doublestar",
              "decoration": "matches",
              "pattern": "**",
              "extras": "{\"untagged\":true}"
          }
        ],
        "params": "{\"latest_pushed_k\":10}"
      }
    ],
    "trigger": {
      "kind": "Schedule",
      "settings": {
        "cron": "0 0 * * *"
      }
    },
    "scope": {
      "level": "system"
    }
  }'

---

Storage optimization tips

Reduce storage before GC

1. Delete unused repositories:

curl -X DELETE "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/projects/{project}/repositories/{repo}" \
  -u "admin:{password}"

2. Delete old tags:

   • delete specific tags manually
   • use retention policies for automatic deletion

3. Clean up untagged artifacts:

   • enable delete_untagged in the GC configuration

Monitor storage trends

# Get storage usage over time
curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/statistics" \
  -u "admin:{password}" | jq '.total_storage_consumption'

Set project quotas

1. Go to Administration → Projects
2. Select the project → Configuration
3. Set the Storage quota (e.g. 50 GB)

Through the API

curl -X PUT "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/projects/{project_id}" \
  -u "admin:{password}" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {
      "storage_limit": "53687091200"
    }
  }'

---

Troubleshooting

Issue 1: GC does not reclaim space

Symptom: storage usage remains high after GC

Solutions:

1. Check whether the artifacts are still referenced by tags
2. Check that delete_untagged is enabled
3. Check for incomplete uploads:

curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/systeminfo" \
  -u "admin:{password}" | jq '.incomplete_upload_count'

4. Run an offline GC for a more aggressive cleanup

Issue 2: GC takes too long

Symptom: GC runs for hours

Solutions:

1. Check the storage size and the number of artifacts
2. Consider splitting into smaller projects
3. Schedule more frequent GC with a smaller cleanup scope
4. Use offline GC instead of online GC

Issue 3: Registry slow during online GC

Symptom: performance degradation during online GC

Solutions:

1. Schedule GC during off-peak hours
2. Use offline GC for large cleanups
3. Increase registry resources (contact support)
4. Reduce the GC frequency

Issue 4: Failed GC task

Symptom: the GC run shows the Failed status

Solutions:

1. Check the GC logs for the error details:

curl -X GET "https://registry-{registryId}.hcp.cloudgouv-eu-west-1.numspot.com/api/v2.0/system/gc/{gc_id}/log" \
  -u "admin:{password}"

2. Check connectivity to object storage
3. Check connectivity to the database
4. Contact support with the error details

---

API reference

Get the GC schedule

GET /api/v2.0/system/gc/schedule
Authorization: Basic {base64(admin:password)}

Update the GC schedule

PUT /api/v2.0/system/gc/schedule
Authorization: Basic {base64(admin:password)}
Content-Type: application/json

{
  "schedule": {
    "type": "Scheduled",
    "cron": "0 2 * * *"
  },
  "delete_untagged": true
}

Trigger a manual GC

POST /api/v2.0/system/gc/schedule
Authorization: Basic {base64(admin:password)}
Content-Type: application/json

{
  "schedule": {
    "type": "Manual"
  },
  "delete_untagged": true,
  "gc_executed_now": true
}

List GC runs

GET /api/v2.0/system/gc
Authorization: Basic {base64(admin:password)}

Get GC run details

GET /api/v2.0/system/gc/{gc_id}
Authorization: Basic {base64(admin:password)}

Get the GC log

GET /api/v2.0/system/gc/{gc_id}/log
Authorization: Basic {base64(admin:password)}

---

Limitations

• Online GC is slower: it can take longer than offline GC
• Offline GC is read-only: the registry is inaccessible for writes during offline GC
• A running GC cannot be stopped: you must wait for it to finish or restart the registry
• Storage is not freed immediately: object storage may have deferred cleanup
• GC does not delete tags: it only removes unreferenced blobs and artifacts

---