Command Info

Name: websites:cleanup-duplicates

Description: Find and clean up duplicate websites (including IDN Unicode/Punycode duplicates)

Status: completed

Start Time: 2026-07-10 15:20:34

End Time: 2026-07-10 15:20:36

Total Time: 2 seconds

Dispatched Jobs Count: 0

Successful Jobs Count: 0

Failed Jobs Count: 0

Output

15:20:34 Lock acquired for command: websites:cleanup-duplicates
15:20:34 🔍 Starting duplicate website cleanup...
15:20:34 Found 1 group(s) of duplicate websites.
15:20:34 Re-keying domain-keyed data onto the keeper BEFORE merging, so nothing is orphaned.
15:20:34 <fg=yellow>Duplicate group for: https://adlogger.org</>
15:20:34 <fg=green>KEEP: ID 3cfc1c33-d85c-456e-8fcb-31cba35e16b8, www.adlogger.org (type LINK, status LIVE)</>
15:20:34 <fg=red>DELETE: ID 3f145f93-5828-402a-b852-856ed442282b, adlogger.org (type LINK, status LIVE)</>
15:20:35 ↳ accuranker_keywords_aggregation: derived view — re-keyed via its source collection
15:20:35 ⚠️ mongo:system.views reassign failed: cannot write to kokos.system.views
15:20:36 ⚠️ mongo:accuranker_keywords_aggregation reassign failed: Namespace kokos.accuranker_keywords_aggregation is a view, not a collection
15:20:36 ⏭ Skipped loser ID 3f145f93-5828-402a-b852-856ed442282b (adlogger.org): reassign had 2 failure(s); left in place, resolve and re-run.
15:20:36 ⚠️ Keeper hostname canonicalize failed (non-fatal; run websites:canonicalize-hosts): SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'adlogger.org' for key 'websites_hostname_unique' (Connection: mysql, Host: 127.0.0.1, Port: 3306, Database: kokos, SQL: update `websites` set `hostname` = adlogger.org, `updated_at` = 2026-07-10 15:20:36 where `id` = 3cfc1c33-d85c-456e-8fcb-31cba35e16b8)
15:20:36 📊 Cleanup complete:
15:20:36 ✅ Groups processed: 1
15:20:36 ⏭ Groups skipped (re-key/keeper failures): 0
15:20:36 ❗ Groups aborted (unexpected error): 0
15:20:36 🔄 Domain-keyed docs re-keyed onto keepers: 16
15:20:36 🧹 Logical duplicates merged: 11
15:20:36 ✅ Losers deleted: 0
15:20:36 ⏭ Losers skipped (reassign failures): 1
15:20:36 🔤 Keepers canonicalized: 1
15:20:36 💡 Re-run websites:audit-idn-duplicate-data to confirm 0 at-risk docs for the merged groups.
15:20:36 Lock released for websites:cleanup-duplicates