dedupe PMTiles #28

Open
opened 2026-07-20 04:15:05 +00:00 by lewismoten · 0 comments
Owner

We should compare each archive’s header values:

number of addressed tiles
number of tile entries
number of unique tile contents

If unique contents are already substantially below addressed tiles, deduplication is working. If they are equal, a post-build deduplication stage could yield savings, but it will probably not dramatically reduce detailed vector levels.

The best approach would be a post-build PMTiles deduplication pass that hashes each compressed tile payload, stores identical payloads once, and points matching entries to the same offset. The official PMTiles CLI performs content deduplication by default when converting archives; its --no-deduplication option disables that behavior.

We should compare each archive’s header values: number of addressed tiles number of tile entries number of unique tile contents If unique contents are already substantially below addressed tiles, deduplication is working. If they are equal, a post-build deduplication stage could yield savings, but it will probably not dramatically reduce detailed vector levels. The best approach would be a post-build PMTiles deduplication pass that hashes each compressed tile payload, stores identical payloads once, and points matching entries to the same offset. The official PMTiles CLI performs content deduplication by default when converting archives; its --no-deduplication option disables that behavior.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lewismoten/qr#28
No description provided.