← All posts

Product update4 min read

A map file went from 133 MB to 44 MB without losing a wall

Two changes to how ICIDO stores a scanned place cut a real map file to a third of its size. What was measured before each change, and what was checked after.

The only compression in the app was the thumbnail

A place in ICIDO is saved as two things: ARKit's own world map, which is what lets the app recognise the room on a later visit, and a filtered mesh of the room's primary surfaces, which is what the app draws. Exported for sharing, they travel together in a single .spacemap file, along with a thumbnail and a manifest.

Until last week nothing in the app had ever tried to make any of that smaller. One real place exported at 202,320,667 bytes, and the only compressed part of the file was the JPEG thumbnail (b3920d8).

Measure first

The mesh file is four flat arrays: vertex positions, vertex normals, triangle indices, and one class byte per face saying whether it is wall, floor, furniture and so on. Before any code was written, each part was compressed on its own, on a real map of 5,337,651 triangles and 1,218 m², to see what would actually shrink (b3920d8):

  • triangle indices: 64.1 MB to 19.6 MB
  • vertex positions: 34.9 MB to 26.3 MB
  • vertex normals: 34.9 MB to 32.3 MB
  • face classes: 5.3 MB to 0.09 MB
  • the ARKit world map: 61.2 MB to 46.1 MB
  • the thumbnail: grew by 12 bytes

Two of those lines decided the design. The thumbnail is already compressed, so compressing it again costs bytes; the format therefore carries a flag byte and stores such blocks raw, rather than assuming compression always wins. And the normals barely moved. At 1.08x they are, as a later commit puts it, float noise (df1544f). No compressor would touch them. Only not storing them would, and that was explicitly set aside as a separate change with its own risk.

Compress each array on its own

The second version of the mesh format compresses each of the four arrays separately rather than the file as a whole. The commit is clear that this is a memory decision more than a size one: each array is compressed straight out of its own storage and read straight back into a correctly sized array, so neither saving nor loading holds a second full-size copy of the mesh (b3920d8).

On the same map the mesh went from 132.8 MB to 74.6 MB. The round trip is byte-identical, and loading takes 0.12 s longer (b3920d8).

One thing was deliberately not done. The mesh block inside a .spacemap package is not compressed a second time, because a package holding an older, uncompressed mesh would need the source, the scratch space and the result resident at once, about 340 MB for this map. Trading an out-of-memory kill for a smaller file is the exact failure an earlier build had shipped to fix (b3920d8). The cost is stated in the commit: a place saved before this change keeps its full-size mesh in packages until something re-saves it.

Then stop storing the normals at all

A normal is the direction a surface faces. For a triangle it can be computed from the triangle itself, from the cross product of two of its edges. The stored normals were 34.9 MB of a 132.8 MB mesh file, the largest thing in it, and the one thing compression could not shrink (df1544f).

They were also already being recomputed in one place. The occupancy map, the top-down grid that routing runs over, falls back to a computed normal whenever a stored one is degenerate, so geometric normals had been trusted on that exact path long before this change (df1544f).

Version three of the format stores positions, indices and face classes, and rebuilds per-vertex normals from the triangles on load, area-weighted, matching the winding the occupancy map's fallback already used (df1544f).

Measured on the same map: 132.8 MB to 43.8 MB. Load takes 0.14 s including the rebuild. Positions, indices, face classes and floor area are exact. The rebuilt normals sit a mean 0.18° from ARKit's (df1544f).

What was checked, and what would have hurt

The number that matters is not the file size. It is whether the map still routes the same.

The routing grid of that map has 327,666 cells. After the change, one of them differed: a cell that had been nothing became floor. That is the direction worth worrying about, and the commit says why it was accepted: the cell survived the island filter, so it sits next to existing floor rather than being a lone patch of phantom ground (df1544f).

Two other things were checked because they carry the most risk. The floor-level estimate and the declutter pass both run at save time and decide what geometry reaches the file at all; nothing downstream can undo a wrong answer there. Neither moved (df1544f).

The test fixtures changed with the format. They used to compare normals; they cannot now, since the normals are rebuilt. They compare the routing grid instead, which the commit calls the thing that would actually hurt someone: if the triangle winding were ever reversed, every floor would become a ceiling and the map would be unwalkable. That check reports zero cells changed (df1544f).

Three versions of the file, all still read

Three formats now exist on real devices: the original raw one, the compressed one with normals, and the current one without. The app reads all three, and none is rewritten unless its owner edits that place (df1544f). An older build of ICIDO handed a newer package refuses it outright and says the sender may be on a newer version, rather than importing a place with no geometry and no explanation (b3920d8).

One inefficiency surfaced afterwards. The compressed reader sliced each block with a copy, and the decoder copied it again, so every compressed block was duplicated on the way in, 19.6 MB on the indices block alone, for nothing. Both now share storage instead (812a911).

Related reading