Special device gets filled from read operations?

[haarp]

I've got a pool which had a special device added after data was already present.

It was my understanding that special devices will only get filled by write operations, i.e. metadata and data (if special_small_blocks is used) only gets added to them if it is (re)written.

Today I ran find/du operations across mounted datasets (while the pool was otherwise idle) and could observe special device usage in zpool list -v increase by a couple of GB (for a couple of million files) and stay there. Presumably it was writing metadata to the special device? Can read operations do this, was my understanding wrong? Is this a recent feature?

Thanks!

OpenZFS v2.3.2, Linux 6.12.48

[no-usernames-left]

Presumably it was writing metadata to the special device? Can read operations do this, was my understanding wrong?

Were access times being updated?

[haarp]

Good point, but no, they weren't.

Also by very rough estimate the increased special disk usage was ~1GB per million files scanned, and 1k per file does not seem reasonable for timestamps alone.

[amotin]

Just for a note, IIRC atimes are stored in dnode's bonus buffer, minimal dnode size is 512 bytes, and metadata usually stored in two copies.

[haarp]

Another oddity I just observed: This read-based filling up only happens as long as the special vdev stays below the 75% threshold (zfs_special_class_metadata_reserve_pct). That would imply whatever is being written msut be actual data.

[amotin]

Theoretically it might be a coincidence. If the free space is significantly fragmented, ZFS might start falling back to the next storage tier before using all free space on the previous, if the block is too big, since it leaves ganging as a last resort.