> The Avogadro number isn't a fundamental physics constant
That's a pretty arbitrary assertion. Who defines what "fundamental" is? The number of periods of <something> of Caesium atoms used to define the duration of one second doesn't sound very fundamental either.
In fact, the second and candela aren't choices aren't that fundamental, mostly reflecting how those are measured. All the others come from the speed of light, fundamental charge, Plank's constant, and gases constant; those are pretty fundamental to physics.
Avogadro's number is used to define the atomic units, what makes a lot more sense than pushing it into metric calculations.
The Avogadro number isn't a fundamental physics constant, it's just a unit conversion constant, and thus, didn't get to define any basic unit.