"Review times vary; some reviews complete in a few hours, others take many days, and in some cases a review can take several weeks.
... If your item's status says "pending review" for more than three weeks, you should contact support."
Reviews are meant to take longer if you ask for e.g. permission to all URLs (which a large number of popular extensions need to because of lack of permission granularity). Maybe your extension isn't impacted here? There's no guarantee the next update won't get trapped in the review queue either, which is not going to be fun when a critical fix is pending.
Maybe it's because I don't have many users, or don't use any minification, or something else?
I'd like to publish for Safari but it's a much higher bar. Similar to the old Edge extensions which required Microsoft Store hoop jumping.