The rapid growth of e-commerce platforms has enabled businesses to implement customized assortment strategies, tailoring their product offerings to different customer groups. While these strategies can enhance profit, they often raise concerns about fairness, especially when particular customer groups receive fewer options. Such practices can provoke customer backlash and damage a company's reputation. Motivated by these issues, we propose a fairness-aware framework for assortment optimization that balances revenue maximization with limits on disparities in assortment sizes across customer groups. By introducing cardinality-based fairness constraints, our framework limits assortment-size differences across customer groups through heterogeneous and asymmetric fairness tolerances. Under this framework, we explore two policies: deterministic and randomized assortment. For any number of customer types, we develop exact polynomial-time algorithms for both settings. For the special case of two customer types, we derive sharper structural characterizations and exploit them to design more efficient algorithms. We also establish revenue guarantees under fairness constraints and characterize the value of randomization. In two-group markets, our analysis shows that these constraints may compel sellers to offer more products for disadvantaged groups, reduce the number of products for privileged groups, or apply both adjustments. Our numerical results further show a limited impact on sellers' revenue in the instances considered, making them less likely to resist such regulations. However, policymakers must weigh the trade-offs carefully, as fairness constraints can reduce the overall customer surplus in some cases.