It’s a total tax on the entire ecosystem. Every merchant that chooses to accept credit cards has to build 2-3% into their margins to cover the interchange fees. It’s often against their agreement with visa/mc to charge the fees for credit transactions only and so they build them into the base cost.
As a side note, the situation is a bit better in Europe, where considerable regulation was introduced recently. With mixed but positive results. Interchange fees were capped to 0.3% for credit cards.