If you've already handled the amount of items per products and you want to restrict the user from adding to cart any additional products, you'll need to listen to an event to manage this. i.e. Can't add Product B, because you already have a Product A in the cart.
There's an event called checkout_cart_add_product_complete
that may be suitable. The problem with this is that the product has already been added to the cart at this point.
Your observer would need to handle the following flow:
- Identify the product that the user tried to add to cart
- Remove this product from the current cart
- Add an error message to the session
This should remove the product every single time someone tries to add it to the cart, and then from this point should inform the user that they've exceeded their maximum.
If you've no experience setting up Events and Observers, I'd recommend reading up on the matter:
http://www.solvingmagento.com/events-and-observers-a-magento-tutorial/
http://code.tutsplus.com/tutorials/understanding-magento-observers--cms-21031