After several research attempts, I still couldn't manage to limit category page to N-number of products to be shown.
I need to limit a specific category page to let's say 180 products only, with pagination as well.
$collection->setPageSize(200);
Will display 200 products but will still display remaining products because of pagination.
To clear things up:
Only list 3 pages (example 180 products regardless) of a specific category to be shown on the front end regardless of how many products are there in that category.