It looks like the default theme has a pager for the <review_product_list>
handle.
<block type="page/html_pager" name="product_review_list.toolbar" />
Make sure if you are using a custom theme that this is included, you can double check with app/design/frontend/base/default/layout/review.xml
to see how the default theme works with the pager.
It should get added to the Mage_Review_Block_Product_View_List
when there is a child block with the correct name.
if ($toolbar = $this->getLayout()->getBlock('product_review_list.toolbar')) {
$toolbar->setCollection($this->getReviewsCollection());
$this->setChild('toolbar', $toolbar);
}