i want to call my custom siderbar on product detail page only below is code i am using in admin > categroy > custom design

<reference name="left">
      <block type="core/template" name="Designer Sidebar"  template="page/design/sidebar.phtml"/>
</reference>

but it add sidebar on all pages how to restrict it for product detail page only as now my custom sidebar also show on product listing page i do not want it there

i want to call my custom siderbar on product detail page only below is code i am using in admin > categroy > custom design

<reference name="left">
      <block type="core/template" name="Designer Sidebar"  template="page/design/sidebar.phtml"/>
</reference>

but it add sidebar on all pages how to restrict it for product detail page only as now my custom sidebar also show on product listing page i do not want it there

based marius answer i update my code to below:in admin > category > custom design

<catalog_product_view>
<reference name="left">
      <block type="core/template" name="Designer Sidebar"  template="page/design/sidebar.phtml"/>
</reference>
</catalog_product_view>

but after that i do not see sidebar.phtml there