The catalog (category) and catalogsearch (results) contexts use the same block classes and templates. Unfortunately there are no public methods for setting/adjusting the
_availableModes property after it's been set - which it is in the constructor for
Mage_Catalog_Block_Product_List_Toolbar. (There is a
setModes() method but it does nothing once the property is populated).
It would be possible to set a custom template using the
catalogsearch_advanced_result handles, and then in your template, check for the
grid array key and
unset() it. Not as clean as being part of the block, but the block wasn't made very friendly anyway.