PHP's memory_limit
setting should have no effect on Magento's performance. It only sets the high threshold of memory usage by PHP processes, and if Magento hogs more memory than that limit, you will get errors, but it changing it won't make things faster.
I would strongly advice against modifying memory_limit
to anything other than the recommended value unless you know what you're doing. Setting it too low may cause some of your Magento pages to crash, and setting it too high might cause your entire server to slow down to a halt in extreme cases.
Instead, I would advise searching the Web on some tips on how to optimize Magento's performance. There are many relevant posts that can point you in the right direction. This post on my company's blog is one example, but there are many others. Enabling various forms of caching, using a CDN and optimizing your DB settings are probably the first places you want to look into.
BTW, I would also strongly advise against posting your server's phpinfo() output publicly for security reasons. You should remember to delete that file as soon as you can.