I have tried the previous solutions and even though they seemed to work they locked me out of admin panel (don't completely understand the reason).
To get into the admin panel again I had to get into phpmyadmin and get into to database table core_config_data and change both paths: web/secure/use_in_adminhtml and web/unsecure/base_url get back into the admin panel again. And don't forget to clear out the cache folder if you have to make edits in the database...
So I applied the following .htaccess solution instead which let me login to admin as well
# AFTER THIS SECTION
############################################
## workaround for HTTP authorization
## in CGI environment
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
# PUT THIS
############# ALL PAGES HTTPS #################
RewriteCond %{HTTPS} off
RewriteCond %{REQUEST_URI} !/admin/
RewriteCond %{REQUEST_URI} !/downloader/
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
############# ALL PAGES HTTPS #################
Hope it helps someone