Magento 2 and ElasticSearch errors. Connection Failed. Test again?

Question

i need some help with configuring my elastic search on magento 2.2.3 and i'm using ES 2.3.

I login and configure here: Click Stores > Settings > Configuration > Catalog > Catalog > Catalog Search.

It's on a remote aws server and i know it's configured on port 443 so i changed the port to 443 rather than 9200 and i've changed the Elasticsearch Server Hostname to the correct endpoint.

• telnet from admin to ES works

as does curl

• so not a network / firewall iss
• Elasticsearch Index Prefix is correct
• cluster name is correct

After that i followed these instructions here: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg-upload-data.html

• I was able to upload a new search document and then I was able to retrieve it as well. So once again the networking between the two is good.

Is there a php dependency that i'm missing or a third party magento plugin that needs to be installed to assist? Please help. Thanks in advance.

Answer 1

Unfortunately I also had this problem, I figured the best solution was to use Nginx to proxy HTTP requests internally through to AWS ElasticSearch. The config is:

server {
    server_name localhost;
    listen 8080;

    location / {
        proxy_pass https://your-amazon-vpc-url-here.es.amazonaws.com;
        proxy_set_header Host $host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Within Magento2 then I set host as localhost and port as 8080 - works fine.

Answer 2

After going through the standard magento docs and aws docs we realized it wasn't connecting successfully. We believe it was because magento is sending non-SSL packets to the ElasticSearch server but ES is only listening for (443) SSL. Changing what Nginx was listening for and to proxy AWS Elasticsearch seemed to fix the problem. This may only be a temp. solution. It would be good to talk to magento and see what they recommend.

If anyone else has any input than that would be great too. Unfortunately it appears in my magento account I can no longer submit tickets so reaching out to magento has become obsolete.

/var/www/html/vendor/magento/module-elasticsearch

  /**
     * @param array $options
     * @return array
     */
    private function buildConfig($options = [])
    {
        $host = preg_replace('/http[s]?:\/\//i', '', $options['hostname']);
        $protocol = parse_url($options['hostname'], PHP_URL_SCHEME);
        if (!$protocol) {
            $protocol = 'http';
        }
        if (!empty($options['port'])) {
            $host .= ':' . $options['port'];
        }
        if (!empty($options['enableAuth']) && ($options['enableAuth'] == 1)) {
            $host = sprintf('%s://%s:%s@%s', $protocol, $options['username'], $options['password'], $host);
        }

        $options['hosts'] = [$host];
        return $options;
    }

If anyone is interested I can post the Nginx configurations too.

Answer 3

Use port 443 and make sure you don't have a trailing slash on the hostname.

This fails: https://vpc-domain-name-identifier.region.es.amazonaws.com/

This succeeds: https://vpc-domain-name-identifier.region.es.amazonaws.com