Magento 2.4 with AWS ES not able to install

Question

I'm trying to install Magento 2.4. I have set up AWS ES that I tasted and it works fine! But I keep getting an error:

  In SearchConfig.php line 81:

  Could not validate a connection to Elasticsearch. null

Is there a special configuration I need to make on AWS es?

Thank you.

Answer 1

i think on aws you can connect to host https://elk-end-point port 443

also you need to use latest AWS ELK 7.6+ stack

Answer 2

I actually spent the last 2 days working on this and I got it to work. You could open AWS ES to the world and call it a day, but of course that's not safe at all.

What I have done is create a new ES domain with VPC access, no fine–grained access control and allow open access to access policy. Now you want only the servers to have access to the ES domain. Create a new Security Group giving 443 access to the Security Group that the servers are part of. By doing this you limit the access to the WEB servers and you don't need an username and password to connect. To connect to AWS ES with username/password you need to send some headers and I am afraid Magento doesn't have this configured out of the box.

But if you decide to go with the steps above, you just need to add this to the Magento installation:

--elasticsearch-host="https://vpc-YOURAWSES-somerandominformation.us-east-1.es.amazonaws.com" \
--elasticsearch-port=443

I am using port 443 because I enable to only accept HTTPS connections.

Just keep in mind that by doing this you won't have access to Kibana using public internet, as being on the VPC means it's on private network. You could create a VPN gateway and connect to it if you really want Kibana access.

Another option is to create a master user in the fine–grained access control option and use the following on the installation:

--elasticsearch-host="https://vpc-YOURAWSES-somerandominformation.us-east-1.es.amazonaws.com" \
--elasticsearch-port=443 \
--elasticsearch-username magento \
--elasticsearch-password 'Thii$$4mePassword!'

As you can see I added single quotes to the password, because AWS ES needs special characters. With this setup you will be able to access Kibana.

I personally want minimum access from the world and don't care to access Kibana for the indexes. I have used Kibana/ES for Magento logs before, but I have found it to be problematic, so I just used EFS for the log folders and called it a day.

Let me know if this works out.

Answer 3

Yes, there is a specific technology stack requirements that you must follow, there are PHP extensions and third-party system to install (e.g. ElasticSearch 7, MySQL 8, etc..).

You will be able to find all the Magento 2.4 technology stack requirements in the link.

The issue that you are having is caused by the missing of the communication with ElasticSearch 7.

In order to test your ElasticSeach, you can run this command below, if needed, you can change the localhost to your ElasticSearch IP and port.

curl -XGET 'localhost:9200/my_index/my_type/_search' -d '{
  "query" : {
    "match" : {
      "testField" : "abc"
    }
  }
}'