Scaling Magento 2 on multiple containers in production with generated code

Question

I'm working on some auto-scaling Kubernetes infrastructure to run Magento 2, and I noticed that when I start new containers, I can't have the generated/ directory automatically compiled as part of the container image build—it seems like it needs a working Magento installation (which I'm not guaranteed to have when building the container) to compile.

So currently what I'm doing is, when a new container starts up (e.g. when autoscaling, or when updating code, and new containers are replacing old ones):

• Check if Magento is installed.
• If it is installed, run:
  • bin/magento module:enable --all
  • bin/magento setup:di:compile

This seems to work okay, and the new container works just like the old/existing ones—if I don't do the compilation step, I just get lots of fatal exceptions until I do the compilation.

But my question is this: is there some other/better way of handling the generated/ directory when you want to run Magento on multiple hosts/containers?

Should I share the generated/ directory via NFS like I do for var/? It seems that reading code out of that directory would be a lot slower with NFS... and maybe the extra minute or so it takes to start a new container is worth the tradeoff to have each container compile/optimize it's own DI code (vs. sharing the directory with other existing containers).

Answer 1

Not sure why this question got downvoted; looking around the Internet there is precious little information about how people should run Magento 2 (Open Source or Enterprise) in a scalable, production, container environment, and there are a lot of little things that are a pain in the butt to handle.

Relating to my question above, I am now compiling the code in the container build process, which seems to work okay; in the Dockerfile, after I've copied or cloned my Magento codebase into the container image:

# Compile code.
RUN magento module:enable --all \
 && magento setup:di:compile

That way I don't have to do that in an entrypoint script.

However, I do still have to run something like the following in my entrypoint—at least after a fresh installation (and maybe also after new deployments):

echo "Setting MAGE_MODE"
magento deploy:mode:set $ENV_MODE --skip-compilation

echo "Cleaning Magento Cache"
magento cache:flush
magento cache:clean

echo "Ensuring permissions are correct on Magento folders - post install."
CHOWN_PATHS_POST_INSTALL="app/etc generated pub/media pub/static var"
chown -R www-data:www-data $CHOWN_PATHS_POST_INSTALL

if [ "$ENV_MODE" = "production" ]; then
    echo "Compiling static assets."
    su -s /bin/bash -c "magento setup:static-content:deploy --jobs 4" www-data
fi

I'm still trying to figure out some other things, but I'm to the point where new containers can start in just a few seconds (still have to do things like chown -R www-data:www-data paths like app/etc and generated after the container starts, and copy/insert a templated app/etc/env.php file into place (to make sure all my site's settings are accounted for and can be driven by environment variables in the container so it's a true 12 factor app deployment).

Answer 2

are you using PVC for data persistence, I am trying to achieve this but am unable to can you please share the Dockerfile