Table of Contents
The stack includes native support for RabbitMQ.
Setup
To enable it, use the following configuration settings within Magento:
Host: queue1.i
Port: 5672
If you have multiple servers, the host
value will change too - for example, stack2 will use host queue2.i
instead.
A RabbitMQ admin
user and virtualhost /
is created on the stack by default - this user's credentials are available within the Sonassi control panel.
You should create a separate RabbitMQ user and virtualhost for each of the stores on your stack, then specify these in your site's env.php
configuration file - this helps to ensure each store's queues are fully isolated from one another (Similar to using a different MySQL or Redis database):
'queue' => [
'amqp' => [
'host' => 'queue1.i',
'port' => '5672',
'user' => 'example_user',
'password' => 'example_password',
'virtualhost' => 'example_virtualhost'
],
'consumers_wait_for_messages' => 1
],
Please note that you'll then need to configure each queue that you'd like to use with RabbitMQ individually in your store's env.php
- otherwise, they'll still default to using MySQL instead.
For reference, please see Magento's documentation at:
This includes an example for the product_action_attribute.update
queue, but you'd need to add additional configuration sections for each of the queues your store is using (based on the output of bin/magento queue:consumers:list
).
After editing your store's env.php
, run bin/magento setup:upgrade --keep-generated
to generate the queues within RabbitMQ.
Accessing the administration panel
The RabbitMQ Administration panel can be accessed at https://queue.magestack.com
If you have multiple servers, you should access the server's respective RabbitMQ instance:
i.e. https://queue.2.magestack.com
! You must be connected to the VPN to reach this administration panel.
Supervisor
We've found that Magento's default consumers_runner
cron task may not always behave as expected, with issues such as:
- Not always reliably starting queue consumers, leaving messages stuck in a
Pending
state - Leaving older queue consumer processes running after a site deployment, which then use stack resources unnecessarily
If you see similar issues to these, we would recommend using Supervisor to manage RabbitMQ message queues instead of Magento's consumers_runner
.
To configure Supervisor as an example (using domain group example
and vhost example.com
):
-
Confirm the message queues that are currently active on your store:
cd /microcloud/domains/example/domains/example.com/http bin/magento queue:consumers:list
-
Open
/microcloud/domains/example/domains/example.com/http/app/etc/env.php
, and disablecron_run
:'cron_consumers_runner' => [ 'cron_run' => false, 'consumers' => [ ] ],
-
Install the
supervisor
package as the root SSH user:su -l root apt install supervisor
-
Create configuration files for each queue - for example, for the
product_action_attribute.update
queue, create/etc/supervisor/conf.d/queue-product_action_attribute.update.conf
with:[program:queue-product_action_attribute.update] process_name=%(program_name)s_%(process_num)02d directory=/microcloud/domains/example/domains/example.com/http command=/usr/bin/fakechroot /usr/sbin/chroot /microcloud/domains/example /usr/bin/php /domains/example.com/http/bin/magento queue:consumers:start product_action_attribute.update --max-messages=100 autostart=true autorestart=true user=www-data redirect_stderr=true stdout_logfile=/microcloud/domains/example/domains/example.com/http/var/log/queue.log
Replace the
example
domain group andexample.com
vhost in theprogram
,directory
,command
, andstdout_logfile
variables for your store as necessary.The
process_name
variable can safely be left as-is: this controls the name that's assigned to the process by Supervisor - for the configuration file above, this would bequeue-product_action_attribute.update_00
(Whereprogram_name
is mapped toqueue-product_action_attribute.update
, andprocess_num
is mapped to00
) -
Create additional configuration files for each of the message queues for your store using the configuration template above - update the file name, and the
program
andcommand
variables for each new configuration file. - Start Supervisor
/etc/init.d/supervisor start
Supervisor will now manage your store's message queues, rather than Magento.
If you add any new message queues, you will need to add a new Supervisor configuration file too.
You can then use Supervisor's supervisorctl
to view the status of queues, or restart them - for example:
su -l root
supervisorctl status
supervisorctl restart all