Help

RabbitMQ Administration

By 14th February 2019October 6th, 2021No Comments

The stack includes native support for RabbitMQ.

Setup

To enable it, use the following configuration settings within Magento:

Host: queue1.i
Port: 5672

If you have multiple servers, the host value will change too - for example, stack2 will use host queue2.i instead.

A RabbitMQ admin user and virtualhost / is created on the stack by default - this user's credentials are available within the Sonassi control panel.

You should create a separate RabbitMQ user and virtualhost for each of the stores on your stack, then specify these in your site's env.php configuration file - this helps to ensure each store's queues are fully isolated from one another:

    'queue' => [
        'amqp' => [
            'host' => 'queue1.i',
            'port' => '5672',
            'user' => 'example_user',
            'password' => 'example_password',
            'virtualhost' => 'example_virtualhost'
        ],
        'consumers_wait_for_messages' => 1
    ],

Accessing the administration panel

The RabbitMQ Administration panel can be accessed at https://queue.magestack.com

If you have multiple servers, you should access the server's respective RabbitMQ instance:

i.e. https://queue.2.magestack.com

! You must be connected to the VPN to reach this administration panel.

Supervisor

We've found that Magento's default consumers_runner cron task may not always behave as expected, with issues such as:

  • Not always reliably starting queue consumers, leaving messages stuck in a Pending state
  • Leaving older queue consumer processes running after a site deployment, which then use stack resources unnecessarily

If you see similar issues to these, we would recommend using Supervisor to manage RabbitMQ message queues instead of Magento's consumers_runner.

To configure Supervisor as an example (using domain group example and vhost example.com):

  • Confirm the message queues that are currently active on your store:

    cd /microcloud/domains/example/domains/example.com/http
    bin/magento queue:consumers:list
  • Open /microcloud/domains/example/domains/example.com/http/app/etc/env.php, and disable cron_run:

    'cron_consumers_runner' => [
        'cron_run' => false,
        'consumers' => [
        ]
    ],
  • Install the supervisor package as the root SSH user:

    su -l root
    apt install supervisor
  • Create configuration files for each queue - for example, for the product_action_attribute.update queue, create /etc/supervisor/conf.d/queue-product_action_attribute.update.conf with:

    [program:queue-product_action_attribute.update]
    process_name=%(program_name)s_%(process_num)02d
    directory=/microcloud/domains/example/domains/example.com/http
    command=/usr/bin/fakechroot /usr/sbin/chroot /microcloud/domains/example /usr/bin/php /domains/example.com/http/bin/magento queue:consumers:start product_action_attribute.update --max-messages=100
    autostart=true
    autorestart=true
    user=www-data
    redirect_stderr=true
    stdout_logfile=/microcloud/domains/example/domains/example.com/http/var/log/queue.log

    Replace the example domain group and example.com vhost in the program, directory, command, and stdout_logfile variables for your store as necessary.

  • Create additional configuration files for each of the message queues for your store using the configuration template above - update the file name, and the program and command variables for each new configuration file.

  • Start Supervisor
    /etc/init.d/supervisor start

Supervisor will now manage your store's message queues, rather than Magento.

If you add any new message queues, you will need to add a new Supervisor configuration file too.