An introduction to MageStack

MageStack is a PaaS; a cloud operating system that consists of multiple open source and licensed applications combined into a single scalable environment that scale horizontally and vertically with ease.

The key to its scalability is that it offers enterprise service architecture within a singular OS.

Abstraction

Large-scale, high performance web applications require levels of separation to perform at their best; separating critical services onto separate machines so that it can scale accordingly. This typically means separating the web and database servers onto separate machines.

Now what happens when you have 2 web servers? You add a load balancer in front of those. And if you add another database server, you add another load balancer. And if your memory based caching (eg. Redis or Memcache) needs its own resources - then you put that on its own server.

Migrating from a single-server environment, to a multi-server distributed environment isn't easy or straightforward. Which is why MageStack is ready to scale from the start.

MageStack is a containerised, cloud operating system that consists of multiple nodes, running on real hardware

  • Dedicated Server (dh)
  • Firewall (fw)
  • Load Balancer (lb)
  • Web Server (web)
  • Database Server (db)
  • Mail Server (mail)
  • Access Server (acc)
  • Monitoring/Management Server (monitor)

These nodes are all containers, which on a single-server deployment, all run on the same system. But when you start to scale out horizontally - MageStack utilises the extra hardware resources and auto-scales out, so that each server is now carrying out its own role(s).

If you were to draw out the layers within a MageStack private cloud, it would look like this

| Physical hardware |      <--- data replication --->       | Physical hardware |  
--------------------------------------------------------------------------------
|                   Containers (span all physical hardware)                     |
---------------------------------------------------------------------------------
| Node Services | Node Services | Node Services | Node Services | Node Services |
---------------------------------------------------------------------------------
|                              Magento Web Store                                |

As each node is merely a container, it can be seamlessly migrated across the physical underlying hardware without complication, re-configuration or downtime. They can also be replicated, so 1 web server can suddenly scale to 10 web servers.

This means that you can start on a single server - and scale to 500 servers; without costly re-configuration or expensive system administrator time.

In essence, MageStack combines what is best about cloud operating systems (like Amazon AWS) in a completely managed, Magento optimised package.

Applications in use

As we mentioned earlier, MageStack consists of multiple applications,

The configuration for these services is managed centrally from a master Git repository (which backs up all configuration files). All MageStack servers receive their software updates automatically.

Management of each of these applications can be done through their respective web portal. Rather than give you a custom made control panel with limited functionality - we've selected best-of-breed pieces of software to manage each respective area. Eg. Kibana for log viewing

Security

Accessing MageStack requires you to connect to a VPN first. We can provide this in one of two ways.

  1. OpenVPN - typically used on a per user basis
  2. IPSEC - typically used for an entire organisation, office or warehouse to be permanently connected

OpenVPN is by far the most common deployment method and is what MageStack natively supports. Without connecting to the VPN first, you won't be able to connect to SSH, FTP, PHPMyAdmin, MySQL or infact any service other than the website itself.

But once you are connected to the VPN, you'll be able to access all the management panels within MageStack.

Access

Each node within MageStack (lb, web, db etc.) runs in its own, completely isolated, Linux environment. You have no SSH access to any of these environments - they are entirely managed by MageStack itself.

You only have access to your access node (acc) - which you can connect to via SSH, FTP, SCP, SFTP. Once connected, you'll have full root access within this environment to do as you please. Whether that means installing useful tools like Git or Subversion - through to installing complete applications/daemons.

The access server is yours to do with as you please.

You have no direct access to any of the other servers. If you are wondering how you restart services or check configuration file syntax, you can do that through Monit.

Load Balancing

MageStack is configured with 1 web server to begin with; if you've got more than 1 physical server (ie. dh) - then we'll configure multiple web servers. The load balancer will automatically detect any new web servers and redirect traffic to them.

If you want to add more servers in the future, be it temporarily (eg. for a Groupon campaign) or permanently - its very easily done; just contact the support team.

High Availability

Every MageStack dedicated server (dh) is configured to be one-of-a-pair. This means that it is configured by default to replicate its entire disk contents and containers to another server - meaning that should it ever fail, the other server will seamlessly take over its role.

This aspect of MageStack is critical in its ability to provide a totally fault tolerant, data-secure environment.

So if you start with a single standalone server, at any point, you can add another dedicated server - and it will immediately replicate its contents to that machine - and should it fail, the other machine will take over all of its services.

We can also use this feature to great effect when conducting any hardware upgrades. As we can simply add another server to your pool - let that take over; whilst we perform maintenance/upgrades on the other server.

Next steps

So now you have a fundamental understanding of MageStack, its time to start using it. Here are the essential topics to getting started with MageStack

  1. Accessing MageStack for the first time
  2. Hostnames and DNS
  3. Directory structure
  4. Database administration with PHPMyAdmin
  5. Server graphs with Munin
  6. Managing services with Monit
  7. Viewing loadbalancer statistics on HAProxy
  8. Apache SOLR Administration
  9. Naming conventions
  10. Memcache instances
  11. Redis instances
  12. Maintenance mode
  13. Varnish cache
  14. Creating subdomains
  15. Debugging Varnish/HAProxy/Nginx
  16. Internal IP addresses
  17. Troubleshooting