ÃÛ¶¹ÊÓƵ

DocumentationCommerceCommerce on Cloud Infrastructure Guide

Pro architecture

Last update: Tue Nov 12 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

  • Admin
  • Developer

Your ÃÛ¶¹ÊÓƵ Commerce on cloud infrastructure Pro architecture supports multiple environments that you can use to develop, test, and launch your store.

  • Master—Provides a master branch deployed to Platform as a service (PaaS) containers.
  • Integration—Provides a single integration branch for development, though you can create one additional branch. This allows for up to two active branches deployed to Platform as a service (PaaS) containers.
  • Staging—Provides a single staging branch deployed to dedicated Infrastructure as a service (IaaS) containers.
  • Production—Provides a single production branch deployed to dedicated Infrastructure as a service (IaaS) containers.

The following table summarizes the differences between environments:

INTEGRATION
STAGING
PRODUCTION
Supports settings management in the Cloud Console
Yes
Limited
Limited
Supports multiple branches
Yes
No (Staging only)
No (Production only)
Uses YAML files for configuration
Yes
No
No
Runs on dedicated IaaS hardware
No
Yes
Yes
Includes Fastly CDN
No
Yes
Yes
Includes New Relic service
No
APM
APM + NRI
Automatic Backups
No
Yes
Yes
NOTE
ÃÛ¶¹ÊÓƵ provides the Cloud Docker for Commerce tool for deploying to a local Cloud Docker environment so that you can develop and test ÃÛ¶¹ÊÓƵ Commerce projects. See Docker development.

Environment architecture

Your project is a single Git repository with three main environment branches: integration, staging, and production. The following diagram shows the hierarchical relationship of Pro environments:

High-level view of Pro environment architecture

Master environment

On Pro projects, the master branch provides an active PaaS environment with your production environment. Always push a copy of the production code to the master environment so that you can debug the production environment without interrupting services.

Caveats:

  • Do not create a branch based on the master branch. Use the integration environment to create active branches for development.

  • Do not use the master environment for development, UAT, or performance testing

Integration environment

The integration environment runs in a Linux container (LXC) on a grid of servers known as PaaS. Each environment includes a web server and database to test your site. See Regional IP Addresses for a list of AWS and Azure IP addresses.

Recommended use cases:

Integration environments are designed for limited testing and development before moving changes to staging and production environments. For example, you can use the integration environment to complete the following tasks:

  • Ensure that changes to continuous integration (CI) processes are Cloud compatible

  • Test critical workflows on key pages like Home, Category, Product Details Page (PDP), Checkout, and Admin

For best performance in the integration environment follow these best practices:

  • Restrict the catalog size - For reference, the Sample Data contains about 2,048 products. Try reducing your catalog size to around 4,000-5,000 products.
    To check the number of products in the catalog, run the following MySQL query:

    code language-sql 
    select distinct count(entity_id) from catalog_product_entity;

  • Reduce the number of customer groups - Having too many customer groups can affect the indexing performance and overall performance.

  • Limit use to one or two concurrent users

  • Disable cron jobs and manually run as needed

Caveats:

  • Fastly CDN and New Relic services are not accessible in an integration environment

  • The integration environment architecture does not match the Staging and Production architecture

  • Do not use the integration environment for development testing, performance testing, or user acceptance testing (UAT)

  • Do not use the integration environment to test B2B for ÃÛ¶¹ÊÓƵ Commerce functionality

  • You cannot restore the database in the integration environment from the database production or staging

NOTE
Projects provisioned before June 5, 2020 had multiple, smaller Integration environments. If you need a larger Integration environment for testing and development, request an upgrade to Enhanced Integration environments. See the Integration Environment request article in the ÃÛ¶¹ÊÓƵ Commerce Help Center for details.

Staging environment

The staging environment provides a near-production environment to test your site. This environment, which is hosted on dedicated IaaS hardware, includes all services, such as Fastly CDN, New Relic APM, and search.

Recommended use cases:

The environment matches the production architecture and is designed for UAT, content staging, and final review before pushing features to the production environment. For example, you can use the staging environment to complete the following tasks:

  • Regression testing against production data

  • Performance testing with Fastly caching enabled

  • Test new builds instead of patching in Production

  • UAT testing for new builds

  • Test B2B for ÃÛ¶¹ÊÓƵ Commerce

  • Customize cron configuration and test cron jobs

See Deployment workflow and Test deployment.

Caveats:

  • After launching the production site, use the staging environment primarily to test patches for production-critical bug fixes.

  • You cannot create a branch from the staging branch. Instead, you push code changes from the integration branch to the staging branch.

NOTE
Some projects demand a more sophisticated development workflow. To support this need, ÃÛ¶¹ÊÓƵ offers an additional staging environment as an add-on option to your cloud infrastructure.

Production environment

The production environment runs your public-facing single and multi-site storefronts. This environment runs on dedicated IaaS hardware featuring redundant, high-availability nodes for continuous access and failover protection for your customers. The production environment includes all services in the staging environment, plus the New Relic Infrastructure (NRI) service, which automatically connects with the application data and performance analytics to provide dynamic server monitoring.

Caveat:

You cannot create a branch from the production branch. Instead, you push code changes from the staging branch to the production branch.

Production technology stack

The production environment has three virtual machines (VMs) behind an Elastic Load Balancer managed by an HAProxy per VM. Each VM includes the following technologies:

  • Fastly CDN—HTTP caching and CDN

  • NGINX—web server using PHP-FPM, one instance with multiple workers

  • GlusterFS—file server for managing all static file deployments and synchronization with four directory mounts:

    • var
    • pub/media
    • pub/static
    • app/etc

  • Redis—one server per VM with only one active and the other two as replicas

  • Elasticsearch—search for ÃÛ¶¹ÊÓƵ Commerce on cloud infrastructure 2.2 to 2.4.3-p2

  • OpenSearch—search for ÃÛ¶¹ÊÓƵ Commerce on cloud infrastructure 2.3.7-p3, 2.4.3-p2, 2.4.4 and later

  • Galera—database cluster with one MariaDB MySQL database per node with an auto-increment setting of three for unique IDs across every database

The following figure shows the technologies used in the production environment:

Production technology stack

Redundant hardware

Rather than running a traditional, active-passive master or a primary-secondary setup, ÃÛ¶¹ÊÓƵ Commerce on cloud infrastructure runs a redundant architecture where all three instances accept reads and writes. This architecture offers zero downtime when scaling and provides guaranteed transactional integrity.

Because of the unique, redundant hardware, ÃÛ¶¹ÊÓƵ can provide three gateway servers. Most external services enable you to add multiple IP addresses to an allowlist, so having more than one fixed IP address is not a problem. The three gateways map to the three servers in your production environment cluster and retain static IP addresses. It is fully redundant and highly available at every level:

  • DNS
  • Content Delivery Network (CDN)
  • Elastic load balancer (ELB)
  • Three-server cluster comprising all ÃÛ¶¹ÊÓƵ Commerce services, including the database and web server

Backup and disaster recovery

ÃÛ¶¹ÊÓƵ Commerce on cloud infrastructure uses a high-availability architecture that replicates each Pro project on three separate AWS or Azure Availability Zones, each zone with a separate data center. In addition to this redundancy, Pro staging and production environments receive regular, live backups that are designed for use in cases of catastrophic failure.

Automatic backups include persistent data from all running services, such as the MySQL database and files stored on the mounted volumes. The backups are saved to encrypted Elastic Block Storage (EBS) in the same region as the production environment. The automatic backups are not publicly accessible because they are stored in a separate system.

NOTE
The mounted volumes only include/refer to the writable mounts and will not include all of your app/ directory. As for the other files, they are created/generated by the build and deployment process, and you will also have to check your Git repository for remaining files.
TIP
On Pro Staging and Production environments, you must submit an ÃÛ¶¹ÊÓƵ Commerce Support ticket to retrieve a specific backup noting the date, time, and timezone in the ticket.
ÃÛ¶¹ÊÓƵ does not restore any environments from an automatic backup. See Restore a DB snapshot from Staging or Production for help choosing a method to restore a Staging or Production snapshot.

You can create a manual backup of the database for your Staging and Production environments using CLI commands. See Back up the database. For integration environments, ÃÛ¶¹ÊÓƵ recommends creating a backup as a first step after accessing your ÃÛ¶¹ÊÓƵ Commerce on cloud infrastructure project and before applying any major changes. See Backup management.

Recovery Point Objective

RPO is six hours maximum time to last backup (for example at 06:00, then 12:00, then 18:00). The frequency of backups depends on the backup schedule of your plan and the volume of changes to write to the storage service.

Retention policy

ÃÛ¶¹ÊÓƵ retains automatic backups according to the following data retention policy:

Time Period
Backup Retention Policy
Day 1 through 3
One backup per hour
Days 4 through 7
One backup per day
Weeks 2 through 6
One backup per week
Weeks 8 through 12
One bi-weekly backup
Month 3 through 5
One backup per month

This policy may vary depending on your cloud infrastructure plan.

Recovery Time Objective

RTO depends on the size of the storage. Large EBS volumes take more time to restore. Restoration times may vary depending on the size of your database:

  • A large database (200+ GB) can take 5 hours
  • A medium database (150 GB) can take 2 1/2 hours
  • A small database (60 GB) can take 1 hour

Pro cluster scaling

The Pro cluster sizing and compute configurations vary depending on the chosen cloud provider (AWS, Azure), region, and service dependencies. ÃÛ¶¹ÊÓƵ cloud infrastructure can scale Pro clusters to accommodate traffic expectations and service requirements as demands change.

The redundant architecture empowers ÃÛ¶¹ÊÓƵ cloud infrastructure to upscale without downtime. When upscaling, each of the three instances rotates to upgrade capacity without impacting site operation. For example, you can add extra web servers to an existing cluster should the constriction be at the PHP level rather than the database level. This provides horizontal scaling to complement the vertical scaling provided by extra CPUs on the database level. See Scaled architecture.

If you expect a significant increase in traffic for an event or other reason, you can request a temporary increase in capacity. See How to request a temporary upsize in the Commerce Help Center.

recommendation-more-help
05f2f56e-ac5d-4931-8cdb-764e60e16f26