Matrix Performance Report: Caching13 Sep
Web caching is a fundamental concept of the Internet today. From large-scale corporate sites to personal web pages and blogs, content caching can greatly improve the performance of a site when it is configured correctly. In turn, users will appreciate a site that is faster to load and will be more inclined to revisit that site in the future.
So, what is the best caching setup for your system? Squiz Matrix provides users with a number of caching options, from configurable caching periods to user group-specific caching rules. With so many possibilities, it can be easy to get confused.
Of course, it is impractical to provide an optimal cache setup that will cater for all users – determining the most suitable cache setup depends on a variety of factors, including how often your content is updated and how many hits your site receives each day. We can, however, point you in the right direction.
Our second Matrix Performance Report focuses on investigating the performance impact of caching within Matrix. Read on as we explore the benefits of caching and how caching affects the user-experience of your sites.
Assets and Their Content
It goes without saying that loading a cached version of an asset is generally going to be more efficient than loading an asset without cache. If the data of a site can be served from cache, the load time to users is going to be comparatively faster than when that data is served directly from Matrix. What we need to determine is exactly how much faster?
We ran a series of tests on both a Standard Page and an Asset Listing, each with a variety of content, and recorded the load time of these assets when cache was enabled and when it was not. These results were then compared to determine how caching effects the performance of these assets.
Please note that these tests were conducted on assets without applied designs. In these tests, we are only investigating the impact that the content of the asset has on caching.
The results of these tests are as follows:
|Asset Type / Content Info||Performance Increase When Cached|
|Standard Page / No content||283%|
|Standard Page / With content||1983%|
|Asset Listing Page / No assets listed||625%|
|Asset Listing Page / 10 assets listed||692%|
|Asset Listing Page / 100 assets listed||1288%|
In these results, a dramatic increase in performance in displayed when caching both a Standard Page and an Asset Listing Page, relative to the content of the asset. A Standard Page with no content saw an already impressive increase of 283% - this sky-rocketed to over 1900% on a page with complex content. Similarly, the Asset Listing Page saw gradual improvements in performance as additional assets were listed (approximately 66% improvement per 10 assets).
These results are due to Matrix being able to serve content data directly from cache rather than from Matrix. The larger the amount of content, the more beneficial serving this content from cache is going to be. This is because, if the asset is not cached, Matrix has to manually process and generate the content each time it is requested. Naturally, the larger the amount of content requiring processing, the longer it is going to take.
The concept of the serving of content with cache and without is shown in the diagram below.
These results overwhelmingly encourage the caching of assets within Squiz Matrix that can be cached. Form assets, such as Custom Forms and Asset Builders, cannot be cached; see the Nested Content DIV section of this report for more information on improving the performance of these assets.
The trick when using cache is determining the setup for your system that is going to be the most beneficial to your users.
Here are a few things to consider when configuring your cache expiry settings:
Ascertain how often your content is updated and how complex that content is. If the content of your assets is simple and already receives an efficient load time, you may not benefit from more aggressive caching strategies. Similarly, if your content is rarely updated, you may only need
to set cache to expire once a day.
On the other hand, if your content is complex and/or is updated often throughout the day, you may benefit from more frequent caching of your assets.
- Take advantage of manual cache clearing. Clearing cache manually can be a viable alternative to regular cache expiry with Matrix's Clear Matrix Cache Tool, allowing you to specify the assets to clear cache for. This is useful if your content is infrequently or only moderately updated throughout the day, allowing you to clear your cache as you go.
- Know your users. Understanding the traffic of your site is key in developing an effective cache strategy. How many users visit your site daily? When do your users most regularly visit? What pages or sections are the hot spots of your site? Develop your cache configuration around this information to provide an efficient user-experience catered specifically to your site.
- Make use of the Cache Manager's Root Node Specific and Type Code Specific settings. This will allow you to configure the cache settings of specific sections and asset types in your system on a case-by-case basis. For more information on how these settings affect performance, see the Cache Manager section of this report.
- Utilise Performance Mode. Squiz Matrix's Performance Mode includes a nifty caching tool that actually allows you to compare the performance of an asset when it is cached and when it is not. Use this tool to determine the benefits of caching for your own individual assets. For more information on the information provided by Performance Mode's caching tool, see the Monitoring Caching in Performance Mode section of this report.
Designs and Design Areas
The next step was to explore how the design of an asset benefits from caching. Specifically, we wanted to really dig deep and examine some of the different design areas on a design and how they are individually affected by caching.
To learn more about the design areas available within Squiz Matrix, visit the Squiz Matrix User Manual Library.
Naturally, caching a design will, in most cases, improve its performance. This is dependent, however, on the individual design areas of the design and the extent of their cacheable content. As a result, the benefits of caching are going to differ greatly from one design to another.
The Nest Content design area benefits from serving its nested content from cache rather than the system having to process this information manually on each load, similarly to our previous test results (Assets and Their Content). As such, it is expected that the complexity of this nested content will correlate to the increase in performance, in the same manner as our previous test results (see below for our investigation into this theory).
To learn more about the Nest Content design area, visit the Squiz Matrix User Manual Library.
Similarly, the Menu design area can also benefit from caching, especially on complex navigation menus containing non-live assets, such as assets that are Under Construction (see below for more information on how caching the Menu design area can improve performance).
To learn more about the Menu design area, visit the Squiz Matrix User Manual Library.
On the other side of the spectrum, many design areas cannot be cached at all, even when caching is enabled. These design areas are as follows:
- Login Form
- Show If
The Body and Exit design areas are not cached as they would not benefit from caching; any body content is cached on the asset level, while the Exit design area contains no cacheable content.
The Login Form design area cannot be cached due to being a form. Forms are not cached as they contain multiple pages and bodycopies. If the login form of a design was cached, it would not be able to complete a submission and, hence, not work. For more information, see the Nesting Form Asset Types section below.
Finally, the Show If design area is a condition based design area that may show alternate content to a variety of different users. If this design area was cached, this content would not be dynamic and would not adhere to set conditions.
Nest Content Design Area
Since we have determined that the Nest Content design area does indeed benefit from caching, further testing is required to investigate to what effect caching can improve performance.
In our previous tests, an asset with complex content saw a 1700% greater improvement when compared to an asset with no content. Do these drastic results also apply when caching nested content?
Also, as many designs contain multiple Nest Content design areas, we were interested in determining how caching more than one Nest Content design areas on a single design would effect performance.
Tests were run on a Nest Content design area, with and without content, to determine the performance increase when the design area was cached and when it was not. These tests were repeated on a design with both two and three Nest Content design areas.
The results of these tests are as follows:
|Performance Increase When Cached|
|1 Design Area||2 Design Areas||3 Design Areas|
|Standard Page / No content||133%||117%||
|Standard Page / With content||814%||1160%||1369%|
These results indicate a sizeable increase in performance when nesting a Standard Page with content; an initial boost in performance of over 800% is shown, with subsequent increases of approximately 200-300% with each additional design area added.
When nesting a Standard Page with no content, our tests display an initial increase in performance of 133%, however this drops by approximately 15% with each subsequent design area added.
These results are indicative of the unique way that Squiz Matrix caches design content. While serving bodycopy content directly from cache eliminates much of the processing time of an asset, serving design content from cache still requires the processing of the design's PHP file.
This extra processing negates some of the performance increase that is gained by serving from cache and explains the drop in performance for each additional design area when our Standard Page contains no content. This drop is also present on our tests with content, however the improvements due to serving the content from cache counter any additional processing time.
Menu Design Area
A site's menu can often be quite complex, containing multiple items over a number of different levels. These menus can potentially cause significant impacts on performance, but even more so when menus contain assets of multiple statuses and permissions.
This is due to the extra processing required to check the permissions of non-live assets to determine whether or not to display these pages in the menu of a site. As we already know that caching can greatly reduce the processing time of assets, we wanted to test just how effective caching would be for these menus containing non-live assets.
Tests were conducted on three different menu configurations (listing 20 assets) as viewed by both a public user and a logged-in user (Backend User with write access on our assets).
Please note that these tests have been conducted on the Menu Normal design area.
The results of these tests are as follows:
|Menu Configuration||Performance Increase When Cached|
|Public User||Logged-in User|
|All assets Live||543%||76%|
|Half Live, half Under Construction||618%||105%|
|All assets Under Construction||671%||146%|
Not surprisingly, a significant improvement is displayed when caching our Menu design area, increasing as more non-live assets are listed on our menu. What is surprising is that our public user tests display a much greater improvement when compared to the results of our logged-in user tests (470-530% greater improvement).
The increasing performance improvements in these results are due to the unique way Squiz Matrix handles permission checks of items on the Menu design area.
After an initial database query to return the assets the public or logged-in user has read access to, Matrix will check the status of each item to be listed on the menu.
If an asset is Live, no further processing is required and the asset is published on the menu. This accounts for the lower performance improvement when all assets are Live, as the non-cached loading time is much lower than when the menu contains non-live assets.
When an asset is non-live (in this case, Under Construction), Matrix runs a readAccess query call to check the permissions of the asset. This extra processing time is eliminated when a menu is cached, explaining the greater performance improvement.
The disparity between our public user and logged-in user results is due to the creation of a cache key, determined by a user's permission key; a public user requires only a public permission key, resulting in Public Level Caching, while a logged-in user requires Matrix to retrieve any user groups the user is under to provide Group Level Caching. This extra processing means that retrieving from cache is marginally slower for our logged-in user, explaining the less impressive performance improvements.
To learn more about Public and Group Level Caching, visit the Squiz Matrix User Library.
Nested Content DIV
Asset content can also be nested in another asset on Nested Content divisions. We have already discussed the improvements caching provides to nested content in a design, but how does it affect content nested on a Nested Content DIV?
We ran a similar set of tests to those of our Nest Content Design Area tests; testing a Nested Content division, with and without content, on a Standard Page to determine the performance increase when the division was cached and when it was not. These tests were then also repeated on a page asset with both two and three Nested Content divisions.
The results of these tests are as follows:
|Performance Increase When Cached|
|1 Division||2 Division||3 Division|
|Standard Page / No content||866%||1567%||2283%|
|Standard Page / With content||2700%||5350%||7683%|
A significant improvement in performance is seen in these results, with a fairly consistent jump for each subsequent page nested on the asset. Increases of approx 700% are seen on divisions nesting a Standard Page without any content, while more impressive increases of over 2000% are seen when nesting pages with complex content.
The results of these tests are slightly more impressive than our previous content caching tests due to the extra divisions on each page asset. Matrix does take a hit to performance as it individually processes each division on an asset (remember that nested assets may also contain multiple divisions that require processing). By serving this content from cache, that processing time is eliminated and, as such, a greater improvement is seen.
Nesting Form Asset Types
Form assets, such as Custom Forms and Asset Builders, cannot be cached within Matrix. This is because, while each form contains multiple pages and bodycopies, only the initial page of the form could be saved to cache. If a form was to be cached, once a user submitted the first page of a form, Matrix would not be able to serve the following page from cache, as it would not exist.
As a result, form asset types should not be used in a Nested Content division of a cached page. Due to the static caching of page assets with Matrix, the system will be unable to determine the type of content that is being nested and will cache the entirety of the page, including the nested form asset.
Alternative implementations involve nesting a form asset in a page via an uncached Nest Content design area or redesigning the page as a form asset and nesting the content of your page asset within. In this later example, the form will not cache, however, the nested content will.
This notion of nesting content within a form is one we wanted to further explore as it allows you to cache specific areas of your form that otherwise would not be cached, while still retaining complete functionality.
To determine the performance impact of nesting cached additional content on form assets, we ran a series of tests on both a Custom Form and an Asset Builder. The performance of each of these assets was recorded when additional content was printed on the form via a cached Nested Content division and compared to the performance of the form when the additional content was uncached, printed directly on the form.
The results of the tests are as follows:
|Form Configuration||Performance Increase|
|Custom Form / No cache||-|
|Custom Form / Nesting a cached Standard Page||52%|
|Asset Builder Page / No cache||-|
|Asset Builder Page / Nesting a cached Standard Page||45%|
Our tests on both the Custom Form and Asset Builder Page assets display a modest performance increase of approximately 50% when additional content is cached via a nested Standard Page.
In our previous Nested Content DIV tests, we saw much greater improvement in performance due to the elimination of DIV processing. This was because our Standard Page asset was serving its entire bodycopy entirely from cache.
As our Custom Form and Asset Builder Page assets are not cached, our Nested Content DIV must still be processed before serving its content from cache. This explains why performance did not dramatically increase when serving the nested content from cache, as any benefits were cancelled out by the extra processing of our nested content division.
The more complex the content being nested is, the greater the improvement in performance will be. The trick is determining if the benefits of caching your content are going to outweigh the extra processing time of the additional division. It is also important to keep in mind that any additional divisions will require extra processing time.
The Cache Manager
Now that we have determined the impact on performance caching has on an asset, its content and design, we must consider the configuration of the Cache Manager, how its settings are evaluated and how this evaluation process affects performance.
The Cache Manager asset is the central hub of cache configuration in your Squiz Matrix system. It allows you to control a vast number of caching options, including activating caching on your system and the system-wide clearing of cache.
In addition to these general caching options, the Cache Manager allows you to configure the caching status and expiry for specific root nodes and asset types in your system. In our tests, we also want to explore how these more advanced cache settings affect the evaluation process of the Cache Manager?
To learn more about the Cache Manager, visit the Squiz Matrix User Manual Library.
A series of tests were run on the Cache Manager with various configuration settings. The evaluation time of these settings was recorded and compared to the evaluation time of the Cache Manager when configured to its default settings.
Our tests indicated that the both the Root Node and Asset Type Specific settings on the Cache Manager did not cause a significant impact on performance when compared to its default settings.
When an asset is viewed on the front end, the Cache Manager's role is to determine if the asset should be served from cache or not.
If the Cache Manager has been configured with Root Node and/or Asset Type Specific settings, extra processing must be run to check the asset against these options. Our tests indicate, however, that these extra checks require minimal processing time, especially when specifying asset type rules.
The performance of the Cache Manager when specifying root node rules is dependent on the number of root nodes specified. This is because the Cache Manager must load the children of each specified root node to establish whether or not the asset is a child of one of them. The more root nodes that have been specified, the longer this processing time is going to take. That being said, our tests indicate that specifying as many as 25 root nodes will only decrease performance by approximately 10%.
It stands to reason that most user's Cache Manager configurations will not be providing any negative impact on performance. Users with unusually large numbers of specified root nodes, however, should keep in mind that these may be causing a very slight performance decrease.
Monitoring Caching in Performance Mode
Squiz Matrix's Performance Mode (v4.4.1+) allows users to analyse the performance of assets within their system. This mode includes a useful Caching tool, enabling the analysis of assets when using a system's default cache settings and when caching is disabled.
Putting this tool into effect, we ran Performance Mode on the News page of a site, analysing and comparing the performance of the page when it was cached and when caching was disabled.
Performance Mode indicated that our page had a total load time of 0.61 seconds when it was not cached and 0.23 seconds when using the system's default cache settings - an improvement of 165%.
While this data represents an impressive performance improvement, the most interesting thing about Performance Mode is that it allows us to actually break down these results to determine which assets on our page benefited the most from caching.
When comparing the analysis of our cached and non-cached performance results, we can note that certain assets that appear on our non-cached results are not appearing within the performance graph of our cached results (assets #666, #407 and #363). This is become these assets are content that has been nested within the other assets on the page. As the content of these assets are called directly from cache, any nested content does not require processing, significantly improving performance.
This is also indicative in the number of queries run during the processing of the page, a drop from 313 to just 108.
Regarding the individual asset loading on our page, Performance Mode tells us that our standard content assets have dropped to virtually no load time at all, as all content is simply pulled directly from cache. The design of our page is the main offender in negatively impacting performance, but even it sees an improvement of 85% when cached.
From these results, it's quite clear that Performance Mode's Caching tool can be effective in determining the impact of caching on your assets. Users should utilise this tool when configuring a caching setup for their system to gain a clearer understanding of how caching can benefit their individual needs.