Measurement Update 4.5.1: More Stable Metrics, Faster Tests

Last week, adjustments in AWS infrastructure led to temporary instability in measurement results, including those from our Watchdog and its notifications.

In response to these unforeseen changes, we diligently worked to ensure more stable results and took the opportunity to implement updates that had been waiting in the wings for some time.

What exactly changed, and how might it affect your graphs? The changes are most noticeable in the FCP metric and also in LCP. All of this, rather interestingly, just on mobile devices:

1. From Saturday, 21st September, you might have noticed deteriorated and unstable FCP and possibly LCP metric values due to test instability on AWS. This change is marked as "AWS Changes (Unstable Results)" in the automatic note we send to everyone.
2. From Friday, 27th September, more stable results have been visible. Various tests at different times are returning similar results. Consequently, the FCP and LCP data will be slightly worse than before.
3. From Tuesday, 1st October, we accelerated the test machine setup to better match current connection speeds in Central Europe, specifically reducing RTT from 150 to 100 ms and slightly speeding up the test machine's connection. This will result in an improvement in metric values once again. The automatic note indicates this as "Measurement Change (Faster Tests)".

Between 21st September and 2nd October, we disabled automatic notifications for these two metrics to prevent the vast majority of PLUS monitoring clients from receiving false positive alerts about speed deterioration.

The new FCP and LCP metric values (as well as others) obtained on mobile devices will be somewhat different from the state before 21st September.

We continue to work on refining measurements and introducing new features that will help you achieve faster websites.