We are sorry, information on this page is available only in Czech. Use Translator Switch to Czech

FileMaker Performance Lab Meeting June 2022 Recap

by HOnza Koudelka

FileMaker Performance Lab Meeting June 2022 Recap - Preview Image

Another Performance Lab Meeting has been great and inspirational again! 27 (out of 52 registered) developers interested in FileMaker performance joined me online to discuss how FileMaker 19.5 compares to the previous versions, how fast it runs on the new Apple silicon Macs, and what's new in BenchTest, our tool for testing and comparing performance of different FileMaker versions and configurations.

 

 

If you missed the meeting or want to recall some specific moments, you can now watch the whole recording:

Watch the meeting recording

Now let me recap the most significant news I shared in the meeting.

New version of BenchTest

I received great feedback from developers who tried BenchTest 2.0, including one bug report. Specifically, when running the standard test on Windows, the testing got sometimes interrupted by an error in the 7th and/or 8th step. I figured out this was due to the Web Viewer not always being ready to perform JavaScript. So I have modified the way how tests involving JavaScript are performed and an updated version of BenchTest is now available for you to download and try.

 

JavaScript fix

 

There are some other minor improvements as well, but the testing and scoring remains unchanged, so results collected by the new version are comparable to those collected by the previous one.

Better display of results on our web

As results kept coming from the community, it became clear that we need to separate them into more categories. For example, many tests were performed in a copy of FileMaker Pro running on the same machine as the FileMaker Server, thus completly eliminating the impact of (even low) network latency. Obviously, these results appeared as the overall leaders. On the other end of the spectrum, tests performed when connecting over the WAN appeared very low in the list due to the high network latency. So we separated these results to their own categories to make the comparison and the leaders in each category much clearer.

 

Improved display of results on the web

 

How fast are Apple silicon Macs

Last year I tested the new Mac mini M1 with both FileMaker 19.2, which was still not native and needed Rosetta, and FileMaker 19.3, which finally utilized the power of the Apple silicon with a native code. This year in January I received the long-awaited MacBook Pro with the M1 Max processor, and just a week before my June lab tests we also got our first Mac Studio. When used as a server for a single client, Mac Studio (with M1 Max) outperformed the Mac mini M1 by nice 33% when running FileMaker Server 19.4 and by 57% when running FileMaker Server 19.5.

 

M1 performance comparison as server for single client

 

Yet more interesting, however, was testing 10 concurrent clients connecting to the Mac Studio. Even when testing the most server-centric operations, which was getting data for virtual list, the Mac Studio remained mostly bored, only utilizing 4-6 processor cores.

 

Mac Studio utilization with 10 concurrent clients

 

This indicates that Mac Studio, even with the M1 Max processor, is likely to easily handle around 50 busy concurrent clients without the hardware becoming the bottleneck.

Compared to the M1 Mac mini, the M1 Max Mac Studio was able to serve the 10 concurrent users almost twice as fast...

 

Comparison with 10 concurrent users

 

Concurrent users testing

To make my test results as close to real use as possible, I run real concurrent clients as VMWare virtual machines. Using the Mac Pro (late 2013) with 96 GB of RAM, up to around 10-12 concurrent clients can be tested without significant degradation of each client's performance.

 

VMWare with 10 concurrent clients

 

To easily spin up multiple concurrent clients, I have created a pre-configured "master" virtual machine with FileMaker Pro set as a startup item and BenchTest automatically opening on launch. I am then using AppleScript to create and run specified number of "linked clones". If you would like to try the same thing, feel free to download my AppleScript scripts below:

Download the scripts

How fast is FileMaker 19.5

Finally, the most awaited comparison was the recently released FileMaker 19.5 compared to FileMaker 19.4. The new version contains around 400 bug fixes and several performance improvements, at least according to the release notes for FileMaker Pro and FileMaker Server.

The good news is that the overall performance score confirmed around 10% improvement on all the configurations I tested.

 

Overall performance of FM 19.5 vs 19.4

 

In specific cases, however, FileMaker 19.4 performed better than 19.5, mostly when running the Replace Field Contents operation. This has already been confirmed as a known issue and is going to be addressed by the 19.5.2 update.

 

Replace Field Contents slower in FM 19.5

 

The most significant improvement was expected from the new ability to evaluate summaries on the server. Unfortunately, none of my test results reliably indicated this was actually happening. There was some difference when running a single VMWare client against Mac Studio, but my expectation was much higher...

 

Comparison of calculating summary (total)

 

Discussing this in the meeting, however, led me to the conclusion that my tests were most likely affected by previous test causing the summarized data to be already downloaded to the client's local cache, so it would be good to try it out again as an isolated test. And since I had planned to be a guest of Richard Carlton's live stream on the same topic just 2 days later, I didn't wait too long to perform these extra tests...

Update from Richard Carlton's live stream

On Thursday the 30th of June it was my honor to talk about FileMaker 19.5 peformance again as a guest of FMTraining TV. If you did not attend and are interested in the whole stream, you're more than welcome to watch the recording:

 

 

The highlight from the session is that FileMaker 19.5 really calculates summaries on server if given conditions are met. Two imporant conditions missing from the Claris support article on this topic are:

  • Both FileMaker Pro and FileMaker Server need to be version 19.5 or later
  • The summarized data must not be already fully downloded to the client's local cache

But the peformance improvement when this actually happens can be huge:

 

Summary calculated by Mac Studio over LAN

 

It even seems the impact of calculating summaries on server is the bigger the slower hardware is used for the server...

 

Summary calculated by Mac Pro 2008 over LAN

 

But the biggest impact of this improvement can be noticed on a slow network (especially with high latency values):

 

Calculating summary by Mac Studio connected over WAN

 

Of course the actual impact on your use case strongly depends on how often the conditions for calculating summaries on server will actually be met. But if you connect to your FileMaker Server over a slow network or large distance (such as when it's hosted in the cloud), and you only need the summaries but not the actual records being summarized, this single new feature can be a game changer for you.

Just beware one key observation I made during my testing. This only applies to summary fields, not to the aggregate functions.

 

Summary vs aggregate

 

While until 19.4 it was safe to say it does not really matter whether you use aggregate functions, such as Sum, over a relationship, or get a value from the remote summary field over the same relationship, as of version 19.5 this is no longer true. Unless/until Claris makes this feature available for aggregate functions as well, it will be much better to create summary fields in many cases.

I have to say this round of tests (and meetings) was really exciting and I am looking forward to seeing what comes in the next version of FileMaker. Until then, please stay tuned and don't hesitate to let me know if you have any questions or comments.

Call us Call
us

+420 608 301 880

Usually available on working days between 7am and 5pm GMT

We'll call you back if you call from a discoverable phone number and fail to reach us

Let us call you Let us
call you

By completing and sending the form you agree that 24U s.r.o., a company established under the laws of the Czech Republic, with its registered office: Zvole u Prahy, Skochovická 88, CZ-25245, registered in the Commercial Register with the Municipal Court in Prague, section C, inset 74920 will use your personal data contained in the form for the purpose of sending 24U’s news, updates and other commercial communications. Providing 24U with personal data for the said purpose is optional. Details on personal data processing and on your rights connected therewith are contained in 24U’s Privacy Policy.

Loader Image