On January 19, 2021 we met online for the second time to discuss my FileMaker performance lab testing results. For the case you either missed the meeting, or attended and want to recall some details, a recording of the meeting is now available. Here's also a brief 4-minute excerpt:
In the meeting I talked about the impact of record size on performance being often higher than impact of any other factor.
As a surprise I also showed my first results of testing actual simultaneous clients connecting to FileMaker Server 19.2 instead of simulating clients by server-side scripts. To perform these tests I created a VMWare virtual machine with FileMaker Pro configured to start and immediately connect to my test database. Then for each test I created as many linked clones of this master virtual machine as I wanted to test in parallel.
My tests revealed, that time needed to perform find in a table with 100,000 records for the first time, when the data probably were not in the local cache of the clients yet, was somewhat proportional to the number of workers (clients). That was not a straight confirmation of the positive impact of the shared lock mechanism introduced in FileMaker Server 19.1. At the time of the meeting, however, I was already running the same set of tests with FileMaker Server 18v4 and promised to share the comparison as soon as I have it, so here it is:
This is something remarkable! It seems that at least the indexed searches, which were supposed to take advantage of the new shared lock mechanism, are really working much better with multiple clients connecting to FileMaker Server 19.2. The obvious next question is how many real solutions and situations will be affected, as other read-only operations don't seem to be able to take advantage of the shared lock so much yet...
In the meeting we also discussed how I could prevent client from using local cache so that I could efficiently test FileMaker Server performance for read-only operations multiple times in a row. Vincent Lugnier suggested trying an AppleScript that empties local caches and locks them to prevent FileMaker from updating them. Unfortunately, when I tested that, I discovered that the locked caches are just the "persistent cache" whose purpose is to preserve locally cached data between sessions. It did not prevent FileMaker Pro from caching data in a temporary file during the session.
The only reliable way to invalidate the whole cache I could find so far is to use Replace Field Contents in server-side script to modify every record, or do delete and re-generate all the test data. These methods are both unfortunately too time consuming for something I would like to do before every single test. So if you have any other ideas, I will appreciate you letting me know.
Finally, I also made a beta version of BenchTest, my testing database, available for download to all meeting participants. While running further test, however, I revealed two bugs in my script that was measuring hard drive reading and writing speeds. So I uploaded an updated version marked as 2.0b2. If you have downloaded the previous beta 2.0b1, please make sure to re-download the database using the same link, to get the fixed version.
Since BenchTest is not finished yet, I decided to share it only with the meeting participants for now. But don't worry, you can still attend the meeting even though it's over now. Just register to watch the recording:
If you have any questions, comments, or ideas, please let me know. I look forward to seeing you at the next FileMaker Performance Lab Meeting.
