Article: Data Lookup Benchmark »
FERDY CHRISTANT - DEC 3, 2004 (06:57:34 PM)
Introduction
This article describes a bechmark that measures the performance of three common ways to retrieve data in Lotus Notes:
- Using @DbLookups
- Using cached @DbLookups
- Using profile lookups
I realize this discussion is rather old, and perhaps many have gone before me with this test, yet I still find developers (including myself) not always using the most effective method to retrieve data. This articles tries to prove which method is the fastest through a benchmark.
Test setup
Before doing any benchmark, it is important to describe the test method and environment, as doing the same benchmark in different conditions may give different results. Let's sum them up:
- The test is done on a big client machine, plenty of memory and speed is available
- The data to look up is on a remote machine, to maximize the difference between caching and not caching.
- The test measures the time it takes to do a certain number of lookups. Each lookup is done to a different document, yet retrieving a constant data size every time.
The results
Let's get right down to it. Here are the results (in seconds):
Analysis:
- @DbLookups without cache are very slow, yet constant. You can almost calculate the result before you even do the test.
- Cached @DbLookups have a significant advantage in effectiveness over regular @DbLookups, yet the advantage quickly dissapears when we increase the number of lookups. It looks like there is a limit to the lookup cache, and it is not big.
- Profile lookups have a significant advantage in effectiveness over both previous methods. It seems there is almost no relationship between the number of lookups, data size to look up and the measured time! The only way to explain this is that the profile is loaded into memory. Memory access speed is so fast that it does not affect the benchmark results. Both other methods have to deal with disk access and/or network access speed, which explains the slow results.
This chart sums it up quite nicely. Click to see the enlarged version:
Conclusion
Ok, so I should always use profile lookups? Not necessarily. Generally, you could follow these guidelines:
- Use @DbLookups without cache when it is absolutely crucial that you retrieve the latest data from the source.
- Use cached @DbLookups when the data you retrieve does not have to be the latest and greatest, yet only use it for this purpose when you are doing a few lookups (<10) with relatively small data sizes. You will have the advantage of increased effectiveness, without the fuss of profile handling.
- Use profiles when you are looking up data that does not change a lot, and when there's lots of it to retrieve. You will have the optimal effectiveness in all cases. When you are only looking up a few values from a profile, you may want to consider cached @DbLookups, which are easier to manage and debug.
Tip: Configuration data typically does not change often. In complex applications, there may be quite a lot of it. In these cases, profiles are the way to go.




Comments: 7
Reviews: 3
Average rating:
Highest rating: 4
Lowest rating: 4
COMMENT: RENE VAN ALEM
DEC 5, 19:58:02
COMMENT: TOM JANSEN
DEC 6, 09:01:39
One thing you forgot to mention I think. When using profiles for lookup, you cannot change the content of the values while the database is open.
Like you said, the profiles are loaded in memory when the db is opened and then they just "sit" there. If the profiles are modified, they are only updated in the db, NOT in memory.
Only after you close the db and re-open it again the chages in the values of the profile document are shown in the lookups you make to them.
Altough profile lookups may be the fastest, this handicap limits their useability. «
COMMENT: MAARTEN DOCTER


DEC 6, 09:40:25
I find it a bit strange to see his comment here, because he's your boss. «
COMMENT: FERDY CHRISTANT
DEC 6, 16:02:12
Is this also the case on the web? I use profiles for this blog template on the web. I can edit values in the application setup screen (web-based) save them to the application setup profile document, and instantly use them.
Maarten,
I think I'm safe. I know what to publish and what not. «
COMMENT: TOM
DEC 7, 08:13:22
Yeah, web and profile docs is a different story. There is no way for the server to know when all sessions to a certain db are closed, so I guess it will always reload the profile doc from disk when a web-session needs it.
I'm not 100% sure on this. Must have been at least 4 years ago that I did a similar benchmark like you did in your article and my web-experience with Domino was very limited back then.
Did you perform your benchmark with both Notes client and browser client? «
COMMENT: FERDY CHRISTANT
DEC 7, 12:43:37
This was a Notes-only benchmark. Assuming you are right about the caching of profiles, I guess it is still suitable to use them for retrieving settings. We usually have quite a few lookups to them, and they hardly ever change. «
COMMENT: NATHAN T. FREEMAN
JAN 11, 10:36:03
Not many people are aware that DbLookups have *named* caches. That is, when you put in string "NoCache" you're activating a reserved keyword. BUT... you can put in "Cache1" and that is stored in a unique cache space that is separate if you specify "Cache2."
I believe that if you were to repeat the test with your code taking into account uniquely named caches (and they're strings, so you can generate them on the fly if you like) you'll get substantially better results for large numbers of cached lookups. «