administration mode
Pssst...Ferdy is the creator of JungleDragon, an awesome wildlife community. Visit JungleDragon

 

Article: Data Lookup Benchmark »

FERDY CHRISTANT - DEC 3, 2004 (06:57:34 PM)

Introduction

This article describes a bechmark that measures the performance of three common ways to retrieve data in Lotus Notes:

  • Using @DbLookups
  • Using cached @DbLookups
  • Using profile lookups

I realize this discussion is rather old, and perhaps many have gone before me with this test, yet I still find developers (including myself) not always using the most effective method to retrieve data. This articles tries to prove which method is the fastest through a benchmark.

Test setup

Before doing any benchmark, it is important to describe the test method and environment, as doing the same benchmark in different conditions may give different results. Let's sum them up:

  • The test is done on a big client machine, plenty of memory and speed is available
  • The data to look up is on a remote machine, to maximize the difference between caching and not caching.
  • The test measures the time it takes to do a certain number of lookups. Each lookup is done to a different document, yet retrieving a constant data size every time.

The results

Let's get right down to it. Here are the results (in seconds):

performance table


Analysis:

  • @DbLookups without cache are very slow, yet constant. You can almost calculate the result before you even do the test.
  • Cached @DbLookups have a significant advantage in effectiveness over regular @DbLookups, yet the advantage quickly dissapears when we increase the number of lookups. It looks like there is a limit to the lookup cache, and it is not big.
  • Profile lookups have a significant advantage in effectiveness over both previous methods. It seems there is almost no relationship between the number of lookups, data size to look up and the measured time! The only way to explain this is that the profile is loaded into memory. Memory access speed is so fast that it does not affect the benchmark results. Both other methods have to deal with disk access and/or network access speed, which explains the slow results.

This chart sums it up quite nicely. Click to see the enlarged version:

graph of benchmark results


Conclusion

Ok, so I should always use profile lookups? Not necessarily. Generally, you could follow these guidelines:

  • Use @DbLookups without cache when it is absolutely crucial that you retrieve the latest data from the source.
  • Use cached @DbLookups when the data you retrieve does not have to be the latest and greatest, yet only use it for this purpose when you are doing a few lookups (<10) with relatively small data sizes. You will have the advantage of increased effectiveness, without the fuss of profile handling.
  • Use profiles when you are looking up data that does not change a lot, and when there's lots of it to retrieve. You will have the optimal effectiveness in all cases. When you are only looking up a few values from a profile, you may want to consider cached @DbLookups, which are easier to manage and debug.

Tip: Configuration data typically does not change often. In complex applications, there may be quite a lot of it. In these cases, profiles are the way to go.

Share |

Comments: 7
Reviews: 3
Average rating: rating
Highest rating: 4
Lowest rating: 4

COMMENT: RENE VAN ALEM rating

DEC 5, 19:58:02

comment » Nice analysis! «

COMMENT: TOM JANSEN

DEC 6, 09:01:39

comment » Nice to get some numbers for these methods with the latest hardware.

One thing you forgot to mention I think. When using profiles for lookup, you cannot change the content of the values while the database is open.

Like you said, the profiles are loaded in memory when the db is opened and then they just "sit" there. If the profiles are modified, they are only updated in the db, NOT in memory.

Only after you close the db and re-open it again the chages in the values of the profile document are shown in the lookups you make to them.

Altough profile lookups may be the fastest, this handicap limits their useability. «

COMMENT: MAARTEN DOCTER emailhomepagerating

DEC 6, 09:40:25

comment » I'm happy to see Rene (comment 1) likes your analysis.

I find it a bit strange to see his comment here, because he's your boss. «

COMMENT: FERDY CHRISTANT email

DEC 6, 16:02:12

comment » Tom,

Is this also the case on the web? I use profiles for this blog template on the web. I can edit values in the application setup screen (web-based) save them to the application setup profile document, and instantly use them.

Maarten,

I think I'm safe. I know what to publish and what not. «

COMMENT: TOM rating

DEC 7, 08:13:22

comment » Ferdy,

Yeah, web and profile docs is a different story. There is no way for the server to know when all sessions to a certain db are closed, so I guess it will always reload the profile doc from disk when a web-session needs it.

I'm not 100% sure on this. Must have been at least 4 years ago that I did a similar benchmark like you did in your article and my web-experience with Domino was very limited back then.

Did you perform your benchmark with both Notes client and browser client? «

COMMENT: FERDY CHRISTANT email

DEC 7, 12:43:37

comment » Tom,

This was a Notes-only benchmark. Assuming you are right about the caching of profiles, I guess it is still suitable to use them for retrieving settings. We usually have quite a few lookups to them, and they hardly ever change. «

COMMENT: NATHAN T. FREEMAN email

JAN 11, 10:36:03

comment » Serious flaw in your analysis, Ferdy, and I hope I'm not too late to make a contribution here.

Not many people are aware that DbLookups have *named* caches. That is, when you put in string "NoCache" you're activating a reserved keyword. BUT... you can put in "Cache1" and that is stored in a unique cache space that is separate if you specify "Cache2."

I believe that if you were to repeat the test with your code taking into account uniquely named caches (and they're strings, so you can generate them on the fly if you like) you'll get substantially better results for large numbers of cached lookups. «

RATE THIS CONTENT (OPTIONAL)
Was this document useful to you?
 
rating Awesome
rating Good
rating Average
rating Poor
rating Useless
CREATE A NEW COMMENT
required field
required field HTML is not allowed. Hyperlinks will automatically be converted.