ORMBattle.NETThe ORM tool shootout

  • Increase font size
  • Default font size
  • Decrease font size
Home FAQs Is it possible to use a specialized API instead of common in tests?

Is it possible to use a specialized API instead of common in tests?

E-mail Print PDF

No. Although exceptions to this rule are possible - see "Exceptions" section below.

Let's define what common and specialized APIs are:

  • Common API is an API you must use by default in a particular ORM. It is the most general API from the point of supported features. It offers rich flexibility. If it is used in 80-90% cases related to a particular operation (e.g. creating an entity), it is a general API.
  • Specialized API is a separated API allowing to optimize execution of a particular operation, or category of operations, offering performance instead of flexibility.

Some examples:

  • If some ORM supports special API for batch insertions and explicitly makes you to create batches, it is a specialized API: it makes you to care about topological sorting of the entities before insertion and forming the batches by your own. But it improves insertion speed.
  • Much more obvious example is direct SQL API: almost any ORM allows you to run direct SQL queries. So in general, if I'd like to get top-level performance on insertion test, I can utilize this API. But it limits you in features: in fact, you loose all ORM features at all. The same is about materialization: I can use this API and materialize the entities by my own using foreach over IDataReader. We think this is not acceptable on performance tests here.

And the most serious reason for denying usage of specialized APIs is the following: if they're enabled, any ORM vendor can write a specialized API for each of our tests and ask us to use it. Obviously, this isn't acceptable, since does not affect on standard ORM features at all, but brings high scores for this ORM.

That's why we'd like to avoid this.

Now let's give some examples of what is acceptable:

  • Changing of some temporary settings allowing to run default API faster on our test. E.g. set appropriate batch, persistence behavior and so on. But only if these settings do not limit the user in available features further! E.g. if it's a transaction scope setting or a session scope setting, and it significantly affects on behavior, likely, we will not accept it. But if it's a statement block scope setting, it's ok.
  • Changing of global settings (e.g. cache kind, cache size), if they'll be the same for all of our tests.

Exceptions

There are few resonable exceptions to this rule. There are frameworks where usage of specialized API is officially recommended practice related to a particular test. Based on this, currently we allow the following exceptions to this rule:

  • BLToolkit tests utilize SqlQuery.Insert/Update/Delete(…, int batchSize, IEnumerable sequence) methods on CUD Multiple sequence. Since BLToolkit does not care e.g. about topological sorting and tracking, this is very natural or it.
    View code.
  • NHibernate tests utilize IStalelessSession on CUD Multiple sequence. Underlying details are described here. We think this is arguable, but since this option allows us to determine its peak performance better, we decided to allow this.
    View code.
Note: results of tests utilizing specialized APIs are marked by dashed frame in Scorecard.
Last Updated on Thursday, 05 November 2009 19:44  

Polls

Which test must we add next?
 

Subscribe to our blog


mother of bride dress