Well written! I don't agree with most of it, but you have done a superb job of defining some of the most pertinent issues. That is, I agree with your problem descriptioin but not with your conclusion. It will take me some time to come up with something that can adequately rebut your post, but until then let me just cheer your very useful description of the problem!

/Mats

Repost. I must have done something wrong when trying the first time something like an hour ago. Or you will soon have two pretty similar comments.
:-)

Hi Clemens,

VERY good, as always!

I'm one of those believing that O/R Mapping is a good tool for many situations. (I also think I'm an exception to your generalization since I have spent lots of time the last 17+ years writing SQL, sprocs, triggers and so on.)

Oh, and I'm a strong believer in using the right tool for the job.

Anyway, let's focus on you instead.
:-)
I'm curious regarding how you see LINQ? Is it a good "compromise" in your opinion or a step in the wrong direction?

Best Regards,
Jimmy
www.jnsk.se/weblog/
###

Hi Clemens,

Very well written and very useful. Thanks.

I totally agree. I think with the tools we have today (VS2005, et al) for integrating data into our applications and the tools we have for abstracting so much of the processing (WF, et al) eliminates most of the need for O\R mapping. Fewer and fewer of the business rules we use in our apps will be written in code by hand, so any (perceived or otherwise) productivity gains provided by giving developers an OO abstraction on top of data will be made moot by the productivity gains provided by the tools that abstract the business rules themselves. The data is just another set of entities that the tools will use in the designers.

Hi Clemens,

I agree with some of what you said, especially the 'silver-bullet syndrom' part you mentioned. But I think there is a place for O/R mappers in todays development. One could even argue that hand-coding a data-access layer to some degree is like writing a (very specialized) O/R mapper and that using a tool to automate this process is just a smart way of coding more efficiently. In my opinion there are often parts of an application that could use such a tool without any major performance implications and a boost in developer productivity. Of course having said this, one has to smart enough to know when not to use O/R mappers, I agree with that:)

Best regards,

Marc

Hi Clemens,

Excellent points. OR-Mapping seems to be inspired by the idea that to be able to write '_customer.Save()' in your code is beautiful as it hides the complexities of data storage. Next thing we know we're adding thousands of lines of code to achieve this beautiful statement, only to find we have to bypass it in the end due to performance issues.

Does that mean that OR-Mapping isn't beautiful? No, it still is. Is it worth the cost? In performance? In development time? In lines of code? In maintenance of the code? Probably not, but for the last decade we've managed to get away with selling this to our customers as the way to go. Customers are wisening up and want their bucks worth.

ADO.NET2.0 and typed datasets offer a significant boost in easy of use and therefore developer productivity. It'll be interesting to see if DLINQ can match this.

- Mark

So how are we supposed to work with data in our applications? If OOP (although not all OOP or O/R paradigms promote "customer.Save" semantics) is such an evil beast for lously couple systems, I would like to see what the alternative is? XML docuemnts directly from the database sent all the way up to the binding in presentation layer? Now how is that for performance?

Or do you promote typed datasets and data adapters? I would find that quite humourus, because ds / da is nothing but a really low-level implementation of o/r, with none of the benefits but all of the drawbacks.

I would very much like to see you talk a bit about the alternatives and why the pro's of thoose overcome the pro's of an O/R mapper.

I think that most of your points are correct. Although I disagree with some of your conclusions, I think there is still hope!

For example taking your point 'share schemas not code': what do you think OR mapping files are? Surely they are just thinly disguised schemas?

The best 'OR Mappers' realise this and build on it, considering a mapping file a unit of re-use. Allowing for additional mapping files (or schemas) to be added later.

If you take this idea a little further and allow one mapping file to reference another then you are essentially providing a declaritive integration model. That allows two data-layers to be integrated together declaratively.

This allows you to 'build out' an applications data model as required. Also even if the actual mapping changes the code using is likely to continue to function just fine.

This is very similar to the WinFS approach. I invite you to look at this: www.base4.net/quickstarts.doc if you get a chance. You might find the section on schemas and extended legacy applications quit interesting.

I've been accused of being a hypocrite because I (used to) advocate O/R mappers, but hate object-to-xml (O/X) mapping. You seem to have the reverse problem ;-) Or do you think that mapping relations to objects is bad, while mapping objects to XML and back is good?

> O/R mapping is one of the unfortunate consequences of trying to apply OOP principles to anything and everything

Rich business logic (or better a domain model based application) and the relational schema often don’t match up. What’s an interesting thought to focus on, is the reason for change and to not let changes to the relational schema ripple through the model. Sure axis of change is an axis of change only if the changes actually occur.

I don’t think O/R Mapping necessarily boosts productivity. I, however, do think O/R Mapping has its own place. One is to let the model schema and relational schema to evolve independently. Talking about opening pandora’s box :D

Kind regards,
Paul Gielens
http://weblogs.asp.net/pgielens/

I agree with you, but I think your 'tenets' about the database edge don't imply that you need to manually write SQL. There are some O/R mapping features like lazy loading that break your rules, but we can find ways to map explicitly that schema and the in memory representation in such a way that we don't need to write the SQL code to load it/save it.

About where to put the business logic... I think in the near future most of the business logic won't be in code, but in data (think WWF), so having a domain model won't help much.

I think Marc hit it on the head.

Let us propose that you are following DDD as specified by Evans. In this scenario an O/R mapper is nothing more than a generic repository; one could quite easily create the named repositories that pass through to the O/R mapper to help decouple the O/R mapper from higher levels. When the case comes that you are *not* dealing with a happy generic mechanism that the O/R mapper is good at ... then create the special repository instead of passing through to the O/R mapper.

There are some O/R mappers that allow you to do things such as you are discussing as well (Wilson O/R comes to mind) using Wilson O/R I can delegate out various accessors as well as CUD to sprocs. Naturally the O/R mapper cannot be as intelligent in many of these cases (nor can it offer the flexibility of dynamic SQL) but there are times when this trade off is preferrable.

It is just a tool ... use it when it helps ... The main factor in the decision making should not be whether to use an O/R mapper or to use self done code that does essentially the same thing. It should be looking at the code saved by using the O/R mapper and comparing that to the possible overhead of having two methodologies to deal with (and vice versa if you are solely using mapping and wish to put in other methods)

One thing that there is absolutely no mention of here is transaction handling ... using DDD combined with a good O/R mapper can save you huge amounts of code when dealing with transactions (especially if that O/R mapper is like nHibernate with AOP based unit of work registration) which removes the registration code from your classes.

A comment on a comment above .. I have *never* created a domain object with a .Save() routine and I recommend you to never do it as well. The concept of a save very rarely encompasses a single object; saves are groups of objects ... thus the save should be pushed to a unit of work or similar object who handles it; this also makes transaction handling quite easy.

Jimmy brings up a great point with LINQ ...

I general I like the DDD approach and I do not think it is a bad idea to store data in objects – it wont work in all scenarios (RTRJ) - but it's the same with most of the things around - if you do not know what happens underneath the surface it is easy to do mistakes.

Hi,

You say that ORM havent taken over the world yet, I beg to differ, ORM in the java world is extremely common.
and ORM will be built into the .net framework when dlinq arrives.

also you say "The sirens are singing: O/R mapping"
isnt that a great sign that ORM is getting more common in .net to?

also, you talk about performance hits using ORM.
per hits compared to _what_?

Id say that in many cases ORM can optimize queries that normally would be unoptimized.
eg say that you have a stored proc that lists all orders in a system , with every field that might be relevant every time.

with any competent ORM you can specify exactly what fields to load in a specific usecase.
in the case of SPROCS you would have to make multiple versions of the same SPROC or pass some sort filter into it, which would make your development process much more tedious..

some ORMappers can also handle caching in various scenarios and thus improving performance.

It would be very interesting to see some common scenarios where you think ORM fails or performs badly compared to whatever approach you like.

//Roger

I must say I wholly disagree.

I'd say the only reason OR mapping hasn't taken over the world yet is the sheer volume of SQL-only trained VB programmers who still think objects are just another data type, and think inheritance is something to do with genetics.

As well, many of these programmers tend to have a view of "objects" which hold more similarities to code libraries than proper-OO objects.

Like you said, we're dealing with data. And the best way to deal with data in an object oriented language is in an object oriented way. Like trying to hammer nails with a screwdriver, using a relational approach isn't as effective. Sure, StoredProcs worked fine when systems were using C or old school ASP. When you can reflect on an object and determine oodles about it, why go the trouble of creating stored procedures?

As for schema changes, from my experience schema changes are almost always driven from the code down, not the other way around. Business logic determines what we need to store, so the coupling has never been an issue. In fact, it's made things much easier for me with ORM, as the compiler will alert me immediately to dependencies elsewhere in the code when I change a persisted object.

-greg

I think this was a great position piece, but then I agree not only with the problem definition but also the position. Most of the ORM adherents seem to take the theoretical/architectural tack and most of the detractors seem to take the practicality/performance tack.

I've hiked up the learning curve for Hibernate (the Java flavor) only to find that what time was saved in mapping basic CRUD functionality got eaten up by out-of-band custom data access (which always seems to be required) and tuning to get performance close to what it would have been with a more specialized, hand-coded DAL.

I also agree with the posters who have pointed out that it's all mapping at some level. And while ORM may be more architecturally and aesthetically pleasing, it definitely hasn't proven itself to be more economical by any means.

-S

Hi Clemens. Nice article. You've put some good thought into pointing out how ORM can be fraught with pitfalls when ORM is done by folks with little experienced in appropriately and effectively using ORM frameworks in specific and Object-Oriented and Domain-Driven approaches in general.

> In short, I think the dream of transparent O/R
> mapping is the same dream that fueled the development
> of fully transparent distributed objects in the early
> days of DSOM, CORBA and (D)COM when we all thought
> that'd just work and were neglecting the related
> issues of coupling, security, bandwidth, etc.
>
> Meanwhile, we’ve learned the hard way that even
> though the idea was fantastic, it was rather naïve to
> apply local development principles to distributed
> systems. The same goes for database programming.

And yet I’m seeing unsettling articles on WCF where authorities – some from within Microsoft – are demonstrating how to use WCF to put local domain objects on the wire and transport them to external concerns and remote execution.

WCF and ORM are chasing the some of the same goals in terms of message-based state transfer as well as schema-based, declarative mapping of local data held in objects to remotable command objects.

At an essential level, ORM transfers and transforms object state to command objects that will be transported to and executed by a SQL database server in SQL 92 formatted messages over whatever protocol the database client provides. In some cases, the metadata and mapping facilities provided by an ORM framework allow this to happen with a good degree of transparency, and in other cases, human thought and human intention has to intercede, to write code, and account for the scenarios that aren’t covered by the basic features of a framework that allow for some transparency. I expect the same will be true for WCF as well.

I don’t see this as quite the same experience as using live object proxies over DCOM to do RMI over a network connection. The implications seem quite different. There are certainly similarities in that there is a network connection in play, and that the response is synchronous, but I don’t see the ORM pattern having as more affinity to RMI than to REST – at least not where simple CRUD is concerned. Analogies of ORM either to RMI or to REST seem to be only of limited applicability, and ultimately good arguments can seem to be made for either analogy.

I don’t see the analogy between DCOM and ORM is effective enough to support your position. Again, I see ORM as a message mapping approach more like WCF than DCOM. Of course data access commands live and execute in a finer scope than the kinds of messages that we often talking about with SOA, but we’re still talking about transforming local data held in objects to command objects that are amenable to wire transportation and remote execution.

ORM is misused and misrepresented in much the same way that WCF will be misused and misrepresented in its role as a mapping, transformation, and command execution framework. It’s a technology that has a following of competent advocates who have made good and appropriate uses of it, as well as a horde of whimpering dogs who got their tails caught on fire for not dealing responsibly with it. It’s the same old story for many technologies – especially where state transformation and transportation are concerned, be it via COM, COBRA, or Continental Air Lines.

I’m greatly looking forward to the evolution of WCF’s object-to-message mapping capabilities so that support for mapping application objects to data access messages will solve many of the challenges that Windows developers currently face with the sometimes arcane tools that we use today to achieve declarative, schema-based mapping of plain-old objects to remotable commands. The WCF has a tremendous opportunity to apply some serious tooling and experience to a persistent and pesky problem than really needs to be taken by the horns and solved once and for all.

To wit, it’s extremely disappointing to see the poor job of handling data access command mapping that the DLinq folks have done. My hopes for a better paradigm as provided by Microsoft will now rest squarely on your shoulders.

Cheers,
Scott

"* Boundaries are explicit"
O/R mapping doesn't "abstract away" data access. It is still explicit, but facilitated by the framework. This respects the Single Responsibility Principle, isolating changes to the data model from the domain model.

"* Services avoid coupling (autonomy)"
An O/R layer exists in order to decouple the domain model from the storage model. I guess I didn't really understand what you meant my this.

"* Share schema not code"
When using O/R mapping, the domain model is decoupled from the data schema and doesn't need to call code on the storage layer (which would occur if using Stored Procedures, for instance). So the layers don't share schema nor code.

I use ORM in my applications and I used to be the biggest evangelist. I still recommend it to people but am now reconsidering if I want an object relational mapper or if what I really want is an object relational "wrapper" such as iBATIS. I haven't used iBATIS but you specify the mapping AND the query in the mapping file. I always hated writing queries with the OO syntax of most ORM libraries. I personally would like write all my queries in SQL so I can optimize it. We'll see what the future holds.

As Alex mentioned in a previous comment, I also disagree that O/RM introduces tight coupling to the data model and maintain that it achieves exactly the opposite. A separate data mapping schema and a query language that supports encapsulation (completely lacking in SQL) are far more loosely coupled than thousands of manually coded SQL statements, especially as you will enmesh assumptions about best retrieval strategies and performance into the business intent of your SQL when you start optimizing your SQL statements.

>Many of the proponents of O/R mapping that I run into (and that is a generalization and I am not trying to offend anyone – just an observation) are folks who don't know SQL and RDBMS technology in any reasonable depth and/or often have no interest in doing so.

Many opponents of O/RM have not seriously tried it out or spent the time to explore the possibilities of O/RM tools. This is why generalized arguments about performance abound.

Also, as Jimmy has already written, O/R mapping is no silver bullet for all jobs: e.g. if you need ETL or reporting, you usually do not want to use O/RM. For OLTP applications where BL is involved on the middle tier however, O/RM decreases LoC and increases decomposition possibilities and maintainability.

> O/R mapping would likely have taken over the world by now

Leaving .NET aside for a moment, O/RM is a commonly used technology in the Java world. That Microsoft developers think they will do better by coding directly in SQL is a legacy of VB and Access.

What I have to agree with is that learning to use O/RM is quite hard, especially for developers who are unsure of how to benefit from OO in general. Another reason why O/RM may be difficult at first is that people not only need to learn a new query language, but that there is no standard language. People stick to SQL because they know it to be a versatile and reusable asset for them as developers. This is one of the reasons why I think LINQ is such an important technology; learning O/RM will not be synonymous with learning a new query language anymore.

Hi Clemens,

I’m totally agreeing with youon this, especially when I consider your perspective as indicated by the title of your blog page. The title clearly indicates that you have an enterprise perspective. Most of your critics seem not to have that perspective but an application perspective.

One of the important differences between these perspectives is that the enterprise perspective is business-oriented while the application perspective is technically oriented. The first one looks in at the business system from an external position; the other is looking out at the business from an internal position.

From an application perspective, domain-driven design (DDD) makes sense, even if not the only approach that does. DDD undoubtedly results in a similarity between the application and the database structures, just as you say, and when this is the case O/R mapping must seem compelling, especially if you’re not proficient in SQL and RDBMS issues.

With an enterprise perspective such as yours, one would be much more interested in the study of business processes, business information, and business rules. One would also want to use the structure of the business as a pattern model for the structure of the organization’s entire IT system portfolio. Its business processes would be modeled as autonomous business process and activity services; its information areas would be modeled as autonomous entity services. All these services would be loosely coupled to each other through message-based service interfaces which typically would be implemented as web services. Such architecture would be vastly different from one created through domain-driven design.

Specification of the service interfaces, including the full contracts (WSDL files and more), would even be available before any implementation of services or service consumption commences. The job of implementers and implementation architects would be to implement these business-driven service interfaces. In such a state-less, loosely coupled, and business-driven environment, domain-driven design of the interior of a service is not nearly as relevant as it is when architecting an application silo.

One would do much better with message classes. Those are totally based on the messages and thus on business usage views rather than on domain objects. Working like that, one would be likely to find there’s not much room for O/R mapping.

This is contract-first development, driven by external business needs. Domain-driven design tends to be implementation-first development, driven by internal object structures. It seems to me that the borderline between your critics and those who agree with you could be drawn between those favoring contract-first and those favoring implementation-first development.

Given your perspective, as I interpret it, I strongly agree with you. And I also agree with your perspective.

I’m sorry that this comment was so long. You’ll find an even longer comment as a blog entry at http://www.2xsundblad.com/blogs/stenandper/archive/2006/03/16/8.aspx.

... I'm still waiting for that OR mapper that actually does some mapping. I dont consider copying a flat table to a round object mapping. In my view a perfect mapper would allow me to define an application data model. A logical data structure for my specific domain. I model data in such a way that it makes the most sense inside my domain or application. Then I tell the mapper from what data stores (yes plural!) it should take this data and "MAP" it to my convinient little domain data model.

Of course if you start to think what you need to make this happen you will realize that it is a mix of distribute data aggregation, record-matching engines and having a 'command interpreter' (I think sql is up for a overhaul too ;-) that lives outside a database.

Oh, did I forget to mention that my perfect mapper any datasource goes, whether its from a 'real' database or a (web) service or the file system or the registry or...

You can't keep a man from dreaming...

Great piece, clemens.

ok