Tuesday, June 23, 2009

Goat Rodeo: a unified data model for Web applications


This is a new data infrastructure (pseudo-transactions, persistence, etc.) for the new Scala-based Lift framework for Web applications.

Lot of fresh ideas.

Wednesday, May 27, 2009

JPA 2.0 almost here

http://www.infoq.com/news/2009/05/jpa20

  • Typesafe API for queries, for those reluctant to use strings.
  • Metamodel API.
  • Bean Validation (JSR #303)

Thursday, May 21, 2009

What's new in Microsoft Data Services

http://oakleafblog.blogspot.com/2009/05/ten-slides-from-whats-new-in-microsoft.html

Granite Data Service 2.0

New version of this Data Service technology, with support for Google Apps and JDO:

Entity Framework 4.0

The new beat is available here.

Microsoft is still on the path to full persistence, with the introduction of POCOs and Design first approach (top down with schema generation) in this version (among several important new features and enhancements).

Sneak preview of new features here:
and here

JDO 2.3

New features of JDO 2.3, by Andy Jefferson of DataNucleus:

Introduction to Data Services

http://www.infoq.com/articles/narayanan-soa-data-services

I agree with the preliminary definitions and rationale at the beginning of the article, but I'm not so sure about the Development section. In particular, I don't think the "contract-first" approach is promoting reusability of data services. It could be nice if you just intend to consume data services only from a single applications, but it does not sustain an enterprise approach.

This sems to enforce the purely static vision of data services, the one traditionally coming from orchestration, BPM vendors, etc.

The future of MySQL?

I don't know how MySQL will evolve after Oracle acquired Sun.
The Open Database Alliance seems to be an open source community supporting independent branches of MySQL, led by some of the historical contributors.

For sure, many customers will hesitate about MySQL in the coming months before clarification from Oracle and others. Could this create space for another open source database player?

Monday, May 18, 2009

New version of EBeam

http://www.theserverside.com/news/thread.tss?thread_id=54593

New features:
  • In-memory filtering and ordering: that's something already avalaible in best ORM products;
  • Autofetch queries based on query execution statistics. That feature is really the nicest of EBeam. It was on Xcalia XIC roadmap but we never developped it. I would love to see it added in DataNucleus.

Friday, May 15, 2009

NeoDatis 1.9 released

I think I've already mentioned that new ODBMS before. This new version has interesting features such as Groovy support (I love Groovy), Scala support, Google Android and pluggable encryption.


Interesting to see that the upcoming DataNucleus 1.1.3 will support this new NeoDatis version.
Once again, it seems that the Cloud, SaaS and Data Services are boosting the market for new specialized databases.

Thursday, May 14, 2009

JDO's not dead?

Seen on Matthew Adams' blog:

I cannot agree more. Interesting to see how this things will evolve.
I've even seen one Oracle document mentioning their JDO implementation, coming from the acquisition of BEA (who acquired SolarMetric a while ago). They simply said you can choose between JPA and JDO. See Oracle's document titled Oracle WebLogic Server: a solid foundation for SOA, June 2008, page 9. When we see how they fought against JDO around 2003, that's a huge move.

In an ideal world, JPA should have just been the ORM part of JDO. Maybe the JDO expert group should have relaxed some of the technical constraints so that Hibernate and TopLink could have found it suitable (mostly byte-code enhancement, binary compatibility plus few other minor things). After discussions with Oracle and Hibernate the JDO expert group was willing to do it, but in the meantime both Oracle and Hibernate convinced Sun to start JPA as part of EJB3.

Monday, May 11, 2009

Data, Context and Interaction

Seen on InfoQ new article by James Coplien and Reenskaug:
http://www.infoq.com/news/2009/05/dci-coplien-reenskau

It is all about a new pattern for better representation of user interactions on data.
It is addressing aspects not covered by typical OO patterns like MVC.

Whether you agree with the ideas, comments, critics, possible implementations or not, all in all this is quite an interesting article.

The article itself is here:

Yet another ORM

Seen on TheServerSide, this new ORM.
Direct link to the vendor: http://www.leeonsoft.com/

Quite frankly, from what I've read on their web site, I'm much more than doubtful about it.
It claims it is much more efficient than JPA and Hibernate, but based on what I see it essentially means the author does not really know JPA.

Thursday, May 7, 2009

Why Data Modeling is the Wrong Tool for Integrating Data Models

Fresh ideas in this article about how to build common data models, even if I wouldn't agree with everything.


Why Data Modeling is the Wrong Tool for Integrating Data Models

Fresh ideas in this article about how to build common data models, even if I wouldn't agree with everything.


Tactical Data Integration with JBoss Federate

Video about Federate, the new open source version of the former MetaMatrix EII product:

http://architects.dzone.com/videos/screencast-tactical-data.

Federate Project. (most of links don't work)

Tactical Data Integration with JBoss Federate

Video about Federate, the new open source version of the former MetaMatrix EII product:

http://architects.dzone.com/videos/screencast-tactical-data.

Federate Project. (most of links don't work)

DryadLINQ

Quite interesting automatisation of distributed queries:
http://www.infoq.com/news/2009/05/DryadLINQ

Microsoft research project.

DryadLINQ

Quite interesting automatisation of distributed queries:
http://www.infoq.com/news/2009/05/DryadLINQ

Microsoft research project.

Tuesday, February 3, 2009

Data Services presentation at The Open Group Conference

I did a presentation about Data Services yesterday at the Open Group Conference in San Diego. The session went very well and we had a series of good questions / feedback:
  • How is security managed in a DSP?
  • What about transaction management?
  • Is that a good platform for mobile applications?
  • Where are metadata deployed?
  • Can we call this a Data Hub?
  • What about deployment of a DSP within an ESB?
  • What are the real infrastructure costs of WOA and how to compute/compare them?
  • What is the link between a DSP and MDM? (see my recent article about this subject)

Seems the benefits of a Data Services Platform are now widely accepted.

Data Services presentation at The Open Group Conference

I did a presentation about Data Services yesterday at the Open Group Conference in San Diego. The session went very well and we had a series of good questions / feedback:
  • How is security managed in a DSP?
  • What about transaction management?
  • Is that a good platform for mobile applications?
  • Where are metadata deployed?
  • Can we call this a Data Hub?
  • What about deployment of a DSP within an ESB?
  • What are the real infrastructure costs of WOA and how to compute/compare them?
  • What is the link between a DSP and MDM? (see my recent article about this subject)

Seems the benefits of a Data Services Platform are now widely accepted.

Pseudo transactions with non-XA resources

I've seen this article about Pseudo-transactions with non XA-resources on TheServerSide. Well, basically what they do is a manual compensation of the non-transactional resources. There are many ways to implement this, some of them could be quite operational, but this is not my point today.

It is quite possible in the future that we'll have to think differently about transactions. First of all, because it will be always difficult to manage non-transactional resources. Second, the real problems will raise when we'll start to deal with asynchronous data access patterns, e.g. when you call a data services and you don't wait for the answer, you'll get it later (maybe).

The multi-agents systems and the actor languages give interesting ideas about this. An actor language is like an object language in which every method call is asynchronous by default, any required synchronisation having to be explicitely defined and managed (continuations, etc.).

I've recently read interesting articles about this. Some people are saying we can just drop the whole notion of transactions. It could be shocking at the beginning but they have good arguments. Other ones think we must change the problem until we can have atomic transactions: see for instance "Life beyond distributed transactions" by Pat Helland (from Amazon) or the Saga transaction system from Arnon Rotem.

Pseudo transactions with non-XA resources

I've seen this article about Pseudo-transactions with non XA-resources on TheServerSide. Well, basically what they do is a manual compensation of the non-transactional resources. There are many ways to implement this, some of them could be quite operational, but this is not my point today.

It is quite possible in the future that we'll have to think differently about transactions. First of all, because it will be always difficult to manage non-transactional resources. Second, the real problems will raise when we'll start to deal with asynchronous data access patterns, e.g. when you call a data services and you don't wait for the answer, you'll get it later (maybe).

The multi-agents systems and the actor languages give interesting ideas about this. An actor language is like an object language in which every method call is asynchronous by default, any required synchronisation having to be explicitely defined and managed (continuations, etc.).

I've recently read interesting articles about this. Some people are saying we can just drop the whole notion of transactions. It could be shocking at the beginning but they have good arguments. Other ones think we must change the problem until we can have atomic transactions: see for instance "Life beyond distributed transactions" by Pat Helland (from Amazon) or the Saga transaction system from Arnon Rotem.

Monday, January 19, 2009

The Role of a Data Services Layer in SOA

UK-trade journal, Database & Network Journal, has published one of my articles in their January issue: “Bringing Data into Service: The Role of a Data Services Layer in SOA”. I'm not aware of any Web version of it, I'm looking for a solution to make it available online. Stay tuned.

You can read the article here.

The Role of a Data Services Layer in SOA

UK-trade journal, Database & Network Journal, has published one of my articles in their January issue: “Bringing Data into Service: The Role of a Data Services Layer in SOA”. I'm not aware of any Web version of it, I'm looking for a solution to make it available online. Stay tuned.

You can read the article here.

How MDM and Data Services can be Complementary

I'm pleased to inform you that DM Review has published my article about Data Services Platforms and Master Data Management.

In the past, I used to think that MDM was mostly an issue for EII products. Today, most EII have turned themselves into Data Services, but there are other approaches to Data Services Platforms. Based on my discussions my some MDM vendors I'm now convinced that DSP platforms have an important role to play in that space. You'll see all the details in the article.

How MDM and Data Services can be Complementary

I'm pleased to inform you that DM Review has published my article about Data Services Platforms and Master Data Management.

In the past, I used to think that MDM was mostly an issue for EII products. Today, most EII have turned themselves into Data Services, but there are other approaches to Data Services Platforms. Based on my discussions my some MDM vendors I'm now convinced that DSP platforms have an important role to play in that space. You'll see all the details in the article.

Thursday, January 8, 2009

SOA is dead?

First of all, I wish you all the best for 2009!

This new year starts with an obituary news, from Ann Thomas Mann (Burton Group): SOA is dead. This blog entry has generated a lot of reactions, and that was probably one of its aims:

(just to mention a few of them).

In fact, what she says is that service-orientation is still needed but the SOA TLA is now too heavily associated with the idea of project failure.

First comments:

  • Seems to me all this is more provocation than anything else, done to generate some noise and visibility. Regarding this, it is a success.
  • She is just writing what many were thinking and even saying since many years.
  • People tend to quickly burn the idols they have created themselves. So that they can quickly idolize another one... with the same unreasonable passion.
  • Sometimes it needs time, even for a good idea, to impose itself. Unfortunately, the World and the software industry are now stuck in short-term visions and immediate ROI, there is no more time for ideas to mature. New concepts now have to be perfect from inception, there is no room for improvement.
  • If we agree that SOA is not a technology but rather a way of thinking software architectures, then saying that "SOA is dead but there is still a need for the idea of services" is a kind of oxymore.
  • If SOA is really wrong, changing it's name won't fix it. WOA, RESTFul, Mashups won't save services if the main idea behind it is not good. The problem cannot just be with the SOA name.

So, what was really wrong with this supposedly dying SOA?

  1. SOA is much more complicated than expected. Remember: the 'S' in SOAP means "simple". SOA was supposed to be simpler than CORBA and J2E. But why SOA would have been less complicated than its predecessors? When you want to design non-trivial business applications by assembling distributed components you will face complexity. Using HTTP / SOAP instead of IIOP and WSDL instead of IDL will not fundamentally change the thing. The problem is still complex and unfortunately (or not) we don't have easy solutions to complex problems yet. Thinking large systems as communicating distributed agents is not so easy. It is all about parallel asynchronous programming. Most developers are not trained for that, as they were not trained to be Java gurus. It is so funny to see that today SOA is attacked on the simplicity front.
  2. SOA is too much process-centric. And once again, it was the same with CORBA and J2E. All distributed arhcitectures are first trying to solve the easy problem first: distribute processes. Then they start to deal with data and the real problems raise.

But it is not necessarily too late for SOA, as far as we can reconcile data and processes, states and behaviors, attributes and methods, however you name it.

So let's go back to the fundamentals. David Linthicum is thinking in the right direction: How to deal with data in SOA?

That's where we are today in SOA. Data Services are clearly going in the right direction. Customers will be able to solve some of their data integration issues with SOA, and they will have time to understand the value of this approach. To some extent, Data Services are at the cross between the promises of SOA and OO: states encapsulated by business methods.

Obviously, we'll need to go further, move from simple data federation to adaptative mediation, but we'll have time as we are solving real problems in the meantime.

Data Services should not be too much focused on names like SOA, WOA, RESTFul, mashups. New architectures are not replacing old ones (even if their proponents always tend to think it, at the beginning). They just complement them. Data Services platforms should equally support all these styles. Data Services Platforms should publish integrated data as SOA-or-not services, but also as POJOs, xDBC... The same way they have to connect to SOA-or-not data services as data sources, but also to RDBMS, packaged applications, mainframe transactions and screens, etc. The real IT world will certainly never be all-SOA, but equally it won't be all-non-SOA.

Services are basically components. There are many deep reasons why imposing a component-oriented design has continuously failed for decades. SOA is a new try and it won't be the last one. Component-based progamming models have failed for cultural reasons (programmers are not ready for that) and also because of the Lego model. You can't build a complex system by manually linking thousands of atomic components. SOA is way too much manually hard-coded today. We need to be able to dynamically generate, publish and compose services at runtime. We need to design new data access patterns, maybe based on asynchronous, reactive models. This will require advanced metadata, more semantic metadata and dynamic computation of chains of service calls at runtime. All this won't be achieved in few years.

In the meantime, programmers will progressively have to accept they won't control the flow of service calls, exactly as DBAs had to accept they won't tune all SQL statements any longer with the raise of persistence technologies. Each time you add abstraction layers you are able to do more, but conversely you lose some visibility and control on the lower layers.

So, the real debate is not what's next after SOA?, or which style of service-orientation will win? REST is not simpler than SOA if you want to manage complex things by manually connecting services. If one day we want to solve complex problems with simple solutions, then we'll need new abstraction layers, we need dynamic SOA. And for that, we need to stop manually calling and connecting services. Regarding this SOA versus WOA or SOAP versus REST is not the point at all. Services are a required piece in the puzzle, but we need new abstractions on top of them.

See the funny REST is dead blog entry here.

SOA is dead?

First of all, I wish you all the best for 2009!

This new year starts with an obituary news, from Ann Thomas Mann (Burton Group): SOA is dead. This blog entry has generated a lot of reactions, and that was probably one of its aims:

(just to mention a few of them).

In fact, what she says is that service-orientation is still needed but the SOA TLA is now too heavily associated with the idea of project failure.

First comments:

  • Seems to me all this is more provocation than anything else, done to generate some noise and visibility. Regarding this, it is a success.
  • She is just writing what many were thinking and even saying since many years.
  • People tend to quickly burn the idols they have created themselves. So that they can quickly idolize another one... with the same unreasonable passion.
  • Sometimes it needs time, even for a good idea, to impose itself. Unfortunately, the World and the software industry are now stuck in short-term visions and immediate ROI, there is no more time for ideas to mature. New concepts now have to be perfect from inception, there is no room for improvement.
  • If we agree that SOA is not a technology but rather a way of thinking software architectures, then saying that "SOA is dead but there is still a need for the idea of services" is a kind of oxymore.
  • If SOA is really wrong, changing it's name won't fix it. WOA, RESTFul, Mashups won't save services if the main idea behind it is not good. The problem cannot just be with the SOA name.

So, what was really wrong with this supposedly dying SOA?

  1. SOA is much more complicated than expected. Remember: the 'S' in SOAP means "simple". SOA was supposed to be simpler than CORBA and J2E. But why SOA would have been less complicated than its predecessors? When you want to design non-trivial business applications by assembling distributed components you will face complexity. Using HTTP / SOAP instead of IIOP and WSDL instead of IDL will not fundamentally change the thing. The problem is still complex and unfortunately (or not) we don't have easy solutions to complex problems yet. Thinking large systems as communicating distributed agents is not so easy. It is all about parallel asynchronous programming. Most developers are not trained for that, as they were not trained to be Java gurus. It is so funny to see that today SOA is attacked on the simplicity front.
  2. SOA is too much process-centric. And once again, it was the same with CORBA and J2E. All distributed arhcitectures are first trying to solve the easy problem first: distribute processes. Then they start to deal with data and the real problems raise.

But it is not necessarily too late for SOA, as far as we can reconcile data and processes, states and behaviors, attributes and methods, however you name it.

So let's go back to the fundamentals. David Linthicum is thinking in the right direction: How to deal with data in SOA?

That's where we are today in SOA. Data Services are clearly going in the right direction. Customers will be able to solve some of their data integration issues with SOA, and they will have time to understand the value of this approach. To some extent, Data Services are at the cross between the promises of SOA and OO: states encapsulated by business methods.

Obviously, we'll need to go further, move from simple data federation to adaptative mediation, but we'll have time as we are solving real problems in the meantime.

Data Services should not be too much focused on names like SOA, WOA, RESTFul, mashups. New architectures are not replacing old ones (even if their proponents always tend to think it, at the beginning). They just complement them. Data Services platforms should equally support all these styles. Data Services Platforms should publish integrated data as SOA-or-not services, but also as POJOs, xDBC... The same way they have to connect to SOA-or-not data services as data sources, but also to RDBMS, packaged applications, mainframe transactions and screens, etc. The real IT world will certainly never be all-SOA, but equally it won't be all-non-SOA.

Services are basically components. There are many deep reasons why imposing a component-oriented design has continuously failed for decades. SOA is a new try and it won't be the last one. Component-based progamming models have failed for cultural reasons (programmers are not ready for that) and also because of the Lego model. You can't build a complex system by manually linking thousands of atomic components. SOA is way too much manually hard-coded today. We need to be able to dynamically generate, publish and compose services at runtime. We need to design new data access patterns, maybe based on asynchronous, reactive models. This will require advanced metadata, more semantic metadata and dynamic computation of chains of service calls at runtime. All this won't be achieved in few years.

In the meantime, programmers will progressively have to accept they won't control the flow of service calls, exactly as DBAs had to accept they won't tune all SQL statements any longer with the raise of persistence technologies. Each time you add abstraction layers you are able to do more, but conversely you lose some visibility and control on the lower layers.

So, the real debate is not what's next after SOA?, or which style of service-orientation will win? REST is not simpler than SOA if you want to manage complex things by manually connecting services. If one day we want to solve complex problems with simple solutions, then we'll need new abstraction layers, we need dynamic SOA. And for that, we need to stop manually calling and connecting services. Regarding this SOA versus WOA or SOAP versus REST is not the point at all. Services are a required piece in the puzzle, but we need new abstractions on top of them.

See the funny REST is dead blog entry here.