Sure, it is possible to use something like MySQL and use that for high-traffic sites. FriendFeed uses it this way, but not with traditional table schemas. You can read this article to learn more about the specifics (if you haven’t done already) -> http://bret.appspot.com/entry/how-friendfeed-uses-mysql

The approach I mentioned about starting with the domain is called Domain-Driven Design (DDD for short). If you want to learn more about this approach, I can highly recommend the book DDD – Tackling Complexity in the Heart of Software (http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215/ref=sr_1_1?ie=UTF8&s=books&qid=1266439284&sr=8-1)

The root of the problem I have is getting devs to differentiate the logical from the physical in the mixed-up world of SQL, RDBMSes, NoSQL, CAP, BASE, and other such complex beasts. There’s so much conflating going on. And, the movement’s extremely poor choice of a name only compounds the issue.

First point: You will not get an argument from me that NoSQL (ugh!) databases currently address the physical, “CAP Theorem” problem better than ol’skool RDBMSes. Many RDBMSes (esp. startup-friendly, popular DBs like MySQL) weren’t built to address distributed petabytes of simple structure data with limited consistency needs but overwhelming partitioning ones.

That doesn’t mean that:
a) all systems have those kinds of FBish needs. In fact, FB and its ilk lie at the far end of the bell curve, and no, the intertubes won’t change that.

b) that RDBMSes can’t be made adaptable to these scenarios. (ex: Drizzle)

c) that we need to push down those CAP needs on to simpler web applications (which I am not sure if that is what you mean by “the largest apps/websites in the world all run from a NoSQL DB so why wouldn’t we learn, pick this up and move on”).

Second point: SQL, and the Relational Model that fathers it, is a different topic. I am not sure why one would think that the Relational Model is not layerable[sic] over distributed key-value stores, but I am open to being beat on the head with a good reason! I am far from a guru in the world of developing NoSQL implementations.

PS: Your comment on your practices of the domain as a starting point is interesting. I naturally gravitate to the exact opposite. I start from both the absolute “bottom” and absolute “top” of a typical web app, namely I use ORM (Object-Role Modeling) to capture the conceptual structure of the data (and many of its constraints), and use cases to capture the behaviors in the system, and from there I use the domain model as the Play-Doh layer to merge those two worlds.

I agree that you can use an RDBMS and use a key/value table and it has been done, but that somewhat mitigates the point of buying Oracle and SQL isn’t going to be much of help either.

When I develop an app, I start with the domain model because that’s where the interesting functionality of the business domain lives. This is the place where I provide a model that corresponds with the feedback of the domain experts. I don’t care about a data model or database because it doesn’t interest the business (nor it shouldn’t), so it doesn’t interest me either. When I’m done with a fully fletched model, the shape of the domain is almost always different from the tables layed out by the DB folks. It just need to be able to store these things and move on. NHibernate reduces some of the pain, but the mismatch is still there and a NoSQL data store reduces it even further.

I want to conclude by saying that the largest apps/websites in the world all run from a NoSQL DB so why wouldn’t we learn, pick this up and move on?

PS: I’m sure you meant no disrespect. Discussions like these can get passionate and intense. Been there

Insulting? My apologies. It was not intended to be personally insulting.

But, if you want to take the aggressive approach that “relational database was just a means for optimizing file storage”, then you best be prepared for like responses *on that topic*, because I will stand by the assertion that that is completely and utterly wrong.

RM, and its derivative language SQL, is very powerful. Good for all scenarios? Absolutely not, but still one of the most powerful tools in any dev’s arsenal. I guess if that is perceived as insulting to Jan to find some*thing* they said or believe wrong, then I am at a loss.

I could just as easily say “NoSQL databases answer the difficulty developers have with designing a suitable structure to data by letting anarchy reign and allowing them to store anything in any unstructured way”, but that wouldn’t be true, would it?

I meant no disrespect to Jan as a person. I’m sure that if we met for a beer, we’d all have a good time, although you probably now would refuse. I shouldn’t have let my frustrations with the rampant NoSQL hype color my post.

@Nick:

Follow up to your analogy:

RDB – After you assemble your car, which the dealer could have assembled for you before shipping, you then order a missing piston. You get the missing piston.

OODB – You want to store just a piston in the warehouse, you can’t. You have to create a fake car around it. Once you store the piston, you order a piston, and you get a car, and instructions on how to trace down the part chart to get to the piston.

I’m sure we can go on and on…

PT’s rudeness was unnecessary but then again, no excuse to dismiss the technical message. I’m not disagreeing with that.

RDB – You order a car, and its posted piece by piece (field) through your letter box where it has to be reassembled.

Oh, we won’t remove your comment, because it should stand in posterity to your rudeness. You should also note at this point that whatever technical message you were trying to make is now lost in this little drama.

Good job, Mr. Professional.

The root of the problem I have is getting devs to differentiate the logical from the physical in the mixed-up world of SQL, RDBMSes, NoSQL, CAP, BASE, and other such complex beasts. There’s so much conflating going on. And, the movement’s extremely poor choice of a name only compounds the issue.

First point: You will not get an argument from me that NoSQL (ugh!) databases currently address the physical, “CAP Theorem” problem better than ol’skool RDBMSes. Many RDBMSes (esp. startup-friendly, popular DBs like MySQL) weren’t built to address distributed petabytes of simple structure data with limited consistency needs but overwhelming partitioning ones.

That doesn’t mean that:
a) all systems have those kinds of FBish needs. In fact, FB and its ilk lie at the far end of the bell curve, and no, the intertubes won’t change that.

b) that RDBMSes can’t be made adaptable to these scenarios. (ex: Drizzle)

c) that we need to push down those CAP needs on to simpler web applications (which I am not sure if that is what you mean by “the largest apps/websites in the world all run from a NoSQL DB so why wouldn’t we learn, pick this up and move on”).

Second point: SQL, and the Relational Model that fathers it, is a different topic. I am not sure why one would think that the Relational Model is not layerable[sic] over distributed key-value stores, but I am open to being beat on the head with a good reason! I am far from a guru in the world of developing NoSQL implementations.

PS: Your comment on your practices of the domain as a starting point is interesting. I naturally gravitate to the exact opposite. I start from both the absolute “bottom” and absolute “top” of a typical web app, namely I use ORM (Object-Role Modeling) to capture the conceptual structure of the data (and many of its constraints), and use cases to capture the behaviors in the system, and from there I use the domain model as the Play-Doh layer to merge those two worlds.