Tuesday, March 20, 2007

A Response to Those Who Hate My "How To Build a Site With No Database" Post

After reviewing many of the comments left on my post about building a site with no database backend, I'm convinced that a lot of people are missing my point.

I just want to be clear about something before I continue on with this post. I'm NOT advocating that you build sites without database backends. It's not my intention to say, "hey look, you don't NEED a stupid database! see...I did it with flat files!" I'm well aware of the inherent limitations you get when you use flat files over a database. You'll get no argument from me about that.

Let me be even clearer: There's a very good reason why using a database to store data is the conventional way of doing business. It's BECAUSE it's a good reason!

So, how can I possibly defend my post? Well, let me try.


My Point


I think the point that's been missed here is that when you look at the requirements of the site, when you look at what it's actually required to do, I think there's a very strong argument for not using a database. If you change the requirements, you change the implementation.

fwgx left a comment which is a good summation of other readers' comments. He says:

"...It's not hard to imagine what other requirements may be required later and as soon as that happens you're stuck. Can you easily get the maximum score per user between two dates? Can you find the average score for users who's name starts with a 'W' and who have played 5 games between midnight and 3 am?All reasonable queries that are easily answered using SQL but that would require tight coulping and nastiness in this solution.Not to mention the lack of type safty, referential integrity checks, FK constraints, transactions, load balancing, query caching, index tuning over time, backup systems, running on redundant servers, triggers etc..."

I'm going to argue here that he's completely missing the point. And believe me, my point is simple.

Don't invent requirements. Don't pre-optimize. Don't build stuff you don't need to. Just. Keep. It. Simple.

The requirements of the site I built don't require me to "find the average score for users who's names start with "W" and who have played 5 games between midnight and 3 am"? Why not? Because the site isn't going to show that information! And for another very good reason.

The site doesn't have users. This means that a user doesn't log in. A user doesn't have an account. If the site required that, then yes, I would have built it with a database. Period.

I appreciate the comment, and I appreciate the example here - but this is exactly what happens when we (developers) stop looking at what we're trying to solve and start inventing possibilities. We make things more complicated. We start talking about triggers, redundant servers, FK constraints.

As far as being "stuck" when/if requirements change later on. Far from it. Changing the code that reads the text file to code that reads a database table will be trivial. In fact, if anyone wants to challenge me - I'll do it and blog about how long it took to do. And I will be completely honest about the experience.

The bottom line here is that I'm not advocating text files as a replacement for databases. Far from it. I'm advocating that you should build software that solves the problem. Not make the problem more complicated.

Let's keep it simple.

49 comments:

Anonymous said...

Unfortunately, you will always get people who hate it and those that will love it.

I see what you have done, I like it.

Anonymous said...

If you're fairly certain your solution won't evolve to need a database then I don't see a problem with the flat file approach, especially if it saves you time or money. When the day comes that you need to upgrade to a database, you can change the data access layer and feel smug that your elegant loosely-coupled design was so easy to update.

John said...

Ciz, thanks for your comment. You've summed up everything I've been trying to communicate!

Anonymous said...

Well, given that the barrier to entry is, essentially zero for finding a web host with at least MySQL support, I think the idea of a database-less backend to be fine as an exercise. However, I think it's a really hard case to claim that building a database in from the start would be a case of premature optimization. Heck, your scoreboard alone could make an argument for having a database so that you don't need to worry about file locking during heavy usage.

If you were dead set against using a true database, then at least using a storage system such as Bamboo Prevalence or db4o would have at least been a more interesting demonstration and handled any persistence issues for you.

I'm all for being lazy, but I see using someone else's solution as the epitome of that, not writing your own.

John said...

Ben,

You're right. It was a mistake to say that using a database was "premature optimization". I'm man enough to admit that :)

And hey, I'm all for being lazy too - but why the heck would I bother using someone else's solution when I could spend some time writing up a blog post about doing it... I mean, this was fun for me, just look at all the hate mail I've been getting! :)

Anonymous said...

You missed the most important piece from my previous comment out:

Software should be SOFT and using this makes the software rigid, highly coupled, inflexible and too much work for it's own good.

As a developer of SOFTware I don't see how you can at all justify not making your design soft and easily expandable. Requirements WILL change. Always.

This isn't about building features that you don't need. It's not premature optimization. It's about making maintainable, flexible systems. The old 80:20 rule applies with software - you (well probably someone else) will spend 80% of your time maintaining the code and only 20% developing it. I think all programmers should follow the rule of "always code as if the person who will maintain your code is a maniac serial killer who knows where you live".

What makes this example especially bad is that you went out of your way to make things more difficult for yourself, gained absolutely nothing, made your solution less maintainable, attempted to reinvent the wheel, and probably did a worse job at it than you could have had off the shelf for free.

Sorry if this comes off as particularly harsh. But you seem to be in complete denial about the nature of requirements and the job and responsibilities of developers.

Andreas Krey said...

I can't let fwgx's comment standing as it is:

Software should be SOFT and using this makes the software rigid, highly coupled, inflexible and too much work for it's own good.

SQL servers are not the only way to produce decoupled components.

As a developer of SOFTware I don't see how you can at all justify not making your design soft and easily expandable. Requirements WILL change. Always.

And Murphy takes care that it always changes in the unanticipated direction.

This isn't about building features that you don't need. It's not premature optimization. It's about making maintainable, flexible systems.

You'd be astonished about the number of platforms that can store files
but do not support any kind of SQL server.

I think all programmers should follow the rule of "always code as if the person who will maintain your code is a maniac serial killer who knows where you live".

That pretty much amounts to 'do not code anything'.

What makes this example especially bad is that you went out of your way to make things more difficult for yourself, gained absolutely nothing, made your solution less maintainable, attempted to reinvent the wheel, and probably did a worse job at it than you could have had off the shelf for free.

To me it looks like the ideal opportunity to try out a nonstandard solution: Tiny problem that you can restart with little overhead if the need should arise.

For our stuff I did a little log-based data storage system (basically: write changes to log, occasionally dump whole state into new log file, process log at startup). Piece of cake. It got interesting when one of the programs should run in rendundant mode on multiple machines. I just took the storage layer and stuffed a distribution layer into it. Now one of the nodes act, and the others keep a replica of the log file, and it just works.

The sad part: We also spend big bucks to get a redundant SQL server for our 'real' database, and this big, experienced vendor's solution does not do it. When, after a power failure, the wrong machine does not come up, the server won't start even though everything that is needed is there.

Anonymous said...

Interesting! :-)

Anonymous said...

WooW =)

Anonymous said...

Interesting!

Anonymous said...

Interesting article, thanks!

Anonymous said...

Great to see this! Thanks!

Anonymous said...

Thanks for interesting article.

Anonymous said...

I like articles like this. Thanks!

Anonymous said...

very good!

Anonymous said...

Write something else. Thanks! Best Blog...

Anonymous said...

Well done. Keep up the great work. Best regards!

Anonymous said...

I like it a lot! Nice site, I will bookmark!

Anonymous said...

Thanks to author! I like articles like this, very interesting.

Anonymous said...

JKhRX3 You have a talant! Write more!

Anonymous said...

JWkfyM Thanks to author.

Anonymous said...

lgfoIa The best blog you have!

Anonymous said...

65BlXM Hello all!

Anonymous said...

Magnific!

Anonymous said...

Nice Article.

Anonymous said...

Nice Article.

Anonymous said...

Magnific!

Anonymous said...

Magnific!

Anonymous said...

Wonderful blog.

Anonymous said...

Magnific!

Anonymous said...

Hello all!

Anonymous said...

Wonderful blog.

Anonymous said...

xwNpTh write more, thanks.

Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

Anonymous said...

Good job!

Anonymous said...

Nice Article.

Anonymous said...

Please write anything else!

Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

Anonymous said...

Nice Article.

Anonymous said...

Hello all!

Anonymous said...

Magnific!

Anonymous said...

Wonderful blog.

Anonymous said...

Wonderful blog.

Anonymous said...

Thanks to author.

Anonymous said...

Wonderful blog.

Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

Anonymous said...

I'm not a complete idiot, some parts are missing!

Anonymous said...

Friends help you move. Real friends help you move bodies

Anonymous said...

I don't suffer from insanity. I enjoy every minute of it.