bithead redux

Going through college, I hated databases. I had this (stupid) belief in college that business was bad, and databases, to me, were about business. I thought I was a Scientist, and I was going to stick to the strict computing side of things. I wanted to work on Operating Systems.

I got over it.

I now find databases fascinating. I love being able to slice and dice large swaths of data with simple commands. The leverage of a database can be spectacular when applied to the right sorts of problems.

However, up until I turned my mind towards SQLite, relational databases always seemed to be too heavyweight. It was much easier to throw something into a text or xml file, and do small bits of slicing and dicing with python or ruby or [insert favorite scripting language here].

With SQLite, I can create a new database quickly, and import data easily. For example, I found myself looking at a list of albums that some magazine thinks are the best of all time. I was curious about a few things, so I did a “view source,” cut the relevant part into a TextMate document, used some regular expressions to clean up the data, and then created a SQLite db and sucked the data in. Now I could ask a bunch of SQL questions and get my answers.

Yeah, SQLite isn’t a complete implementation of SQL, it plays fast and loose with types, and won’t scale like MySQL or other major databases, but it is really good enough.

It doesn’t need a server running in the background. You can back up your data by just copying the file somewhere safe. The looseness in typing is an advantage, in that you don’t have to clean the data up completely to do useful things with it.

The bottom line is that I don’t really have to think before I just suck data into a database.

I absolutely adore SQLite.