In the time between finishing at Twilio and starting at Exec, I had four days off. A few days prior, Posterous had been acquired by Twitter. This gave me an idea, and it was one I knew I could build in those four days. This post will explore how I built and launched ExportMyPosts.com in those four days, and what building-blocks I used to get there.

The Idea

On March 12, Posterous was acquired. This was great news for Posterous and Twitter, but many users were left wondering about what would happen to their blogs and their data. Posterous' response at the time was very vague (their response since then has cleared a little bit).

Their main response at the time was, "If you want to export your data, you can use our API for that." This is true, of course. It is possible to get one's data out that way, but most users wouldn't even know where to begin with that recommendation.

However, I was familiar with Posterous' API from previous projects, so I knew what would need to be done in order to extract all the information and files you would want out of your blog. I thought it would be neat to build a tool that could be used by anyone to export their blog in the simplest way possible: the click of a button.

The "New Thing"

I have a rule for every personal side-project that I build. There must be at least one "new thing" I have to learn that I have never done before in other projects. I have had this rule for years. After dozens of personal projects under my belt, I now have an entire arsenal of skills to use on future projects. I recommend this rule to everyone. This is how you can become The Starter, The Architect, The Debugger and The Finisher all in one person and accomplish things very quickly (though I am more of a Starter than the other three).

There were a few "new things" in this project, but main one was using Stripe to take payments on the website.

I have used PayPal before and taken credit card payments with a Merchant and Gateway account for a previous business, but I had yet to play with Stripe to really see how easy it was. That was the "new thing" this time.

Building Blocks

Here is a list of the pieces I put together to assemble the site:

Servers

Rackspace Cloud virtual servers running Linux. I use Apache2 as my web server, PHP as my language, and Redis as my datastore.

Dev setup: One 512mb slice which runs a web server, the export worker and email daemons, and the redis db.

Production setup: One 512mb slice to run the web server. One 512mb slice to run the redis db. One or more (depending on load) 256mb slices to run the export worker and email daemons.

PHP on CodeIgniter 2.x

Haters gonna hate, but I love PHP. I learned it in 2002, and I know it backward and forward. I know how to configure it with Apache to get maximum performance without even thinking about it anymore. This means I can write PHP code very fast. Believe it or not, it is actually possible to write good PHP code.

A couple years ago I came across CodeIgniter, which is a pretty nice MVC framework for PHP web apps. The community is a little devided on some issues, but the code is nice and makes creating websites quick.

Redis

I love Redis. You love Redis. We all love Redis. Here's why:

Having a schema-less datastore meant that I could iteratively tweak and adjust my models without having to worry about updating schemas or tables or rows or anything. Do I need to store another attribute about this user? Great, just set another key and I'm done. Everything I need to know about users, blogs, payments, promo codes, and jobs can be expressed as key-value pairs, lists, and sets.

(Let's not start a war about NoSQL vs RDBMS, ok? I still love SQL, too. There is a time and place for everything. Use the right tool for the job, and all that.)

However, there is another awesome power that Redis had for me in this project: queues. Antirez published a post about Redis reliable queues in the middle of this project, so I took a few pointers and tweaked my queue implementation.

Because the lpush and rpop commands are atomic, it meant that I could use a Redis list as a distributed queue to dispatch export jobs to worker daemons. In addition, because of the atomic incr and decr commands, I could use a Redis key as a locking mechanism between worker processes to serialize API requests to Posterous so I would play nice with their rate-limits.

Email

At first I hadn't even thought about adding email to the site. It's not something that most projects need. Then I thought how strange it would be for users to click "Export" and have to wait around while their blog was queued for processing. How would they know it was done? So, it became clear I would need some notification mechanism: email.

I have configured and stood up more Postfix servers than I care to remember. That was once a "new thing" that I learned, but after dealing with it again and again, I have finally taken down the "Get Off My Lawn" sign and decided to go with a shiny web 2.0 Email API service from here on out.

My personal favorite service is Mailgun. There are many similar services out there, but Mailgun happens to work really well for my use-cases, and likewise for this project. I had to write a simple PHP wrapper for their API, but it still took about 10 minutes to start delivering email to my inbox from the app.

Stripe for Payments

I have dealt with web-based payments before. PayPal is nice, but they are becoming increasingly scary. I have had online Merchant and Gateway accounts. Those are a giant mess and rife with fees and shady practices. I consider that whole world to be the seedy underbelly of the internet.

After hearing much praise about Stripe (not to mention knowing the founders through many YC connections and events), I decided to give them a spin to see if they lived up to the hype.

The short answer is: yes. The long answer is: yes, but it still takes lots of effort to properly integrate payments into a website. Much of the logic and implementation is application specific, and no company can fully remove that pain. Stripe certainly eases the pain (a lot), but the greatest thing they provide is a one-stop service. They act as both the Merchant and Gateway accounts. In fact, you don't even have to know what those mean to use Stripe. All you have to know is, "people pay with their credit card on my site, money lands in my bank account." What normally takes three separate services is rolled up into one.

The second greatest thing is that there are no monthly fees. I only pay Stripe when I am paid. So, if I go a whole year without making a sale, I am out nothing. I don't know of any other payment systems with that deal.

Anyway, I don't want this post to turn into a Stripe ad, but it was the "new thing," so I paid special attention to it.

Worker Daemons

I have two types of worker daemons (again, written in PHP) that run in the background: an email daemon and an export daemon. The email daemon is just a simple loop that pulls email jobs off another Redis queue and sends them through Mailgun.

The export daemon does the heavy lifting for the whole project. It is responsible for communicating with the Posterous API to extract blog data and download associated media files (images, videos, audio, etc). Then it zips them all up into one file and uploads it to S3. Then a link is sent to the user to let them know their blog is ready for download.

I designed the export worker to be scalable, so that more can be run at the same time if there are many pending jobs. The only tricky part was to serialize the Posterous API requests betwen the workers because the API has a very strict requests per hour limit with second granulatiry. As I mentioned before, I just use a Redis key as a mutex to make sure that only one worker is making a request at any one time and sleeps a bit before giving up the lock.

Bootstrap and Bootswatch

Twitter's Bootstrap seems to be popping up everywhere now, but for good reason. It is a great frontend framework for non-designers (like me) to make a decent looking website without having to worry too much. I don't begrudge anyone using Bootstrap, but I don't like seeing the same default theme everywhere. Thankfully, there are sites like Bootswatch which provide some different themes and ability to customize your Bootstrap look and feel. I also added a textured background image from Subtle Patterns to give the site some extra depth.

The Four Days

Thursday

I started work on the site. First I wrote the worker daemon to see if it was even possible to export the blog data the way I wanted to. This was also the first day of the NCAA Basketball Tournament, and I also run my own Bracket Challenge. So I wrote code, setup the dev server, and watched basketball.

Mostly I watched basketball.

Friday

More basketball.

Started work on the website frontend. Played around with Bootstrap and Bootswatch until I found a theme and design I liked. Then I added enough code to be able to login via Posterous and export a blog with the website as a GUI instead of running the worker from the command line. At this point I integrated Mailgun emails so users would get an email when their blog was queued, started exporting, and finished exporting. I had a couple friends test exporting their blogs to make sure the system was working as expected.

Saturday

More basketball.

Payment integration day. If I didn't want to take payments for this site, it would have been mostly done by now. Adding payments to a website means inserting payment-related logic to lots of things in the app. Is the user allowed to do this action? Have they paid for it? Have they paid for one of these items, but not two? What items do I display to the user based on what they have or have not already purchased? Make sure a user can "purchase" the same item again, but this time don't charge them for it. Etc, etc... All of these issues are answered with code that isn't even related to integrating with your payment service!

Integrating with Stripe was fairly straightforward. Their test mode is amazing, so you can try out every conceivable user interaction flow and see how their system will respond before flipping to production.

Sunday

More basketball and a Dim Sum lunch with the Exec team.

More payment integration coding. Once I was satisfied that I had all my bases covered, I went about thoroughly testing the system. I nuked the database and started from scratch to make sure that nothing was dependent on data I had manually manipulated in the database. I added a Promo Code system so that I could give away some free exports at launch. I spun up my production servers and configured them depending on which role they played. I added Terms of Service and a Privacy Policy. Everything was go for launch.

The Launch

On Monday, I rested. Well, I rested from the project, but it was also my first day at Exec, so there wasn't much rest at all!

I have learned to wait an extra day between "finishing" a project and launching it. There are always one or two things you think of that need fixing, and you'll be thankful you waited. And so it was with this site. Monday night I came home and tweaked a few things, ready to go on Tuesday morning.

Tuesday (yesterday) morning I hit the Big Red Launch Button by sending out a few tweets, posting to Hacker News, and emailing a story to TechCrunch and Mashable.

Some Stats

The initial response has been ok but not overwhelming. The HN post only garnered a meer 34 points. I was expecting a bigger reaction there since the original Posterous acquisition thread got close to 600 points.

After the first day, only 27 people logged into the site out of ~1500 visits.

Four people actually bought export packages (all within the first hour of launch), for a grand total of $39.00 in revenue so far. One of those purchases came from someone who bought an upgrade after using one of the promo codes, and that made me happy.

I gave out a Promo Code on HN good for the first 25 people to use it. So far, only 14 have claimed it. Same story with a promo code I gave on Twitter several times. Only 3 of 10 claimed. It's somewhat frustrating to think that I can't even give this service away.

The Future?

I'm not sure if this project has legs, but it was very fun to build, and I have very positive feedback from the people that have used it. I think it would be a valuable service even if Posterous wasn't acquired. Being able to export your own data has value.

I could see it expanding to interface with other services that hold users' data (blogs, photos, etc).

As always, building is the easy part, marketing and finding users who actually want your product is the tricky bit.

Submit to Hacker News.

— Fin.

Like this post? Please share the love!