A few weeks ago, I decided to create an email-powered blog that is hosted in an Amazon s3 bucket as a website. I wanted to be able to email new posts to the blog and have them published automatically. I use the email subject line as the title of the post and the email body as the text of the post (written in markdown).

This blog (the one you're reading right now) is powered by jekyll and hosted on s3, so I decided to reuse most of that setup and add some extra glue to handle posts by email. It does require an extra server to be running on the open internet (to handle incoming mail webhooks and publish new content to s3), but it can be a small, cheap 256MB slice since it does not have to handle any blog traffic (s3 takes care of that).

The result is http://mailblog.jazzychad.net. I am hoping to use it when I want to quickly publish small posts or passing thoughts.

Overview

Email is sent to a special address handled by Mailgun.

Mailgun posts the email data to a webhook on my mailblog server. PHP/Apache is running on the server to handle the incoming data.

The PHP script formats the email data into a new post markdown file, writes it into my jekyll _posts folder, and calls a do_blog.sh script.

The do_blog.sh script causes jekyll to compile the blog and then sync the files to my s3 bucket.

DIY

The whole process took about 3 hours to complete. Most of that time was spent struggling with RVM because ruby and multi-user rvm are nuts. I managed to distill the commands down to the magic incantation in this post.

All of the commands required to reproduce this setup are listed step-by-step below. I tested them in a brand new server instance, so it should work as advertised. If not, please let me know so we can debug.

Initial Setup

Disclaimer: These setup instructions are specific for setting up a server on the Rackspace Cloud. I used a 256MB RAM instance of Ubuntu 10.04 LTS.

On Rackspace, when you create a new instance, you are given a root account and password. This works differently on other cloud hosting services, so your initial setup steps may be different if you use a different provider. The goal is just to create a user account for yourself and give yourself sudo privileges.

Boot up a new server, login as root, change your password, update the system, then reboot.

# initial root login
   passwd
   apt-get update
   apt-get upgrade
   # reboot
   shutdown -r now

Log back in as root, add new user ('chad' in this example), and add to sudoers file.

   adduser chad
   apt-get install emacs22 -y
   chmod u+w /etc/sudoers
   # edit file to add new user
   emacs -nw /etc/suoders
   # return permissions
   chmod u-w /etc/sudoers
   exit

Install required packages

Login as new user and install these packages.

# install stuff
   sudo apt-get install git-core -y
   sudo apt-get install s3cmd -y
   sudo apt-get install php5 -y
   sudo apt-get install php5-cli -y
   sudo apt-get install build-essential -y

Install RVM and gems

#install rvm and gems
   curl -s -o rvm-installer https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer
   chmod u+x rvm-installer
   # install globally as sudo. VERY IMPORTANT
   sudo ./rvm-installer stable
   source /usr/local/rvm/scripts/rvm
   sudo usermod -a -G rvm chad
   newgrp - rvm # the - is VERY important
   rvm pkg install zlib
   rvm install 1.9.2
   rvm use 1.9.2
   gem install compass --no-ri --no-rdoc
   gem install liquid --no-ri --no-rdoc
   gem install maruku --no-ri --no-rdoc
   gem install rdiscount --no-ri --no-rdoc

Additional Steps

# let www-data use rvm
   sudo usermod -a -G rvm www-data

# /var/www/ writeable
   sudo chmod o+w /var/www
   sudo /etc/init.d/apache2 restart

# configure s3cmd
   # make sure to have aws access keys setup at:
   # https://aws-portal.amazon.com/gp/aws/securityCredentials
   s3cmd --configure
   mv .s3cfg .s3cfg_mailblog
   chmod g+r .s3cfg_mailblog

# install blog template, mailblog-files, and jekyll
   git clone git://github.com/jazzychad/mailblog.git blog
   chmod g+w blog
   chmod g+w blog/_posts
   mkdir -p scripts/ruby/gems
   cd scripts/ruby/gems
   git clone git://github.com/jazzychad/jekyll.git jc-jekyll
   cd ~
   git clone git://github.com/jazzychad/mailblog-files.git

Configure you S3 Bucket

• Go to https://console.aws.amazon.com/s3/home
• Create a bucket named exactly like desired url domain (e.g. mailblog.example.com)
• Click on your bucket
• Click on Properties button
• Click on Website tab
• Check Enabled
• Set Index Document: index.html
• Set Error Document: 404.html
• Click Permissions tab
• Click Add Bucket Policy
• Paste in bucket policy from mailblog-files/bucket_policy.txt,
• Insert proper bucket name in policy
• Update your domain's DNS to add CNAME to point to bucket domain

Configure and Customize Your Blog

You will want to replace all references to 'example.com' and system paths which have 'chad' in them. You can also customize the sass and layout files to style your blog however you like.

# configure and customize blog directory
   cd ~/blog
   grep -rHn "example\.com" .
   grep -rHn "chad" .
   # edit files to customize blog, update paths in Rakefile, etc

Setup do_blog.sh and Test

The do_blog.sh script will be called by your incoming mail webhook (this is why we added www-data to the rvm group). Place it in your home directory. You can test that everything is working so far by running it. This will publish your blog to your s3 bucket. If everything was setup correctly, you should be able to go view it at your mailblog domain (if the DNS has updated already) or the full s3 bucket url.

# get do_blog.sh
   # add do_blog.sh to home dir
   cp ~/mailblog-files/do_blog.sh ~/do_blog.sh
   chmod u+x do_blog.sh
   chmod g+x do_blog.sh
   # test setup so far
   ./do_blog.sh

Configure Mailgun

Tell Mailgun how to route incoming mail to your mailblog server. You can choose an email address you want to use and set the webhook on your server. You can choose any webhook address you want, just make sure it matches the path you use in the next step.

• Login to Mailgun.net
• Go to https://mailgun.net/cp/routes
• Click Create Route
• Filter expr: match_recipient("mailblog@domain.tld")
• Add Action: forward("http://mailblogserver.domain.tld/mailgun/hook.php")
• Add Action: stop()
• Click Save

Add Webhook to Your Server

# add mailgun hook.php
   mkdir -p /var/www/mailgun
   cp mailblog-files/hook.php /var/www/mailgun/hook.php

You're Done!

Go ahead and try to send an email to your new mailblog address. A few seconds after sending, you should see your new post appear in your s3 bucket and your mailblog homepage update to show your new post.

Caveats

A few warnings. First, there is no access control. This means that anyone can post to your blog if they know your blog's email address. You could add some basic security to this by adding a check of the from address in the hook.php script (and also verifying the webhook signature value to make sure the call is authentic). So, a bit of security by obscurity (which is admittedly terrible and lazy).

Second, this setup only handles text posts right now. Eventually I would like to support handling image attachments to the emails and using the email text as the caption or something.

There are many improvements that can be made on this system, but this is a good start. Try it, and let me know what you create!

Repos

Custom jekyll gem: https://github.com/jazzychad/jekyll

Mailblog template: https://github.com/jazzychad/mailblog

Mailblog-Files: https://github.com/jazzychad/mailblog-files

Further discussion on Hacker News.