Create Your Own Posterous with Jekyll, PHP, Mailgun, and S3– 12 March 2012
– 1174 words
A few weeks ago, I decided to create an email-powered blog that is hosted in an Amazon s3 bucket as a website. I wanted to be able to email new posts to the blog and have them published automatically. I use the email subject line as the title of the post and the email body as the text of the post (written in markdown).
This blog (the one you're reading right now) is powered by jekyll and hosted on s3, so I decided to reuse most of that setup and add some extra glue to handle posts by email. It does require an extra server to be running on the open internet (to handle incoming mail webhooks and publish new content to s3), but it can be a small, cheap 256MB slice since it does not have to handle any blog traffic (s3 takes care of that).
The result is http://mailblog.jazzychad.net. I am hoping to use it when I want to quickly publish small posts or passing thoughts.
Email is sent to a special address handled by Mailgun.
Mailgun posts the email data to a webhook on my mailblog server. PHP/Apache is running on the server to handle the incoming data.
The PHP script formats the email data into a new post markdown file, writes it into my jekyll _posts folder, and calls a do_blog.sh script.
The do_blog.sh script causes jekyll to compile the blog and then sync the files to my s3 bucket.
The whole process took about 3 hours to complete. Most of that time was spent struggling with RVM because ruby and multi-user rvm are nuts. I managed to distill the commands down to the magic incantation in this post.
All of the commands required to reproduce this setup are listed step-by-step below. I tested them in a brand new server instance, so it should work as advertised. If not, please let me know so we can debug.
Disclaimer: These setup instructions are specific for setting up a server on the Rackspace Cloud. I used a 256MB RAM instance of Ubuntu 10.04 LTS.
On Rackspace, when you create a new instance, you are given a root account and password. This works differently on other cloud hosting services, so your initial setup steps may be different if you use a different provider. The goal is just to create a user account for yourself and give yourself sudo privileges.
Boot up a new server, login as root, change your password, update the system, then reboot.
# initial root login passwd apt-get update apt-get upgrade # reboot shutdown -r now
Log back in as root, add new user ('chad' in this example), and add to sudoers file.
adduser chad apt-get install emacs22 -y chmod u+w /etc/sudoers # edit file to add new user emacs -nw /etc/suoders # return permissions chmod u-w /etc/sudoers exit
Install required packages
Login as new user and install these packages.
# install stuff sudo apt-get install git-core -y sudo apt-get install s3cmd -y sudo apt-get install php5 -y sudo apt-get install php5-cli -y sudo apt-get install build-essential -y
Install RVM and gems
#install rvm and gems curl -s -o rvm-installer https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer chmod u+x rvm-installer # install globally as sudo. VERY IMPORTANT sudo ./rvm-installer stable source /usr/local/rvm/scripts/rvm sudo usermod -a -G rvm chad newgrp - rvm # the - is VERY important rvm pkg install zlib rvm install 1.9.2 rvm use 1.9.2 gem install compass --no-ri --no-rdoc gem install liquid --no-ri --no-rdoc gem install maruku --no-ri --no-rdoc gem install rdiscount --no-ri --no-rdoc
# let www-data use rvm sudo usermod -a -G rvm www-data # /var/www/ writeable sudo chmod o+w /var/www sudo /etc/init.d/apache2 restart # configure s3cmd # make sure to have aws access keys setup at: # https://aws-portal.amazon.com/gp/aws/securityCredentials s3cmd --configure mv .s3cfg .s3cfg_mailblog chmod g+r .s3cfg_mailblog # install blog template, mailblog-files, and jekyll git clone git://github.com/jazzychad/mailblog.git blog chmod g+w blog chmod g+w blog/_posts mkdir -p scripts/ruby/gems cd scripts/ruby/gems git clone git://github.com/jazzychad/jekyll.git jc-jekyll cd ~ git clone git://github.com/jazzychad/mailblog-files.git
Configure you S3 Bucket
- Go to https://console.aws.amazon.com/s3/home
- Create a bucket named exactly like desired url domain (e.g. mailblog.example.com)
- Click on your bucket
- Click on Properties button
- Click on Website tab
- Check Enabled
- Set Index Document: index.html
- Set Error Document: 404.html
- Click Permissions tab
- Click Add Bucket Policy
- Paste in bucket policy from mailblog-files/bucket_policy.txt,
- Insert proper bucket name in policy
- Update your domain's DNS to add CNAME to point to bucket domain
Configure and Customize Your Blog
You will want to replace all references to 'example.com' and system paths which have 'chad' in them. You can also customize the sass and layout files to style your blog however you like.
# configure and customize blog directory cd ~/blog grep -rHn "example\.com" . grep -rHn "chad" . # edit files to customize blog, update paths in Rakefile, etc
Setup do_blog.sh and Test
The do_blog.sh script will be called by your incoming mail webhook (this is why we added www-data to the rvm group). Place it in your home directory. You can test that everything is working so far by running it. This will publish your blog to your s3 bucket. If everything was setup correctly, you should be able to go view it at your mailblog domain (if the DNS has updated already) or the full s3 bucket url.
# get do_blog.sh # add do_blog.sh to home dir cp ~/mailblog-files/do_blog.sh ~/do_blog.sh chmod u+x do_blog.sh chmod g+x do_blog.sh # test setup so far ./do_blog.sh
Tell Mailgun how to route incoming mail to your mailblog server. You can choose an email address you want to use and set the webhook on your server. You can choose any webhook address you want, just make sure it matches the path you use in the next step.
- Login to Mailgun.net
- Go to https://mailgun.net/cp/routes
- Click Create Route
- Filter expr: match_recipient("firstname.lastname@example.org")
- Add Action: forward("http://mailblogserver.domain.tld/mailgun/hook.php")
- Add Action: stop()
- Click Save
Add Webhook to Your Server
# add mailgun hook.php mkdir -p /var/www/mailgun cp mailblog-files/hook.php /var/www/mailgun/hook.php
Go ahead and try to send an email to your new mailblog address. A few seconds after sending, you should see your new post appear in your s3 bucket and your mailblog homepage update to show your new post.
A few warnings. First, there is no access control. This means that anyone can post to your blog if they know your blog's email address. You could add some basic security to this by adding a check of the from address in the hook.php script (and also verifying the webhook signature value to make sure the call is authentic). So, a bit of security by obscurity (which is admittedly terrible and lazy).
Second, this setup only handles text posts right now. Eventually I would like to support handling image attachments to the emails and using the email text as the caption or something.
There are many improvements that can be made on this system, but this is a good start. Try it, and let me know what you create!
Custom jekyll gem: https://github.com/jazzychad/jekyll
Mailblog template: https://github.com/jazzychad/mailblog
Mailblog-Files: https://github.com/jazzychad/mailblog-files— Fin.