Ruby Red Bricks

Ruby, Lego™ and other things I dig

Nanoc-novel

Much to my wife’s chagrin, I’ve been writing an online novel. My good lady doesn’t mind me writing, it’s the constant fiddling with the technology that drives her crazy. And to be honest, tweaking layouts, trying out new frameworks, etc is the e-version of sorting out your pens.

I finally settled on nanoc a while ago, building a very rough draft using nanoc and Maruku to convert numerous ‘pages’ of Markdown content into a website. You’re welcome to read it, though bear in mind it is still a draft, still very rough - and certainly not to everyone tastes.

The chief problem with writing like this is managing content split up over many files. If I want to insert a page, or move pages around, there’s a fair bit of manual renumbering. It’s also hard to edit, as I can’t just read through the text as it flows. It’s much easier to work with a single text file - and somehow break it up into individual pages for the final online version.

To this end I built a very simple gem called breakdown, designed to split a large Markdown file into many smaller ones - based on directives within the file.

Given my own novel is a work in progress I wanted to try this approach on a more substantial piece of text. Trawling through the Gutenberg Project, I came across a wonderful set of short stories by Edward Page Mitchell. Mitchell wrote most of his stories in the 1870s and 1880s, making him an early contributor to American science fiction literature.

The Gutenberg Project version of Mitchell’s work is collected as a set of short stories, entitled The Tachypomp and Other Stories. As such it made a good candidate for breaking into smaller pieces using the breakdown gem. Because the gem splits on normal Markdown horizontal rules, marking up the original text - and adding small amounts of formatting - took only a few minutes. The single-file version of the text looks something like this:

*** index

| Title | The Tachypomp and Other Stories | 
| Author | Edward Page Mitchell |
...
...

*** table-of-contents

THE TACHYPOMP
THE SOUL SPECTROSCOPE
THE MAN WITHOUT A BODY
...
...

*** section-1

THE TACHYPOMP

A Mathematical Demonstration

There was nothing mysterious about Professor Surd's dislike for me. I
was the only poor mathematician in an exceptionally mathematical
class. The old gentleman sought the lecture-room every morning with
...
...

Once converted into several files, nanoc does the heavy lifting of converting the files into HTML. In the example above, the text is split into index.md, table-of-contents.md and section-1.md and placed in nanoc’s content folder. From there nanoc processes the files, adds styles and other artefacts as determined in its Rules file, and builds the website.

The process is still reasonably manual at present, requiring an import step and then build. By extending nanoc’s command mechanism it should be relatively easy to wrap all the calls together compile the entire site from a single source file in one step.

The resulting online book can be found here: http://tachypomp.rubyredbricks.com/.

The source code for the books can be found here: https://github.com/ferrisoxide/tachypomp.

None of this is particularly clever or (ahem) novel. The breakdown gem is fairly naive - it’s the first gem I’ve ever written and I’m already slightly embarrassed by it. But as someone who no longer codes professionally, it was a bit of fun getting back into hacking.

There are a number of ways to improve this approach, but I’m already finding it paying dividends. Apart from the small amount of coding that went into the gem, I’m less inclined to fiddle with technology and more motivated to actually finish This Purple World. And who knows, I might even make my wife happy in the process - though that’s not a sure thing. For the record, she hates the novel.

Google Chrome Took Over My Desktop

Gosh.

Somehow I managed to get Chrome into full-screen mode, some automatic gesture that’s natural within OS X that you do without thinking… that sensibly placed button in the top-right-hand corner.

Could I get out again? Not naturally. I started poking through menus, trying to find some switch to get my desktop back to normal. Nada, though going through the settings made me think about how much Chrome logs you into Google-space.

I did manage to google my way out, thanks to OSX Daily. Apparently it’s a known issue with OS X Lion and Chrome. To sum up, hitting Command+Shift+F in Google Chrome swaps the screen mode to get you out of full-screen.

I had to hit Command+Shift+F at least twice - once to bring Chrome into hell-mode where it takes even more of your eyeball space, then stepping back into good-desktop-citizen mode again.

The complete asymmetry of the swapping modes is what got me. Once in, harder to get out. I don’t expect that sort of behaviour in an application, though I’m sure I’ve been guilty of writing code that works that way.

The fun fact is that the OSX Daily article appeared in August 2011. I don’t know the Chrome codebase, but at a hunch I’d say there’s been plenty of time to fix this. Maybe Google’s Chrome team have other challenges to deal with, and making migrating back out of Google-space simple is just not a priority at the moment.

Google could have easily buried the article. As it is googling for “os x take google chrome out of full screen mode” brings the OSX Daily page as the first hit - without advertising (at least at time of writing) to obfuscate the message.

I still suspect Google’s behaviour. Everyone is vying for the desktop at present. It’s a natural survival of the fittest that goes beyond the desktop, to phones, to tablets, to apps, to your dreams. Making it harder for people to swap focus makes commercial sense, so neither the asymmetry in swapping modes nor the longevity of the problem surprises me.

More fun facts: Firefox also exhibits this behaviour. On OX Lion you have to hit Command+Shift+F to get back out of full-screen - though thankfully just the once. Only Safari drops back out on hitting the Esc key, which feels the most natural way of doing things to me.

Tin-foil hat theories are best left to another post.

Minecraft the Movie on TPB - How Much Does ‘Free’ Cost?

NB: written back in December 2012, only published Feb 22, 2013

While I can’t find anything official on the 2 Player Productions, there’s a buzz about the movie Minecraft: The Story of Mojang being put up on The Pirate Bay by its producers. (Google search)

My kids backed this as a Kickstarter project. Getting access to goodies was a nice part of the deal, the real goal was to see that movie fully realised - i.e. on our TV. We got the link to the digital copy, downloaded an SD version (shitty, edge of the exchange net access) and waited. Like a slowly warming room on a cold winter’s night. “37 minutes to go! No.. 33. No.. 42”

We watched the movie as a family. I won’t spoil anything in saying you probably won’t learn anything you didn’t already know - but you’ll have a smile on your face nonetheless. It’s quite inspiring in many ways - particularly the advice about not accepting advice and the need to make stuff happen (not just dream it). The choice comment from the night, having seen the boys’ names in the credits, was from my oldest: “I feel like everything has changed.” He’s now part of gaming history.

It was a wonderful film. And now it’s released you have the option of either buying it - or just downloading it from The Pirate Bay. Assuming the comments on TPB’s page is true, it sounds like the folks from 2PP are accepting a blatant reality - one that traditional media seems to be fighting tooth and claw to deny. People will copy your stuff. And sometimes they won’t pay for it.

I’m intrigued by this. It appeals to my thinking, about how the machinations of intellectual property pundits are being disrupted by smaller groups intent on creating some kind of meaningful dialog with the consumers of their product. It’s a new kind of capitalism - writ small and with a human face. Yes, I’m sure there will be people who try and rort things. But they have less chance of doing large scale damage (relative to, say, a large financial institution) and more chance of being caught out.

But how do I feel about it? When I heard about Minecraft: The Story of Mojang being released on TPB I have to say I felt a little twinge. Here was something that we’d spent money on, had waited and waited until we could finally watch it, now available for anyone to just download for free at a whim.

Maybe more of a cringe than a twinge: a horrible feeling that things just aren’t right, like bogans turning at your dinner party. That feeling lasted oh, a good hour or two, with minor flashbacks throughout the night. When I woke up on Christmas Day however, it had vanished. Maybe due to severe sleep withdrawals, having written a web app that displayed the number of hours until the kids could wake Mum and Dad (“No earlier than 6am!”). What worked on my box of course failed once I pushed it up to the server. Time difference. Different version of Ruby. Grrr.. By the time I got everything working the page read “4 hours and 22 mins 50 secs until you can wake Mum and Dad”.

But now, a day and a sleep-in later, I’m still feeling fine. People were always going to pirate this movie. By getting in ahead 2PP are creating an opportunity to engage with a new potential market. It makes sense - both from a business perspective and from a human one. It’s taking me back, to when I first read the Cluetrain Manifesto and thought “Yep, this is the world I want to live in.”

There are two kinds of people 2PP will engage with via The Pirate Bay: those who will buy the product, and those who won’t. In either case there will be a range of reasons people either decide to give back or not. I suspect that choosing not to contribute will boil down to one of three reasons: too poor, too miserly or being a brain-dead zombie.

I’ve met plenty of brain-dead zombies: people who just download out of some compulsive habit, without any conscious realisation that the sheer volume of content they have siting on their multi-terabyte drives cannot be watched within a single lifetime. Yet still they download. I’m sure in the future we’ll come up with medical term for this, but for now “brain-dead zombie” will do as a placeholder.

Of those too poor, well.. having once been a university student / worker of crappy jobs I understand the feeling of going without. It’s hard to judge, when there’s so much on offer, so much available and being consumed by the rest of society. How can anyone be expected to resist? And why resist? Who really is the victim? The only difference is that when I was younger we didn’t rip CDs. We made mixtapes. Of course, we were intentionally violating IP laws in the pursuit of turning our friends on to new music. We knew every crackle and pop, the scratches where the needle would jump the vinyl. But maybe none of this happened. We were under the radar and no statistics were gathered. We bought the posters, went to the gigs. Our relationship with our pop culture heroes was writ across bedrooms, on the old fridges we bought for the first flat we rented. It’s all different now, of course. But I remember what it was like to be young.

As to the miserly, again it’s hard to judge. I used to get irritated by well-off dudes limewiring (or whatever old dudes do) music, but then you get to talk about their collection of hard-to-get blues that you just can’t find on iTunes. And I get it. But then you have a whole crowd of casual downloaders who just do it because they can. There’s not even a “Fuck it to the Man!”. Popular culture is just a noisy post-modern backwash to their lives that they feel as entitled to as air or thought.

Taking a purely freeloader approach to online content creates very little value. Maybe a link in a tweet is good advertising. But that’s more check out my cool that check out this cool thing. Again, no judgment. But here’s the sticker. Someone who just downloads an infinite stream of candied culture into their brains will never know the joy, the expectation, of watching and waiting for something you care about come into fruition. They’ll never have that giddy delight come over them when a package arrives in the mail.. from overseas! (more joy for little people). They’ll not be engaging in the creation of something new, something good. Or they may just get their joy from somewhere else. We’re all different. But fundamentally they won’t have the opportunity to access some of the value of this film. Not everything can be downloaded.

So if you didn’t contribute to the Kickstarter project, and you didn’t buy a copy from 2 Player Productions, but you downloaded and watched the movie anyway.. good on you. It’s a wonderful film, isn’t it? I hope you didn’t mind missing out on the extras, and I really hope you enjoyed the film.

Anna Calvi: Desire

Oh my.. how did I miss this? Like PJ Harvey and Siouxsie Sioux got together and had a baby..

It might not be to everyone’s taste, but I’m sure you’ll understand that magic when you hear a song that just gets you excited about music. It’s like falling in love all over again.

Day Zero of Giving Up Smoking

Reasons for Quitting

For myself. I’m a slow dive. Smoking and stressing and not sleeping all feed into themselves. There’s at least one thing I can yank out of the cycle.

For my family. Only thing worse than having a dad who stinks of cancer breath - is not having him around at all. I’m in this for the long haul. Love my kids. Love my wife. Love my friends and family.

For the hip pocket. Man smoking is expensive. I could have bought a really nice guitar for what I’ve spent on smokes. Or one heck of a lot of Lego.

Because enough is enough. I’ve tried all sorts of ways to give up. Kidded myself enough times that ‘this is the last packet because…’. Time’s up.

To be a better corporate citizen. Apart from the regular anti-social “thinking time” outside, nothing says “lack of respect” like stinking up the office.

Because it’s lost its cachet. I took up smoking to give up. 10 years without smoking I started again so I could experience going through withdrawals. A bit odd perhaps. But it’s given me things to write about. So write..

Reasons for Quitting: more than a badly framed nerd joke.

Getting Started With Taco, More Links

I’ve been in touch with Matt Sears, of the Matt Sears Todo.rb Gist fame, he tells me that he’s been working on taco - a gem-ified version of todo.rb with tweaks and extra features.

Taco looks pretty neat. From the code, it looks like Matt’s is moving towards alternative repos for taco’s list - which will help with the cross-computer, mobile access requirements listed earlier. But the basics are all there.

Installing taco

The taco site recommends installing as a gem:

1
gem install taco

I couldn’t get this to work, as it seemed to pull in an older version. I’ve reported it back to Matt, though it might be something stupid in my RVM setup. I don’t feel in total control of RVM yet, but I’m learning (I am but a humble manager).

Installing from repo worked:

1
2
3
4
5
6
git clone git://github.com/mattsears/taco.git
cd taco
bundle install
rake build
rake test 
rake install

As did adding it to a Gemfile:

1
2
source "http://rubygems.org"
gem 'taco', :git => 'git://github.com/mattsears/taco.git'

Using taco

Once installed, you can query taco directly, without any parameters:

1
2
3
4
5
6
$ taco

TACOS:
--------------------------
[1] Add an item!     empty
[2] Add an item!     empty

Help is in the logical place:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ taco help

Usage: taco [options] [command]
Taco holds stuff in a shell

Commands:
  add       Adds a new item
  del       Removes an item
  list      Prints all items
  init      Bootstraps tacos in this directory
  bump      Moves an item to the top of list
  settings  Moves an item to the top of list

Options:
    -h, --help                       Display this screen
    -v, --version                    Display the current version

Dupe in the ‘settings’ description there, but these are minor. Taco has what I expect from an app: sensible defaults and some form of navigation/support. No mention of the ‘@’ tagging though, e.g.

1
taco add tidy desk @work

All the commands operate as you’d expect, though there are some quirks.

Taco automatically generates two empty items within your list:

1
2
3
4
5
6
$ taco

TACOS:
--------------------------
[1] Add an item!     empty
[2] Add an item!     empty

Add a new items wipes out one of the placeholder items:

1
2
3
4
5
6
7
$ taco add have breakfast
Added: have breakfast: @

TACOS:
----------------------------
[1] Add an item!       empty
[2] have breakfast

But doing it a second time doesn’t:

1
2
3
4
5
6
7
8
$ taco add organise brain
Added: organise brain: @

TACOS:
----------------------------
[1] Add an item!       empty
[2] have breakfast
[3] organise brain

Minor. And easy to snip out:

1
2
3
4
5
6
7
$ taco delete 1
Deleted: Add an item!: @empty

TACOS:
-----------------------
[1] have breakfast
[2] organise brain

The bump and list commands are fairly self evident. There is a neat done command to mark tasks as complete, though it didn’t work exactly as expected:

1
2
3
4
5
6
7
$taco done have breakfast
Done: organise brain: @done

TACOS:
---------------------------
[1] have breakfast
[2] organise brain     done

OK, my bad - you have to use indexes throughout. But it’s not responding to garbage input. Minor, minor. The correct way to mark an item as done - or delete/bump it - is to use the item’s index:

1
2
3
4
5
6
7
$taco done 1
Done: have breakfast: @done

TACOS:
---------------------------
[1] have breakfast     done
[2] organise brain

Taco’s repo is in $HOME/.tacos.yml, so you can access your lists in any dir.

There’s also a $HOME/.tacorc file for storing configuration data:

1
2
3
storage: 'yaml'
foo: 'bar'
1: '1'

I think I understand what the storage: property is for, but the rest look like some kind of index. I’m sure will be revealed as I get into the code.

Other links

Some useful Reddit comments:

metamorfos points us to us his rubdo project on Github. Use of Dropbox a great idea.

seyday points us to ditz on Rubyforge. Heavier tool than what I need right now, but one to come back to.

That’s it. Spent enough time on this over the weekend and now it’s time for the family.

Cheers folks -Tom

TODO: Find a Todo Tool

This is a follow-on from an earlier post about getting organised using Ruby scripts. The first step is to get some kind of system in place to managing the things I need to remember - a Todo list tool.

Sourcing a Todo Tool

First off, I’m not going to build if something already exists. I think that it’s great that we as a community - and I’m thinking of the larger programming community - try to solve the same problems in different ways. It may look inefficient from an external point of view, but it’s really actually the most optimal way of solving problems - follow as many paths as possible to find the best reusable one.

But that’s the topic for another post.

A quick google reveals numerous Todo list applications written in Ruby. The topic seems a perennial favourite in Ruby coding competitions. Chances are looking good I’ll find a suitable starting point for Argh!

But before I start looking, some constraints. What do I want from a tool?

Requirements

  • MUST be written in Ruby
  • MUST be able to capture new todo items via the a unix shell
  • MUST be able to categorise items (priority, project, focus, groupings, etc)
  • MUST be able to manage items (list, delete, change category - meta tasks)
  • MUST be simple to use

  • SHOULD work across different computers

  • MAY be accessible from a mobile web interface.

The last two come from the folks at Reddit. Makes sense.

As for time contraints, I can’t spend more than about 30 mins having a look - it’s the weekend and there’s stuff to do with the family. Here’s the top hits:

Candidate Tools

Public Gist from Matt Sears

https://gist.github.com/1259080

A small, simple Todo manager. Exists as a Gist, but with a couple of forks that might be worth following. Has a basic tagging system using ‘@’ identifiers.

Can create, delete, list etc as expected. Can change priority using ‘bump’ command. Defers filtering on ‘@’ identifiers to the system’s grep.

Pros:

  • Deadly simple; Small code footprint.

Cons:

  • Outsourcing filtering out to grep is a great idea, but a reminder in the help wouldn’t hurt.
  • Only one identifier per item (as far as I can tell)

Examples:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Add todo items
$ todo add Check out rubyrags.com
Add: (1) Check out rubyrags.com

# Create a todo with context
$ todo add Buy Duck Typing shirt from rubyrags.com @work
Add: (2) Buy Duck Typing shirt from rubyrags.com @work

$ todo add Buy Ruby Nerd shirt from rubyrags.com
Add: (3) Buy Ruby Nerd shirt from rubyrags.com

# List todo items
$ todo list
1. Check out rubyrags.com
2: Buy Ruby Nerd shirt from rubyrags.com       @work
3: Buy Duck Typing shirt from rubyrags.com     @work

$ todo list @work
1: Buy Ruby Nerd shirt from rubyrags.com       @work
2: Buy Duck Typing shirt from rubyrags.com     @work

Dooby

https://github.com/rafmagana/dooby

Interesting approach - essentially one project per directory. Has a Twitter-like tagging model; people (@joan, @peter), contexts (#important, #home) and item sets (%fix_computer, %do_many_small_tasks).

I tried installing Dooby under Ruby 1.9.3 and had a few issues. Not sure what’s going on but didn’t have the time to get bogged down in debugging. The exercise is to see what’s on offer, kick the tyres later.

Pros:

  • Quite feature-rich
  • Comes as a gem

Cons:

  • Not convinced by the one-project-per-directory
  • Larger code-base

Examples:

1
2
3
4
5
6
7
8
9
10
# Add todo items, using @people, #context and %item_sets tags
$ dooby add "#fix the email error in %website, check   this out with @peter #today"
$ dooby a "learn to use the #aliases of the #commands  "
$ dooby a "#pair with @jim on the %tickets module"

# List todo items
$ dooby list @peter
$ dooby l today
$ dooby l "#today"
$ dooby l %website

Others

I wanted to review at least three, but the more I looked the more sketchy the Todo apps looked. I also chewed up my allotted time too quickly. Still easily distracted.

I did look at the todo gem, but it doesn’t look like it’s been touched in a while.

There was also another command line app Tudu that sprung up in searches. But once I started leaning out of Ruby-space it really was time to stop.

As noted before, Todo apps appear in Ruby comps fairly regularly. I might need to look further into the results of the CodeBrawl comp - NB Matt Sears’ entry came in second in that comp.

If anyone knows any code I should have a good look at let me know!

Way Forward

Right now, I want to start with something simple. Matt Sears’ gist looks pretty decent for a one pager (not including the tests). Dooby looks like fun but, as commented on by Bill Dueber in the previous post, the whole point of this exercise is to get things together and happening. I’ve love to have Dooby’s multiple tags model in Matt’s code though.

Neither of the two solutions I’ve looked at can work across computers, nor have a mobile interface. The winner of the CodeBrawl comp seems to use Github Gists as the repo model. Interesting, but not sure I’d want all my data public. Anyway, the next step is to actually get something installed, and we can book-strap from there. Onwards, forwards.

Future post: Argh! Coding Begins

Ordering My Brain With Ruby

My brain is chaos. I have so many things to do, so many things to remember, so many time critical moments; it’s an unfortunate fact that things get missed.

Life’s like juggling kittens.. that have been set on fire.

I need a way to order my brain. And I want to do it in Ruby.

I’m pretty sure I’m not alone in finding the work/life/lego balance hard to achieve. I’ve looked at all sorts of tools to arrange my brain - read books, installed software, dumped everything in one convenient bucket labelled “To Be Sorted”. Great way to feel you’re doing stuff when you’re not doing stuff.

I don’t think there is one tool or process that can cover all bases. We all have a massive range when it comes to the kind of detail we need to deal with. Some things are microscopic, others very general. Some are time/space/money critical, some are just nice-to-haves.

Often I find myself with some golden hammer application, that works for a certain set of tasks or bits of data but slowly gets munged into performing tasks it’s ill-suited for. A classic example is the todo list that becomes a project management tool - or even the reverse, where a complex system is made to fit a simple job. Help-desk applications become CRM systems. And so on.

Part of the problem is that many of these systems have just enough to perform some of the functions of other systems, albeit is a slightly brain-damaged way. The other part of the problem is that once stuck using any system it’s hard for us to break habits and try something new - even if it may make things orders of magnitude easier.

I blame UML (but I blame UML for a lot of things).

I know a guy who arranges his life using Perl scripts. Really intriguing concept; he palms off a lot of the things you and I struggle with - stuff he has to do, important dates, things he’s borrowed or lent to people, etc, building web site - to bunches of code. If you have any dealings with him sooner or later you’re going to get some automatic email letting you know for example, that you still have that book he lent you.

It’s kind of beautiful. Never do I feel like I’ve just been instructed by a robot - the sense of the person is embodied in the scripts he uses to extended himself. The language in his messages are polite, and respectful. And not demanding or abrasive. Much like the man himself.

Side note: if you’re reading this RW, let me know if I’ve misrepresented you.

I’m not privy to the codebase. I’m not sure if I’d want to be, as reading Perl makes me giddy and sick. Ruby’s my fave and Ruby’s where I’ll stay. Yes, possibly even if something better comes along.

To explain: I’m a manager of sorts. I don’t code commercially any more - not much at least. My opportunities for learning new programming languages are limited. Maybe one day I’ll get a chance to get my hands dirty with code again, but right now I’m learning different stuff. Ruby is my COBOL.

But I still spend a lot of my in and out of servers, pulling data out of databases, tweaking apps. When I need to remember stuff I waste time jumping out to a web app, or even writing stuff down on paper. I’m at the keyboard, in a shell, and I just want to go ‘argh, remember this’ and the computer does it.

I don’t know where the ‘argh’ came from just then. Argh like a pirate argh? Argh, I can’t believe I just did that. Argh, argh. OK, I’m having one of those moments where a word stops having meaning from being repeated too often. Argh. The Urban Dictionary has a reasonable definition:

argh N. Used to describe a feeling of frustration or confusion. Often used for school work or trying to understand someone’s lack of words. It’s simple and straightforward.”

Argh, that’s right on the mark. Welcome to Project Argh.

Future Post - TODO: find a todo list tool

More ePub With Nanoc

A little way back I posted a lengthy article on how to create a e-book using ePub, the static website compiler nanoc and some quick and dirty code. In this much shorter post I’ll aim to clean up the solution a little.

nanoc meta-data

One of the nice things about nanoc is the capacity to add arbitrary meta-data to each page. In the original version of the e-book I used a convention based on the URL and nanoc identifier to link from one page to the next: page ‘/001/’ precedes page ‘/002/’ and so forth. It was crufty and required more gsubbing than was sound.

A cleaner approach is to put the data directly inside the page, within nanoc’s meta-data block:

1
2
3
4
5
6
--- 
title: Epithelium
chapter: 1
page: 1
---
... page goes here ...

There are certain pages that I want to exclude from the ePub version (e.g. the website home page). Again, using meta-data the ePub compiler can directed to ignore specific content:

1
2
3
4
--- 
title: Home
exclude_from_epub: true
---

Using the preprocessor

Another useful feature in nanoc is the preprocess directive you can add to the /Rules file. Using this I can build an outline of the e-book from meta-data and hive it off to a .yaml for other parts of the code to use later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
preprocess do
  outline = {}

  @items.each do |item|
    unless item[:exclude_from_epub]
      chapter = item[:chapter]
      page = item[:page]
      title = item[:title]

      if chapter
        outline[chapter] = {} unless outline[chapter]
        outline[chapter][page] = {
          :label => item[:title],
          :file => item.identifier.gsub(/^\/|\/$/, '') + '.html' }
      end
    end
  end

  File.open("tmp/outline.yaml", "w") {|file| file.write outline.to_yaml }
end

The full code for the e-book can be found on Github.

Five more pages have been added to both the web and ePub versions of the book and are available online. I’ll be releasing more on a regular basis, so if you’re interested in following the book’s progress feel free to follow me on Twitter or look for the hashtag #thispurpleworld.

Quick and Dirty ePub With Nanoc

Bit of background

First off, this is a bit of a hack. At best it’s a jump headfirst into publishing an ePub document using nanoc, while knowing very little of the former and only a fraction more of the latter.

We’re aiming for a quick way of producing ePub books, using nanoc to do the bulk of the heavy lifting while other gems and custom code fill in the rest.

A sample project demonstrating this approach can be found at thispupleworld.com, with the source code available on Github.

An ePub Primer

ePub is a set of standards produced by the International Digital Publishing Forum. There are two main versions currently in use: ePub 2.0.1 and ePub 3.0. ePub 3.0 introduces some interesting features, including interactivity via Javascript and support for SVG and MathML, but the tools and content we’ll be looking at here use version 2.0.1

ePub is essentially a packaging model for electronic books. Content (pages, chapters, etc) is represented as XHTML documents, styled with CSS and bundled together in a zip file. Two special xml-based files, toc.ncx and content.opf, contain the table of contents and overall descriptive data for your book.

It is possible to bundle fonts, images and other elements into your ePub package. For now we’re just going to worry about text content. It’s also possible to create nested documents in ePub, but in our first cut we’ll be assuming a flat structure.

Step 1: make it work. Step 2: make it pretty.

Setup nanoc

Let’s set up an empty nanoc site to start filling with the content.

1
2
3
gem install nanoc
nanoc create_site my-epub-project
cd my-epub-project

Create a /Gemfile in the same directory.

/Gemfile
1
2
3
4
5
6
7
source "http://rubygems.org"

gem 'nanoc'         # here for convenience
gem 'adsf'          # simple web server for previewing content
gem 'kramdown'      # used to generate Markdown content
gem 'systemu'       # used when deploying via rsync
gem 'eeepub'        # ePub generator

The eeepub gem provides a simple DSL for packaging XHTML documents. It also provides lower level tools for manipulating toc.ncx and content.opf files, but we won’t be using these.

Install the gems using bundler:

1
bundle install

Setup eeepub

nanoc allows for multiple layouts. Create an alternative layout for generating the ePub XHTML documents in the project’s /layouts directory. NB: <meta> tags need to be terminated with a closing /> in XHTML.

/layouts/epub.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
  <head>
    <title>
          <meta name="generator" content="*nanoc* <%= Nanoc::VERSION %>" />
    </title>
  </head>
  <body>
      <div id="main">
        <%= yield %>
      </div>    
  </body>
</html>

nanoc also allows for multiple representations, meaning the same content item can be rendered in different formats. We’ll make use of this to render the website in both HTML (for viewing online) and XHMTL (for ePub). In the project’s /Rules file add the following compile declaration to the rules nanoc created by default:

/Rules
1
2
3
4
5
6
7
8
9
compile '*', :rep => :epub do
  if item.binary?
    # don’t filter binary items
  else
    filter :erb
    filter :kramdown
    layout 'epub'
  end
end

Note we’re using both :erb and :kramdown filter types. We’ll come back the reason for this later. The compile declaration includes a :rep => :epub option and changes the layout to use the epub.html template created before.

In the same /Rules file add the following routing rule:

/Rules
1
2
3
4
5
6
7
8
route '*', :rep => :epub do
  if item.identifier == '/'
    # Exclude the root index.html from the epub version
  else
    # Write item with identifier /foo/ to /epub/foo.html
   '/epub' + item.identifier.chop + '.html'
  end
end

Again, the route includes the :rep => :epub option. The route rule explicitly removes the default /index.html created by nanoc when initially generating the site, as this will only be used in the web version. We may change this later, but I wanted to illustrate that you can vary the content between representations using route and compile declarations.

We’ll store the configuration data eeepub and our custom code will need in /epub.yaml. Create this file in the project’s root directory.

/epub.yaml
1
2
3
4
5
6
7
8
9
10
11
12
meta:
  title: My ePub Project
  creator: Me!
  publisher: Me!
  date: 2012-09-07
  identifier:
    url: http://my-epub-project.com/book
    id:: BookId
  uid: BookId

filename: my-epub-project.epub
output_dir: output/book

I’ll have to admit, I don’t really know what the identifier and uid attributes are for - other than they end up included in the generated ePub meta data. I plan on looking into the ePub meta data further.

The filename and output_dir attributes will be used to generate the actual .epub file.

Creating Pages

While ePub allows for hierarchical ordering of content, we’re going to use a flat structure for now. In your project root, execute the following on the command line:

1
2
nanoc create_item "001"
nanoc create_item "002"

nanoc will have created two files in the /content folder. Naming each page ‘001’, ‘002’ and so on is just a convention we’ll make use of custom code later on. We could have just as easily use ‘Page 1’, ‘Page 2’, etc for our convention.

Edit both with some random text for now, but I’ll ask you to do something a bit odd with them. Somewhere in text add an erb style tag like below. Do the same for the text you add to the second file.

/content/001.html
1
2
3
4
5
6
7
8
--- 
title: 001
---
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla 
imperdiet est sit amet quam placerat ac mollis est viverra. Nunc 
adipiscing purus sit amet lorem imperdiet vitae aliquet sem tincidunt. 
Proin pretium ultrices nulla, at tincidunt lacus tempus in. 
Etiam sollicitudin odio <%= link_to_next_page 'in dui elementum vulputate' %>

The link_to_next_page method is not built into nanoc - it’s a style of linking between pages I’ve used in this purple world. It may not be useful to you, but I want to show how we can control how content is generated in different representations.

This also illustrates how we can mix and match template types in the same file. Here we are using both erb and markdown content together - made possible by the multiple filter properties set in the /Rules configuration file.

Representation-Aware Content

One way we can extend nanoc is by adding helpers to the /lib folders. These helpers are similar to helpers in Rails - methods to generate smaller pieces of content. Here’s the link_to_next_page method used above:

/lib/link_to_page.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
include Nanoc3::Helpers::LinkTo

module LinkToPage

  def link_to_next_page(text)
    if @item_rep.name == :epub
      text
    else
      next_page_num = @item.identifier.gsub(/\//, '').to_i + 1
      next_page_ident = next_page_num.to_s.rjust(3, '0')
      # check if last page
      next_page = @items.find { |i| i.identifier == "/#{next_page_ident}/" }
      next_page ? link_to(text, "/#{next_page_ident}") : text
    end
  end
end

include LinkToPage

The code is a little crufty, but what it’s basically doing is creating an href link between pages - but only in the default representation. With ePub we can assume that readers will navigate from page to page using the controls built into their ePub readers, so we don’t want the link_to_next_page to do anything much at all, apart from returning the text passed to it.

Add the link_to_page.rb code to /lib and run the command line run the following:

1
nanoc compile

Have a look at the HTML and XHTML nanoc generates in the /output and /output/epub folders. Note how the same source files have produced two quite different sets of documents.

Converting output to epub

We need a mechanism to convert the generated XHTML into the final ePub format. Another way of extending nanoc is by adding to the set of command line actions. Create a /commands folder in the project root and add something like this:

/commands/convert_to_epub.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
require 'eeepub'
require 'yaml'

usage       'convert_to_epub'
summary     'build epub file from output/epub'
description 'This is a rough and ready post-compile tool to build epub files from content.'

run do |opts, args, cmd|
  root_dir = File.join(File.dirname(__FILE__), '..')
  config = YAML.load(File.read(File.join(root_dir, 'epub.yaml')))

  nanoc_output_dir = File.join(root_dir, 'output') # TODO get this from *nanoc* config 
  file_list = Dir.glob("#{nanoc_output_dir}/epub/*.{html}")

  epub = EeePub.make do
    title       config['meta']['title']
    creator     config['meta']['creator']
    publisher   config['meta']['publisher']
    date        config['meta']['date']
    identifier  config['meta']['identifier']['url'], :scheme => 'URL', :id => config['meta']['identifier']['id']
    uid         config['meta']['uid']

    files file_list
    nav file_list.collect {|f| {:label => "Page #{File::basename(f, '.html')}", :content => File::basename(f) }}
  end

  FileUtils.mkdir_p(File.join(root_dir, config['output_dir']))
  epub_filename = File.join(root_dir, config['output_dir'], config['filename'])
  epub.save(epub_filename)
  FileUtils.rm_rf("#{nanoc_output_dir}/epub/") # remove epub XHTML files created by nanoc

  puts "epub saved to #{epub_filename}"
end

Again, the code is pretty ordinary - but it servers to demonstrate how we can extend nanoc to our own ends.

Run the following from the command line:

1
nanoc convert_to_epub

The XHTML files in /output/epub/ will have been removed and the packaged version of your .epub file will now be available in /output/book/. Open the file in Calibre, or one of the eBook readers available.

You can also look at the content online. From the project root execute the following:

1
nanoc view

Open a web browser and navigate to http://localhost:3000. Same content, two different representations.

Example Project - This Purple World

The sample project used to test this technique is something I started years ago, when I was more vigorously engaged with writing. When I read the first few pages out at a writers’ group a little old lady said, “It made me feel sick”.

The writing may not be your cup of tea, but the exercise was to see how quickly we could get something built - in the small amount of time I have for coding (moved into management, trying to keep it real).

For the record, at the coffee break the little old lady told me, “I think you were trying to make me sick, so I guess it’s effective writing”.

You can check out a draft version at thispupleworld.com. The source code is available on Github. Bear in mind, this is still very much a work in progress. I’ve put 5 (out of around 100) pages up. It really needs a decent edit, so I’ll put up more as I go through it - hopefully cleaning up the nanoc / eeepub project in the process as I get to understand both toolkits more throughly.

Next Steps

This really is a quick and dirty project. There are number of ways to improve the product, and I’ll look at this in a later (and hopefully shorter) post.

Some things that come to mind:

  • Making use of ePub’s hierarchical model for publications, separating the content into chapters and pages.
  • Replace the “001”, “002” page numbering convention with use of nanoc meta data.
  • Styling ePub documents using CSS
  • That neglected Step 2: make it pretty