DoggyBook update

I have been making progress on converting the Handbook of Biomass Engine Systems. I have actually been doing it off-site and updating the site periodically, since I came across some really good free tools for doing the job locally. One is a program called Calibre. It can convert any ebook format into any other, and has an excellent book organizer/library system. I would rank it as the Itunes of ebooks. The other tool is a program called Sigil. This is a full up ebook editor. The program is basic but easy to use. Sigil uses the Epub format, which seems to be the most common type. It takes a little poking around to figure out what an ebook is actually made of. The bottom line surprised me a little. Most ebooks are actually based on HTML, the same as a webpage. The specific type is called XHTML, and an ebook is just a collection of these pages, usually one for each chapter, plus some labeling/metadata, a stylesheet, and whatever images you included. All zipped up in a containing folder. So the nitty-gritty of editing can be a lot like editing basic HTML (no fancy webpage stuff, it's mostly text); Sigil offers WYSIWYG as well as straight code editing. It is easy to work with. Unfortunately, trial-and-error can be slow, because the file must be converted to MOBI for the Kindle (done with Calibre), then the old file must be manually deleted and the new file inserted. Kindle is not really designed to have "updated" versions of books; the assumption is that once a book is loaded you don't need to change it.

Another issue I have run into is the e-conversion of tables. The Kindle's screen being very small, anything that doesn't "wrap" well has to either shrink, making it hard to read, or be reformatted somehow. For now I have just inserted them as images, meaning they shrink to fit the page. At least the information is there, and will be distributed with the book. As for reformatting all the tables, the brilliant idea lightbulb has yet to go off. I may think of something yet.

Since ebooks are based around HTML, it seems like there should be a good way do make a simple website into an ebook fairly quickly. I am thinking of the MetroWiki, of course. And if DoggyBook could have a dynamic link that allowed me to grab the current content for compiling and ebook, I would go back to online editing.

Rambling thoughts.....

DoggyBook

DoggyBook is an open project to digitize books, pamphlets and papers for use in E-readers like Kindle and Nook.

I have recently gotten very interested in the availability of classic texts in digital format. The reason? Dad just bought himself an Amazon Kindle 3 e-reader, and he lets me use it as well. Most of my reading list is older technical manuals, but many of them are PDF format. The text is "frozen", and will not wrap to fit a small page.  Many of these PDFs are actually scanned as images, making them very inflexible. You can view a page "fit to screen" (very small) or full size (hard to navigate). So I have been converting  the Handbook of Downdraft Gasifiers Engine Systems to a more "e-friendly" format. This is a government publication in the public domain.

After looking around, I discovered that the process of digitizing an old book consists of three major steps.

1) Take a snapshot of the physical page, using a scanner or similar device. This can be exported directly for "snapshot" PDFs.

2) "Read" the text on the page using Optical Character Recognition (OCR), which leaves you with an unformatted mass of misspelled text - but text nonetheless. All images, tables and equations must be skipped by the OCR, as they only produce garbled nonsense. 

3) The text must be spell-checked (mostly by hand), edited back to original formatting as much as possible, and the images and tables added back in as graphics. This is the human element, and this is why so many older texts are only available as PDF snapshots. Once formatted properly, it can be saved out to an e-book format like epub or mobi.

It is a lot of work, especially if you are not in the business of doing it. I have found some free tools to help with the job. First is Adobe Reader. Most versions will export to a text file. The only catch is, there has to be some text to export. "Snapshot" PDFs are only images, no text. So instead, we need FreeOCR, reportedly the heart of the Google Books OCR software. It is plain and unadorned; you see a panel with the PDF page, draw a box around the text area, and hit "Convert". Depending on the amount of text, each chunk takes about a minute to process. Then you see in the other panel, the plain text output. Copy this to a Word document, then clear the cache and do some more. I did a 140 page book in about 2 hours. Then you have a massive .doc file. Books this size are difficult to work on with a slow PC like mine, so breaking it into chapters makes sense. 

Then I decided to move the whole project out into the open, so that more eyes can help spot the typos. I created a wiki called DoggyBook, and the whole text is posted on various pages there. I will continue to edit it online as I can, and maybe interested folks will join me and speed this thing along. Periodically I will compile the whole thing as a .doc file and post it for download on the front page. Eventually, all the typos will get smoothed out, and images will be inserted in the right places.

This is only worth the effort for a special book. In the area of biomass gasification, this is certainly one of the classic texts. In fact, it is still in print - the Biomass Energy Foundation will sell you a spiral bound photocopy for $35. But nobody has it available for e-readers yet. I aim to fix that.