eBooks: How to Free and Consolidate your Digital Library

Image Credit: ceslava from Flickr

First off, I apologize for the amount of time it took me to get this information into a post.  3 months is WAY too long to be working on something off and one… That said, for the past few months, I have been writing a continual series regarding eReaders and digital books. The benefit of the portability and easy access to your library is offset by Digital Rights Management and different formats. These two factors alone can cause a person to have eBooks

The most frustrating aspects of digital books is the digital rights management (DRM) that is typically attached to the book file itself to prevent the mass distribution of this copyrighted material. Not only are there different eBook formats that publishers use, there are different forms of DRM that get attached to these books. Additionally, because the publishing industry overall is in a state of turmoil two new concerns have arisen. The first is, if I purchase an eBook, will the publisher still be around in the future for me to get my eBook back incase something happens to my reader. The second is, as eReaders continue to advance (Kindle, iPad, Galaxy Tab, Xoom, Playbook, and the host of unannounced Android based readers that are sure to come) and new products are brought  to the market, what happens when I choose to switch readers and the new one uses a different format?!

Especially, when highlighting the first concern is the fact that Borders (Borders and WaldenBook Stores) is in the process of filing for bankruptcy.

Borders is preparing for a costly and time-consuming trip through bankruptcy court, where it will seek to close about a third of its 674 Borders and Waldenbooks stores, the people familiar with the matter said. Borders also would cut swathes of its 19,500 staff as it attempts to reinvent itself to compete with Amazon and its hot-selling Kindle reader, and Barnes & NobleInc., the nation’s largest bookstore chain and maker of the Nook e-reader. …

…Whether it can restructure and emerge as a stand-alone company is unclear. Many Wall Street bankers and lawyers who have studied the chain believe it may not be able to avoid liquidation.

With the possibility of liquidation even mentioned, this fuels the need for readers to be able to take full control of their ever increasing digital library. What remains in the way has been  a way to accomplish this relatively easily. The remainder of this post will be dedicated to helping accomplish that task.

After playing with different tools for months, loading “python scripts”, reading blogs, looking at DeDRM tools, and generally going through many hoops to convert my personal digital library from, Microsoft Reader, MobiPocket, Fictionwise, Barnes and Noble (original and Nook formats), Kindle, and Borders formats. I tripped across an amazing tool called Calibre. Calibre is an eBook library management application that work on Windows, OS X, and Linux operating systems. Here is a review I wrote about this software back in November. At that time, Calibre could convert DRM-free formats, however, recently Calibre and a host of other frustrated by digital rights management  have come together to create “plug-ins” which will strip away most DRM form the various formats in use by publishers and eBook distributors like: Kindle, Barnes and Noble, Borders, and more.

Some may be concerned about the difficulty of getting the system and the plug-ins setup. My opinion is that there are two slightly more challenging steps. I will work to make these instructions as easy as possible. I also want to give credit where credit is due. much of the following is based off of this post on Apprentice Alf’s Blog. He did a great job laying out the steps I am building upon on in this post.

The first thing you need to do is download and install Calibre onto your computer. (Note: Calibre is updated often so check back occasionally for the latest update.)

Here is a like to Calibre’s download page. Choose the appropriate download for your system:

(Note: I have linked to the download page itself as opposed to the file in case in future updates, the file name changes.)

When going through the Calibre install, the install “Wizard” will help to walk you through installing Calible.  This will include a choice of the format you would want Calibre to convert all eBooks you bring into the system into. If you chose one format (say Kindle) and then upgrade to a different reader (say the iPad) you can re-run this wizard from the Preferences area.

Once you have installed Calibre, you need to download the combined tools package which include the DeDRM resources that will allow Calibre to remove eBook DRM as it pulls your eBook into its library.

Download the Combined Tools Package.

Click here to access the Combined Tools download Page

(Note: I have again linked to the download page itself instead of a file to prevent linking to old information as updates are made.)

You will want to click on link called http://www.datafilehost.com/download-385939ba.html on that page to move to the download.  That link will take you to the actual file you need to download. Unfortunately, the file host service they are using confuses things a bit with a big ad so here is an image for just what you need to click on:

Download file image - Just ignore the "Free Download" of PDF Creator

Download file image - Just ignore the "Free Download" of PDF Creator

After you download the Combined Tools Package you will need to “unzip” it. this is different on different systems. Mac will automatically unzip the folder containing the files. On a Windows PC, you should be offered the choice of where you want to “save” the file. Choose your desktop for easy access. Once the file is downloaded, right-click on the Tools.zip file and choose “Extract All” to get the files you need uncompressed.

While there will be a number of files and folders, the only one you are interested in is Calibre_Plugin:

Combined Tools Package Folder

The only folder you are interested in is Calibre Plugin

Now that you have Calibre installed and Combined Tools Package unzipped, you need to install the plugins from the Tools into Calibre:

Running Calibre and Installing the DeDRM Plugins

First Run Calibre and then click on Preferences in the upper right corner:

Calibre Screen - Preferences

Now Scroll down and click “Plugins”

Calibre - Preferences

Click on “Add a new Plugin” to install the 5 DeDRM plugins:

Calibre - Add a Plugin

Navigate to the Tools>Calibre Plugins folder that you downloaded and put onto your desktop earlier. Choose one of the .zip files and click “Open”.  This will begin the Calibre install process for that plugin:

Calibre - Plugin Install

Once you click “open” you will receive a warning dialogue about “Security risk” possible when installing plugins, go ahead and say “Yes” you wish to proceed.

Then you should receive a “Success” dialogue box regarding the plugin you selected to install, along with a message about having to restart Calibre for the plugin to be able to function. Click “OK” but do not re-start Calibre yet. Wait until you have installed each of the 5 DeDRM Plugins and then just restart the one time.

Repeat the above process for the remaining 4 DeDRM Plugins (there are 5 total at this time). After you install the 5th plugin, close and restart Calibre.

That was not so bad right? Now comes the hard part, configuring the DeDRM with YOUR specific information which will allow the plugins to actually strip away the Digital Rights Management and allow Calibre to convert your newly freed eBooks to be opened in the reader of your choice. (My preference is Apple’s iBooks.)

Configuring the DeDRM Plugins

Now, depending on the type of eBooks you have purchased, depends on which plugin you need to configure. and each has its own specific needs.  I will work to be as specific as possible here, but you might experience some trial and error…

To configure the plugins, go back into Preferences and scroll down to Advanced and click on Plugins and look for File Type Plugins

Calibre - File Type Plugins

Click on the “>” and all of the plugins will be displayed. Below is a list of what the plugins cover and the information you need to configure each with to enable the DeDRM for that type.

Choose the plugin you wish to configure. Click on it once to highlight it blue, and then click on Customize Plugin. This will popup a window for you to enter the DRM Unlock information. Enter the required information and click OK.

  • Barnes and Noble (including Fictionwise.com purchases). Here is where things are the most complicated in my opinion. BN uses its own proprietary version of ePub DRM, and I have found that they are ALSO selling Nook capable versions of titles that they acquired through the purchase of fictionwise.com that leverage a different version of ePub DRM. Both are highlighted in the screen shot below. While the second mainly only impacts older fiction titles this can cause frustration if you do not configure two different plugins.

Calibre - Configure BN Plugins

    • eReader PDB 2 PML is the BN eReader/ fictionwise based plugin. If you have eReader ebooks from Barnes and Noble or Fictionwise you must enter your name and the last 8 digits of your credit card number into the customisation field. This is the Name you have on file at your ebook retailer’s website. For me this is both my first and last name, and the last 8 digits of my credit card number separated by a comma.
      • example: John Smith,12345678
    • lgnoble ePub DeDRM is the BN / Nook Secured based plugin. While this information may be similar be sure you have it correct. Here you will need to enter your full name and the full credit card number. If you are unsure what this is, you can get the name from your Nook Library page (either on your nook or on the BN Nook Library website.
      • example: John Smith,1234567891234567
  • Kindle and MobiPocket eBooks use the mobi format which leverages a serial number or a PID (Personal Identification) that resides with the device itself.

Calibre - Kindle Customize Kindle DeDRM Plugin

    • K4PC,K4Mac, Kindle Mobi and Topaz DeDRM is the Kindle based plugin. Kindle and MobiPocket eBooks use the mobi format.
    • The easiest way to DeDRM Kindle books is by downloading the Kindle for PC or Kindle for Mac applications and then bring your book into the appropriate app for your system. After that simple locate it on your hard drive and drag it into Calibre.  If you do this, you will not need to “customize” the Kindle Plugin at all.
    • However, if you have Amazon Kindle ebooks that were downloaded to your Kindle, you must enter your Kindle’s serial number into the customisation field of the K4MobiDeDRM plugin. If you have already entered a PID there, add the Kindle serial number as well, separating it from the PID with a comma. Again you do not need to do this if you first download the eBook into Kindle for PC or Kindle for Mac.
      • example: 1234567891234567
  • Adobe Digital Editions – If you have either ePub or PDF formatted books in ADE, you do not need to put anything into the customization field for the plugins
  • If you have ePub books from Apple iBooks, there is currently no way to strip the DRM from the eBook.  Apple is using a technology called Apple FairPlay and this format of Digital Rights Management has not been broken at the time of this writing.
  • If you have an eBook that uses Microsoft LTE, there is NO Calibre plugin to remove the DRM.

Once you have customized the plugins appropriate, click “Apply” and “Close” and you are ready to remove the DRM from your ebooks by importing them into Calibre.  The easiest way I have found to accomplish this is to open Calibre and simply drag and drop them into the main area of Calibre.

Calibre - Main Screen

Now all you need to do is download your books to your computer and then drag and drop them into Calibre.  However, if you already have ebooks on your computer, locate your books on the hard drive. Here is a great snip from the Apprentice Alf Blog on this:

Q: Where are my ebooks?
Your ebooks are stored on your computer or on your ebook reader. You need to find them to be able to remove the DRM. If they are on your reader, you should be able to locate them easily. On your computer it’s not so obvious. Here are the default locations.

Macintosh
Navigating from your home folder,
Kindle books are in either Library/Application Support/Kindle/My Kindle Content or Documents/My Kindle Content, depending on your version of Kindle for Mac.
Adobe Digital Editions ebooks are in Documents/Digital Editions

Windows
Navigating from your My Documents folder
Kindle books are in My Kindle Content
Adobe Digital Editions ebooks are in My Digital Editions
Mobipocket Reader ebooks are in My eBooks

A final thought.  Calibre has an eReader built into the application itself.  After you import an eBook that needed DRM stripped off, test it by double-clicking to launch the book.  If all went well, the book should open without any issue.  If something happened and the DRM was not stripped off, you will get an error relating to this fact. Please be sure to be patient as part of this process. If for some reason the DRM does not get stripped off delete the eBook from Calibre, and then you need to look back at how you customized the plugin. There must be something missing (or incorrect) in the information the plugin needs. (This is how I discovered the two variants for Barnes and Noble DRM). Correct the information and try importing the eBook again.

I hope that this helps.  If you have questions please feel free to comment.

Advertisements

28 thoughts on “eBooks: How to Free and Consolidate your Digital Library

  1. Pingback: eBooks: How to Free and Consolidate your Digital Library … | digipreneur.co.cc

  2. Pingback: World Spinner

  3. Pingback: Easily open Barnes and Noble eBooks in Apple’s iBooks [updated] « VanishingPoint

  4. Pingback: Opening “other” eBooks in the Apple iBooks Reader « VanishingPoint

  5. Pingback: Calibre – eBook Management and Conversion Tool « VanishingPoint

  6. Fantastic and deeply appreciated! Thank you for your kind service. Finally I can consolidate my ebooks and use an ONE elegant reader. What a difference.

  7. Pingback: Free eBooks on the iPad « VanishingPoint

  8. I installed the plug-ins in Calibre & downloaded Kindle for PC however Calibre will not convert them, I get a warning that they are locked by DRM. Any thoughts? Is there an update or something that I am missing?
    L

    • Sorry for the delay, it has been a miserable couple of weeks here… I have a bunch of questions to try and help. You mentioned downloading Kindle for the PC. So, from there I am going to assume that you have a kindle book that you are reading. Do you also have a Kindle reader? You downloaded the book to the PC as well correct? I know a silly question, but I need to ask. Did you open it in the kindle reader in the PC at all? I don’t remember if I had to do that or not, but it can’t hurt.

      From there you need to use My Computer and go to the folder that contains the book. Now open Calibre in a window and arrange it so you can see both Calibre and the file explorer window where the ebook is. Then click an hold the mouse button to drag and drop the book into the Calibre window. This should copy the book into the Calibre library and strip the DRM. From there you just need to run the conversion to build the ePub version that should work on the iPad.

      If you do have a Kindle reader, (I don’t) there should be a settings menu. On there should be the serial number of the Kindle. Yu can also try putting this in the Kindle DeDRM plugin and see if that helps. But even if you do that, you will still need to re-drag and drop the book into Calibre because it only strips the DRM during the import. Let me know if this doesn’t work, and maybe we can arrange a time to have a Skype call or something. Also, if is doesn’t the more information you can provide with how you are doing things, or what is happening when you try the better I can see how to help.

      Scott

  9. I too (like Laurie above me) followed all the steps in your post but am getting an error saying they are locked by DRM. I downloaded them from barnes and noble and they are epub files. I followed the instructions for customizing the plug-ins that are applicable for Barnes and Noble ebooks. WHat could I be doing wrong? Any help would be appreciated. Thanks! And thanks for the guide!

    • Alex,

      B and N was really the hardest one for me to configure as well. This is because of the two different types of DRM they are using because of the purchase of Fictionwise. (either the last 8 of the credit card used to by the book, or the name in the Nook Library on their site) The biggest problem is that you never know which one it is… So here are a couple things that you can try. First, login to the BN site by signing in. Then click on my account in the upper right. From there scroll down to the “My Nook Library” link and click on it. This will show you the last 4 of the credit card that is registered to the account, and the “Name” you have on the account.

      Here are a couple of screen shots that might help. First, if you click on the little ? Next to the words unlock code it provides this message: http://gallery.me.com/scottweidig#100004/Nook%20Library

      Verify the credit card you are using there, and be sure that you have correctly entered the last 8 into the DeDRM plugin to calibre. You can also see the name you have on the account on this screen too:
      http://gallery.me.com/scottweidig#100004/Nook%20Library%202

      Be sure that you have both in correctly separated by a comma. Also, if you use a first name and last name, be sure you put the space between the two names.

      Another idea is to only put one in the plugin. Say the last 8 of the credit card. Then try dragging and dropping the book into Calibre and see if it strips the DRM. If it doesn’t, remove that one and try the account name. Then try dragging and dropping into calibre again. Remember, it is ket that after you make changes you re-drag and drop the book into the library because calibre will only strip the DRM when it is importing the book.

      Let me know if this helps, or what issues you run into. The more information the better. Finally, I am willing to make the same Skype call offer is this does not work for you. Sometimes it is hard to diagnose things text.

      Scott

  10. Scott. Thanks so much for the reply. So…the credit card on the account is not the same as the credit card I used to buy the books (there were fraudulent charges on that card so the bank closed it and sent me a new one). Do I need to use the card on the account? Or the card that I bought the books with? Also. The name on the card is Alexan En……., but the name on the account is alex En……. Which name should I use? I’ll try doing it one plug in at a time. Thanks again for all your help! Sincerely, alex

    • Alex,

      Check the account at the BN site. Again this will be weird in a way… if the account number (you can see the last 4 digits next to your name on the My Nook Library page) is the OLD one, that will be the one you need to use for the book you have already downloaded… What it is looking for really is the card number that it was purchased with… That said, however, there is the possibility that if you update the credit card on file with the BN site and then re-download the book to your computer the NEW credit card information will come this that download… I am not positive about this, but there is a good chance that it will. Also, the your new card would be linked to BN for future purchases…

      Regarding the name, use what is on the account not the card. So, use alex En…… <- make sure it is spelled just the way you have it on the account with capitalizations… i.e. if it is "alex En……" on the account use that. if it is "Alex En…." use that if it is "alex en……." use that.

      Trying it one at a time is best to troubleshoot. Just remember you need to make the change in Calibre then re-import the book… if it doesn't work, delete the book from Calibre and then make the change and then re-import the book… repeat until it works. Once the DRM is stripped from the book, it will stay stripped so if you need to change the information in the plugin to say the new credit card number for future purchases it won't matter for that book you already converted.

  11. Hi Scott. I just wanted to say thanks for all your help. I wasn’t able to get the books to convert using Calibre….but I used a script that I found online that worked. So everything’s great now. Thank you again for all your help! God bless you! Sincerely, alex

    • Alex, was it a python script that worked? If so, what were the credentials you used for it? Those should be the same that would work in Calibre. Also, sorry that you couldn’t
      Get it to work, it really is a great conversion tool.

      S

  12. Scott, it was DeDrm 2.6 that came in the packet i downloaded. I just dragged the ebooks on it after configuring it and it did it’s thing. One reason why I think it worked is because I could see all the different variables that I was typing in…that why I made sure to include all possible combinations of name and credit card. I dunno, but I’m just glad it worked. Thanks for all your help!

  13. Thank you so much for this! I had purchased so many eBooks from Borders for my iPhone, but then my wife got me a Kindle for my birthday, and I definitely didn’t want to have to re-buy every book just because I got a new reader. You’re a lifesaver, sir!

  14. I have an iBooks ePub book. I understand at this time I cannot strip the drm from such a book. I mistakenly bought the wrong format and only have my Mac and iPhone. What can I do to read it on a screen bigger than my iPhones?

    • I apologize, but unfortunately at this time Apple has not put out a Mac based version of iBooks. So, unless you have an iPad (or access to an iPad) the iPhone is the only screen that you will be able to read this book from.

      That said, there are a few few things that you can do to make the reading experience on the iPhone a bit better. After you open your book, tap the aA (little A big A) button this will allow you to increase the size of the font to a more comfortable level. Additionally, you can adjust the actual font. You can try each to see what you like best, I have found Veranda easier on tired eyes as I read quite a bit at night on my iPhone (also, it hurts less than when I drop my iPad on my face if/when I drift off). I also turn off sepia to go with an exclusive B/W theme. While I find that B/W works better for daytime reading it is perfect for when I use the built-in accessibility feature “Turn Black and White on” which reverses the color scheme to white text on a black background. Having white text on a black background when I read at night in bed helps eliminate all of that glare from that stark white screen. Finally, you can tap the “sun” to lower the screen brightness, again to cut down on the harshness from the whites.

      If you did not know about the “reverse” screen colors, in iOS’s accessibility features, to access it you can simply triple-click the home button and it will bring up a choice for accessibility. Simply tap black and white. Do the same thing to turn it off. You can also get to these features from Settings>General> and scrolling down to Accessibility.

      I read A LOT at night in my iPhone. Almost all the books I read are on my iPhone while getting relaxed in bed so, the iPhone is perfect. It is light, I can turn pages with a simple tap or swipe of a single finger so I don’t have to use two hands to manage my book. To make it better I do use most of the options I list above: I increase the font size, flip to white text on black, and personally I use landscape mode to made the iPhone slightly easier to hold and turn pages with one hand.

      I know that this might not be the greatest answer, but hopefully with some of these options you can enjoy your ebook from your iPhone with a bit less frustration. I am not sure of apple will ever release a Mac version of iBooks. I know the screen seems small, but a font change may make a world of difference. I actually enjoy reading on the iPhone. It is light and easy to hold in one hand nd still turn pages when I am sleepy.

      Oh, and if you wear glasses as I do, try taking them off if you read on the iPhone at night. For my eyes, the font actually gets just a bit bigger without them on. I can’t do that with my iPad as it is actually harder for me to read with them off on it, but for the iPhone it works nicely.

    • Like an iBooks version for a kindle version? I don’t want to dash your hopes, but Apple has not been very open to something like that in the past.

      Sent from my iPhone

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s