Generating book lists from my Goodreads profile

If you read books you have perhaps heard about Goodreads.com, the site has book reviews, season awards, and more. The most interesting part is that you can track you read, currently reading and want to read books.

Yes, you could use a notepad or a spreadsheet 🤷‍♂️ but using the site is nice. You can review the books you read, rate them and so on.

🔒 Owning your data

They key thing is not to give your data away with risk of losing it. I want to create a backup / export of my data, should Goodreads eventually go away. Simple, don't lose your data 😉

🤓 RSS XML

The good folks at Goodreads are nice enough to offer RSS XML files for your profile ❤️ For each of the mentioned shelves (Currently Reading, Read, and Want to Read) 💯

We can use these 3 XML to store the output in .txt files. The text files can be used (regardless of the RSS XML availability) to generate a nice HTML output (should you want to showcase the lists in your site).

https://www.goodreads.com/review/list_rss/YOUR_PROFILE_ID?key=&shelf=currently-reading
https://www.goodreads.com/review/list_rss/YOUR_PROFILE_ID?key=&shelf=to-read
https://www.goodreads.com/review/list_rss/YOUR_PROFILE_ID?key=&shelf=readCode language: HTML, XML (xml)

📚 Fetching the RSS

While we wait for Node 17.5 to have native fetch() support we need to find a workaround. This nice NPM package node-fetch will provide what we need ✅

Additionally we need to parse de XML, arraybuffer-xml-parser is the other package we make use of ✅

With these two we can do something like:

fetch(url)
  .then(response => response.text())
  .then(data => {
    const xmlData = encoder.encode(data);
    const object = parse(xmlData);
    const items = object.rss.channel.item;

    items.forEach(item => {
      console.log(item.title)
    })
  });Code language: JavaScript (javascript)

🆕 Creating files

Ok, so we've fetched the RSS files and after parsing them we need to create a file. With Node is easy:

import fs from 'fs';

fs.writeFile(fileName, data, (err) => {
  if (!err) {
    console.log('File created: ' + fileName);
  }
});Code language: JavaScript (javascript)

The content of the .txt files look like this:

book_id|year|Book Name|URL
book_id|year|Book Name|URL
book_id|year|Book Name|URL

👓 Reading files

Reading your files with Node is trivial:

import fs from 'fs';

function readFile(shelve) {
  return fs.readFileSync(`./database/${shelve}.txt`, { encoding:'utf8', flag:'r'});
}Code language: JavaScript (javascript)

Reading the lines in the file

Once you read the file with fs you can get the lines with a simple split:

function getLines(content) {
  return content.split(/\r?\n/);
}Code language: JavaScript (javascript)

🔥 GitHub Actions

You know me, I'm a fan of Github Actions 🤩 We'll use this simple action to run our Node.js script and deploy the output to a different branch 🚀

The idea is to have our scripts create the files in a the public folder (or whatever name you want it to be!). Then after running the script copy that folder to another branch, for persistence, history, etc.

Create the file in your repo: .github/workflows/build.yml

name: Build and Deploy
on:
  push:
    branches:
      - main
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout 🛎️
        uses: actions/checkout@v2
        with:
            persist-credentials: false
      - name: Install and Build
        run: |
          npm ci
          npm run build
      - name: Deploy 🚀
        uses: JamesIves/github-pages-deploy-action@releases/v4
        with:
          BRANCH: output # The branch the action should deploy to.
          FOLDER: public # The folder the action should deploy.Code language: YAML (yaml)

🤔 One possible improvement could be to have the action do a commit to the main branch with the changes. Otherwise I have to manually copy the updated files from the output branch to the main branch. Small price to pay, though.

👨‍💻 Conclusion

Now you know how to easily fetch, write files to disk and read them with NodeJS.

The rest is up to you know to build your app. Make use of the free and powerful GitHub Actions and own your data!

😎 Live

See it live here ricard.blog/books

🍴 GitHub

Play with it, modify it, fork it!

github.com/quicoto/goodreads-profile-generator