Sun 13 October 2024 | -- (permalink)

The following is a lightly-edited version of the rant I composed after a volunteer group I work with decided that the reason our volunteers weren't writing documentation for most of our projects was that using a text editor is just too hard, and that all would magically become well if we started over on Google Drive. I have doubts, but I arrived late to that meeting (airline having a bad day), and by the time I walked in it was clear that fighting this would be a long and thankless upwind battle into a large and fragrant cow pasture.

The original title of this piece was considerably less polite.

How to use Emacs with Google Drive as the Goddess Intended

All this point and drool stuff making you crazy? Trying to be a good sport about it for the good of the team but you miss Emacs? Want all those old Wiki files we painfully converted from Trac to Markdown available in the brave new world? Just want to keep your existing workflow, or want to be able to stash a copy of the whole freaking doc tree on your laptop so you can read the documentation on how to fix the network when when the network is broken and you're trying to read the documentation to figure out how to fix it?

Fear not, Gentle Reader, you've come to the right place. Let me introduce you to our friend rclone.

Interacting with Google Drive: rclone is your best friend

rclone is basically rsync for weird filesystems, clouds, and other annoyances. It has a driver for Google Drive, and among its other capabilities it knows how to convert (or ask Google Drive to convert, same difference from the user standpoint) between whatever GUI abomination Google is using internally and various other formats, including .pdf and .docx. By default, when exporting "normal" documents from Google Drive, rclone will convert them to .docx, but if you just want to read the things, you can ask for .pdf, or ask for OpenOffice formats, all kinds of fun.

Before using rclone on a Google Drive, you need to set it up. Basically, run rclone config and follow the nice instructions. I call mine gdrive: to tell it apart from other rclone remotes I use to track other projects that have gone over to the dark side.

Once you've done that, copying the entire shared Google Drive with PDF conversion can be as simple as:

#!/bin/sh -
rclone \
    --drive-export-formats pdf \
    --drive-skip-dangling-shortcuts \
    --verbose \
    --progress \
    --fast-list \
    --copy-links \
    sync \
    gdrive: \
    $HOME/gdrive

Authoring documents in Markdown: rclone again

From the above, you may suspect that conversion also works in the other direction, and indeed it does. Turns out the rclone drive documentation is a little out of date as of this writing, and md works fine for both import and export:

So upload may be as simple as writing your document as Markdown in Emacs as the Goddess intended, then doing:

rclone -vP \
    --drive-export-formats md \
    --drive-import-formats md \
    copyto \
    mydoc.md \
    gdrive:mydoc.md

You can of course also download in Markdown:

rclone -vP \
    --drive-export-formats md \
    --drive-import-formats md \
    copyto \
    gdrive:mydoc.md \
    mydoc.md

Alert readers may wonder why we're specifying both import and export formats in both directions. Glad you asked: beats me. It just seems to work better this way. There's probably some complex reason that makes internal sense to Google, if you figure it out tell me and we'll both know.. You might want to experiment to figure out what works best for you.

The rclone documentation notes that rclone never performs conversion on upload by default (that is: the default value of --drive-input-formats is the empty string), because conversion is an inherently lossy process. Fair enough.

rclone is a bit more willing to convert on download (default value of --drive-export-formats is not empty), but if you use this in the hope of generating documents in an editable format you'll want to be careful, since a simple format like Markdown will obviously lose all the pretty fonts and pictures somebody else may have spent time on. Conversion of editable formats between toolsets is a fraught process anyway, with or without rclone or Google Drive. Conversion between document formats is like the old numerical analysis warning about repeated operations on floating point numbers: every time you convert a document, you pick up a little dirt.

Mirroring everything locally as PDF is harmless, and PDF is a fairly good format if you just want to view and search the content. You should probably limit conversion between editable formats to your own documents, or documents where you and your collaborators have an agreement on how you're going to do this.

Bonus tool: pandoc is your other friend

While it's not always needed here, given rclone's format conversion capabilities, no discussion of document format conversions would be complete without at least a brief mention of pandoc, the Swiss Army Chainsaw of the document format conversion world. If the conversion tools built into Google Drive and rclone let you down, you might give pandoc a try.

How tasty is this dogfood?

This document was prepared in Markdown with Emacs. When I uploaded it to Google Drive using the procedure above, the Markdown conversion did a tolerable job on basic text, but sucked caterpillar snot when it came to converting code blocks. No idea why.

Using pandoc to convert from Markdown to .docx worked somewhat better. Not all that surprising, given that the main purpose of the whole tool suite of which Google Drive is part is to help Google eat Microsoft's lunch, so the .docx conversion support is probably better than the support for most other formats. OpenOffice .odt didn't work at all for me. Again, no idea why.