POSTS

Optimal Mastodon Tools

Blog

I believe understanding this technical setup is best assisted by understanding how I see the design of the Mastodon network. Those ideas are covered in a previous post

I Don’t Like Mastodon’s River; Introducing Newsraft

Since Mastodon federates micro-blog posts from across the universe of collaborating nodes, I think the ideal interface for subscribed-to Mastodon content is a big “inbox” that’s created when some code pulls your messages from all those nodes to your computer. Then, the code should help you swiftly tear through your unread content: mark important, mark read, mark all read, peek at an image, etc.

I think it should look something like this:

The explore menu in Newsraft is like a federated inbox of all followed content. My unread posts are yellow on blue, read content reverts to a standard high-contrast play gray on black, and important posts (for further review) are marked purple. All of these colors are configurable as is the view and the date format.

The explore menu in Newsraft is like a federated inbox of all followed content. My unread posts are yellow on blue, read content reverts to a standard high-contrast play gray on black, and important posts (for further review) are marked purple. All of these colors are configurable as is the view and the date format.

The “infinite scroll” interface of Facebook or Twitter was designed to keep punters suckers users insecure so that they came back often like a cocaine-addicted lab rat tapping a button for another pellet. Despite the cognitive and productivity flaws this design brings about, those sites' ubiquity has coached many to believe “this is what social networking as to feel like,” and Mastodon has emulated it, warts and all.

But this interface fails to show off what can be so glorious about Mastodon: you can get done with “doing social networking” on Mastodon without fear of missing out. Content lands in a bucket and you can peruse it or mark it read or important on your schedule. The software that lets me have such a calm, cool experience is Newsraft.

The software I wanted is:

  1. Text-based, therefore fast, like my beloved tin news reader from yesteryear
    tin in all its lightweight glory

    tin in all its lightweight glory

  2. Flexible: let me organize, but also give me a single virtual inbox through which I can traverse quickly, like my beloved mutt mail reader from yesteryear
  3. Maintainable by Unix tools – I don’t want to have to log into something, I want to run it as an application on my computer

Solution: Newsraft + RSS feeds of my followed content

Fact 1: Mastodon automatically publishes every user’s posts as an RSS feed

I didn’t know about this, but my “node-mate” @linuxgal@techhub.social clued me in. All Mastodon users’ posts by default are published to an RSS feed. RSS is a simple and well-understood technology for publishing updates. RSS makes many news tickers or regularly updated media releases (e.g. podcasts) on the internet work.

Thus for any Mastodon person, you can get an RSS feed of their content at https://<home-node/@<id>. For someone on another server you simply take their ID of @someid@someOther.server and turn it into https://someOther.server/@someid. In fact, in the Appendix code sample I provide below, I’ve written that exact logic in JavaScript.

It’s important to note that an RSS feed is meant to be read by a machine. Humans would find the feed’s text dull and confusing. So an RSS reader takes the machine-friendly information and makes it friendly to the human eye. Some applications might go for a full native application experience, some others might go for a more-minimal, text-oriented approach. I prefer the latter.

Armed with a list of RSS publication URLs, all you need is an RSS reader to process these machine-friendly feeds of Mastodon friends’ posts into something humans can process.

Newsraft is just such an application.

Fact 2: Newsraft is a text-based console RSS reader program

newsraft is built explicitly to honor the design of the console-based mutt mail reader.(goal 1, ✅) Longtime readers of the site will know that I ❤️ mutt (1, 2, 3, 4).

Additionally, from a programming perspective, it’s designed to be small with a minimum of outside dependencies. It’s just a plain old C program that’s been carefully written and which adheres to the Unix model of simplicity and configuration by text files.

As a result, all one has to do is take the aforementioned list of RSS feeds, put it in a file called $HOME/.newsraft/feeds and violà, Newsraft is ready to start presenting your feeds instead of forcing you to rely on a web application.

Newsraft lets you organize your feeds into a hierarchical view of sections if that’s comfortable for you:

Default entrypoint to Newsraft, your feeds in grouped subsections

Default entrypoint to Newsraft, your feeds in grouped subsections

Want one big “unread” bucket instead? Newsraft supports that. It’s called the “Explore” menu and it’s one keystroke (e) away and it is awesome. It’s my favorite interface in Newsraft and is what I’ve been using for screenshots in this post:

The aggregated inbox; color scheme is configurable

The aggregated inbox; color scheme is configurable

It’s worth noting that configuring colors, date format, display, key bindings is just a question of editing a textfile, as we expect from a Unix utility. Simply edit a file called $HOME/newsraft/config. While the defaults are sensible, you can have it your way. Obviously goal 3 was also covered ✅. I’ll post my config at the end of this post.

Technical note about local caching mechanism: sqlite3 A great improvement since the `mutt` era is that `sqlite3` has become a lightweight database that's present everywhere. So instead of the fetched content being stored in some strange flat file archive they're stored by [Newsraft] in an SQL compatible database. To wit:
➜  ~ sqlite3 ~/.newsboat/cache.db "SELECT DISTINCT COUNT(*) AS subscribed_feeds
FROM rss_feed"
27

While I most often work in the Explore menu, let me show you how you might work with hierarchically categorized feeds.

To view a post, step into the provider’s mailbox:

Posts for the account Minimus, a Latin-speaking mouse

Posts for the account Minimus, a Latin-speaking mouse

Then Return on the post and get a text preview:

Post for Minimus about pot-making

Post for Minimus about pot-making

In the post, the text-oriented reader puts a numbered lists of external assets and images as “footnotes” at the end of the post. This allows me to focus on the content first.

I can then strike 2,i and launch a local preview of the image listed by footnote 2 in my iTerm (“What, display an image in the terminal? How? See below!”).

Minimus at his craft; quick-render

Minimus at his craft; quick-render

Hitting Return returns me back to the listing of Minimus’ posts.

I can optionally hit o to open in browser if I decide the post needs to be seen in Mastodon’s web interface (perhaps I want to reply).

Minimus at his craft; full

Minimus at his craft; full

The majority of Mastodon posts are not just captions around an image so this particular workflow isn’t the default. Consider this post from Anne Applebaum, journalist who adds some commentary and then links a relevant URL at some other site:

Covering Taiwan + US / China relations

Covering Taiwan + US / China relations

Here I see Applebaum’s observation and I also see where the article is linked. If I strike 2,o, Newsraft will open link 2 in my browser and I’ll hit the Wall Street Journal’s paywall.

News Corp&rsquo;s civilized façade, the Wall Street Journal, and its paywall

News Corp’s civilized façade, the Wall Street Journal, and its paywall

Conclusion

Unlike Twitter, whose content is worse, with the use of a dedicated reader and Mastodon like an RSS distribution hub, you can actually “read it all” (provided you subscribe judiciously).

I’m still keeping the Mastodon iOS application on my phone for idle time, but I’m going to try to avoid it and prefer to consciously “Do Mastodon” with Newsraft.

Lastly, the developer of Newsraft has been very open to feedback and has quickly implemented all of my [feature requests]! I’ve been collaborating with Newsraft’s author about how to improve the app and he’s shipped shipped a number of my requests. I’m sure we’re iterating to an ever-better approach.

Appendix 1: Code for Generating RSS URLs

Expand for instructions and code
  1. Visit your Mastodon home page: https:///home. Mine is https://techhub.social/home
  2. Swap home for relationships
  3. You’ll be given a table of your relationships
  4. Use the following code to extract RSS endpoints for the individuals listed:
    1. Open the JavaScript console (command+option+i on Mac on Firefox)
const localInstance = "techhub.social"; // Change to be your local instance;
const displayNameToRSSURL = (x) => {
  const xs = x.split("@");
  const onMyLocal = xs.length === 2;
  return (onMyLocal ? `https://${localInstance}/${x}` : `https://${xs[2]}/@${xs[1]}`) + '.rss'
}
alert(
  [...document.querySelectorAll(".account .display-name span")]
    .map( a => a.textContent)
    .map(displayNameToRSSURL)
    .join("\n");
);
  1. Copy and paste the code above into the console. You should generally be wary of people asking you to do this. But I’m a good guy.
  2. An alert box should pop up with all the names in a copy-and-paste-able format
  3. Sadly, this code is brittle and probably won’t work for the long term, but it shows the general algorithm for generating a list of your RSS feeds

Appendix 2: Displaying Images in iTerm

Add image display into iTerm and integrate with Newsraft
  1. First acquire imgcat and put it in your PATH. I put mine in ~/bin
  2. Verify you can imgcat an image on your machine by imgcat-ing a file to make sure things work. This is a custom extension to iTerm
  3. Let’s tell Newsraft to pass selected URLs or filepaths to imgcat when we invoke a command called i on it. This configuration will be explained below
  4. View images with happiness in our hearts

In Newsraft, we specify custom behavior, custom key-bindings, custom colorization inside a file called ~/.newsraft/config. Here’s the relevant line:

bind i exec clear; imgcat --url "%l"; read

Per the man page of Newsraft, bind i sets i as a command key. The command it runs is to execute a screen clear; then to run imgcat by passing along the link variable %l; then it does a shell read to wait for input (i.e. strike enter to continue) before returning to Newsraft

Appendix 3: My Newsraft Configuration

My Newsraft configuration with colors; shell out to imgcat etc.
# Set up "<link number>i" to pipe the link to `imgcat`
bind i exec clear; imgcat --url "%l"; read

# Above clobbers 'i' as mark important; restore it
bind z mark-important
bind ! mark-important
bind - mark-unimportant

# Colors
# Section listing
set color-list-section-unread-fg "cyan bold"
set color-list-section-fg "cyan"

# Within the feeds comprising a section
set color-list-feed-fg "cyan"
set color-list-feed-unread-fg "cyan bold"

# Individual posts
set color-list-item-fg "white"
set color-list-item-bg "black"
set color-list-item-unread-fg "yellow bold"
set color-list-item-unread-bg "blue"