❌

Reading view

There are new articles available, click to refresh the page.

Managing music with rclone

I've been listening to my music via Navidrome for a bit now and it's working quite well. To manage my music, I use rclone locally and on my host machine.

When I add new music, I update the tags in Mp3tag, save the art to the same directory as the audio files and maintain a directory structure of Artist/Release Year-Album-Name/Encoding format/Files. These files reside on a drive I've creatively named Storage at /Volumes/Storage/Music which is backed up to Backblaze B2 via Arq. My music bucket is mounted to the server running Navidrome and made available to Navidrome as the music volume for the container, for example:

services:
  navidrome:
    image: deluan/navidrome:latest
    user: 1000:1000 # should be owner of volumes
    ports:
      - "4533:4533"
    restart: unless-stopped
    environment:
      # Optional: put your config options customization here. Examples:
      # ND_LOGLEVEL: debug
    volumes:
      - "/path/to/data:/data"
      - "/mnt/music:/music:ro" # I'm a mount that has your music in it

The B2 mount is run as a systemd service:

[Unit]
Description=Mount B2 /Music with Rclone
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStartPre=/bin/sh -c 'if mountpoint -q /mnt/music; then fusermount -u /mnt/music; fi'
ExecStart=/usr/bin/rclone mount b2:cdransf-music /mnt/music \
    --allow-other \
    --async-read=true \
    --dir-cache-time=72h \
    --vfs-cache-mode full \
    --vfs-cache-max-size 50G \
    --vfs-read-chunk-size 512M \
    --vfs-read-chunk-size-limit 5G \
    --buffer-size 1G \
    --vfs-read-ahead 4G \
    --vfs-cache-poll-interval 5m \
    --tpslimit 10 \
    --tpslimit-burst 20 \
    --poll-interval 5m \
    --no-modtime
ExecStop=/bin/fusermount -uz /mnt/music
Restart=always
User=root
Group=root

[Install]
WantedBy=multi-user.target

We've got a description, directives telling the service when to start (e.g. when the network is up and that it wants the network up but not to fail if it's not).

A check to see if the mount is already mounted and unmounts it if so. And then, then, we start the mount.

--allow-other: let other users access the mount. --async-read: read asynchronously for better performance. --dir-cach-time=72h: cache the directory structure for 3 days. --vfs-cache-mode full: allows full file caching and helps with seeking and access. --vfs-cache-max-size 50G: a generous cache but safely within the limits of the host machine. --vfs-read-chunk-size 512M: initial chunk read size. --vfs-read-chunk-size-limit 5G: max chunk read size. --buffer-size 1G: the amount of data to buffer in memory per file. --vfs-read-ahead 4G: how far to read ahead for streaming. --vs-cache-poll-interval 5m: how often to check the cache and clean it up. --tpslimit 10 and --tpslimit-burst 20: request throttling so things don't get out of hand. --poll-interval 5m: check B2 every 5 minutes for changes. --no-modtime: disable modtime support because B2 doesn't support it particularly well.

Finally, we unmount when the service stops and restart the service automatically (handy when I reboot the host).

The final lines run the service as root and ensure the service is started at system boot.


Uploading music

To upload music from my local machine to B2 I also use rclone via a function sourced into my zsh config. This β€” again β€” creatively titled upload_music function does a few things:

local src="$1" is the path to the local directory I want to upload. I typically run this using upload_music . after navigating to the directory that contains the files I'm uploading.

base is the hardcoded root directory for my stored music.

Next, if I pass an empty or invalid directory, the script errors out with a message explaining how to run it properly.

We use realpath to resolve our $src and $base paths, allowing us to determine that the former is within the latter (and erroring out if not). We then compute the relative path and use all of the above to upload the music to my B2 bucket, preserving a directory structure that mirrors what I have locally.[1]

# upload music to navidrome b2 bucket
upload_music() {
    local src="$1"
    local base="/Volumes/Storage/Media/Music"

    if [[ -z "$src" || ! -d "$src" ]]; then
        echo "Usage: upload_music <local_directory>"
        return 1
    fi

    # resolve absolute paths
    src="$(realpath "$src")"
    base="$(realpath "$base")"

    if [[ "$src" != "$base"* ]]; then
        echo "Error: '$src' is not inside base directory '$base'"
        return 1
    fi

    # compute relative path
    local rel_path="${src#$base/}"

    echo "Syncing '$src' to 'b2:cdransf-music/$rel_path'..."
    rclone sync "$src" "b2:cdransf-music/$rel_path" --progress --transfers=8 --checkers=16 --bwlimit=off
}

Finally, now that the music has been uploaded, the service is defined and running, I'll run a script from the host machine to prompt the whole thing to read in the new music:

#!/bin/bash

echo "Restarting the rclone mount..."
systemctl restart rclone-b2-music.service

echo "Restarting the Navidrome container..."

# find container name running from the navidrome image
navidrome_container=$(docker ps -a --filter "ancestor=deluan/navidrome:latest" --format "{{.Names}}" | head -n 1)

if [[ -n "$navidrome_container" ]]; then
    docker restart "$navidrome_container"
    echo "Navidrome container '$navidrome_container' restarted."
else
    echo "Error: Could not find a container using the image 'deluan/navidrome:latest'."
fi

echo "New music should now be available in Navidrome!"

We restart the mount to invalidate the cache and restart the Navidrome container so that it recognizes that the mount has been restarted. Once Navidrome restarts (which it does quickly) it kicks off a new scan and the music is available when the scan concludes.


  1. One of the nice parts about using rclone for this is that it will only update files after the initial upload. If I change metadata (or anything else minor) the subsequent upload run is extremely quick. β†©οΈŽ

❌