rclone and ScaleWay Storage as backup strategy

Exploring a few changes to how, and where, I host my own data, I revisited my backup strategy.

For years I’ve been using SpiderOak ONE and my main complain was the pricing slide. At 500GB backed up, I can’t be at the 400GB plan, so end up forking over US$150 a year for a 2TB plan.

I probably don’t need those 500GB, as I know from their UI that it once backed up some useless data like logs and save games. The tool did a great job on data de-duplication, only counting once for every copy of the same file across multiple devices.

Sitting and figuring out if I could find over 100GB of data to lower my plan to 400GB demands more time than I’d like to afford, so it kept there for years. Until now, it became a more pressing problem.

As I double checked backups, in order to safely move away from cloud providers storing my data (blog post coming soon), SpiderOak wasn’t properly running on my laptop. This meant, for most of the time, my main data wasn’t being backed up at all!

Now the opportunity cost of this equation favoured exploring moving away from SpiderOak, together with expanding my home server’s storage to support the move.

I can’t find a definitive source for the quote “Put all your eggs in the same basket and watch that basket closely” but this methodology tends to work for many things. For my data placement strategy, the model is evolving towards:

  • mobile: NextCloud client, auto-sync for Documents, upload-only camera roll
  • laptop: NextCloud client, auto-sync for Documents
  • home server: NextCloud server, storing data on an NVMe device on ZFS
  • to be decided if on NextCloud: email, calendar, contacts

For backups, I did investigate whether https://www.tarsnap.com/ was my way to go. Colin knows what he is doing, and putting more money towards a FreeBSD-first company is worth my attention, but I was still uncomfortable with with US$ 0.25 / GB-month. Keeping in mind the original number of 500 GB, that’s US$125 a month, or twelve times what I currently pay a year.

The majority of these 500 GB are the camera roll: pictures and films of family and friends we recorded over the years. Cleaning and tidying up these to removed blurred photos is, again, another thing hard to find the time to do. But the question then remains, how can I still keep them safe on the 3-2-1 rule?

Close friends have all mentioned they push it to a cloud storage glacier tier. Why am I not doing the same?

As of this week, I am.

I created a new account at ScaleWay, the French cloud aspirant. Creating the account was trivial, not much different than what AWS asks: your email, a password, and our credit card.

They have a decent Terraform support, had no surprises in getting credentials from the website and start recording elements on code. As with AWS, navigating directly into code isn’t as efficient as it sounds, so I adopted a “ClickOps first” approach.

Created a bucket in the Paris, France region, created an Application with rights to write to this bucket only, and configured rclone according to Scaleway’s own instructions. The instructions are a little bit out-of-date but not a showstopper.

I am impressed with how easy to operate rclone is. After creating a “scaleway” (type s3) backend, setting storage_class = GLACIER, then linking a new “secret” backend (type = crypt), I was ready to upload data.

rclone copy --progress --s3-chunk-size=20M /local/data secret:/encrypted/data

In fact, I did just that: took about 9 hours to upload 276 GB of encrypted photos. Funny enough to see that it also encrypts the filename on the Storage Bucket, giving an extra degree indirection on metadata analysis.

A friend then quoted a sage, “backing up is easy, restoring is hard”.

rclone, which proposes itself to be “The Swiss-army of cloud storage”, does not disappoint here. It really shows it can deliver:

  • Creating some local data
cd /tmp
mkdir -p local/data
sysctl hw > local/data/sysctl-hw.txt
  • Copy from local to Scaleway, encrypted
rclone copy --progress local secret:backup-of-local

Outputs:

Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         1.2s
  • Check how the directories appear on the Scaleway bucket itself. Notice how I don’t even bother installing AWS CLI S3 or Scaleway’s CLI tool as rclone can do that for me.
rclone lsf scaleway:felipe-my-bucket

Outputs:

2nltok1e55r9hpocr4l77d6k00/

The directory structure is mirrored but all filenames are encrypted based on the passphrase I set up my “secret” backend earlier on.

Querying via the “secret” backend, the magic gives me the original filenames:

rclone ls secret:backup-of-local

Shows:

    17808 data/sysctl-hw.txt

Even on failure, rcopy did a good job. Scaleway doesn’t allow download from Glacier:

rclone copy --progress secret:backup-of-local restore-local

The wall of errors clearly shown that… (I’ve redacted part of the filenames just in case).

2025/03/27 19:05:54 ERROR : data/sysctl-hw.txt: Failed to copy: failed to open source object: Object in GLACIER, restore first: bucket="felipe-my-bucket", key="2nltok1e55r9hpocr4l77d6k00/7guc526cxxxxxxxxx4squel9ko/4bj8xxxxxxxxxxga5h3u1lh014"

A quick search shown that rclone backend restore was the way forward:

rclone backend restore scaleway:felipe-my-bucket/2nltok1e55r9hpocr4l77d6k00/7guc526cxxxxxxxxx4squel9ko/4bj8xxxxxxxxxxga5h3u1lh014

But that doesn’t work, because…

2025/03/27 19:10:26 NOTICE: Failed to backend: is a file not a directory

So, turns out that we can only remove entire folders from glacier at a time. I suppose it’s good to know this early on, as it is a critical decision when having to do restores in the future. Curiously enough, the Scaleway Web UI only allows me to restore a single file, not directories.

Unfortunately, I haven’t yet found the way to get the IAM Permissions correct to allow my Application to restore from Glacier:

[
        {
                "Status": "operation error S3: RestoreObject, https response error StatusCode: 403, RequestID: txg270f8a03xxxxxxxxbfc2-0067e5a73e, HostID: txg270f8a03xxxxxxxxxfc2-0067e5a73e, api error AccessDenied: Access Denied",
                "Remote": "4bj8xxxxxxxxxxga5h3u1lh014"
        }
]

It took about 17 hours from the request (19:33 UTC) to restore from glacier until it eventually got moved to Standard class (12:33 UTC, next day). Confirmed with a regular script checking the metadata:

while true ; do date ; rclone  lsjson scaleway:felipe-my-bucket/2nltok1e55r9hpocr4l77d6k00/7guc526cxxxxxxxxx4squel9ko; sleep 60 ; done | tee -a glacier-restore.txt

After this time, the rclone copy from secret: to a local folder completed immediately.

The 17 hours to restore is aligned to their marketing, “you are willing to accept 24 to 48 hour latencies to first byte”, and highlights why the 3-2-1 policy is crucial: having the chance to just connect a spare HDD on USB and copy the files directly will beat Glacier-to-Standard time and also transfer time from France to home. Plus cost of restoring (€ 0.009/GB) and egress fees (€0.01/GB).

Overall, I’m satisfied with the offering, and looking forward for potential cost saving of just € 0.002 GB/month on Scaleway’s Glacier offering, which for my 500GB should be at €1/month.

Later, I intend to investigate the backup of other parts of the system, say files with more frequent updates and system configuration, using restic, which is a close relative to rclone.