| 1 | --- |
| 2 | title: "NAS Part I - The Build & Borgbackup Initial Impressions" |
| 3 | date: 2023-05-15T20:40:00+03:00 |
| 4 | description: "Almost a year later, the rumours about untimely death of my NAS build have been greatly exaggarated" |
| 5 | tags: [backup, linux, review, opensource, opinion, nas] |
| 6 | type: blog |
| 7 | draft: false |
| 8 | --- |
| 9 | |
| 10 | # NAS - Part I |
| 11 | |
| 12 | If you have been reading since last year July, you may know of my "Project ReuseNAS" - you may also |
| 13 | know that my build date estimate for "end of 2022" or "Q1 2023" was never met as per obvious lack of posts on that front. |
| 14 | |
| 15 | When I was originally planning the NAS build, I was very much all over the idea, that I could reuse all of my old PC components. |
| 16 | Over time after discussing with members of the based.quest community, we came to the conclusion that I would be severly handicapping |
| 17 | myself with the hardware I already had laying around. |
| 18 | |
| 19 | I was unable to find a satisfactory case for the NAS build that could hold the HDDs on them - apparently cases with 4x 3.5" + 2x 2.5" bays are quite uncommon, |
| 20 | as such I ended up buying a brand new case. |
| 21 | |
| 22 | Further I did not trust getting an used power supply or used SSD/HDD drives. CPU cooler I most likely could have found used, but the project was starting to drag |
| 23 | on for far too long - so I ended up getting a brand new one. I know, impatient. |
| 24 | |
| 25 | My original plan was also to buy 4x4TB IronWolf Pro drives - as of right now, I only got 2x4TB IronWolf (non-Pro) drives. |
| 26 | |
| 27 | The final rig ended up being: |
| 28 | |
| 29 | CPU: i5-2400 (Second-hand)\ |
| 30 | CPU cooling: be quiet! Pure Rock 2 (Brand new)\ |
| 31 | Motherboard: MSI H61MA-E35(b3) (Second-hand)\ |
| 32 | RAM: 4x4GB DDR3 from my old "Celery Fujitsu" PC (Second-hand)\ |
| 33 | PSU: Corsair CV550 (Brand new)\ |
| 34 | Case: Krux Naos TG (Brand new)\ |
| 35 | HDD: 2x4TB Seagate IronWolf (Brand new)\ |
| 36 | Boot SSD: 2x120GB Patriot (Brand new) |
| 37 | |
| 38 | As of writing this, a few of these are still being shipped, but that didn't stop me from starting the experimentation, using my current PC as the test-bench. |
| 39 | |
| 40 | ## The borgbackup |
| 41 | One simple Friday evening, as I had finished getting final parts of the NAS - I asked the matrix chat for suggestions regarding backup solutions I could use. |
| 42 | My criteria was simple - it must be able to do incremental backups and potentially deduplicate. I was thinking of some hacky rsync solution already until |
| 43 | Kerum stepped in with the idea of using [borgbackup](https://www.borgbackup.org) - I was intrigued, it offers space efficient storage of backups, LZ4 compression, |
| 44 | mountable backups with FUSE and claims to be easy to setup. |
| 45 | |
| 46 | Without wasting further time - I decided to take my 2x4TB drives out, attach them to my main PC, build a software RAID 1 and install borgbackup. |
| 47 | What I discovered was everything I could have asked for from a free solution and beyond. |
| 48 | |
| 49 | Let's start with the extras I wasn't expecting for -- the backups are all encrypted. If your HDDs get stolen for whatever reason, your backups are secure. |
| 50 | Further you are secure from malicious actors - you do NOT need borg on the remote site even, you do not need to store any passwords on the repository server. |
| 51 | |
| 52 | About performance - your initial pass is always going to be slower as borgbackup has to make note of blocks for deduplication and also encrypt the data. |
| 53 | However, this is why I wanted the ability to do incremental backups - you only have to move what you need. Your next passes however, will be a lot quicker - depending on the |
| 54 | amount of inodes in your server. For me, an incremental backup takes about ~ minutes presently. Here's a summary from the last backup I just took (presently manually, going full-automatic |
| 55 | once the NAS final components have shipped!). |
| 56 | |
| 57 | ``` |
| 58 | Mon 15 May 2023 07:10:29 PM CEST Starting backup |
| 59 | |
| 60 | /path/to/<redacted>.log: file changed while we backed it up.log |
| 61 | ------------------------------------------------------------------------------ |
| 62 | Repository: ssh://<user>@<nas>:<port>/path/to/borg-repo |
| 63 | Archive name: cernodile-2023-05-15T19:10:29 |
| 64 | Archive fingerprint: 42cc287287f3c2f9d34308844ad3f18b15f53333d9eeb3e9b71ad1c89c33b063 |
| 65 | Time (start): Mon, 2023-05-15 19:10:30 |
| 66 | Time (end): Mon, 2023-05-15 19:16:53 |
| 67 | Duration: 6 minutes 22.94 seconds |
| 68 | Number of files: 741792 |
| 69 | Utilization of max. archive size: 0% |
| 70 | ------------------------------------------------------------------------------ |
| 71 | Original size Compressed size Deduplicated size |
| 72 | This archive: 197.53 GB 149.67 GB 1.58 GB |
| 73 | All archives: 971.53 GB 754.30 GB 151.90 GB |
| 74 | |
| 75 | Unique chunks Total chunks |
| 76 | Chunk index: 530571 4076474 |
| 77 | ------------------------------------------------------------------------------ |
| 78 | terminating with warning status, rc 1 |
| 79 | |
| 80 | Mon 15 May 2023 07:16:59 PM CEST Pruning repository |
| 81 | |
| 82 | Keeping archive (rule: daily #1): cernodile-2023-05-15T19:10:29 Mon, 2023-05-15 19:10:30 [42cc287287f3c2f9d34308844ad3f18b15f53333d9eeb3e9b71ad1c89c33b063] |
| 83 | Pruning archive (1/1): cernodile-2023-05-15T06:15:29 Mon, 2023-05-15 06:15:31 [4266d525a871b036cb71f0ba0f8930e1dcd04499fa5293deba87305497fe957d] |
| 84 | Keeping archive (rule: daily #2): cernodile-2023-05-14T01:08:46 Sun, 2023-05-14 01:08:48 [0dc46a3106d1df91170fb60d0661a1b6ca92fc50694acfa8833745dd302d1fd6] |
| 85 | Keeping archive (rule: daily #3): cernodile-2023-05-13T09:09:41 Sat, 2023-05-13 09:09:41 [f0be5704c565202b405ac833e1256fd320ce0968946bfabfd3312d3a1aa3afc2] |
| 86 | Keeping archive (rule: daily[oldest] #4): cernodile-2023-05-13T00:51:50 Sat, 2023-05-13 00:51:51 [14be5350fa55603c65e897c5a6f37925943b7c5eed793db386d5090464d6a18a] |
| 87 | terminating with success status, rc 0 |
| 88 | |
| 89 | Mon 15 May 2023 07:17:18 PM CEST Compacting repository |
| 90 | |
| 91 | |
| 92 | Mon 15 May 2023 07:17:34 PM CEST Backup, Prune, and/or Compact finished |
| 93 | ``` |
| 94 | |
| 95 | I seem to face a visual bug regarding the warning status (no actual warning is output!), but other than that, I've had no real issues so far - I have verified integrity of the backups (each one!), all match up. |
| 96 | |
| 97 | Note the original and compressed sizes here. The original backed up data (aka actual FS usage) is 197 GB which gets compressed down to 149.67GB! With 3 other archives visible here, you can see that the incrementals have only added up about 3GB (which roughly aligns with the daily amount of data added too ~1GB/day). |
| 98 | |
| 99 | ## FUSE the backups! |
| 100 | As mentioned before, borgbackup supports mounting your backups with FUSE provided you have the space for it. Let's say I accidentally make changes I didn't really intend to today and I didn't make a backup of it. |
| 101 | So I want to naturally take my nginx.conf back from the previous day. |
| 102 | |
| 103 | To do that, I go back to my PC terminal and check for my existing restore points with `borg list /path/to/repo`: |
| 104 | ``` |
| 105 | $ borg list /path/to/repo: |
| 106 | Enter passphrase for key /path/to/repo: |
| 107 | cernodile-2023-05-13T00:51:50 Sat, 2023-05-13 01:51:51 [14be5350fa55603c65e897c5a6f37925943b7c5eed793db386d5090464d6a18a] |
| 108 | cernodile-2023-05-13T09:09:41 Sat, 2023-05-13 10:09:41 [f0be5704c565202b405ac833e1256fd320ce0968946bfabfd3312d3a1aa3afc2] |
| 109 | cernodile-2023-05-14T01:08:46 Sun, 2023-05-14 02:08:48 [0dc46a3106d1df91170fb60d0661a1b6ca92fc50694acfa8833745dd302d1fd6] |
| 110 | cernodile-2023-05-15T19:10:29 Mon, 2023-05-15 20:10:30 [42cc287287f3c2f9d34308844ad3f18b15f53333d9eeb3e9b71ad1c89c33b063] |
| 111 | ``` |
| 112 | |
| 113 | As I said before, I want to see /etc/nginx from previous day, in this case, May 14th. To do that, I'll have to use `borg mount` as such: |
| 114 | ``` |
| 115 | $ borg mount /path/to/repo/::cernodile-2023-05-14T01:08:46 /mnt/ /etc/nginx |
| 116 | Enter passphrase for key /path/to/repo: |
| 117 | $ cd /mnt/ |
| 118 | $ ls -lh etc/nginx/nginx.conf |
| 119 | -rw-r--r-- 1 root root 1.5K 26. apr 23:33 etc/nginx/nginx.conf |
| 120 | ``` |
| 121 | |
| 122 | Voilah! We have retrieved the configuration file that was present that day! Now we can retrieve the contents of it using whichever preferred utility you wish or transport it directly |
| 123 | to the server with scp/ftp/sftp. |
| 124 | Once you're done, run `borg umount /mnt`. |
| 125 | |
| 126 | ## Future concepts to look into |
| 127 | I want to see if it's possible to make my backups immutable in event of cernodile.com or based.quest being hacked. |
| 128 | |
| 129 | ## Conclusion |
| 130 | Overall my first impressions so far are very positive, there is so much I haven't discovered yet with this backup solution and it is everything I could have asked for. |
| 131 | For my use case, this is ideal. The backup script I use is available in borgbackup's own documentation. I recommend you read up on that as well if you are as interested |
| 132 | in borgbackup as I am. |
| 133 | |
| 134 | I am glad to finally have my NAS and backup solution set up, it's been a long time coming. |
| 135 | |
| 136 | Thank you so much for reading, hopefully without this long of a delay next time, |
| 137 | - Cernodile |