Skip to content

Commit

Permalink
fix(guide): Fix broken links in the hardlinks index page (#2178)
Browse files Browse the repository at this point in the history
  • Loading branch information
TRaSH- authored Dec 24, 2024
1 parent 41b5b34 commit 0f67f42
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 35 deletions.
16 changes: 8 additions & 8 deletions docs/File-and-Folder-Structure/Check-if-hardlinks-are-working.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ You've followed the guide step by step but still want to check if hardlinks are

You can use 3 options to check if you got working hardlinks.

_All methods require you to log in to your terminal with PuTTY or similar software._
_All methods require logging in to your terminal with PuTTY or similar software._

!!! warning

Expand All @@ -16,15 +16,15 @@ _All methods require you to log in to your terminal with PuTTY or similar softwa

## Usenet

!!! info "If you are using Usenet, these examples won't work due to making use of instant/atomic moves and not hardlinks.<br><br>Still want to test if it works?<br><br>Test an import of a 4k remux, or any other big file, and you should notice it's almost instant and not a slower and more I/O intensive copy + delete."
!!! info "If you use Usenet, these examples won't work because you use instant/atomic moves and not hardlinks.<br><br>Still want to test if it works?<br><br>Test an import of a 4k Remux or any other big file, and you should notice it's almost instant and not a slower and more I/O intensive copy + delete."

## Mac or Linux Method 1: Using ls

This is the easiest to check in our opinion.
This is the easiest to check, in our opinion.

In your terminal `cd` to your download location and run `ls -al` or type `ls -al /path/to/your/download/location/`
In your terminal, `cd` to your download location and run `ls -al` or type `ls -al /path/to/your/download/location/`

You will get a listing of all your files and on the left side you will see a couple of numbers, every file with a number above 1 are hardlinks.
You will get a listing of all your files and on the left side you will see a couple of numbers, every file with a number above 1 is hardlinked.

![!Hardlinks check ls -al](images/hardlinks-ls-al.png)

Expand All @@ -38,8 +38,8 @@ You will get a listing of all your files and on the left side you will see a cou

This way requires a bit more work.

- In the terminal type: `stat /path/to/your/download/location/file.mkv`
- In the terminal type: `stat /path/to/your/media/location/file.mkv`
- In the terminal, type: `stat /path/to/your/download/location/file.mkv`
- In the terminal, type: `stat /path/to/your/media/location/file.mkv`

You will get 2 results you can use to compare several things.

Expand All @@ -52,7 +52,7 @@ You will get 2 results you can use to compare several things.

## Mac or Linux Method 3: Using Inode copies

- In your terminal `cd` to your download location and run `ls -i file.mkv` or type `ls -i /path/to/your/download/location/file.mkv`
- In your terminal, `cd` to your download location and run `ls -i file.mkv` or type `ls -i /path/to/your/download/location/file.mkv`

This will reveal the inode of your file on the left side of the file.

Expand Down
14 changes: 7 additions & 7 deletions docs/File-and-Folder-Structure/Hardlinks-and-Instant-Moves.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,28 +27,28 @@ Then Continue to [How to set up](/File-and-Folder-Structure/How-to-set-up/) for

### Hardlinks Limitations

!!! danger "- You <u>CAN'T</u> create hardlinks for directories :bangbang:<br>- You <u>CAN'T</u> hardlink across separate file systems, partitions, volumes or mounts :bangbang:<br>- Some file systems, such as exFAT, are known not to support hardlinks and should be avoided (double-check if you are unsure!)"
!!! danger "- You <u>CAN'T</u> create hardlinks for directories :bangbang:<br>- You <u>CAN'T</u> hardlink across separate file systems, partitions, volumes or mounts :bangbang:<br>- Some file systems, such as exFAT, are known not to support hardlinks and should be avoided (double-check if you are unsure!)."

### What are Hardlinks

??? question "**What are hardlinks?** - [Click to show/hide]"

- [**Short answer**] Having a file in multiple locations without using double your storage space.
- [**Long answer**] Hardlinks are a way for a copy operation to be instant and not consume space for those additional copies.
- [**Long answer**] Hardlinks allow a copy operation to be instant and not consume space for those additional copies.

Every file is a hardlink: some metadata that points at blocks on the file system, **which is why they're restricted to the same file system**. There can be as many metadata files pointing at those blocks as needed and the blocks know how many links point to them. Once the blocks have 0 links, they're considered deleted.
Every file is a hardlink: some metadata that points at blocks on the file system, **which is why they're restricted to the same file system**. There can be as many metadata files pointing at those blocks as needed, and the blocks know how many links point to them. Once the blocks have 0 links, they're considered deleted.

This means:

- You can delete any "copy" without impacting other instances (hardlinks) of it.
- You can delete any "copy" without impacting other instances (hardlinks).

Your download client can remove its "copy" without impacting the library "copy". The library "copy" can also be removed by Plex, Sonarr/Radarr, or manually without impacting your download client's "copy".

- Space is only reclaimed once all "copies" of data that are hardlinked are deleted.
- Space is only reclaimed once all "copies" of hardlinked data are deleted.

- Modifying any copy of a hardlinked file will impact all "copies".

For example, modifying the id3 tags of a .mp3 download after import would modify the download client "copy", resulting in breaking the torrent.
For example, modifying the id3 tags of a .mp3 download after import would change the download client "copy", breaking the torrent.

[More info from Wikipedia, the free encyclopedia](https://en.wikipedia.org/wiki/Hard_link){:target="_blank" rel="noopener noreferrer"}

Expand All @@ -58,7 +58,7 @@ Then Continue to [How to set up](/File-and-Folder-Structure/How-to-set-up/) for

??? question "**What are Instant Moves (Atomic Moves)?** - [Click to show/hide]"

A real move and not a copy file from the download folder to the media folder and then deleting the file from the download folder.
This is a real move, not a copying of a file from the download folder to the media folder and then deleting the file from the download folder.

### What are the Starr Apps

Expand Down
4 changes: 2 additions & 2 deletions docs/File-and-Folder-Structure/How-to-set-up/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How To Set Up

In this section, you will find several how-to guides for installation of the Starr Apps.
This section will show several how-to guides for installing the Starr Apps.
This is all done in a way to achieve [Hardlinks and Instant Moves (Atomic-Moves)](/File-and-Folder-Structure/Hardlinks-and-Instant-Moves/).

Select your desired method of installation:
Expand All @@ -10,7 +10,7 @@ Select your desired method of installation:
- [Native](/File-and-Folder-Structure/How-to-set-up/Native/)
- [Synology](/File-and-Folder-Structure/How-to-set-up/Synology/)
- [TrueNAS Core](/File-and-Folder-Structure/How-to-set-up/TrueNAS-Core/)
- TrueNAS Scale users can follow the same concepts as the Docker method, until someone provides a guide and is willing to maintain and give support for it in the Guide Discord.
- TrueNAS Scale users can follow the same concepts as the Docker method until someone provides a guide and is willing to maintain and support it in the Guide Discord.
- [Unraid](/File-and-Folder-Structure/How-to-set-up/Unraid/)
- [Windows](/File-and-Folder-Structure/How-to-set-up/Windows/)

Expand Down
20 changes: 10 additions & 10 deletions docs/File-and-Folder-Structure/Replace-copies-with-hardlinks.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Replace copies with hardlinks

Have you recently switched to a setup that supports hardlinks and Instant Moves (Atomic-Moves), and would like to replace duplicated files with hardlinks?
Have you recently switched to a setup that supports hardlinks and Instant Moves (Atomic-Moves) and would like to replace duplicated files with hardlinks?

If your Operating System supports it you could make use of [jdupes](https://codeberg.org/jbruchon/jdupes/releases).
If your operating system supports it, you can make use of [jdupes](https://codeberg.org/jbruchon/jdupes/releases).

The latest version's binaries are available for Windows at the link above. You can use a package manager, such as [homebrew](https://formulae.brew.sh/formula/jdupes), `apt`, or `pacman`, to install the latest available version for Mac or your flavor of Linux.

Expand All @@ -12,13 +12,13 @@ The latest version's binaries are available for Windows at the link above. You c

We won't cover every command :bangbang:

If you want to know what else [jdupes](https://codeberg.org/jbruchon/jdupes) can do please [read the usage manual](https://codeberg.org/jbruchon/jdupes#usage).
If you want to know what else [jdupes](https://codeberg.org/jbruchon/jdupes) can do, please [read the usage manual](https://codeberg.org/jbruchon/jdupes#usage).

!!! tip

This process can put a significantly large load on your system resources for an extended period of time depending on the size of your library. As duplicates are discovered, and hardlinks are made, the process becomes more efficient - as linked and different-sized files are not repeatedly checked against eachother. This leads to subsequent runs potentially finishing faster.
This process can put a significantly large load on your system resources for an extended time, depending on the size of your library. As duplicates are discovered and hardlinks are made, the process becomes more efficient - as linked and different-sized files are not repeatedly checked against each other. This leads to subsequent runs potentially finishing faster.

To speed this process up _significantly_ more, you can leverage a hash database. This will store information about the files, including their signatures, across runs of `jdupes` - vastly increasing the speed at which runs are finished.
You can leverage a hash database to speed this process up _significantly_. This will store information about the files, including their signatures, across runs of `jdupes` - vastly increasing the speed at which runs are finished.

Simply use the following additional option _BEFORE YOUR DIRECTORIES_ with a path that is always available and persistent:

Expand All @@ -33,7 +33,7 @@ jdupes [options] DIR1 DIR2
```

The example below will do a dry run and summarize at the end.
!!! info "Folder paths should be adjusted to match your directory structure"
!!! info "Folder paths should be adjusted to match your directory structure."

=== "Without Hash Database"

Expand All @@ -49,8 +49,8 @@ The example below will do a dry run and summarize at the end.

---

The example below will hard link all duplicate files without prompting.
!!! info "Folder paths should be adjusted to match your directory structure"
The example below will hardlink all duplicate files without prompting.
!!! info "Folder paths should be adjusted to match your directory structure."

=== "Without Hash Database"

Expand All @@ -66,8 +66,8 @@ The example below will hard link all duplicate files without prompting.

---

!!! bug "Windows only allows a maximum of 1023 hardlinks per file"
!!! bug "Windows only allows a maximum of 1023 hardlinks per file."

!!! Warning "The `-Q` or `--quick` option only reads each file once, hashes it, and performs comparisons based solely on the hashes. There is a small but significant risk of a hash collision which is the purpose of the failsafe byte-for-byte comparison that this option explicitly bypasses. Do not use it on ANY data set for which any amount of data loss is unacceptable. You have been warned!"
!!! Warning "The `-Q` or `--quick` option only reads each file once, hashes it, and performs comparisons based solely on the hashes. There is a small but significant risk of a hash collision which is the purpose of the failsafe byte-for-byte comparison that this option explicitly bypasses. Please do not use it on ANY data set for which any data loss is unacceptable. You have been warned!"

--8<-- "includes/support.md"
16 changes: 8 additions & 8 deletions docs/File-and-Folder-Structure/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@

## Introduction

It is important that your media server has a well organised file and folder structure. Along with generally easier file and folder management, you will benefit from:
It is important that your media server has a well-organized file and folder structure. Along with generally easier file and folder management, you will benefit from:

- Improved security, by only granting the appropriate applications and tools access to your files.
- [Hardlinks](/File-and-Folder-Structure/Hardlinks-and-instant-moves#what-are-hardlinks), so that the same file can appear in multiple places whilst only taking up the disk space of one copy.
- [Instant moves](/File-and-Folder-Structure/Hardlinks-and-instant-moves#what-are-instant-moves-atomic-moves) (also known as 'Atomic Moves') so that files can be moved to other parts of the file system instantaneously.
- Improved security is achieved by only granting the appropriate applications and tools access to your files.
- [Hardlinks](/File-and-Folder-Structure/Hardlinks-and-Instant-Moves/#what-are-hardlinks), so the same file can appear in multiple places while only taking up one copy's disk space.
- [Instant moves](/File-and-Folder-Structure/Hardlinks-and-Instant-Moves/#what-are-instant-moves-atomic-moves) (also known as 'Atomic Moves') so that files can be moved to other parts of the file system instantaneously.

The first requirement is that all your media files and folders should be part of the same file system. That is, everything must be contained on a single physical or virtual drive. Second, all of your applications should have a consistent view of where your files and folders are - i.e., your files should appear in the same place to all of your applications.
The first requirement is that all your media files and folders be in the same file system. Everything must be contained on a single physical or virtual drive. Second, all your applications should have a consistent view of where your files and folders are - i.e., your files should appear in the same place as all your applications.

We recommend setting up a file and folder structure on your host server that looks like this:

{! include-markdown "../../includes/file-and-folder-structure/docker-tree-full.md" !}

The `data` folder can be placed wherever you like. As an example, in Unraid, you would set up a share called `data`. This would then be accessible within Unraid's file system at `/mnt/user/data`.
The `data` folder can be placed wherever you like. For example, in Unraid, you would set up a share called `data`. This would then be accessible within Unraid's file system at `/mnt/user/data`.

If you are installing applications directly on the host system (natively), then they will already have visibility of that file and folder structure, assuming permissions are set correctly. If you are installing applications non-natively, for example via Docker, then each application should be granted access to the lowest level folder that is required whilst maintaining consistent pathing to the top level folder, which here is `data`. For example, a torrent client installed via Docker would have `/mnt/user/data/torrents` mapped to `/data/torrents`. This means that the download client would see the contents of the host's `/mnt/user/data/torrents` folder in the `/data/torrents` folder inside the container.
If you install applications directly on the host system (natively), they will already have visibility of that file and folder structure, assuming permissions are set correctly. Suppose you are installing applications non-natively, for example, via Docker. In that case, each application should be granted access to the required lowest-level folder while maintaining consistent pathing to the top-level folder, which here is `data`. For example, a torrent client installed via Docker would have `/mnt/user/data/torrents` mapped to `/data/torrents`. This means the download client would see the contents of the host's `/mnt/user/data/torrents` folder in the `/data/torrents` folder within the container.

More detailed examples can be found in the [How To Set Up](/File-and-Folder-Structure/How-to-set-up/) section.
The [How To Set Up](/File-and-Folder-Structure/How-to-set-up/) section provides more detailed examples.

--8<-- "includes/support.md"

0 comments on commit 0f67f42

Please sign in to comment.