Yandex.Disk¶
Two options are available to synchronize over a Yandex.Disk storage space.
The recommended approach consists of using the rclone-based backend provided in Escale. See the Synchronizing with rclone section.
Synchronizing with rclone¶
This section details how to use the partially-native client for Yandex.Disk. This approach has the disadvantage of requiring the rclone utility that in turn depends on the Go toolchain.
Installing the Go toolchain may add a significant amount of used space (like 160MB on Linux for example).
The backend in Escale that makes use of the rclone utility is referred to as a native backend because data are not buffered and all file transfers and accesses to your remote data are performed at call time.
Requirements¶
You can either install and set up rclone following this tutorial or let escale’s configuration wizard do it for you.
Note however that escale will not install the Go toolchain for you. Ensure that the go command is available:
$ go version
This should show the version number of your installed Go distribution. Otherwise please install Go.
You will also need a dedicated folder in your Yandex.Disk storage space. It will temporarily accommodate the files to be transfered and will permanently accommodate some meta files.
In this tutorial we make an Escale Repository folder at the root of the storage space.
Configuring Escale¶
If not already done, install Escale as detailled elsewhere:
$ pip install --user escale
or:
$ git clone https://github.com/francoislaurent/escale $ cd escale $ pip install --user -e .
Run the Escale configuration wizard in a terminal:
$ escale -i
You may run into a license acceptance step that requires to answer two yes-no questions. You must accept the terms of the license if you want to use Escale.
Do you have knowledge of the CeCILL-C license and accept its terms? [NO/yes] yes Acceptance stored in file: '$HOME/.config/escale/acceptance'
Note
User-supplied text is shown in bold characters.
Carriage returns are indicated by <enter>.
The configuration wizard offers an expert mode and a simplified mode.
The simplified mode is considered self-explanatory and most users may safely follow the steps of this mode.
This tutorial covers the expert mode.
Run in simplified mode: [Y/n] n <enter>
If you set Escale up for the first time, you will first be requested the path of the folder you want to synchronize:
Editing configuration file '$HOME/.config/escale/escale.conf' Path of your local repository (required): ~/Shared Files <enter>
where ~/Shared Files is the folder which content will be synchronized in this example.
If the directory you specify does not exist, Escale will make it for you.
Answer yandex://remote/Escale Repository to the second question,
where remote is the remote name as defined for rclone
and Escale Repository is the name of the folder that will accommodate the relay repository in your Yandex.Disk space:
Is the relay repository locally mounted in the file system? [N/y] <enter> Request help with '?' Path of the locally accessible relay repository (required): yandex://remote/Escale Repository <enter>
You will have to choose a name for the synchronized repository and a client name.
The repository name - or configuration section name - should be different from a local repository to another, on your local host.
The client name should be different from a client to another, for a given relay repository. The default client name is the hostname of your local host.
Choose a name for this configuration section. Prefer ascii names. Section name (required): my-first-repository <enter> Request help with '?' Client name: [poincare] <enter>
If you want to encrypt the data before they are sent to the relay storage, you will need to share a same encryption key between all the clients that synchronize over the same relay repository.
The first client to be set up can generate a key. This key will be available in a file that can be copied into the other clients.
The first client would perform the following steps:
Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>
'$HOME/.config/escale/my-first-repository.passphrase' file does not exist yet
Generate a new key? [Y/n] <enter>
key: b'_0Uozrh6S6RJ0ybOlJvZWreGFl6NffMUbU8xGvZ5QJw='
while the others would get a copy of the generated file - here called x.passphrase -
and perform the following steps:
Encryption: [on] <enter> Passphrase filename [my-first-repository.passphrase]: <enter>
In the above example, the my-first-repository.passphrase file is expected to be in the current directory or the configuration directory.
You can alternatively specify a filepath.
Note
A passphrase file is not a text file.
Especially, the passphrase stored in it should not be followed by a newline or carriage return character, otherwise these trailing characters would be included in the passphrase.
Important
Passphrases are private and should not be sent in plain emails. Consider encrypted emails or services like onetimesecret.com.
Pay special attention not to introduce newline or carriage return while editing a passphrase file.
You are then requested to specify in which mode synchronize should operate:
Synchronization mode can be 'upload', 'download', 'shared' or 'conservative' Request help with '?' Which mode for this client? [shared] <enter>
More information about synchronization modes can be found in this page.
Especially, if you want to implement a simple backup setup, you may have two clients:
- the one that accommodates the active data will be set in upload mode,
- the other one that accommodates the backup copy will be set in download mode.
The next few steps are straightforward:
Refresh interval (in seconds): [10] <enter> Request help with '?' Disk quota: [] 1 <enter>
Setting disk quota to 1 will make Escale use no more than one gigabyte in your Dropbox space.
Note that larger files will not be transferred due to insufficient space, as long as Escale does not feature automatic split-and-recombine mechanics. This value should be large enough so that your files can be piped through. It can be arbitrarily large but will usually be smaller that the available storage space.
The wizard will now assist you in installing the rclone utility, if missing.
If you don't have 'rclone' installed, leave it empty: RClone binary: <enter> The 'rclone' Go package is going to be installed. Do you want to continue? [Y/n] <enter> Cannot find the 'GOPATH' environment variable. Where do you want Go packages to be installed? [~/golang] <enter> go get -u github.com/ncw/rclone ... 'rclone' installed. Running 'rclone config' See also: https://rclone.org/yandex/ ...
From the last ellipsis begins the output of the rclone config command.
As instructed, please follow the steps described in the tutorial for Yandex Disk.
Exit the configuration wizard:
Do you want to add/edit another section? [N/y] <enter>
Your client is ready and can be launched with:
$ escalectl start
You can make your terminal continuously flush the logs with:
$ tail -f ~/.config/escale/escale.log
Beware that the escalectl stop command does not work properly on Windows. It actually kills all the Python processes instead of stopping Escale only.
Attention
You may have to empty the trash from time to time for escale to work smoothly.
Synchronizing with the native WebDAV backend¶
Attention
The native WebDAV client no longer works properly with Yandex.Disk. Please favor the rclone backend.
Escale can operate with the Yandex.Disk WebDAV server at webdav.yandex.ru.
If not already done, install Escale as detailled elsewhere:
$ pip install --user escale
or:
$ git clone https://github.com/francoislaurent/escale $ cd escale $ pip install --user -e .
Run the Escale configuration wizard in a terminal:
$ escale -i
You may run into a license acceptance step that requires to answer two yes-no questions. You must accept the terms of the license if you want to use Escale.
Do you have knowledge of the CeCILL-C license and accept its terms? [NO/yes] yes Acceptance stored in file: '$HOME/.config/escale/acceptance'
Note
User-supplied text is shown in bold characters.
Carriage returns are indicated by <enter>.
The configuration wizard offers an expert mode and a simplified mode.
The simplified mode is considered self-explanatory and most users may safely follow the steps of this mode.
This tutorial covers the expert mode.
Run in simplified mode: [Y/n] n <enter>
If you set Escale up for the first time, you will first be requested the path of the folder you want to synchronize:
Editing configuration file '$HOME/.config/escale/escale.conf' Path of your local repository (required): ~/Shared Files <enter>
where ~/Shared Files is the folder which content will be synchronized in this example.
If the directory you specify does not exist, Escale will make it for you.
You have to choose a name for the directory that will accommodate the relay repository.
Here we choose Escale Repository as a name:
Is the relay repository locally mounted in the file system? [N/y] <enter> Request help with '?' Address of the relay repository (required): https://webdav.yandex.ru/Escale Repository <enter>
Note that you will need to manually make this directory, for example from the web interface to your Yandex.Disk space.
You will have to choose a name for the synchronized repository and a client name.
The repository name - or configuration section name - should be different from a local repository to another, on your local host.
The client name should be different from a client to another, for a given relay repository. The default client name is the hostname of your local host.
Choose a name for this configuration section. Prefer ascii names. Section name (required): my-first-repository <enter> Request help with '?' Client name: [poincare] <enter>
The remote service may request a username and password.
If the credentials are available in a file, leave the following field empty. Authentification username: [] my-username <enter> Password: [] <type password> <enter> Writing new credential file '$HOME/.config/escale/my-first-repository.credential'
Important
Passwords and credential files are private and should not be sent in plain emails. Consider encrypted emails or services like onetimesecret.com.
If you want to encrypt the data before they are sent to the relay storage, you will need to share a same encryption key between all the clients that synchronize over the same relay repository.
The first client to be set up can generate a key. This key will be available in a file that can be copied into the other clients.
The first client would perform the following steps:
Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>
'$HOME/.config/escale/my-first-repository.passphrase' file does not exist yet
Generate a new key? [Y/n] <enter>
key: b'_0Uozrh6S6RJ0ybOlJvZWreGFl6NffMUbU8xGvZ5QJw='
while the others would get a copy of the generated file - here called x.passphrase -
and perform the following steps:
Encryption: [on] <enter> Passphrase filename [my-first-repository.passphrase]: <enter>
In the above example, the my-first-repository.passphrase file is expected to be in the current directory or the configuration directory.
You can alternatively specify a filepath.
Note
A passphrase file is not a text file.
Especially, the passphrase stored in it should not be followed by a newline or carriage return character, otherwise these trailing characters would be included in the passphrase.
Important
Passphrases are private and should not be sent in plain emails. Consider encrypted emails or services like onetimesecret.com.
Pay special attention not to introduce newline or carriage return while editing a passphrase file.
You are then requested to specify in which mode synchronize should operate:
Synchronization mode can be 'upload', 'download', 'shared' or 'conservative' Request help with '?' Which mode for this client? [shared] <enter>
More information about synchronization modes can be found in this page.
Especially, if you want to implement a simple backup setup, you may have two clients:
- the one that accommodates the active data will be set in upload mode,
- the other one that accommodates the backup copy will be set in download mode.
The next few steps are straightforward:
Refresh interval (in seconds): [10] <enter> Request help with '?' Disk quota: [] 1 <enter>
Setting disk quota to 1 will make Escale use no more than one gigabyte in your Dropbox space.
Note that larger files will not be transferred due to insufficient space, as long as Escale does not feature automatic split-and-recombine mechanics. This value should be large enough so that your files can be piped through. It can be arbitrarily large but will usually be smaller that the available storage space.
Exit the configuration wizard:
Do you want to add/edit another section? [N/y] <enter>
Your client is ready and can be launched with:
$ escalectl start
You can make your terminal continuously flush the logs with:
$ tail -f ~/.config/escale/escale.log
Beware that the escalectl stop command does not work properly on Windows. It actually kills all the Python processes instead of stopping Escale only.