Google Drive

Three options are available to synchronize over a Google Drive storage space.

  • you can either mount your Google Drive storage space in your local file system with an external tool (see the Mounting locally section; not recommended)
  • or you can use the Google Drive backend provided in Escale (see the Synchronizing with drive section)
  • or else you can use the rclone backend provided in Escale (see the Synchronizing with rclone section)

Mounting locally

The approach described in this section consists of mounting your Google Drive storage space as a folder accessible in your local file system.

Google offers for download its Drive app for macOS. macOS can therefore go for this solution.

However Linux users may fall back on the following open-source solutions:

You may find the following guide interesting.

Configuring Escale

If not already done, install Escale as detailled elsewhere:

$ pip install --user escale

or:

$ git clone https://github.com/francoislaurent/escale
$ cd escale
$ pip install --user -e .

Run the Escale configuration wizard in a terminal:

$ escale -i

You may run into a license acceptance step that requires to answer two yes-no questions. You must accept the terms of the license if you want to use Escale.

Do you have knowledge of the CeCILL-C license and accept its terms? [NO/yes] yes
Acceptance stored in file: '$HOME/.config/escale/acceptance'

Note

User-supplied text is shown in bold characters.

Carriage returns are indicated by <enter>.

The configuration wizard offers an expert mode and a simplified mode.

The simplified mode is considered self-explanatory and most users may safely follow the steps of this mode.

This tutorial covers the expert mode.

Run in simplified mode: [Y/n] n <enter>

If you set Escale up for the first time, you will first be requested the path of the folder you want to synchronize:

Editing configuration file '$HOME/.config/escale/escale.conf'
Path of your local repository (required): ~/Shared Files <enter>

where ~/Shared Files is the folder which content will be synchronized in this example.

If the directory you specify does not exist, Escale will make it for you.

Let’s now assume that your Google Drive is mounted on $HOME/GoogleDrive.

You need to create a folder in there so that Escale can temporary stored files and meta-information. Let’s call this folder Escale Repository.

Respectivelly answer y and ~/GoogleDrive/Escale Repository to the next two questions:

Is the relay repository locally mounted in the file system? [N/y] y <enter>
Request help with '?'
Path of the locally accessible relay repository (required): ~/GoogleDrive/Escale Repository <enter>

You will have to choose a name for the synchronized repository and a client name.

The repository name - or configuration section name - should be different from a local repository to another, on your local host.

The client name should be different from a client to another, for a given relay repository. The default client name is the hostname of your local host.

Choose a name for this configuration section.
Prefer ascii names.
Section name (required): my-first-repository <enter>
Request help with '?'
Client name: [poincare] <enter>

If you want to encrypt the data before they are sent to the relay storage, you will need to share a same encryption key between all the clients that synchronize over the same relay repository.

The first client to be set up can generate a key. This key will be available in a file that can be copied into the other clients.

The first client would perform the following steps:

Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>
'$HOME/.config/escale/my-first-repository.passphrase' file does not exist yet
Generate a new key? [Y/n] <enter>
        key: b'_0Uozrh6S6RJ0ybOlJvZWreGFl6NffMUbU8xGvZ5QJw='

while the others would get a copy of the generated file - here called x.passphrase - and perform the following steps:

Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>

In the above example, the my-first-repository.passphrase file is expected to be in the current directory or the configuration directory. You can alternatively specify a filepath.

Note

A passphrase file is not a text file.

Especially, the passphrase stored in it should not be followed by a newline or carriage return character, otherwise these trailing characters would be included in the passphrase.

Important

Passphrases are private and should not be sent in plain emails. Consider encrypted emails or services like onetimesecret.com.

Pay special attention not to introduce newline or carriage return while editing a passphrase file.

You are then requested to specify in which mode synchronize should operate:

Synchronization mode can be 'upload', 'download', 'shared' or 'conservative'
Request help with '?'
Which mode for this client? [shared] <enter>

More information about synchronization modes can be found in this page.

Especially, if you want to implement a simple backup setup, you may have two clients:

  • the one that accommodates the active data will be set in upload mode,
  • the other one that accommodates the backup copy will be set in download mode.

The next few steps are straightforward:

Refresh interval (in seconds): [10] <enter>
Request help with '?'
Disk quota: [] 1 <enter>

Setting disk quota to 1 will make Escale use no more than one gigabyte in your Dropbox space.

Note that larger files will not be transferred due to insufficient space, as long as Escale does not feature automatic split-and-recombine mechanics. This value should be large enough so that your files can be piped through. It can be arbitrarily large but will usually be smaller that the available storage space.

Exit the configuration wizard:

Do you want to add/edit another section? [N/y] <enter>

Your client is ready and can be launched with:

$ escalectl start

You can make your terminal continuously flush the logs with:

$ tail -f ~/.config/escale/escale.log

Beware that the escalectl stop command does not work properly on Windows. It actually kills all the Python processes instead of stopping Escale only.

Synchronizing with drive

This section details how to use the partly-native client for Google Drive. This approach has the inconvenient of requiring the drive utility that in turn depends on the Go toolchain.

Installing the Go toolchain may add a significant amount of used space (like 160MB on Linux for example). If you already have it installed, then the approach described below is advised.

Note

you will need Go>=1.7. On Ubuntu 16, the officially supported Go version is too old.

You can alternatively install a compiled drive package. This procedure is described in this link.

The backend in Escale that makes use of the drive utility is referred to as a native backend because data are not buffered and all file transfers and accesses to your remote data are performed at call time.

Note that Escale will assist you in installing drive but not the Go toolchain.

Please first install Go. On Ubuntu 17 or later, this is as simple as:

$ sudo apt install golang

If not already done, install Escale as detailled elsewhere:

$ pip install --user escale

or:

$ git clone https://github.com/francoislaurent/escale
$ cd escale
$ pip install --user -e .

Run the Escale configuration wizard in a terminal:

$ escale -i

You may run into a license acceptance step that requires to answer two yes-no questions. You must accept the terms of the license if you want to use Escale.

Do you have knowledge of the CeCILL-C license and accept its terms? [NO/yes] yes
Acceptance stored in file: '$HOME/.config/escale/acceptance'

Note

User-supplied text is shown in bold characters.

Carriage returns are indicated by <enter>.

The configuration wizard offers an expert mode and a simplified mode.

The simplified mode is considered self-explanatory and most users may safely follow the steps of this mode.

This tutorial covers the expert mode.

Run in simplified mode: [Y/n] n <enter>

If you set Escale up for the first time, you will first be requested the path of the folder you want to synchronize:

Editing configuration file '$HOME/.config/escale/escale.conf'
Path of your local repository (required): ~/Shared Files <enter>

where ~/Shared Files is the folder which content will be synchronized in this example.

If the directory you specify does not exist, Escale will make it for you.

Answer googledrive://Escale Repository to the second question:

Is the relay repository locally mounted in the file system? [N/y] <enter>
Request help with '?'
Path of the locally accessible relay repository (required): googledrive://Escale Repository <enter>

You will have to choose a name for the synchronized repository and a client name.

The repository name - or configuration section name - should be different from a local repository to another, on your local host.

The client name should be different from a client to another, for a given relay repository. The default client name is the hostname of your local host.

Choose a name for this configuration section.
Prefer ascii names.
Section name (required): my-first-repository <enter>
Request help with '?'
Client name: [poincare] <enter>

If you want to encrypt the data before they are sent to the relay storage, you will need to share a same encryption key between all the clients that synchronize over the same relay repository.

The first client to be set up can generate a key. This key will be available in a file that can be copied into the other clients.

The first client would perform the following steps:

Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>
'$HOME/.config/escale/my-first-repository.passphrase' file does not exist yet
Generate a new key? [Y/n] <enter>
        key: b'_0Uozrh6S6RJ0ybOlJvZWreGFl6NffMUbU8xGvZ5QJw='

while the others would get a copy of the generated file - here called x.passphrase - and perform the following steps:

Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>

In the above example, the my-first-repository.passphrase file is expected to be in the current directory or the configuration directory. You can alternatively specify a filepath.

Note

A passphrase file is not a text file.

Especially, the passphrase stored in it should not be followed by a newline or carriage return character, otherwise these trailing characters would be included in the passphrase.

Important

Passphrases are private and should not be sent in plain emails. Consider encrypted emails or services like onetimesecret.com.

Pay special attention not to introduce newline or carriage return while editing a passphrase file.

You are then requested to specify in which mode synchronize should operate:

Synchronization mode can be 'upload', 'download', 'shared' or 'conservative'
Request help with '?'
Which mode for this client? [shared] <enter>

More information about synchronization modes can be found in this page.

Especially, if you want to implement a simple backup setup, you may have two clients:

  • the one that accommodates the active data will be set in upload mode,
  • the other one that accommodates the backup copy will be set in download mode.

The next few steps are straightforward:

Refresh interval (in seconds): [10] <enter>
Request help with '?'
Disk quota: [] 1 <enter>

Setting disk quota to 1 will make Escale use no more than one gigabyte in your Dropbox space.

Note that larger files will not be transferred due to insufficient space, as long as Escale does not feature automatic split-and-recombine mechanics. This value should be large enough so that your files can be piped through. It can be arbitrarily large but will usually be smaller that the available storage space.

The wizard will now assist you in installing the drive utility, if necessary.

If you don't have 'drive' installed, leave it empty:
Drive binary: <enter>
The 'drive' Go package is going to be installed.
Do you want to continue? [Y/n] <enter>
Cannot find the 'GOPATH' environment variable.
Where do you want Go packages to be installed? [~/golang] <enter>
go get -u github.com/odeke-em/drive/drive-google
...
'drive' installed.
Do you want to add/edit another section? [N/y] <enter>

Exit the configuration wizard:

Do you want to add/edit another section? [N/y] <enter>

Your client is ready and can be launched with:

$ escalectl start

You can make your terminal continuously flush the logs with:

$ tail -f ~/.config/escale/escale.log

Beware that the escalectl stop command does not work properly on Windows. It actually kills all the Python processes instead of stopping Escale only.

When you will run the escale command for the first time without the -i option, you will be instructed to copy and paste a link into a web browser, so that you can log in with Google services.

Onced logged-in, the webpage will display a single line you can copy and paste back onto the command-line. This will permit Escale to connect to your Google Drive space.

Synchronizing with rclone

This section details how to use the partially-native client for Google Drive. This approach has the inconvenient of requiring the rclone utility that in turn depends on the Go toolchain.

Installing the Go toolchain may add a significant amount of used space (like 160MB on Linux for example).

The backend in Escale that makes use of the rclone utility is referred to as a native backend because data are not buffered and all file transfers and accesses to your remote data are performed at call time.

Requirements

You can either install and set up rclone following this tutorial or let escale’s configuration wizard do it for you.

Note however that escale will not install the Go toolchain for you. Ensure that the go command is available:

$ go version

This should show the version number of your installed Go distribution. Otherwise please install Go.

You will also need a dedicated folder in your Google Drive storage space. It will temporarily accommodate the files to be transfered and will permanently accommodate some meta files.

In this tutorial we make an Escale Repository folder at the root of the storage space.

Configuring Escale

If not already done, install Escale as detailled elsewhere:

$ pip install --user escale

or:

$ git clone https://github.com/francoislaurent/escale
$ cd escale
$ pip install --user -e .

Run the Escale configuration wizard in a terminal:

$ escale -i

You may run into a license acceptance step that requires to answer two yes-no questions. You must accept the terms of the license if you want to use Escale.

Do you have knowledge of the CeCILL-C license and accept its terms? [NO/yes] yes
Acceptance stored in file: '$HOME/.config/escale/acceptance'

Note

User-supplied text is shown in bold characters.

Carriage returns are indicated by <enter>.

The configuration wizard offers an expert mode and a simplified mode.

The simplified mode is considered self-explanatory and most users may safely follow the steps of this mode.

This tutorial covers the expert mode.

Run in simplified mode: [Y/n] n <enter>

If you set Escale up for the first time, you will first be requested the path of the folder you want to synchronize:

Editing configuration file '$HOME/.config/escale/escale.conf'
Path of your local repository (required): ~/Shared Files <enter>

where ~/Shared Files is the folder which content will be synchronized in this example.

If the directory you specify does not exist, Escale will make it for you.

Answer rclone://remote/Escale Repository to the second question, where remote is the remote name as defined for rclone and Escale Repository is the name of the folder that will accommodate the relay repository in your Google Drive space:

Is the relay repository locally mounted in the file system? [N/y] <enter>
Request help with '?'
Path of the locally accessible relay repository (required): rclone://remote/Escale Repository <enter>

You will have to choose a name for the synchronized repository and a client name.

The repository name - or configuration section name - should be different from a local repository to another, on your local host.

The client name should be different from a client to another, for a given relay repository. The default client name is the hostname of your local host.

Choose a name for this configuration section.
Prefer ascii names.
Section name (required): my-first-repository <enter>
Request help with '?'
Client name: [poincare] <enter>

If you want to encrypt the data before they are sent to the relay storage, you will need to share a same encryption key between all the clients that synchronize over the same relay repository.

The first client to be set up can generate a key. This key will be available in a file that can be copied into the other clients.

The first client would perform the following steps:

Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>
'$HOME/.config/escale/my-first-repository.passphrase' file does not exist yet
Generate a new key? [Y/n] <enter>
        key: b'_0Uozrh6S6RJ0ybOlJvZWreGFl6NffMUbU8xGvZ5QJw='

while the others would get a copy of the generated file - here called x.passphrase - and perform the following steps:

Encryption: [on] <enter>
Passphrase filename [my-first-repository.passphrase]: <enter>

In the above example, the my-first-repository.passphrase file is expected to be in the current directory or the configuration directory. You can alternatively specify a filepath.

Note

A passphrase file is not a text file.

Especially, the passphrase stored in it should not be followed by a newline or carriage return character, otherwise these trailing characters would be included in the passphrase.

Important

Passphrases are private and should not be sent in plain emails. Consider encrypted emails or services like onetimesecret.com.

Pay special attention not to introduce newline or carriage return while editing a passphrase file.

You are then requested to specify in which mode synchronize should operate:

Synchronization mode can be 'upload', 'download', 'shared' or 'conservative'
Request help with '?'
Which mode for this client? [shared] <enter>

More information about synchronization modes can be found in this page.

Especially, if you want to implement a simple backup setup, you may have two clients:

  • the one that accommodates the active data will be set in upload mode,
  • the other one that accommodates the backup copy will be set in download mode.

The next few steps are straightforward:

Refresh interval (in seconds): [10] <enter>
Request help with '?'
Disk quota: [] 1 <enter>

Setting disk quota to 1 will make Escale use no more than one gigabyte in your Dropbox space.

Note that larger files will not be transferred due to insufficient space, as long as Escale does not feature automatic split-and-recombine mechanics. This value should be large enough so that your files can be piped through. It can be arbitrarily large but will usually be smaller that the available storage space.

The wizard will now assist you in installing the rclone utility, if missing.

If you don't have 'rclone' installed, leave it empty:
RClone binary: <enter>
The 'rclone' Go package is going to be installed.
Do you want to continue? [Y/n] <enter>
Cannot find the 'GOPATH' environment variable.
Where do you want Go packages to be installed? [~/golang] <enter>
go get -u github.com/ncw/rclone
...
'rclone' installed.
Running 'rclone config'
See also: https://rclone.org/drive/
...

From the last ellipsis begins the output of the rclone config command. As instructed, please follow the steps described in the tutorial for Google Drive.

Exit the configuration wizard:

Do you want to add/edit another section? [N/y] <enter>

Your client is ready and can be launched with:

$ escalectl start

You can make your terminal continuously flush the logs with:

$ tail -f ~/.config/escale/escale.log

Beware that the escalectl stop command does not work properly on Windows. It actually kills all the Python processes instead of stopping Escale only.