One way upload to specific remote OneDrive folder (not home)

[VidarParry]

Hi everyone.
I'm not an expert, so please be kind:-)

I have installed OneDrive Client for Linux on my RPi and successfully have it uploading-only to my OneDrive Home. I have 4 RPI's and my aim is to back each of them up to OneDrive (upload only) to specific OneDrive folders.

So in this case I want OneDrive Client to upload a number of folders on the RPi to my remote cloud OneDrive folder called "OneDrive/System Library/RPI Backups/DevTest".

The problem I am experiencing is that the RPi end seems to work fine, but it always syncs to the Home of the remote cloud OneDrive. I haven't been able to find a working way of uploading to (in this case) "OneDrive/System Library/RPI Backups/DevTest-31" in the OneDrive cloud.

My understanding is that using "--single-directory "System Library/RPi Backups/DevTest-31"" will achieve what I want, but

My current setup is as follows. I have created a sync_list file and changed the config parameters as follows:

force_session_upload = "true"
local_first = "true"
no_remote_delete = "true"
sync_dir = "/"
sync_root_files = "true"
upload_only = "true"

Note the sync_dir is set to "/", being the root of the RPi. I'm relying on sync_list to select the actual folders to upload.

The sync_list file has entries such as:
/opt/RPi-Reporter-MQTT2HA-Daemon/*
The command I'm using is
onedrive --sync --verbose --single-directory "System Library/RPi Backups/DevTest-31" --dry-run
and the error is

The OneDrive Client was asked to search for this directory online and create it if it's not located: .System Library.RPi Backups.DevTest-31

ERROR: The requested --single-directory path to sync has generated an error. Please correct this error and try again.

It looks like the path has been changed to replace the slashes with dots.

Appreciate any advice to get this working.
Cheers
...Steve

Here is an example of a sync output:

Reading configuration file: /home/XX/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/XX/.config/onedrive
WARNING: D-Bus message bus daemon is not available; GUI notifications are disabled
DRY-RUN Configured. Output below shows what 'would' have occurred.
DRY-RUN: Copying items.sqlite3 to items-dryrun.sqlite3 to use for dry run operations
DRY-RUN: Not creating backup config file as --dry-run has been used
DRY-RUN: Not updating hash files as --dry-run has been used
Using IPv4 and IPv6 (if configured) for all network operations
Attempting to contact the Microsoft OneDrive Service
Successfully reached the Microsoft OneDrive Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
WARNING: Application has been configured NOT to cleanup remote files that are deleted locally.
Application Version:   onedrive v2.5.9-1+np1+1.2
Account Type:          personal
Default Drive ID:      819558xxxx82509
Default Root ID:       819558xxx982509!seaxxxxxdb43d7976fbc7da445c639
Microsoft Data Centre: Australia East
Remaining Free Space:  533.60 GB (572948637286 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /
The OneDrive Client was asked to search for this directory online and create it if it's not located: .System Library.RPi Backups.DevTest-31

ERROR: The requested --single-directory path to sync has generated an error. Please correct this error and try again.

[abraunegg]

@VidarParry

Oneiway upload to specific remote OneDrive folder
....
I have 4 RPI's and my aim is to back each of them up to OneDrive (upload only) to specific OneDrive folders.

For awareness, this question comes up from time to time and has been covered in previous discussions. It’s also documented here:
https://github.com/abraunegg/onedrive/blob/master/docs/advanced-usage.md#upload-data-from-the-local-onedrive-folder-to-a-specific-location-on-onedrive

My understanding of your requirement(s)

1. You want to upload data to a specific folder structure online: OneDrive/System Library/RPI Backups/DevTest-31 as illustrated below:

2. You want to upload into this specific online folder, data from this local source:

/opt/RPi-Reporter-MQTT2HA-Daemon/*

and have that data in the DevTest-31 folder online

3. You want to perform upload only, and not delete any data online.

Notes

1. Upload only is local-first by default - there is no online check - nothing ... so 'local_first' is a no-op
2. Strictly, you do not need a 'sync_list' file .. more on this below
3. Your resulting 'config' should only be the following

upload_only = "true"
no_remote_delete = "true"

Your Configuration

1. Do not set your 'sync_dir' to '/' .. remove that entry.
2. On your local system, create /opt/OneDrive and inside this folder, create the entire path you wish to have online, so in your case OneDrive/System Library/RPI Backups/DevTest-31 .. which (and I dont know your reasoning for this) - your full local path would be /opt/OneDrive/OneDrive/System Library/RPI Backups/DevTest-31/
3. Next .. symbolically link DevTest-31 to point at /opt/RPi-Reporter-MQTT2HA-Daemon/ .. therefore all your data in this MQTT directory is now visible under the DevTest-31 folder. You can verify this to ensure this is working as expected
4. Next, symbolically link ~/OneDrive to point at your /opt/OneDrive location. Alternatively, configure 'sync_dir' to point at /opt/OneDrive
5. Optionally, create a 'sync_list' file to only sync the structure you need - which is relative to your effective 'sync_dir'

/OneDrive/System Library/RPI Backups/DevTest-31/*

If you are adding other data .. then this is where 'sync_list' is helpful to guard what is uploaded. If this system / path is the only thing being uploaded from this device, then a 'sync_list' file is not needed.

Your resulting configuration:

# Optional
# sync_dir = "/opt/OneDrive"
# Required
upload_only = "true"
no_remote_delete = "true"

When you do this - your MQTT data will be uploaded to the location you have specific online.

This is the only way this works.

[VidarParry]

Thank you for your detailed reply! I had read that section of the docs but couldn't get my head around it (sorry!).
I have implemented and experimented with your approach. It is certainly nearly working now, just one final glitch I can't sort out.

Whatever I do, it creates and uploads to a new folder called OneDrivein my remote cloud Onedrive .

I actually want it to use my existing System-Library folder which also exists at the root of my Cloud OneDrive.

(To make it more Linux friendly, I have renamed my OneDrive folders to replace spaces, and this is reflected in my RPi file structure as well. Full remote cloud path path is now OneDrive/System-Library/RPi-Backups/DevTest-31)

So my config is just:

no_remote_delete = "true"
upload_only = "true"
# because there are only 4 threads in RPi
threads = "4"

And the commands I have used to set up the directories and symbolic links are:

sudo mkdir /opt/OneDrive
#create the entire path in the local opt/OneDrive:
sudo mkdir /opt/OneDrive/System-Library
sudo mkdir /opt/OneDrive/System-Library/RPi-Backups/
sudo mkdir /opt/OneDrive/System-Library/RPi-Backups/DevTest-31
#Symbolically link DevTest-31 to point at the RPi folder (s)you want to backup
sudo ln -s /opt/RPi-Reporter-MQTT2HA-Daemon/  /opt/OneDrive/System-Library/RPi-Backups/DevTest-31
#Symbolically link symbolically link ~/OneDrive to point at the /opt/OneDrive location
sudo ln -s /opt/OneDrive ~/OneDrive

NOTE: That these commands were not using "OneDrive/OneDrive" in the path you suggested as that also gave the same result. I had hoped that taking one of the OneDrive levels out would place System-Library at the root of the cloud OneDrive, but it didn't.
So the results of the run are:

onedrive --resync --sync --verbose

Reading configuration file: /home/pi/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/pi/.config/onedrive
WARNING: D-Bus message bus daemon is not available; GUI notifications are disabled

The usage of --resync will delete your local 'onedrive' client state, thus no record of your current 'sync status' will exist.
This has the potential to overwrite local versions of files with perhaps older versions of documents downloaded from OneDrive, resulting in local data loss.
If in doubt, backup your local data before using --resync

Are you sure you wish to proceed with --resync? [Y/N] y

Deleting the saved application sync status ...
Using IPv4 and IPv6 (if configured) for all network operations
Attempting to contact the Microsoft OneDrive Service
Successfully reached the Microsoft OneDrive Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
WARNING: Application has been configured NOT to cleanup remote files that are deleted locally.
Application Version:   onedrive v2.5.9-1+np1+1.2
Account Type:          personal
Default Drive ID:      81955xxxx982509
Default Root ID:       819558Dxxxx509!sea8cc6beffdb43d7976fbc7da445c639
Microsoft Data Centre: Australia East
Remaining Free Space:  415.20 GB (445817605324 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/pi/OneDrive
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: 81955xxxx982509
Processing: ~/OneDrive
The directory has not changed
Scanning the local file system '~/OneDrive' for new data to upload
New directories to create on Microsoft OneDrive: 27
OneDrive Client requested to create this directory online: ./OneDrive
The requested directory to create was not found on OneDrive - creating remote directory: ./OneDrive
Successfully created the remote directory ./OneDrive on Microsoft OneDrive
OneDrive Client requested to create this directory online: ./OneDrive/System-Library
The requested directory to create was not found on OneDrive - creating remote directory: ./OneDrive/System-Library
Successfully created the remote directory ./OneDrive/System-Library on Microsoft OneDrive
OneDrive Client requested to create this directory online: ./OneDrive/System-Library/RPi-Backups
The requested directory to create was not found on OneDrive - creating remote directory: ./OneDrive/System-Library/RPi-Backups
Successfully created the remote directory ./OneDrive/System-Library/RPi-Backups on Microsoft OneDrive
OneDrive Client requested to create this directory online: ./OneDrive/System-Library/RPi-Backups/DevTest-31
The requested directory to create was not found on OneDrive - creating remote directory: ./OneDrive/System-Library/RPi-Backups/DevTest-31
Successfully created the remote directory ./OneDrive/System-Library/RPi-Backups/DevTest-31 on Microsoft OneDrive
OneDrive Client requested to create this directory online: ./OneDrive/System-Library/RPi-Backups/DevTest-31/RPi-Reporter-MQTT2HA-Daemon
The requested directory to create was not found on OneDrive - creating remote directory: ./OneDrive/System-Library/RPi-Backups/DevTest-31/RPi-Reporter-MQTT2HA-Daemon
<snip - more of the same>

Appreciate your further advice!

Many thanks.
Cheers
...Steve

[abraunegg]

@VidarParry

My understanding is that using "--single-directory "System Library/RPi Backups/DevTest-31"" will achieve what I want, but

To clarify this understanding:

• When using --single-directory the path is relative to the configured 'sync_dir'. In your attempt, your 'sync_dir' is configured to '/' , thus, the relative path the application is seeking is /System Library/RPi Backups/DevTest-31/. This path also needs to 100% exist locally as a relative path to 'sync_dir' for the client to operate when using that option.

The OneDrive Client was asked to search for this directory online and create it if it's not located: .System Library.RPi Backups.DevTest-31

Now .. this is an odd one ... the / in the path should not be replaced with a . ... so either that is a bug or this is a consequence of something else such as your 'sync_dir' setting. I am unsure which at this stage - so will need to test and validate.

[VidarParry]

Thank you. Let me know if you want any further information for your test/validation from me.
Cheers
...Steve

[abraunegg]

@VidarParry

Whatever I do, it creates and uploads to a new folder called OneDrivein my remote cloud Onedrive .

What will be occurring is that your configured 'sync_dir' sees a OneDrive folder .. and that is what is being uploaded .. so essentially one of your symbolic links is pointing at an incorrect point for your data.

In your configured 'sync_dir' .. what you want to do is change to that directory, then:

• run ls -la
• run tree -a

Do you see the expected files / structure that you want replicated?

Essentially all you have to do is fix the symbolic links, then the correct structure will be replicated to the location you are wanting online.

[VidarParry]

Not sure. What I see is 👍

Should I be calling the symbolic link System-Folder instead of OneDrive?

[abraunegg]

So what you have done is created a symbolic link inside the ~/OneDrive folder .. hence that is why it is showing twice.

The ~/OneDrive folder is what should be pointing at /opt/OneDrive

The easiest option(s) here are:

• recreate ~/OneDrive
• Edit config to point at /opt/OneDrive

recreate ~/OneDrive

rmdir ~/OneDrive
ln -s /opt/OneDrive ~/OneDrive
ls -ld ~/OneDrive

edit 'sync_dir' config

sync_dir = "/opt/OneDrive"
no_remote_delete = "true"
upload_only = "true"
# because there are only 4 threads in RPi
threads = "4"

Remove any online folder structure as well

[VidarParry]

That seems to have fixed it - thank you again. The subtleties tripped me up:-)
Have a great evening!
Cheers
...Steve