Using duplicity directly, is it possible to have multiple backup plans?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.
Is it possible to set up multiple backup plans with different schedules and sources/targets?
In particular I would like to:
- regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
- back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
But will duplicity be able to handle this? Or will it get confused with the different backups?
Also: what will the performance for the remote backups be like?
How will the actual diff+copy mechanism work in the remote case?
Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...
backup duplicity
add a comment |
If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.
Is it possible to set up multiple backup plans with different schedules and sources/targets?
In particular I would like to:
- regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
- back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
But will duplicity be able to handle this? Or will it get confused with the different backups?
Also: what will the performance for the remote backups be like?
How will the actual diff+copy mechanism work in the remote case?
Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...
backup duplicity
add a comment |
If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.
Is it possible to set up multiple backup plans with different schedules and sources/targets?
In particular I would like to:
- regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
- back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
But will duplicity be able to handle this? Or will it get confused with the different backups?
Also: what will the performance for the remote backups be like?
How will the actual diff+copy mechanism work in the remote case?
Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...
backup duplicity
If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.
Is it possible to set up multiple backup plans with different schedules and sources/targets?
In particular I would like to:
- regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
- back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
But will duplicity be able to handle this? Or will it get confused with the different backups?
Also: what will the performance for the remote backups be like?
How will the actual diff+copy mechanism work in the remote case?
Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...
backup duplicity
backup duplicity
edited Apr 13 '17 at 12:23
Community♦
1
1
asked Sep 26 '12 at 11:22
black_puppydogblack_puppydog
4531315
4531315
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.
1
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
add a comment |
Is it possible to set up multiple backup plans with different schedules and sources/targets?
Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:
Make a bash script that contains the Duplicity command:
#!/bin/bash
# Filename: myBackup.sh
duplicity --full-if-older-than 1M
--exclude-filelist excludeList.txt
/ sftp://uid@other.host/some_dir
This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (
/
), but will exlude/
include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.
Make an exclude list (
exludeList.txt
):
**[Cc]ache*
**[Hh]istory*
**[Ss]ocket*
**[Tt]humb*
**[Tt]rash*
**.kvm
**.local/share/icons
**.rpmdb
**.thumbnails
**_NOBACKUP*
/home/user/VirtualBox VMs/**
/home/user/Downloads/**
+ /home/user
+ /var/www
+ /root
+ /etc
**
In our Duplicity command we state that the source is
/
, and if you don't have an exclude file that means everything below the root dir is included in the backup.
The double asterisk
**
substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with+
). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is**
.
Schedule this with Cron by creating the following file in your
/etc/cron.daily
to run it once every day:
#!/bin/bash
myBackup.sh
regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:
Put a script in
/etc/cron.hourly
that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.Write a service that runs when you plug your backup drive.
back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.
Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.
Let me know if there's any info you need that's missing in this answer.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f193149%2fusing-duplicity-directly-is-it-possible-to-have-multiple-backup-plans%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.
1
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
add a comment |
In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.
1
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
add a comment |
In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.
In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.
answered Sep 26 '12 at 14:15
JanuaryJanuary
25.9k116789
25.9k116789
1
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
add a comment |
1
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
1
1
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
I made the performance question into an own AU quesion.
– black_puppydog
Sep 27 '12 at 12:31
add a comment |
Is it possible to set up multiple backup plans with different schedules and sources/targets?
Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:
Make a bash script that contains the Duplicity command:
#!/bin/bash
# Filename: myBackup.sh
duplicity --full-if-older-than 1M
--exclude-filelist excludeList.txt
/ sftp://uid@other.host/some_dir
This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (
/
), but will exlude/
include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.
Make an exclude list (
exludeList.txt
):
**[Cc]ache*
**[Hh]istory*
**[Ss]ocket*
**[Tt]humb*
**[Tt]rash*
**.kvm
**.local/share/icons
**.rpmdb
**.thumbnails
**_NOBACKUP*
/home/user/VirtualBox VMs/**
/home/user/Downloads/**
+ /home/user
+ /var/www
+ /root
+ /etc
**
In our Duplicity command we state that the source is
/
, and if you don't have an exclude file that means everything below the root dir is included in the backup.
The double asterisk
**
substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with+
). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is**
.
Schedule this with Cron by creating the following file in your
/etc/cron.daily
to run it once every day:
#!/bin/bash
myBackup.sh
regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:
Put a script in
/etc/cron.hourly
that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.Write a service that runs when you plug your backup drive.
back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.
Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.
Let me know if there's any info you need that's missing in this answer.
add a comment |
Is it possible to set up multiple backup plans with different schedules and sources/targets?
Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:
Make a bash script that contains the Duplicity command:
#!/bin/bash
# Filename: myBackup.sh
duplicity --full-if-older-than 1M
--exclude-filelist excludeList.txt
/ sftp://uid@other.host/some_dir
This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (
/
), but will exlude/
include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.
Make an exclude list (
exludeList.txt
):
**[Cc]ache*
**[Hh]istory*
**[Ss]ocket*
**[Tt]humb*
**[Tt]rash*
**.kvm
**.local/share/icons
**.rpmdb
**.thumbnails
**_NOBACKUP*
/home/user/VirtualBox VMs/**
/home/user/Downloads/**
+ /home/user
+ /var/www
+ /root
+ /etc
**
In our Duplicity command we state that the source is
/
, and if you don't have an exclude file that means everything below the root dir is included in the backup.
The double asterisk
**
substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with+
). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is**
.
Schedule this with Cron by creating the following file in your
/etc/cron.daily
to run it once every day:
#!/bin/bash
myBackup.sh
regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:
Put a script in
/etc/cron.hourly
that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.Write a service that runs when you plug your backup drive.
back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.
Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.
Let me know if there's any info you need that's missing in this answer.
add a comment |
Is it possible to set up multiple backup plans with different schedules and sources/targets?
Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:
Make a bash script that contains the Duplicity command:
#!/bin/bash
# Filename: myBackup.sh
duplicity --full-if-older-than 1M
--exclude-filelist excludeList.txt
/ sftp://uid@other.host/some_dir
This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (
/
), but will exlude/
include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.
Make an exclude list (
exludeList.txt
):
**[Cc]ache*
**[Hh]istory*
**[Ss]ocket*
**[Tt]humb*
**[Tt]rash*
**.kvm
**.local/share/icons
**.rpmdb
**.thumbnails
**_NOBACKUP*
/home/user/VirtualBox VMs/**
/home/user/Downloads/**
+ /home/user
+ /var/www
+ /root
+ /etc
**
In our Duplicity command we state that the source is
/
, and if you don't have an exclude file that means everything below the root dir is included in the backup.
The double asterisk
**
substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with+
). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is**
.
Schedule this with Cron by creating the following file in your
/etc/cron.daily
to run it once every day:
#!/bin/bash
myBackup.sh
regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:
Put a script in
/etc/cron.hourly
that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.Write a service that runs when you plug your backup drive.
back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.
Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.
Let me know if there's any info you need that's missing in this answer.
Is it possible to set up multiple backup plans with different schedules and sources/targets?
Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:
Make a bash script that contains the Duplicity command:
#!/bin/bash
# Filename: myBackup.sh
duplicity --full-if-older-than 1M
--exclude-filelist excludeList.txt
/ sftp://uid@other.host/some_dir
This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (
/
), but will exlude/
include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.
Make an exclude list (
exludeList.txt
):
**[Cc]ache*
**[Hh]istory*
**[Ss]ocket*
**[Tt]humb*
**[Tt]rash*
**.kvm
**.local/share/icons
**.rpmdb
**.thumbnails
**_NOBACKUP*
/home/user/VirtualBox VMs/**
/home/user/Downloads/**
+ /home/user
+ /var/www
+ /root
+ /etc
**
In our Duplicity command we state that the source is
/
, and if you don't have an exclude file that means everything below the root dir is included in the backup.
The double asterisk
**
substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with+
). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is**
.
Schedule this with Cron by creating the following file in your
/etc/cron.daily
to run it once every day:
#!/bin/bash
myBackup.sh
regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:
Put a script in
/etc/cron.hourly
that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.Write a service that runs when you plug your backup drive.
back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)
This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.
Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.
Let me know if there's any info you need that's missing in this answer.
answered Mar 24 at 21:09
PetaspeedBeaverPetaspeedBeaver
1175
1175
add a comment |
add a comment |
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f193149%2fusing-duplicity-directly-is-it-possible-to-have-multiple-backup-plans%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown