Using duplicity directly, is it possible to have multiple backup plans?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







3















If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.



Is it possible to set up multiple backup plans with different schedules and sources/targets?
In particular I would like to:




  • regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.

  • back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)


I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
But will duplicity be able to handle this? Or will it get confused with the different backups?
Also: what will the performance for the remote backups be like?
How will the actual diff+copy mechanism work in the remote case?



Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...










share|improve this question































    3















    If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.



    Is it possible to set up multiple backup plans with different schedules and sources/targets?
    In particular I would like to:




    • regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.

    • back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)


    I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
    But will duplicity be able to handle this? Or will it get confused with the different backups?
    Also: what will the performance for the remote backups be like?
    How will the actual diff+copy mechanism work in the remote case?



    Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...










    share|improve this question



























      3












      3








      3


      1






      If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.



      Is it possible to set up multiple backup plans with different schedules and sources/targets?
      In particular I would like to:




      • regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.

      • back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)


      I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
      But will duplicity be able to handle this? Or will it get confused with the different backups?
      Also: what will the performance for the remote backups be like?
      How will the actual diff+copy mechanism work in the remote case?



      Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...










      share|improve this question
















      If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.



      Is it possible to set up multiple backup plans with different schedules and sources/targets?
      In particular I would like to:




      • regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.

      • back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)


      I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup.
      But will duplicity be able to handle this? Or will it get confused with the different backups?
      Also: what will the performance for the remote backups be like?
      How will the actual diff+copy mechanism work in the remote case?



      Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...







      backup duplicity






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Apr 13 '17 at 12:23









      Community

      1




      1










      asked Sep 26 '12 at 11:22









      black_puppydogblack_puppydog

      4531315




      4531315






















          2 Answers
          2






          active

          oldest

          votes


















          2














          In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.






          share|improve this answer



















          • 1





            I made the performance question into an own AU quesion.

            – black_puppydog
            Sep 27 '12 at 12:31



















          0















          Is it possible to set up multiple backup plans with different schedules and sources/targets?




          Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:





          1. Make a bash script that contains the Duplicity command:



            #!/bin/bash
            # Filename: myBackup.sh
            duplicity --full-if-older-than 1M
            --exclude-filelist excludeList.txt
            / sftp://uid@other.host/some_dir


            This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (/), but will exlude/
            include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.




          2. Make an exclude list (exludeList.txt):



            **[Cc]ache*
            **[Hh]istory*
            **[Ss]ocket*
            **[Tt]humb*
            **[Tt]rash*
            **.kvm
            **.local/share/icons
            **.rpmdb
            **.thumbnails
            **_NOBACKUP*
            /home/user/VirtualBox VMs/**
            /home/user/Downloads/**
            + /home/user
            + /var/www
            + /root
            + /etc
            **


            In our Duplicity command we state that the source is /, and if you don't have an exclude file that means everything below the root dir is included in the backup.



            The double asterisk ** substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with +). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is **.




          3. Schedule this with Cron by creating the following file in your /etc/cron.daily to run it once every day:



            #!/bin/bash
            myBackup.sh




          regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.




          Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:




          • Put a script in /etc/cron.hourly that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.


          • Write a service that runs when you plug your backup drive.




          back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)




          This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.



          Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.



          Let me know if there's any info you need that's missing in this answer.






          share|improve this answer
























            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "89"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f193149%2fusing-duplicity-directly-is-it-possible-to-have-multiple-backup-plans%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            2














            In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.






            share|improve this answer



















            • 1





              I made the performance question into an own AU quesion.

              – black_puppydog
              Sep 27 '12 at 12:31
















            2














            In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.






            share|improve this answer



















            • 1





              I made the performance question into an own AU quesion.

              – black_puppydog
              Sep 27 '12 at 12:31














            2












            2








            2







            In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.






            share|improve this answer













            In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Sep 26 '12 at 14:15









            JanuaryJanuary

            25.9k116789




            25.9k116789








            • 1





              I made the performance question into an own AU quesion.

              – black_puppydog
              Sep 27 '12 at 12:31














            • 1





              I made the performance question into an own AU quesion.

              – black_puppydog
              Sep 27 '12 at 12:31








            1




            1





            I made the performance question into an own AU quesion.

            – black_puppydog
            Sep 27 '12 at 12:31





            I made the performance question into an own AU quesion.

            – black_puppydog
            Sep 27 '12 at 12:31













            0















            Is it possible to set up multiple backup plans with different schedules and sources/targets?




            Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:





            1. Make a bash script that contains the Duplicity command:



              #!/bin/bash
              # Filename: myBackup.sh
              duplicity --full-if-older-than 1M
              --exclude-filelist excludeList.txt
              / sftp://uid@other.host/some_dir


              This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (/), but will exlude/
              include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.




            2. Make an exclude list (exludeList.txt):



              **[Cc]ache*
              **[Hh]istory*
              **[Ss]ocket*
              **[Tt]humb*
              **[Tt]rash*
              **.kvm
              **.local/share/icons
              **.rpmdb
              **.thumbnails
              **_NOBACKUP*
              /home/user/VirtualBox VMs/**
              /home/user/Downloads/**
              + /home/user
              + /var/www
              + /root
              + /etc
              **


              In our Duplicity command we state that the source is /, and if you don't have an exclude file that means everything below the root dir is included in the backup.



              The double asterisk ** substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with +). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is **.




            3. Schedule this with Cron by creating the following file in your /etc/cron.daily to run it once every day:



              #!/bin/bash
              myBackup.sh




            regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.




            Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:




            • Put a script in /etc/cron.hourly that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.


            • Write a service that runs when you plug your backup drive.




            back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)




            This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.



            Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.



            Let me know if there's any info you need that's missing in this answer.






            share|improve this answer




























              0















              Is it possible to set up multiple backup plans with different schedules and sources/targets?




              Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:





              1. Make a bash script that contains the Duplicity command:



                #!/bin/bash
                # Filename: myBackup.sh
                duplicity --full-if-older-than 1M
                --exclude-filelist excludeList.txt
                / sftp://uid@other.host/some_dir


                This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (/), but will exlude/
                include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.




              2. Make an exclude list (exludeList.txt):



                **[Cc]ache*
                **[Hh]istory*
                **[Ss]ocket*
                **[Tt]humb*
                **[Tt]rash*
                **.kvm
                **.local/share/icons
                **.rpmdb
                **.thumbnails
                **_NOBACKUP*
                /home/user/VirtualBox VMs/**
                /home/user/Downloads/**
                + /home/user
                + /var/www
                + /root
                + /etc
                **


                In our Duplicity command we state that the source is /, and if you don't have an exclude file that means everything below the root dir is included in the backup.



                The double asterisk ** substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with +). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is **.




              3. Schedule this with Cron by creating the following file in your /etc/cron.daily to run it once every day:



                #!/bin/bash
                myBackup.sh




              regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.




              Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:




              • Put a script in /etc/cron.hourly that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.


              • Write a service that runs when you plug your backup drive.




              back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)




              This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.



              Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.



              Let me know if there's any info you need that's missing in this answer.






              share|improve this answer


























                0












                0








                0








                Is it possible to set up multiple backup plans with different schedules and sources/targets?




                Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:





                1. Make a bash script that contains the Duplicity command:



                  #!/bin/bash
                  # Filename: myBackup.sh
                  duplicity --full-if-older-than 1M
                  --exclude-filelist excludeList.txt
                  / sftp://uid@other.host/some_dir


                  This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (/), but will exlude/
                  include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.




                2. Make an exclude list (exludeList.txt):



                  **[Cc]ache*
                  **[Hh]istory*
                  **[Ss]ocket*
                  **[Tt]humb*
                  **[Tt]rash*
                  **.kvm
                  **.local/share/icons
                  **.rpmdb
                  **.thumbnails
                  **_NOBACKUP*
                  /home/user/VirtualBox VMs/**
                  /home/user/Downloads/**
                  + /home/user
                  + /var/www
                  + /root
                  + /etc
                  **


                  In our Duplicity command we state that the source is /, and if you don't have an exclude file that means everything below the root dir is included in the backup.



                  The double asterisk ** substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with +). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is **.




                3. Schedule this with Cron by creating the following file in your /etc/cron.daily to run it once every day:



                  #!/bin/bash
                  myBackup.sh




                regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.




                Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:




                • Put a script in /etc/cron.hourly that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.


                • Write a service that runs when you plug your backup drive.




                back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)




                This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.



                Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.



                Let me know if there's any info you need that's missing in this answer.






                share|improve this answer














                Is it possible to set up multiple backup plans with different schedules and sources/targets?




                Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:





                1. Make a bash script that contains the Duplicity command:



                  #!/bin/bash
                  # Filename: myBackup.sh
                  duplicity --full-if-older-than 1M
                  --exclude-filelist excludeList.txt
                  / sftp://uid@other.host/some_dir


                  This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (/), but will exlude/
                  include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.




                2. Make an exclude list (exludeList.txt):



                  **[Cc]ache*
                  **[Hh]istory*
                  **[Ss]ocket*
                  **[Tt]humb*
                  **[Tt]rash*
                  **.kvm
                  **.local/share/icons
                  **.rpmdb
                  **.thumbnails
                  **_NOBACKUP*
                  /home/user/VirtualBox VMs/**
                  /home/user/Downloads/**
                  + /home/user
                  + /var/www
                  + /root
                  + /etc
                  **


                  In our Duplicity command we state that the source is /, and if you don't have an exclude file that means everything below the root dir is included in the backup.



                  The double asterisk ** substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with +). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is **.




                3. Schedule this with Cron by creating the following file in your /etc/cron.daily to run it once every day:



                  #!/bin/bash
                  myBackup.sh




                regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.




                Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:




                • Put a script in /etc/cron.hourly that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.


                • Write a service that runs when you plug your backup drive.




                back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)




                This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.



                Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.



                Let me know if there's any info you need that's missing in this answer.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Mar 24 at 21:09









                PetaspeedBeaverPetaspeedBeaver

                1175




                1175






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Ask Ubuntu!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f193149%2fusing-duplicity-directly-is-it-possible-to-have-multiple-backup-plans%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    How did Captain America manage to do this?

                    迪纳利

                    南乌拉尔铁路局