Ask your questions here.
Post a reply

Re: Squeeze snapshot

Fri Mar 29, 2013 2:10 am

I reinstalled snapshot and ran it again. This time it came out to over 39gb!!! It seems that nothing in Videos was excluded. At some point I just wanted to kill the process because it was taking so long. Cancel didn’t work. xkill didn't work. Nothing would stop the process!! Dunno whether these issues are with squeeze or in the script itself. It seemed to work OK on my minimal wheezy install.

Re: Squeeze snapshot

Fri Mar 29, 2013 3:51 am

There were also some hefty files in /root/.local/share/Trash/files/ almost 4gb total! Have to figure out how to empty root trash . . .

Question of the month is why there is something in root's Trash at all.
There should not be anything.
iow: You have started X as root.
(ups: or with gksu. Not sure what i consider worse. starting thunar at all or running X as root ...).

Post the excludes file.

Re: Squeeze snapshot

Fri Mar 29, 2013 4:01 am

nadir wrote:
There were also some hefty files in /root/.local/share/Trash/files/ almost 4gb total! Have to figure out how to empty root trash . . .

Question of the month is why there is something in root's Trash at all.
There should not be anything.
iow: You have started X as root.

Post the excludes file.

I have NEVER started x as root. Sometimes I use gksu nautilus though. There were some .debs in there. Some directories etc. deleted from htdocs in the xampp server in /opt. Can't remember what else.

Code:
# rsync excludes file for refractasnapshot and refractasnapshot-gui
# version 9.0.8


# Exclude some system files. These are required, and you probably
# shouldn't change them.

- /dev/*
- /cdrom/*
- /media/*
- /swapfile
- /mnt/*
- /opt/*
- /sys/*
- /proc/*
- /tmp/*
- /live
- /boot/grub/grub.cfg
- /boot/grub/menu.lst
- /boot/grub/device.map
- /etc/udev/rules.d/70-persistent-cd.rules
- /etc/udev/rules.d/70-persistent-net.rules
- /etc/fstab
- /etc/mtab
- /etc/blkid.tab
- /home/snapshot

# Added for newer version of live-config/live-boot in sid (to become
# Jessie.) These are only relevant here if you create a snapshot while
# you're running a live-CD or live-usb.
- /lib/live/overlay
- /lib/live/image
- /lib/live/rootfs
- /lib/live/mount
- /run/*


## Entries below are optional. They are included either for privacy
## or to reduce the size of the snapshot. If you have any large
## files or directories, you should exclude them from being copied
## by adding them to this list.

# Uncomment this to exclude everything in /var/log/
- /var/log/*

# The next three lines exclude everything in /var/log
# except /var/log/clamav/ (or anything else beginning with "c") and
# /var/log/gdm (or anything beginning with "g").
# If clamav log files are excluded, freshclam will give errors at boot.
- /var/log/[a-b,A-Z]*
- /var/log/[d-f]*
- /var/log/[h-z]*

- /var/cache/apt/archives/*.deb
- /var/cache/apt/pkgcache.bin
- /var/cache/apt/srcpkgcache.bin
- /var/cache/apt/apt-file/*
- /var/lib/apt/lists/*
- /var/lib/dhcp/*
- /var/spool/mail/*
- /var/mail/*

#- /root/.bash_history
- /root/.ssh
- /root/.local/share/Trash/*

- /home/*/.Trash*
- /home/*/.local/share/Trash/*
- /home/*/.mozilla/*/Cache/*
- /home/*/.mozillabackup/*
- /home/*/.adobe
- /home/*/.aptitude
#- /home/*/.bash_history
- /home/*/.cache
- /home/*/.fonts/*
- /home/*/.gksu*
- /home/*/.gvfs
- /home/*/.macromedia
- /home/*/.nano_history
- /home/*/.recently-used
- /home/*/.recently-used.xbel
- /home/*/.local/share/recently-used.xbel
- /home/*/.thumbnails/large/*
- /home/*/.thumbnails/normal/*
- /home/*/.vbox*
- /home/*/.wine*
- /home/*/.xchat2
- /home/*/.xsession-errors*
- /home/*/.ICEauthority
- /home/*/.Xauthority
- /home/*/.ssh

# Examples:
- /home/*/Downloads/*
- /home/*/Documents/*
#- /home/*/Music/*
#- /home/*/Pictures/*
- /home/*/Videos/*

# To exclude all hidden files and directories in your home, uncomment
# the next line. You will lose custom desktop configs if you do.
#- /home/*/.[a-z,A-Z,0-9]*
.

Re: Squeeze snapshot

Fri Mar 29, 2013 4:36 am

- /home/*/Videos/*

Good. Same here. Am running the snapshot right now.

Perhaps post
ls ~
and
ls -ahl ~
too, just to be sure.

Re: Squeeze snapshot

Fri Mar 29, 2013 4:42 am

If you have a saved filesystem snapshot rsyncs / to that with the --delete option.

(man rsync)
--delete
This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized

Therefore if you already got "/*/monster_mpegs" in a saved fs, even if you change the excludes and do another rsync, they will stay there

Two workarounds: delete stuff manually in the saved fs and rerun snapshot (faster) or delete saved fs and start again with the right custom excludes for you

If you're up for script-hacking you could add < --delete-excluded>
--delete-excluded
In addition to deleting the files on the receiving side that are not on the sending side, this tells rsync to also delete any files on the receiving side that are excluded (see --exclude).

Maybe there is a good case to add --delete-excluded to the official script.. or not. 39gig filesystems are a special case and won't produce an iso anyway

The snapshot script works, I tested it many times. Unless this is some bug the crystal ball failed to spot. Much work has gone into making snapshot user-customizable.. however it is impossible to set defaults that suit every special case..

This thread definitely reinforces my policy of keeping a large "data" partition, separate from /home

Re: Squeeze snapshot

Fri Mar 29, 2013 11:11 am

If you want to add --delete-excluded to the rsync command, look around line 360 of /usr/bin/refractasnapshot-gui for this:
Code:
# Copy the filesystem
rsync -av / myfs/ --delete --exclude="$work_dir" \
--exclude="$snapshot_dir" --exclude-from="$snapshot_excludes" \
| tee >($DIALOG --title="Copying filesystem..." --progress --pulsate --width 300)
kill $(pgrep $DIALOG)


And change it to:
Code:
# Copy the filesystem
rsync -av / myfs/ --delete-excluded --exclude="$work_dir" \
--exclude="$snapshot_dir" --exclude-from="$snapshot_excludes" \
| tee >($DIALOG --title="Copying filesystem..." --progress --pulsate --width 300)
kill $(pgrep $DIALOG)


Killing the script in the middle of a run can be tricky. I think I do it with
Code:
pkill rsync
pkill mksquashfs
pkill genisoimage
(and maybe)
pkill refractasnapshot

Re: Squeeze snapshot

Fri Mar 29, 2013 2:22 pm

nadir wrote:Perhaps post
ls ~
and
ls -ahl ~
too, just to be sure.

Here ya go.

ls ~
Code:
bin
Documents
dwhelper
Videos
viewonecache
Desktop
Downloads
p.v1
viewone


ls -ahl ~
Code:
drwxr-xr-x 65 xxxxx xxxxx 4.0K Mar 28 23:07 .
drwxr-xr-x  6 root   root   4.0K Mar 28 20:41 ..
drwx------  3 xxxxx xxxxx 4.0K May 19  2011 .adobe
drwx------  2 xxxxx xxxxx 4.0K May 19  2011 .aptitude
-rw-r--r--  1 xxxxx xxxxx  392 May  8  2011 .aspell.en.prepl
-rw-r--r--  1 xxxxx xxxxx  126 May  8  2011 .aspell.en.pws
drwxr-xr-x  2 xxxxx xxxxx 4.0K Mar 28 21:04 .audacity1.2-xxxxx
drwxr-xr-x  6 xxxxx xxxxx 4.0K Mar 28 21:04 .audacity-data
drwxr-xr-x  4 xxxxx xxxxx 4.0K Mar 28 22:48 .avidemux
-rw-r--r--  1 xxxxx xxxxx 6.3K Mar 28 12:10 .bash_history
-rw-r--r--  1 xxxxx xxxxx  220 May 17  2011 .bash_logout
-rw-r--r--  1 xxxxx xxxxx 3.2K May 17  2011 .bashrc
drwxr-xr-x  2 xxxxx xxxxx 4.0K Jul 16  2011 bin
drwxr-xr-x 11 xxxxx xxxxx 4.0K Mar 27 00:50 .cache
drwx------  3 xxxxx xxxxx 4.0K May 26  2011 .compiz
drwxr-xr-x 29 xxxxx xxxxx 4.0K Jan 23 20:04 .config
drwx------  3 xxxxx xxxxx 4.0K May 17  2011 .dbus
drwxr-----  3 xxxxx xxxxx 4.0K Mar 28 19:21 Desktop
drwxr-xr-x  2 xxxxx xxxxx 4.0K May 23  2011 .devilspie_old
-rw-r--r--  1 xxxxx xxxxx   41 Mar 28 12:24 .dmrc
drwxr-xr-x 11 xxxxx xxxxx 4.0K Mar 22 13:48 Documents
drwxr-xr-x  2 xxxxx xxxxx 4.0K Feb 13 16:00 Downloads
drwxr-xr-x  2 xxxxx xxxxx  12K May  2  2012 .dvdcss
-rw-r--r--  1 xxxxx xxxxx  965 Mar 26 22:41 .dvdstyler
drwxr-xr-x  2 xxxxx xxxxx 4.0K Sep 12  2012 dwhelper
-rw-------  1 xxxxx xxxxx   16 May 17  2011 .esd_auth
drwx------  5 xxxxx xxxxx 4.0K Dec  8 23:16 .evolution
drwxr-xr-x  2 xxxxx xxxxx 4.0K Nov  4 09:32 .fontconfig
drwxr-xr-x  4 xxxxx xxxxx  12K Jun 10  2011 .fonts
drwx------  4 xxxxx xxxxx 4.0K Mar 28 16:33 .galeon
drwx------  6 xxxxx xxxxx 4.0K Mar 28 12:24 .gconf
drwx------  2 xxxxx xxxxx 4.0K Mar 29 09:04 .gconfd
drwx------  4 xxxxx xxxxx 4.0K May 18  2011 .gegl-0.0
drwx------  3 xxxxx xxxxx 4.0K Sep  5  2009 .gftp
drwxr-xr-x 22 xxxxx xxxxx 4.0K Jan 28 11:42 .gimp-2.6
-rw-r-----  1 xxxxx xxxxx    0 Mar 28 13:25 .gksu.lock
drwx------  3 xxxxx xxxxx 4.0K Jun 24  2011 .gnash
drwx------ 13 xxxxx xxxxx 4.0K Mar 28 11:55 .gnome2
drwx------  2 xxxxx xxxxx 4.0K May 17  2011 .gnome2_private
drwx------  2 xxxxx xxxxx 4.0K Apr 19  2012 .gnupg
drwx------  2 xxxxx xxxxx 4.0K Jan 23 20:08 .gphoto
drwx------  2 xxxxx xxxxx 4.0K Oct 31  2007 .grsync
drwxr-xr-x  2 xxxxx xxxxx 4.0K Mar 27 12:01 .gstreamer-0.10
-rw-r--r--  1 xxxxx xxxxx  281 Mar 28 12:24 .gtk-bookmarks
drwx------  2 xxxxx xxxxx 4.0K May 17  2011 .gvfs
drwxr--r--  2 xxxxx xxxxx 4.0K Dec 15  2011 .hardinfo
drwxr-----  2 xxxxx xxxxx 4.0K Mar 26 23:12 .hplip
-rw-------  1 xxxxx xxxxx  60K Mar 28 12:24 .ICEauthority
drwxr-xr-x  3 xxxxx xxxxx 4.0K Mar 18  2011 .icedteaplugin
drwxr-xr-x  5 xxxxx xxxxx 4.0K Oct 15 18:56 .icons
drwxr-xr-x  4 xxxxx xxxxx 4.0K Jan 19 19:56 .java
drwx------  3 xxxxx xxxxx 4.0K Feb 26  2012 .liferea_1.6
drwx------  3 xxxxx xxxxx 4.0K May 17  2011 .local
drwx------  3 xxxxx xxxxx 4.0K May 19  2011 .macromedia
drwx------  3 xxxxx xxxxx 4.0K May 27  2011 .mission-control
drwx------  4 xxxxx xxxxx 4.0K Aug 15  2012 .mozilla
drwx------  3 xxxxx xxxxx 4.0K Sep 25  2012 .mozillabackup
drwxr-xr-x  2 xxxxx xxxxx 4.0K May 17  2011 .nautilus
-rw-r--r--  1 xxxxx xxxxx 1.2K Feb 21 07:49 .nvidia-settings-rc
drwxr-xr-x  3 xxxxx xxxxx 4.0K May 21  2011 .openoffice.org
drwx------  3 xxxxx xxxxx 4.0K Jun  8  2011 .pki
-rw-r--r--  1 xxxxx xxxxx   37 May 19  2011 .printer-groups.xml
-rw-r--r--  1 xxxxx xxxxx  675 May 17  2011 .profile
drwx------  2 xxxxx xxxxx 4.0K Mar 28 20:21 .pulse
-rw-------  1 xxxxx xxxxx  256 May 18  2011 .pulse-cookie
-rw-r--r--  1 xxxxx xxxxx  175 Aug  5  2011 p.v1
drwxr-xr-x  2 xxxxx xxxxx 4.0K Jun  3  2011 .qt
-rw-------  1 xxxxx xxxxx 1.9K Nov 19 12:52 .recently-used
-rw-------  1 xxxxx xxxxx 445K Mar 28 23:07 .recently-used.xbel
-rw-r--r--  1 xxxxx xxxxx  170 Aug 11  2011 .rgb.txt
drwxrwx---  3 xxxxx xxxxx 4.0K Aug  1  2007 .sane
drwxr-xr-x  3 xxxxx xxxxx 4.0K May 28  2011 .screenlets
drwxr-xr-x  5 xxxxx xxxxx 4.0K Jan 23 14:16 .shotwell
drwx------  6 xxxxx xxxxx 4.0K Jul 30  2012 .Skype
drwx------  2 xxxxx xxxxx 4.0K May 20  2011 .ssh
-rw-------  1 xxxxx xxxxx  12K Nov  2  2011 .swp
drwxr-xr-x  5 xxxxx xxxxx 4.0K May 31  2011 .themes
drwx------  5 xxxxx xxxxx 4.0K Jan 23 20:04 .thumbnails
drwx------  2 xxxxx xxxxx 4.0K May 17  2011 .update-notifier
drwxr-xr-x  2 root   root   4.0K Jul 14  2011 .ure
drwxr-xr-x  4 xxxxx xxxxx 4.0K Mar 28 20:35 Videos
drwxr-xr-x  2 xxxxx xxxxx 4.0K Aug  5  2011 viewone
drwxr-xr-x  5 xxxxx xxxxx 4.0K Jan 28 11:37 viewonecache
drwxr-xr-x  3 xxxxx xxxxx 4.0K May 18  2011 .vlc
drwxr-xr-x  4 xxxxx xxxxx 4.0K Mar 28 23:05 .wine
-rw-------  1 xxxxx xxxxx  633 Mar 16  2012 .wvdial.conf
-rw-------  1 xxxxx xxxxx    0 Dec  8 14:12 .Xauthority
-rw-r--r--  1 xxxxx xxxxx  439 Oct  5 23:26 .xchm
-rw-------  1 xxxxx xxxxx  16M Mar 28 23:04 .xsession-errors
-rw-------  1 xxxxx xxxxx 3.9K Mar 28 12:23 .xsession-errors.old

Re: Squeeze snapshot

Fri Mar 29, 2013 2:36 pm

@dzz . . . Before I ran this last snapshot, I reinstalled refractasnapshot and deleted the saved filesystem. So I assume it started from scratch. There were no monster .mpgs in that original snapshot - just a lot of audio and other basic editing files in VIdeos. After that I started the current editing project which now has close to 80gb in Videos. Snapshot didn't exclude any of them in either run. Maybe I need to move - /home/*/Videos/* to the top of the exclude list or at least before the entries that are commented out.

@fsmithred - I'll switch that out and try later (yard work most of today). At least I know how to kill the processes now. This last time it went on for hours which really sucked. Probably would have even restarted after a reboot. Note: the cancel button probably shouldn't be there if it doesn't work.

Re: Squeeze snapshot

Fri Mar 29, 2013 4:27 pm

For me it's a miracle.

You excludes looks fine, i tested the very same (with data at Videos and the excludes you got, sure enough it worked), the idea of dzz (which sounded quite promisising) doesn't seem to be the problem. No clue what else to look at.

Re: Squeeze snapshot

Fri Mar 29, 2013 4:44 pm

nadir wrote:For me it's a miracle.

You excludes looks fine, i tested the very same (with data at Videos and the excludes you got, sure enough it worked), the idea of dzz (which sounded quite promisising) doesn't seem to be the problem. No clue what else to look at.

Did you try it on squeeze? Could be a squeeze thing.
Post a reply