Squashed portage tree

Introduction

The Gentoo package manager, Portage, uses a lot of disk space mostly due to a high number of small files. By storing the tree in a SquashFS file the used disk space goes down from about 740MB to 70MB !! It is also a lot faster due less number of times the disk is accessed.

Another nice side effect is that distributing portage locally by copying the squashfs file or even exporting the SquashFS file via NFS (read-only) is now much faster and efficient.

Some personal choices first:
I use a dedicated volume to store all portage related things mounted under /mnt/portage. Therefore I chose the default path to store the squashFS backing files to be /mnt/portage/portage.sqfs.
The squashed tree will be mounted at the portage default location /usr/portage.
If your setup differs from that layout please adjust various commands below accordingly.

Requirements

  • Kernel support for Squash and Overlay file systems:
File systems -> Miscellaneous filesystems -> SquashFS 4.0 - Squashed file system support 
File systems -> Overlay filesystem support
  • Installed sys-fs/squashfs-tools

  • Distfiles and if used also PKGDIR moved outside of the portage tree, e.g. in /etc/portage/make.conf:
DISTDIR="/mnt/portage/distfiles"
PKGDIR="/mnt/portage/packages"

 

Preparation

  • required kernel modules are loaded, if not compiled in statically:
# modprobe loop squashfs overlay

in case you are use OpenRC make sure /etc/conf.d/modules contains contains:

modules="loop squashfs overlay"

  • /etc/fstab entry for squashfs portage tree:
/mnt/portage/portage.sqfs /usr/portage squashfs ro,defaults,nosuid,nodev,noexec 0 0

 

Squash it

To create the initial squashed portage tree:

# update_portage.sh -p <portage tree> -s <squashfs file>

In case your portage location is NOT /usr/portage, make sure your portage file system location matches the entry in /etc/portage/repos.conf/gentoo.conf and matches the -p option calling update_portage.sh !

 

Updating portage

Either manually execute the script or to update daily via cron:

# ln -s /usr/local/sbin/update_portage.sh /etc/cron.daily/update_portage.sh


Update:

Added proper cleanup in case the script gets interrupted!

Ebuilds for RegRipper, DNSEnum, fierce, theHarvester and Dradis

While doing some security consulting I always update some ebuild while doing so.
Most ebuilds are based on the work from the guys at pentoo.ch !

  • app-forensics/regripper ** ( git master only )

DESCRIPTION="Perl scripts to parse Windows registry files"
HOMEPAGE="https://regripper.wordpress.com/regripper/"

  • net-analyzer/dnsenum 1.2.4.2

DESCRIPTION="A perl script to enumerate DNS from a server"
HOMEPAGE="https://github.com/fwaeytens/dnsenum"

  • net-analyzer/theHarvester  ** ( git master only )   

DESCRIPTION="The Harvester is a tool designed to collect email accounts of the target domain"
HOMEPAGE="http://www.edge-security.com/theharvester.php"

    • net-analyzer/fierce   1.0.3

DESCRIPTION="Fierce is a DNS reconnaissance tool written in perl"
HOMEPAGE="http://ha.ckers.org/fierce/"

  • app-misc/dradis

DESCRIPTION="A framework for effective information sharing"
HOMEPAGE="http://dradisframework.org/"

 
Check out my overlay.

 

Using the Android SDK on Gentoo

Installation

1. Unmask and install the Android SDK update manager:

emerge dev-util/android-sdk-update-manager

In case you run into issues during compilation of any java ebuilds, try using the oracle-jdk-bin (1.7) !
For some reason on my machine some ebuilds failed using the icedtea-bin.

2. Add all users who should be allowed using it to the 'android' group:

gpasswd -a <user> android

3. In case you added yourself relog now, then run:

export ANDROID_SWT=/usr/share/swt-3.7/lib
/opt/android-sdk-update-manager/tools/android

and install at least the platform-tools package.

ADB will be at /opt/android-sdk-update-manager/platform-tools/adb

5. Finally edit your .bashrc file to add adb to your PATH:

 export PATH="/opt/android-sdk-update-manager/platform-tools/:${PATH}" 

Connecting

Add the following udev rules to e.g. /etc/udev/rules.d/80-android.rules:

SUBSYSTEM=="usb", ATTR{idVendor}=="22b8", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="0bb4", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="04e8", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0666", GROUP="plugdev"
Restart udevd and make sure you are member of the plugdev group.

Plugin your Android device via usb

Test the connection

adb devices

That should give you a list of connected Android devices

Now what ?

Install SystemRescueCD on USB

On Hardened Gentoo Linux usb_inst.sh script fails because it attempts to run binaries which use RWX mmap and thus they are killed by the kernel due to PaX mprotect:

/mnt/iso # bash ./usb_inst.sh 
Device [/dev/sde] detected as [ Patriot Memory ] is removable and size=15258MB
* Device [/dev/sde] is not mounted
PROT_EXEC|PROT_WRITE failed.
PROT_EXEC|PROT_WRITE failed.

To work around this we need to run paxctl-ng -m on a few binaries:

--- /tmp/usb_inst.sh	2014-06-22 18:33:26.000000000 -0700
+++ /mnt/cdrom/usb_inst.sh	2013-03-24 11:53:05.000000000 -0700
@@ -565,6 +565,9 @@
         PROG_MKVFATFS="${TMPDIR}/mkfs.vfat"
         PROG_SYSLINUX="${TMPDIR}/syslinux"
         PROG_DIALOG="${TMPDIR}/dialog"
+        paxctl-ng -m ${PROG_DIALOG}
+        paxctl-ng -m ${PROG_INSTMBR}
+        paxctl-ng -m ${PROG_MKVFATFS}
         # syslinux requires mtools
         ln -s mtools ${TMPDIR}/mcopy
         ln -s mtools ${TMPDIR}/mmove

Either patch the script in place ( mount RW first ), or copy the script to e.g. /tmp:

/mnt/iso # bash -c '. /tmp/usb_inst.sh' usb_inst.sh

My overlay is now in offical layman list

How to use my overlay

As my overlay was recently added to the layman list of overlays you can now easily add it running:

layman -S && layman -a quarks

or you checkout out the repository manually:

git clone git://www.startux.de/quarks.git
git clone http://www.startux.de/git/quarks.git