The Innovation Lover

chroot() is a useful system call available in most UNIX like operating systems. I have been using chroot to do several hacks for the past few years. Mostly people use chroot for fixing boot loader / GRUB. To fix GRUB, a live cd can be used to boot into a GNU/Linux system, then run

# chroot /mnt/root_partition

~# echo chrooted environment

Then, execute grub-install or any command to update GRUB configs.

Basically chroot makes the environment believe provided path is the root “/” of the filesystem.

We can exit from chrooted environment by pressing Ctrl-D.

chroot can be used to build chroot jail to protect server services for preventing attacker to gain complete access to the server by creating chroot jails.

Last day, I was working on my GSOC project Live Installer for Pardus. It was the first time, I was using chroot() in python. It had to execute a few statements in chroot environment and come back to prevous environment. But exiting from chroot environment found to be difficult and there were no direct methods to exit from it. So I had to do a little hack. I would like to share the hack so that you can reuse it without going for long search on how to do it.

import os
real_root = os.open("/", os.O_RDONLY)
os.chroot("/mnt/new_root")
# Chrooted environment
# Put statements to be executed as chroot here
os.fchdir(real_root)
os.chroot(".")

# Back to old root
os.close(real_root)

chroot() is provided by os module
The major player of this hack is fchdir() which can take file descriptor has argument and change to that directory as current working directory. We open our real root using real_root = os.open(”/”, os.O_RDONLY) and its file descriptor is stored in real_root. Now chroot to new file system. Execute all the required statements. After that execute fchdir() to change current directory to old_root using real_root descriptor. Then chroot to current directory to switch back to real root.

Happy Hacking

Recently I were looking for youtube downloaders over the web. I found many websites filled with lot of ads. Few websites where there which gave some neat interface to download videos. I got interested and decided to write my own shell script to parse and download videos. By looking into HTTP requests, I came through how downloaders work.

The logic is pretty simple. By passing the video_id which is received as v=videoid from youtube video URLs, to? http://www.youtube.com/get_video_info?video_id=videoid, we obtain a metadata file which contain metadata about the video we need to download. We extract a parameter called tokenid. Again pass the video_id and token to the same URL to obtain the video. We can also specify formats in which it is to be downloaded such as mp4,flv or 3gp in different video qualities. fmt=id parameter is passed to specify file format. By carefully watching the HTTP requests from youtube page, I collected variety of fmt arguments for different formats and quality. I have compiled all these info to write a youtube video downloader shell script. Download it and have fun.

#!/bin/bash
#Description: Youtube video downloader script
#Author: Sarath Lakshman
#url: http://sarathlakshman.info

if [ $# -ne 3 ];
then
    echo -e "Usage: $0 URL -format FORMAT\nFormats of different video qualities:\n1080 (mp4) - highest\n720  (mp4) - higher\n360  (flv) - high\n480  (flv) - low\n240  (flv) - lower\n3gp  (3gp) - least\n\nEg: $0 http://www.youtube.com/watch?v=yZPSx2r3TiY -format 1080\n"
    exit 0
fi
url=$1

declare -A formats;
declare -A extension;
formats[1080]=37;
formats[720]=22;
formats[480]=35;
formats[360]=34;
formats[240]=5;
formats[3gp]=13;

extension[1080]=mp4;
extension[720]=mp4;
extension[480]=flv;
extension[360]=flv;
extension[240]=flv;
extension[3gp]=3gp;

vid=`echo $1 | cut -d= -f2`

wget -O /tmp/meta.data "http://www.youtube.com/get_video_info?video_id=$vid&el=vevo" &> /dev/null

token=`sed 's/.*token=\([^&=]\+\).*/\1/g' /tmp/meta.data`
title=`sed 's/.*title=\([a-zA-Z0-9%+-]\+\).*/\1/g; s/-//g; s/[%+0-9]\+/_/g' /tmp/meta.data`

echo "Downloading..."

wget -o /tmp/log.$$ -O "$title.${extension[$3]}" "http://www.youtube.com/get_video?video_id=$vid&t=$token&fmt=${formats[$3]}"

if grep -q "Not Found" /tmp/log.$$ ; then
    echo "Unsupported format. Please try again with lower video quality format"
    rm $title.${extension[$3]} ;
else
echo Download complete. Play $title.${extension[$3]} and enjoy \).
fi

Download the scrpt from here : youtube_dl.sh

slynux@slynux-laptop:~/scripts$ ./youtube_dl.sh
Usage: ./youtube_dl.sh URL -format FORMAT
Formats for different video qualities:
1080 (mp4) - highest
720  (mp4) - higher
360  (flv) - high
480  (flv) - low
240  (flv) - lower
3gp  (3gp) - least

Eg: ./youtube_dl.sh http://www.youtube.com/watch?v=yZPSx2r3TiY -format 1080

Hey,

I have been working on GSOC project on Pardus Live installer. I work on a separate svn branch for GSOC. When working across many files, we many need to generate diff between numerous files recursively working directory and some other branch directory. svn diff command helps only if the directories are fork of same svn repo and with change in revisions or so.

I needed a generic tool to make diff out of two directories. But googling a few minutes didn’t give me pleasing results and hence I am here with my own script.

Try out and comment.

#!/bin/bash
#Filename: diffdir.sh
#Description: A utility to take diff of files recursively between two directories
#Author: Sarath Lakshman
#e-mail: sarathlakshman@slynux.com
#Homepage: http://www.sarathlakshman.info

olddir=$1
newdir=$2

if [ $# -ne 2 ];
then
	echo -e "\nUsage: $0 [DIR OLD] [DIR NEW] > data.diff\n\n"
	exit 0
fi

(cd $olddir; find . -type f ! -regex ".*\.svn.*" ) > /tmp/files.$$
(cd $newdir; find . -type f ! -regex ".*\.svn.*" ) >> /tmp/files.$$

sort /tmp/files.$$ -u -o /tmp/reqfiles.$$

cut -c3- /tmp/reqfiles.$$ > /tmp/uniqfiles.$$

for file in `cat /tmp/uniqfiles.$$`;
do

	[ -e "$olddir/$file" ] && [ -e "$newdir/$file" ]

	if [ $? -ne 0 ];
	then

		echo [NEW] $file\:
		echo ===================================================================
		[ -e "$olddir/$file" ] && cat "$olddir/$file"
		[ -e "$newdir/$file" ] && cat "$newdir/$file" 

	else

		diffed_data=`diff -u "$olddir/$file" "$newdir/$file"` 

		if [[ -n $diffed_data ]];
		then

			echo $file\:
			echo ===================================================================
			echo "$diffed_data"
			echo
		fi
	fi
done

Download the script : diffdir.sh

UPDATE : diff -Naur olddir newdir will do the stuff :p

Sometimes we get shocked when something unexpected occurs. We get stuck and feels like everything is over. Today I also had such a situation.

I was writing an important document over the last couple of days. I hadn’t kept a backup or draft, I had only single Open office text document file. While I was writing, my laptop got switch off suddenly due to some power button issue. When I powered it on and looked at the file, I got stuck. It shows 0 byte as size.
I ran file filename.odt, it shows empty data. OMG. It was around 12 Pages of text and WTH !

I lost my hope. But I decided to go for an experiment. Installed TestDisk, which the GNU/Linux recovery suite.
Ran the Photorec command. I selected the disk drive where the data is located, then chose scan unallocated space. I received 1000s of files of different format.

Ran,

 find . -type f -name "*.odt" | wc -l

gave me around 200 files.
I thought of greping the files to find the right file. But I understood that grep over binary files won’t work.
So I moved odt files to a directory.

mkdir odts
find . -type f -name "*.odt" -exec mv {} odts \;

Then I manually opened some of the files. Then I got earlier versions of the file which contains initial text only. So my breath became normal. I looked at the file size and noticed that it is of the size 27KB.
So again I filtered out files using size as a contraint.

mkdir filtered_odts
find . -type f -name "*.odt" -size +27k \
-exec mv {} filtered_odts \;

So I got a list of fewer number of files. I manually found out highest size which in the order of 27K, which was the file with all text I wrote. Voila, thanks to find and testdisk.