Shell - wait for keypress to continue

read -p "Press any key to continue..."

Vim - search and replace in multiple files

You can use :argdo command once you have opened multiple files using for example vim -p file1 file2 etc. Example:

:argdo %s/text_to_replace/new_text/gc

Pipe comand output to scp

If you want to pipe the shell command output to scp, eg. output of ls, you can use the following example:

scp $(ls -1 *.txt | head -1000) user@host:directory

Shell if statement

Shell if statement example1:

#!/bin/sh

if [ "$PASSWORD" == "PASSWORD" ]; then
echo "You have access!"
else
echo "ACCESS DENIED!"
fi


Example2:

#!/bin/sh

if [ "$N" -lt 20 ] || [ "$N" -ge 50 ]; then
echo "Sorry. Out of range."
elif [ "$N" -ge 20 ] && [ "$N" -lt 30 ]; then
echo "Ok"
elif [ "$N" -ge 30 ] && [ "$N" -lt 40 ]; then
echo "Ok"
elif [ "$N" -ge 40 ] && [ "$N" -lt 50 ]; then
echo "Ok"
fi

Block ip address with iptables

To block a traffic from a certain ip address to your machine because eg. it's doing a hacking:

• Check DNS of this address to be sure you're not blocking someone important:
  $ dig ip_address
• Add new rule in iptables:
  $ iptables -A INPUT -s ip_address -j DROP
• In Redhat based distributions (Fedora, CentOS, Enterprise Linux):
  $ service iptables save

Shell for loop

General for loop structure in shell:

for { variable } in { list }
do
#loop body eg.
echo list_element
done


Here's the example:

for i in 1 2 3 4 5 6 7 8 9 10
do
echo "Variable i = "$i
done


To loop over a range with a certain limits:

for (( i = 0 ; i <= 5; i++ ))
do
echo "Variable i = "$i
done

grep - display lines before and after

Display n lines before and m lines after result:
$ grep -B n -A m

Wildcards in wget in http

Use

wget -r -l1 --no-parent -A.jpg http://www.site.com

to download all *.jpg files from http://www.site.com to level 1 in subdirectories tree.

Linux - find files larger than

Find files in the directory larger than 500M:

find /path -size +500M


Shell script - loop over files from the text file

In shell script to loop over the files from the list in the text file simply do:

#!/bin/sh

#Get the file list from the command line
filelist=$1; shift

#Loop over the files from the list
while read f
do
echo $f
#Put your commands here...
done < $filelist


cvs - show differences between revisions/releases

To show the differences between two file revisions:

$ cvs diff -kk -u -r 1.14 -r 1.15 file.c

To show the differences between two tagged versions/releases of the project (in each of the files):

$ cvs diff -r RELEASE_1_0 -r RELEASE_1_1

A context diff between two releases redirected to the file:

$ cvs diff -c -r RELEASE_1_0 -r RELEASE_1_1 > diffs


Ubuntu - startup programs

To add new program to the startup:

1. Go to System > Preferences > Sessions.


2. ‘Startup Programs’ tab.


3. Click on Add


4. Give the name and command, and hit OK

tee command in linux

Redirecting the output (standard output = stdout) of the program both to the file and to the screen:

program | tee outputfile.txt

Forcing both the standard error and the standard output to be written in the file:

program 2>&1 | tee outputfile.txt

Appending into the file instead of rewriting it:

program | tee -a outputfile.txt


C - parsing command line options with getopt

Very useful way to parse the command line options in C/C++:


void usage(){
std::cout << "Usage: Program.exe [options] inputFile" << std::endl;
std::cout << "Options are:" << std::endl;
std::cout << "-n x : read only n entries" << std::endl;
std::cout << "-o FILE: set output file" << std::endl;
}

int main(int argc, char* argv[]){

std::string outputFileName;
int nEvents = -1;

//Parse options from the command line
for(;;){
//Get option
int c = getopt(argc, argv, "n:o:");

//If end of options break...
if (c<0) break;

switch(c){

case 'n':{
nEvents = atoi(optarg);

std::cout << "Number of entries to read : " << nEvents << std::endl;
break;
}

case 'o':{
//std::cout << "Optarg:" << optarg << std::endl;
outputFileName = std::string(optarg);

std::cout << "Output filename is : " << outputFileName << std::endl;
break;
}

case '?':{
usage();
exit(0);
}//end of case

}//end of switch
}//end of for loop

if((argc - optind) < 1){
usage();
exit(1);
}
}


Colored svn diff

svn diff command shows the differences between two versions of the file. If you want to get the nice, colored output of this command just do:

1. Install colordiff (eg. on Ubuntu):
   sudo apt-get install colordiff
2. Simply redirect svn diff output to colordiff:
   svn diff -r Rev1:Rev2 file | colordiff
3. You can declare a simple function in your ~/.bashrc file:
   svndiff() {
svn diff "${@}" | colordiff
}
4. To test your new svndiff function run:
   svndiff -r Rev1:Rev2 file
5. To color longer outputs you can use vi editor:
   svn diff -r R1:R2 file | vim -

SSH without a password

To avoid entering your password every time you log in via ssh, add your public key to the .ssh/authorized_keys file in your home directory on the remote machine.

On the local machine:

• Generate pair of keys:
  ssh-keygen
  and enter the name of the key (default is id_rsa), then hit enter twice (password prompts).
• Append the public key (key_name.pub) to the list of keys on the remote machine:
  < cat ~/.ssh/key_name.pub | ssh user@remote_machine "cat >> ~/.ssh/authorized_keys"
• Make sure that ~/.ssh/authorized_keys has the 600 permissions!
• If you entered the name of the key different that default one (id_rsa) then you need to connect/copy with the -i option (ssh -i ~/.ssh/key_name or scp -i ~/.ssh/key_name)

MySQL create user

To create user:


CREATE USER 'username'@'host' IDENTIFIED BY 'password';

To grant all privileges to user on a specific database:

GRANT ALL PRIVILEGES ON database_name.* TO 'username'@'host';


MySQL import dump

To import database from a dumped sql file:

• Login to mysql


• Create database with the specific name:
  
  
create database database_name;

  


• Exit mysql.


• Import the contents of the database from a file:
  
mysql -u user -p database_name < database_dump.sql

  

MySQL - creating backup

Simplest solution is to use mysqldump tool.

• To create backup of a specific database (database_name):
  
  
mysqldump -u root -p database_name > dumpfile.sql

  and enter your root password.
  


• To backup all databases:
  
  
mysqldump -u root -p --all-databases > dumpfile.sql

  

MySQL - resetting root password

In order to reset your root password for mysql:

1. Find mysqld_safe application (usually in /usr/bin/mysqld_safe)
2. 
   
3. Stop currently running mysqld process:
   
/etc/init.d/mysqld stop

   


4. Start mysql in a safe mode with --skip-grant-tables option using previously found path of mysqld_safe. This gives full access for all users with no passwords:
   
/usr/bin/mysqld_safe --skip-grant-tables &

   


5. Enter mysql:
   
mysql

   


6. Type:
   UPDATE mysql.user SET Password=PASSWORD('MyNewPass') WHERE User='root';
FLUSH PRIVILEGES;

   Mind that UPDATE and FLUSH should be in different lines.
   


7. Restart mysql with usual settings:
   
/etc/init.d/mysqld restart

   

Mail from command line

To send an email from command line you can use "mail" command.
Example:
echo "Message body" | mail -s "Test_subject" example@exampledomain.com
Sends message "Message body" with a subjects "Test_subject" to example@exampledomain.com

Testdisk - good and simple recovery tool

Testdisk is a very nice data recovery tool: http://www.cgsecurity.org/wiki/TestDisk.


Step by step guide how to recover your files from damaged disk using PhotoRec from Testdisk suite: http://www.cgsecurity.org/wiki/PhotoRec_Step_By_Step

Python - remove leading zeros

To remove one or more leading zeros from the string eg. convert 0040 into 40:

import re
oldstring = '0040'
newstring = re.sub(r"\b0{2}","",oldstring)

To remove different number of zeros change the number in {} in the code.

EDIT:
Better way:


import re
oldstring = '0040'
newstring = re.sub("^0+","",oldstring)


Checking ntfs partition on linux

Use ntfsfix application from ntfs-3g package eg.:

$ sudo ntfsfix /dev/sdc1


Python - get date and time

To get current date and time:

from time import strftime

strftime("%Y/%m/%d %H:%M:%S")

'2010/07/28 21:50:30'


Format drive to ntfs in Ubuntu

If you want to format eg. external usb drive to NTFS in Ubuntu you can use gparted. You will also need ntfsprogs package:


sudo apt-get install gparted ntfsprogs


Python - replace character in a string

To replace a character("a") in a string with the other one ("b"):

s = s.replace('a', 'b')


Python - get index in the list

To get the index of the element in the list:

list.index(element)


Linux - finding last reboot/shutdown time

To find last reboot time:

last reboot | less

To find last shutdown time:

last -x | grep shutdown | less


Python and config file parser

Very useful Python module for parsing config files:
http://docs.python.org/library/configparser.html

ssh access using public and private keys

1. Generate a key pair on your machine:
   
   $ ssh-keygen
   
   Enter the name of the files with the keys and passphrase (not necessary).


2. Add your public key to the list of authorized keys on the remote machine:
   
   
   cat ~/.ssh/id_rsa.pub | ssh user@machine "cat >> ~/.ssh/authorized_keys"


3. Set permissions on the remote machine:
   
   
chmod 700 ~/.ssh

chmod 600 ~/.ssh/authorized_keys

   

Tracking changes between file revisions in CVS

• To check file history (and revisions):
  
$ cvs log filename

  


• To compare two revisions (REV1 and REV2) of the file:
  
$ cvs diff -u -r REV1 -r REV2 filename

  

Passing arguments to awk from shell

To pass argument SHELLARGUMENT to awk from a shell do:

$ awk -v x="$SHELLARGUMENT" '{print x}'

eg.:

#!/bin/sh

SHELLARGUMENT=test

awk -v x="$SHELLARGUMENT" '{print x}'



Echo without newline

To print a text without a newline in shell script:

$ echo -n "Text"

diff print only unmatched lines

To print only unmatched lines from file2:

$ diff file1 file2 | grep ">" | awk -F '>' '{print $2}'

Loop over a files in directory in shell


#!/bin/sh

for f in `ls`

do

echo “Processing $f file …”

#Here you can put your commands

done


You can loop over files with specific names changing ls to ie. ls *.txt etc.

Awk - powerful tool

Awk is a scripting language that can be used in shell. Please find here some examples:

1. We've got a file that contains lines and columns (separated with spaces). We want to print only the contents of 3rd column.
   
   cat file.txt | awk '{print $3}'
   


2. Print last column:
   
   cat file.txt | awk '{print $NF}'
   
   $NF means for awk the last argument (here: column)


3. Print the first expression before '/' sign:
   
   cat file.txt | awk -F '/' '{print $1}'
   
   Using -F option you can set different separators for awk arguments.


4. Another example. Printing last column + some text before it + some text after:
   
   cat file.txt | awk '{print "cp /home/"$1" "$1}'
   
   If you have filenames from /home directory in the file.txt this will prepare copying those file to current directory.

Grep: get patterns from a file

Having a list of patterns in patternsfile and needed to search for those patterns in datafile use:

$ grep -f patternsfile datafile

Double pipe

Nice feature of double pipe "||":


$ command1 || command2


This means that command2 will be run if the command1 fails.

Clickable links in Latex table of contents

1. Use hyperref package:
   
   \usepackage{hyperref}


2. Set link colors:
   
   \hypersetup{

colorlinks,

citecolor=black,

filecolor=black,

linkcolor=black,

urlcolor=black

}


3. Link only the page numbers and not the entire table of contents:
   
   \hypersetup{linktocpage}

Change color in OpenOffice formula

color <colour>{expression}

File extension in shell

FILENAME="file.txt"

1. To print file's extension:
   
   $ echo ${FILENAME#*.}
   
   txt


2. To print file name without extension:
   
   $ echo ${FILENAME%.*}
   
   file

Tracking memory leaks

Very powerful tool for checking memory leaks and speeding up the applications:

1. Valgrind (http://valgrind.org)


2. Valkyrie - front-end for Valgrind


3. Callgrind (valgrind -tool=callgrind): CPU-time profiling tool


4. kcachegrind - front-end for callgrind

User management in terminal

1. Adding new user:
   
   $useradd user


2. Setting password:
   
   $passwd user


3. Assigning user to a group:
   
   $usermod -a -G group user

Update-alternatives

If you have installed two versions of the application and you need to switch between them - there's a nice tool: update-alternatives. How to use it? Here's an example: switching between gcc-4.3 (and simultaneously g++-4.3) and gcc-4.4 (g++-4.4):

1. $ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.3


2. $ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 40 --slave /usr/bin/g++ g++ /usr/bin/g++-4.4


3. $ sudo update-alternatives --config gcc