Posts Tagged ‘Linux’


March 4th, 2014


  • find /home -iname “*.log” -exec rm {} \;

    Delete file recursively by extension.


linux batch rename files

October 28th, 2011

Use this command to rename a set of files:

for file in * ; do mv $file `echo $file | sed 's/OLD_STRRING/NEW_STRING/'` ; done

And here is a batch file:


if [ $# -ne 3 ]; then
        echo "Usage: $0 'FilePattern' SearchString ReplaceString";
        echo "Eg: $0 'r-*.sh' 5 6";
        for file in $1 ; do mv $file `echo $file | sed s/$2/$3/` ; done

If the 2nd or 3rd parameter contains space, use ‘ ‘ to surround the parameter.

mysqlpp connection timeout problem

October 3rd, 2011

If you have a application having long time connection with mysql using mysqlpp. You may at first think of using Connection::connected() to determine whether the connection is still valid. But actually this doesn’t work if the connection gets timeout.

After checking the reference, I found out Connection::ping() would be a good way to determine timeout.

Here is a piece of test code:

#include <iostream>
#include <string>
#include <mysql++.h>

using namespace std;

int main() {
	string db_schema("DB_SCHEMA");
	string db_host("DB_HOST");
	string db_user("DB_USER");
	string db_pass("DB_PASS");
	string qry_str("SELECT MAX(user_id) FROM users");

	mysqlpp::Connection cn;
	cn.connect(db_schema.c_str(), db_host.c_str(), db_user.c_str(), db_pass.c_str());
	mysqlpp::Query qry = cn.query(qry_str);
	mysqlpp::StoreQueryResult res =;
	cout << "frist query result: " << res[0][0] << endl;

	string tmp;
	cout << "Wait for a period of time to let the connection timeout, and then input whatever a string to continue: " << endl;
	cin >> tmp;

	cout << "ping result: " << << endl;
	cout << "connected: " << cn.connected() << endl;
		cn.connect(db_schema.c_str(), db_host.c_str(), db_user.c_str(), db_pass.c_str());
	mysqlpp::Query qry2 = cn.query(qry_str);
	res =;
	cout << "second query result: " << res[0][0] << endl;

	return 0;	

But of course, I’m still new to mysqlpp. There may be a better way to do this. Please comment below if you have a better way.

NOTE: Here is how to change mysql connection idle timeout.

UPDATE: Just found another way to handle the timeout problem: use Connection::set_option(new mysqlpp::ReconnectOption(true));. This is much more elegant.
Here is a piece of sample code:

mysqlpp::Connection cn;
cn.set_option(new mysqlpp::ReconnectOption(true));
cn.connect(db_schema.c_str(), db_host.c_str(), db_user.c_str(), db_pass.c_str());

bash script for killing all mysql processes / connections

September 29th, 2011

Here is a small bash script for killing all mysql processes (or connections).

The basic idea is to select all process IDs into a tmp file (each process ID a line), and then read the file line by line and kill the process, finally remove the tmp file.

This bash only works for MYSQL >= 5.1.7. Becuase before 5.1.7 process list is not stored in the table INFORMATION_SCHEMA.PROCESSLIST.


  • either by providing the MYSQL root password as a parameter to the bash as:
    user> MysqlRootPass
  • or by editing the password the line mysqlpass=$1 as:
    mysqlpass=MysqlRootPass and then excute the bash without parameter:

    Of course, you can also edit the other variables in the bash file.



# mysql host/user/pass who has the privilege to read schema INFORMATION_SCHEMA
# tmp file path

# select all process IDs into the tmp file. each line has a process id.
echo "SELECT id FROM INFORMATION_SCHEMA.PROCESSLIST INTO OUTFILE '$tmpprocesslistfile';" | mysql -h$mysqlhost -u$mysqluser -p$mysqlpass INFORMATION_SCHEMA

# read the file line by line and kill the process
while read qryid
echo "kill $qryid;" | mysql -h$mysqlhost -u$mysqluser -p$mysqlpass
done < "$tmpprocesslistfile"

# remove the tmp file
echo 'All connections have been killed.';
echo 'Please enter sudo password to remove the tmp file if necessary.';
sudo rm $tmpprocesslistfile

System V message queue example (msgget, msgctl, msgsnd, msgrcv)

September 21st, 2011

Found a very good example of message queue showing how to use msgget, msgctl, msgsnd, and msgrcv at Here I list the code with tiny fix and some of my comments.

The linux man page of msgsnd & msgrcv can be found here:


extern int errno;       // error NO.
#define MSGPERM 0600    // msg queue permission
#define MSGTXTLEN 128   // msg text length

int msgqid, rc;
int done;

struct msg_buf {
  long mtype;
  char mtext[MSGTXTLEN];
} msg;

int main(int argc,char **argv)
  // create a message queue. If here you get a invalid msgid and use it in msgsnd() or msgrcg(), an Invalid Argument error will be returned.
  if (msgqid < 0) {
    printf("failed to create message queue with msgqid = %d\n", msgqid);
    return 1;
  printf("message queue %d created\n",msgqid);
  // message to send
  msg.mtype = 1; // set the type of message
  sprintf (msg.mtext, "%s\n", "a text msg..."); /* setting the right time format by means of ctime() */

  // send the message to queue
  rc = msgsnd(msgqid, &msg, sizeof(msg.mtext), 0); // the last param can be: 0, IPC_NOWAIT, MSG_NOERROR, or IPC_NOWAIT|MSG_NOERROR.
  if (rc < 0) {
    perror( strerror(errno) );
    printf("msgsnd failed, rc = %d\n", rc);
    return 1;

  // read the message from queue
  rc = msgrcv(msgqid, &msg, sizeof(msg.mtext), 0, 0); 
  if (rc < 0) {
    perror( strerror(errno) );
    printf("msgrcv failed, rc=%d\n", rc);
    return 1;
  printf("received msg: %s\n", msg.mtext);

  // remove the queue
  if (rc < 0) {
    perror( strerror(errno) );
    printf("msgctl (return queue) failed, rc=%d\n", rc);
    return 1;
  printf("message queue %d is gone\n",msgqid);

  return 0;

————– Update 14/Dec/2011 ——————
A note about the struct msg_buf. From: “The Linux Programmer’s Guide”

The ability to assign a given message a type, essentially gives you the capability to multiplex messages on a single queue. For instance, client processes could be assigned a magic number, which could be used as the message type for messages sent from a server process. The server itself could use some other number, which clients could use to send messages to it. In another scenario, an application could mark error messages as having a message type of 1, request messages could be type 2, etc. The possibilities are endless.

On another note, do not be misled by the almost too-descriptive name assigned to the message data element (mtext). This field is not restricted to holding only arrays of characters, but any data, in any form. The field itself is actually completely arbitrary, since this structure gets redefined by the application programmer. Consider this redefinition:

struct my_msgbuf {
long mtype; /* Message type */
long request_id; /* Request identifier */
struct client info; /* Client information structure */

Here we see the message type, as before, but the remainder of the structure has been replaced by two other elements, one of which is another structure! This is the beauty of message queues. The kernel makes no translations of data whatsoever. Any information can be sent.

command line network tools on linux

July 29th, 2011


  • netstat -tulpan:
  • netstat -tl
  • netstat -anp –tcp –udp: will show all open tcp and udp connections.


  • lsof -i
  • lsof -i | grep 80
  • lsof -i | grep apache2


display bandwidth usage on an interface by host

Install & use mysql++ on linux(debian squeeze, ubuntu)

July 28th, 2011

The key to use mysql++ on debian/ubuntu system is to find where the mysql++ .h head files and the .so binary files.

Firrst install mysqlclient and mysql++ on debian squeeze:

root# apt-get install libmysqlclient-dev libmysqld-dev libmysql++-dev libmysql++-doc libmysql++3

This will install the mysql head files in /usr/include/mysql, mysql++ head files in /usr/include/mysql, and and in /usr/lib. These are all the files we need.

Now let us use the mysql++ library.

Save the simple code below as test.cpp

#include <iostream>
#include <mysql++.h>

int main() {

mysqlpp::String s(“hello, world”);

std::cout << s << std::endl;

return 0;


And save the makefile as Makefile:

CXX := g++
CXXFLAGS := -I/usr/include/mysql -I/usr/include/mysql++

LDFLAGS := -L/usr/lib -lmysqlpp -lmysqlclient -lnsl -lz -lm

all: test
clean:        rm -f $(EXECUTABLE) *.o

Finally, excute

make all

The compile result would be test.

compress & decompress files under Linux

February 24th, 2011



zip -r dir





rar a xxx.rar dir


unrar e xxx.rar



bzip2 file


bunzip2 xxx.bz2 (NOTE: use the switch -k to keep the .bz2 file)



gzip file (NOTE: -r, --recursive   operate recursively on directories)


gunzip xxx.gz



tar -cjf xxx.tar.bz2 dir


tar -jxvf xxx.tar.bz2



tar -czf xxx.tar.gz dir


tar -zxvf xxx.tar.gz

set up postfix on debian

July 13th, 2010

Suppose you have 2 sites: & And you would like to set as a local domain, and as a virtual domain. That means emails of will go to the system user’s personal mail dir while has no actual corresponding system user. Details about local domain & virtual domain, please see paragraph “Virtual domains versus local domains” at

part 1: configure local domain

  • install postfix
    apt-get install postfix
  • config postfix
    add the domain to config files using psotconf:
    postconf -e "myorigin ="

    add hostname:
    postconf -e ""
  • Reload Postfix Server:
    postfix reload

Now you can try to send a email via telnet.
telnet localhost 25
You should see:

Connected to localhost.
Escape character is ‘^]’.
220 ESMTP Postfix (Debian/GNU)

Now send a test mail to yourself:

mail from:<>
rcpt to: <>
Subject: First test mail to yourself
Damn it! It worked! Hahahahaha…

End the input with an enter, then a dot “.”, and an enter again.

And then quit:

Now use mail to check your email.

part 2: configure virtual domain

  • create a user and take his home dir as the virtual server’s base dir.
    #> groupadd -g 5000 vmail
    #> useradd -g vmail -u 5000 vmail -d /home/vmail -m
  • set the virtual_uid_maps and virtual_gid_maps to these IDs:
    #> postconf -e virtual_uid_maps=static:5000
    #> postconf -e virtual_gid_maps=static:5000
  • add as a virtual domain into virtual_mailbox_domains
    #> postconf -e
  • set the base dir
    #> postconf -e virtual_mailbox_base=/var/spool/vmail
  • reload postfix
    #> postfix reload;
  • add virtual email addresses
    • tell which file the email addresses are stored in
      #> postconf -e virtual_mailbox_maps=hash:/etc/postfix/vmailhost
    • create the file and add some addresses in there.
      #> vi /etc/postfix/vmailhost
      with content like this:

      So, the mail files for service will actually be stored in /home/vmail/, and mail files for webmaster will be in /home/vmail/
    • Now, map the file into DB
      #> postmap /etc/postfix/vmailhost
  • Probably you also need to create the dirs for these 2 addresses
    mkdir /home/vmail/
    #> mkdir /home/vmail/
    #> cd /home/vmail/
    #> mkdir new curr tmp
    And do the same for
  • change the owner and permissions if you created this folders manually and with ROOT privilege:
    #> chown -R vmail:vmail /home/vmail/
    #> chmod -R 700 /home/vmail/

OK, now you can send mail via telnet to have a test.

part 3: configure alias for

If you want to forward received mails of, you can use a alias to forward to your another email address, let’s say

  • tell where is the alias configuration file
    #> postconf -e "alias_maps = hash:/etc/postfix/aliases"
    postfix reload;
  • add forward rules in the file:
    #> vi /etc/postfix/aliases
    with the conetent:
    and map it:
    #> postmap /etc/postfix/aliases


  • Every configuration done by postconf, you can actually do it by editing /etc/postfix/ directly.
  • Every time you change with command postmap, do not forget to reload postfix with: #> postfix reload.
  • Every time you change a db file, do not forget to re-postmap it with: #> postmap /etc/postfix/xxxxx.
  • If you want to set both sites into virtual server, only set localhost as your destination and both sites into virtual domain.

Important debug method

First, all check the log which should be /var/log/mail.log, no matter you want to find the cause of the failure or to check that your email is sent/received successfully.

Second, use mutt to try to send/receive emails

Important references


———————– v1.1: 27/08/2010 ————————

How to add SASL auth

atually a complete instruction for install postfix on debian
official postfix manual

———————– v1.2: 27/08/2010 ————————

How to set up SPF

———————– v1.3: 03/09/2010 ————————

How to set up domain keys & dkim

Setting up domain keys with dkfilter:

Note: as the dkfilter author has said, dkfilter is deprecated now and DKIMproxy is recommended.

Follow this link : except the settings in file /etc/postfix/ should be:

pickup    fifo  n       -       n       60      1       pickup
    -o content_filter=dksign:[]:10027

And, you may also need to add these as said in

# modify the default submission service to specify a content filter
# and restrict it to local clients and SASL authenticated clients only
submission  inet  n     -       n       -       -       smtpd
    -o smtpd_etrn_restrictions=reject
    -o smtpd_sasl_auth_enable=yes
    -o content_filter=dksign:[]:10027
    -o receive_override_options=no_address_mappings
    -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject

# specify the location of the DomainKeys signing filter
dksign    unix  -       -       n       -       10      smtp
    -o smtp_send_xforward_command=yes
    -o smtp_discard_ehlo_keywords=8bitmime

# service for accepting messages FROM the DomainKeys signing filter
# inet  n  -      n       -       10      smtpd
    -o content_filter=
    -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
    -o smtpd_helo_restrictions=
    -o smtpd_client_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o mynetworks=
    -o smtpd_authorized_xforward_hosts=

How to generate key:

perl -MMIME::Base64 -e 'print encode_base64("cihangcihangpassword");'

How to test:

telnet localhost 25
ehlo localhost
auth plain Y2loYW5nAGNpaGFuZwB4TDM4MjVNdjZ1

Setting up Domain Keys and/or DKIM with DKIMproxy:

didn’t try. see here: Mail-DKIM and DKIMproxy.

setting java environment under ubuntu

May 20th, 2010

you can set all the environment vars in /etc/environment.

for example:



use bash> source /etc/environment to refresh the system variables.