Crontab every seconds

Welcome again!

Today I will post a quick tip, something useful with a “clever” solution. Crontab every X seconds, let’s see the script.

Your script
Just modify the run function of what you want to do it every $SECLAUNCH seconds. This will be lauched once, even if the run functions needs more than $SECLAUNCH seconds. If it take less than, the function run will be launched at $SECLAUNCH – timeelapsed.

#!/bin/bash
# USING runScriptEverySeconds.sh
# Load defaults config
. /scripts/config # Loading defaults configurations
 
##########################
# Seconds between launch #
##########################
SECLAUNCH=5
     
###################
# FUNCTION TO RUN #
###################
     
function run {
   runYourScriptEvery5seconds.sh
}
     
. /scripts/runScriptEverySecond.sh #Script who make it possible running every seconds

Here is the script you have to set as /scripts/runScriptEverySecond.sh

#!/bin/bash

########### SCRIPT RUN COMMAND EVERY $SECLAUNCH SECONDS #################
#									#
# This scripts will launch function run() every SECLAUNCH seconds. 	#
# If the scripts need more than SECLAUNCH will be launched as soon 	#
# as the last ends.							#
#									#
# 			-- IMPORTANT --					#
#									#
# User needs to add .defaultScripts/runScriptEverySeconds.sh at the 	#
# end and define SECLAUNCH and run funcion.				#
#									#
#########################################################################

##########################
# Seconds between launch #
##########################
# SECLAUNCH=5

#######################
# FUNCTION TO DEFINE  #
#######################

# function run {
# 	date
# 	sleep $(($RANDOM%5+1))
# }



###############################################
#		DO NOT EDIT		      #
###############################################

DATE=`date +%s`

# Launch command

for (( ; ; ))
do
	NOW=`date +%s`
	RES=$(($NOW - $DATE))
	#echo $RES $SECLAUNCH
	if [ $RES -ge $SECLAUNCH ]
	then
		#echo "INSIDE"
		DATE=`date +%s`
		run
	else
		sleep 1
	fi
done

If you have questions, don’t worry! ask me here or at twitter @becloudin

Stay cloud!

Arbre Empathique Brest – Disable Failsafe Grub on Ubuntu Server

Hi everybody!

Today I will talk a something it happend me at my job. First I will explain in what I was working, later I will explain the problem and the easy solution.

If you follow me at LinkedIn you will know that I work for ADmira Digital Networks, one of the best digital signage companies. This weeks I’ve been working for a project called Arbre Empathique. This project consist in a very big iron tree placed in Brest with hydrophonic plants, cameras, screens, wifi and sensors. The cameras provide real time video to the screens, player takes images from the cameras, data from sensors sensors and shows different content at the same time everything at realtime!. The Wi-Fi was the method to connect people with the tree to get more information from it. If you want to know more, check the ADmira post or just ask ;).

The Wi-Fi was the problem, well, the ubuntu server that is behind the access point. As you probably don’t know, the digital signage  hardware and software has to be prepared to be placed in the worst sites you could ever imagine, and I can promise Brest is one of the worsts I’ve ever see. You have to be prepared to get energy fails, cold, warm, windy, raining, and everything has to work. That’s the reality of digital signage, it’s quite different from a normal software company, but you have to be prepared for the worst.

The problem was the ubuntu server 12.04, it works really great, but it held in Grub window waiting for a keyboard keystroke choosing a boot option. That couldn’t be a problem, if you get your servers in your office, but when your server is located in more than 1200Km, yes, that’s a problem.

The solution, change the Grub script adding a timeout when recordfail gets 1. Let’s see what was there, and what I’ve done it.

/etc/grub.d/00_header

make_timeout ()
{
    cat << EOF
if [ "\${recordfail}" = 1 ]; then
  set timeout=-1 # This tells Grub to wait the keystroke choose
else
  set timeout=${2}
fi
EOF
}

Here the solution, change the -1 to 10.

make_timeout ()
{
    cat << EOF
if [ "\${recordfail}" = 1 ]; then
  set timeout=10 # Here we can just wait 10 seconds 
else
  set timeout=${2}
fi
EOF
}

Remember to run update-grub as root and you’ll never be stucked again.

Just with that change our ubuntu will work although any kind of energy failure happens. Of course, hardware fails and you have to be prepared! That’s digital signage 😉

To finish you can see the video from Cloud9 the architects of that project.

 

Stay in the cloud!

Log rotate

Hi Cloud!

In this article we will explain an example of how to rotate logs. Ubuntu linux includes Logrotate a powerful tool that will help us to take everything under control.

Let’s starts.

Logrotate it’s commonly used to pack and save logs for a defined period of time. For example, logs of Apache webserver could be useful for just 1 month. Once we have one month and one day, the oldest log will be automatically deleted. Other logs could be saved more time, for example auth.log. This is completely configurable and really easy to set.

Other common configurations are the compress. The most of the logs are simple text lines one beside other. This kind of file are easy to compress and save lot of space. Logrotate has the option to do it. Other option is to run a script after o before logrotate starts. This is useful for example to save logs into other devices like tapes, other disks or uploaded to cloud services.

Other options, like notify by e-mail, notify if empty, are very interesting and you could try to use the manual and get more information. Remember, man logrotate and you will find everything you need.

One important thing with I get some problems is that logrotate could save gzip logs in other directory, BUT, and that’s very important, in the SAME DEVICE. If you want to save it elsewhere, you will have to write your own script to do it. Quite easy, but you will have to know it.

Let’s see the examples:

Log rotate NGINX
/etc/logrotate.d/nginx


/usr/local/nginx/logs/access.log {
        # We run logrotate every day
        daily 
        # If the log file is missing, go on to the next one without issuing an error message. See also nomissingok.
        missingok
        # Delete logs after 7 days in store
        rotate 7
        # Compress this logs
        compress
        # Delay this compress. This will perform access.log.1 and the next one will be compressed. This helps when you have to take a look of the day before log without having to uncompress the file.
        delaycompress
        # Do not rotate the log if it is empty
        notifempty 
        # Create new log with permisions 640, user root and group adm
        create 640 root adm
        # Needed to run ownscripts
        sharedscripts
        # Run it after rotate
        postrotate
                # Script to save Nginx logs in other device
                /scripts/logNGINXRotate.sh
                # Restart Nginx
                /usr/local/nginx/sbin/restartNginx.sh
        endscript
}

Scripts to save Nginx logs to other device.
This script save the rotated logs directly to S3 using s3fs mounted directory.
/scripts/logNGINXRotate.sh

#!/bin/bash
LogDate=$(date +"%Y-%m-%d.%H:%M")

logdir='/usr/local/nginx/logs/'
logdestdir='/mnt/s3/logs/api/nginx'

for filename in `ls $logdir | grep gz`
do

	mv $logdir$filename $logdestdir/access.$LogDate.tar.gz
done;

Scripts that restarts Nginx
/usr/local/nginx/sbin/restartNginx.sh

#!/bin/bash

pid=`cat /usr/local/nginx/logs/nginx.pid`
date=`date`

echo $date" ## RESTARGING NGINX ## PID:"$pid
kill -HUP $pid

Apache
Example scripts for Apache webserver.

/var/log/apache2/*.log {
        daily
        missingok
        rotate 30
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
                        /scripts.servers/api/logApacheRotate.sh
                        /etc/init.d/apache2 reload > /dev/null
                fi
        endscript
}

Scripts to rotate Apache webserver.
/scripts.servers/api/logApacheRotate.sh


#!/bin/bash
LogDate=$(date +"%Y-%m-%d.%H:%M")

logdir='/var/log/apache2/'
logdestdir='/mnt/s3/logs/api/apache'

for filename in `ls $logdir | grep gz`
do

	mv $logdir$filename $logdestdir/$filename.$LogDate.tar.gz
done;

The final step is to verify that everything do what you expect. With this command you can try at any time every change you do.

 logrotate --force /etc/logrotate.d/yourLogRotateFile

Stay cloud!

Generate SSH keys

Hi cloud!

Today I explain how to generate public/private keys. This is very useful for a lot of services that require something more secure than a typed password. The most common usage for private Key is the ssh connection. If we use private key we will never need to type password again, saving hundreds of hours and more than one security problem.

How we can create our new key?

 $ ssh-keygen 

Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
a1:04:aa:73:5a:8f:24:14:70:24:ca:30:37:ca:92:c1 ubuntu@ip-10-22-22-22
The key's randomart image is:
+--[ RSA 2048]----+
|Oo= .            |
|=E.o .           |
|=+.   . .        |
|o.   . . .       |
|+ +   . S        |
| B o             |
|. . .            |
|                 |
|                 |
+-----------------+

That’s it! We will have our key generated. We can find them at ~/.ssh/id_rsa.pub (Public key) and ~/.ssh/id_rsa (Private key). The private key is the file you NEVER provide to anyone. And remember to get correct permisions (600). If not, you will be noticed of incorrect permissions.

To change that permissions:

 $ chmod 600 ~/.ssh/id_rsa 

In next posts we will explain new techniques to save even more time in your normal commands.

But this will be in a few days!

Stay cloud!

Shell – How to Bookmark your commands ( ctrl + r )

Hi!

This is my first real post related with something useful. I hope this blog will be provably not a reference, but somewhere to find interesting things, more o less a “tips & tricks” of hundreds of different themes.

I will explain something useful, a few tips that save me hundreds of hours.

Search into your bash history file with command line.

 

$ ssh devel@yourserver.com #develop
$ (reverse-i-search)`dev': ssh devel@myserver.com #develop

 

What you have to do is to add #bookmark word. Every time you want to launch this command you just need to press ctrl + r and start typing the keyword. The shell will try to find in bash history any command containing this words. So once you start writing dev you will get the command containing dev word, in this case the ssh command to connect to your server.

Ctrl + r

If you have for example #develop and #develop2 you just need to press 2 times ctrl + r and you will get the occurrences of this words.

I promise if you try to use this frequently  you will save hundreds of hours typing the same command once, twice, …

 

Stay tuned!

Uso de cookies

Este sitio web utiliza cookies para que usted tenga la mejor experiencia de usuario. Si continúa navegando está dando su consentimiento para la aceptación de las mencionadas cookies y la aceptación de nuestra política de cookies, pinche el enlace para mayor información.plugin cookies