Apache 2.4 mod ldap authnz_ldap debian squeeze closed (connection lost)

It does not work out of the box.

You need to open and edit

/etc/ldap/ldap.conf

and add:

# TLS certificates (needed for GnuTLS)
TLS_CACERT /etc/ssl/certs/ca-certificates.crt

Let me know if this helped.

Docker cheat-sheet

Docker is great for development, but also, it gives you the superpower of extremely small images to easily move around docker containers.

I need this for personal stuff, but I think it can be useful to others, so let’s share it.

It’s a living list so I’m gonna add here stuff as soon as I need it.

Foreplay (Composer) – a.k.a. Install fest
#Install composer:
sudo apt-get -y install python-pip
sudo pip install docker-compose
Images
#Build an image from a dockerfile in pwd:
docker build --rm=true -t tagname .
#--rm=true -> Remove intermediate containers after a successful build
#-t, --tag=[] -> Name and optionally a tag in the 'name:tag' format
#-f, --file -> Name of the Dockerfile (Default is 'PATH/Dockerfile')
Docker Composer
#Start a container with:
docker-compose up -d
#up -> Create and start containers
#-d -> Detached mode: Run containers in the background,
       print new container names.
       Incompatible with --abort-on-container-exit.
Containers
#Check what's running:
docker ps

#Stop container:
docker stop {containerid}

 

 

Letsencrypt is broken on Debian Jessie

You need backports,

so first add the jessie-backports to your favourite apt source.list

# Backport
deb http://http.debian.net/debian jessie-backports main

Then apt-get update, and then install back letsencrypt forcing the use of the backports:

apt-get install -t jessie-backports letsencrypt

That’s it,

Giuseppe

Extend Xen guest partition on a live system

nanites

No restart of the guest VM is required

on Dom0:

lvextend /dev/lvm/domU.lan-var -L +5G
Extending logical volumedomU.lan-var  to 7.00 GiB
Logical volume domU.com-var successfully resized

On DomU

resize2fs /dev/xvdaX #(where X is a number)
resize2fs 1.42.5 (29-Jul-2012)
Filesystem at /dev/xvdaX is mounted on /var; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/xvdaX to 1835008 (4k) blocks.
The filesystem on /dev/xvdaX is now 1835008 blocks long.

 

 

Installing and configuring Bareos (Bacula fork) with web front-end on Debian Wheezy.

And taking nice backups easy to be restored.

The story is quiet simple: some times ago I would have liked to have a kind of 8 months old backup for a friend of mine, but it was too late. So now, I want backups with a decent retention. So I can go “back in time” and recover files. Another choice could have been Subversion, but I’m going to backup images as well so, for this reason I’ll not take Subversion into consideration.

I’ll proceed with Bareos as it seems to be a Free Software fork of Bacula (so I hope it will be stable and robust as Bacula)  but with features that normally are sold as commercial by the Bacula project team.

I’ll use the Bareos official repository, where to download and install Bareos from.

Components

Bareos is made out of three main components and console:

  • Director
  • Storage Service
  • File Daemon
  • – Console

Not all of these components are needed for a client machine, let’s see what do we need in order to run the each component efficiently and start taking backups.

Before starting I’ll use this fantastic feature offered by Transip: the Snapshot. So if something goes wrong it will be very easy to rollback to the pre-screw-up image.

Some pre-requirements

These are required packages, if you already have them the following command doesn’t hurt.

apt-get -y install mysql-server apache2 php5 php5-mysql sudo

Then I will install their repositories and install Bareos from them like this:

URL=http://download.bareos.org/bareos/release/latest/Debian_7.0/
printf "deb $URL /\n" > /etc/apt/sources.list.d/bareos.list

# add package key
wget -q $URL/Release.key -O- | apt-key add -

apt-get update
apt-get install bareos bareos-database-mysql

If you are going to use the instructions on the Bareos website, please watch out the debian version which now is 6.0. So if you use Wheezy, just change the 6 into 7, as I did. Oh yes and in their example PostgreSQL is used in place of Mysql.

Some configuration to do.

For completing the Bareos configuration, we should setup the database and the tables and so there are some sql files to be executed, but for these files to work we need passwordless access to mysql for root. For some minutes only. If this was mysql5.6+ we could use the special variable –login-path=local, but we are on 5.5 so we can either create a .my_cnf in the root home directory or restarting mysql using the –skip-grant-tables option which will make mysql go with no passwords.

I’ll chose the second one. By the other things, this procedure is very useful in order to reset the mysql password without having a valid mysql account. So what to do is:

  • Stop mysql
  • Restart mysql with the –skip-grant-tables option
  • Run scripts below
  • Kill mysqld_safe
  • Restart Mysql in normal mode using the usual init.d script.
/etc/init.d/mysql stop
/usr/bin/mysqld_safe --skip-grant-tables

At this point you should HIT CTRL+Z so you can continue using your terminal.
When finished you can get back to the mysqld_safe process by typing ‘fg‘ as requested later in this post. (CTRL+Z sends the process in background, ‘fg‘ instead brings it back to ForeGround).

/usr/lib/bareos/scripts/create_bareos_database
/usr/lib/bareos/scripts/make_bareos_tables
/usr/lib/bareos/scripts/grant_bareos_privileges

This last command gives some problems because you started mysql with the skip-grant-tables option so it cannot use those tables for executing queries.

Kill the mysql with –skip-grant-tables, as we don’t need it any more with:

fg
CTRL + C

Open the file /usr/lib/bareos/scripts/grant_mysql_privileges and change the line

if $bindir/mysql $* -uroot  -f <<END-OF-DATA

into

if $bindir/mysql $* -uroot -p -f <<END-OF-DATA

So the password will be requested as user input.

The Director

The file in /etc/bareos-dir.conf contains the main part of the configuration.  The main important sections and some very small explanations of them are:

  • JobDefs
    This defines element in common for two or more job so there is less configuration to write. They can contain almost all is contained in a Job
  • Job
    This define the client to backup, the fileset, the storage to use etc
  • Schedule
    This controls the timing of the jobs, define a schedule with a name here, and then use it by name in Job or JobDefs
  • FileSet
    This define a fileset to be used for a job, then the name of this fileset is used in Job or JobDefs
  • Client
    This define the client by name to be used in Job or JobDefs
  • Storage
    This Defines a storage by name to be used in Job or JobDefs
  • Catalog
    This defines the database where to store information about for example owner and file permissions.
  • Messages
    The Notification configuration
  • Pool
    The way storage devices (tapes or files) are managed.
  • Console
    This configures credentials for using the console and interacting with the director.

What to configure

After a fresh install, there is some basic configuration for Bareos in /etc/bareos. This configuration wants to illustrate how to set up various component but it is also working for making a backup of Bareos itself and the MySQL catalog.

Precisely there are 5 files and each of them has a sample configuration for the director itself in  order to backup both catalog and files.

I want to add pretty much the same thing : mysql database dumps and files coming from web dirs. But before doing this I want to know how much disk space a full backup is going to take so I can configure the storage and so.

1.3G web sites folder/

320M mysql dump of all dbs

So I’m going to need some space too have a long retention. Let’s just start with a ‘normal’ retention then we will always be able to increase  it. But meanwhile we can keep an eye on the backup files created on this side, monitoring their growth and being able so to have an estimation about the initial disk space needed.

I’m not going to use tapes for my backups, but a second machine will store the backups  (as data files) and those backup are nightly transferred on the NAS in my house through a cron job which runs on a NAS running Linux.

Speaking about the web folders and MySQL  here, means speaking about WordPress blogs, like this one, so what they do is basically adding new contents into the database, and new images and media on the disk. So the only part which could be neglected is WordPress itself, but since the disk space taken for those files is infinitely small compared to the images, we are going to backup those files as well, thus, by the way, we will backup plug-ins and themes instead of  having to remember their names. So yes, it is definitely worth to save those file as well.

There are some configuration keys referenced in other configuration files. I’ll keep it consistent so do the same. These are for example, the storage.storage name, the storage.device name the fileset.name and so. The Device is defined in /etc/bareos/bareos-sd.conf, we’ll see later.

This is the Mysql part.

/etc/bareos/bareos-dir.conf

JobDefs {
    Name = MySqlDefs
    Type = Backup
    Level = Incremental
    Accurate = Yes
    Write Bootstrap = "/var/lib/bareos/%c-%n.bsr"
    Fileset = MysqlDumps
    Pool = MysqlPool
    Schedule = "WeeklyCycle"
    Storage = MysqlBackupStorage
    Messages = Standard
    Priority = 10
    Write Bootstrap = "/var/lib/bareos/%c.bsr"
    Write Bootstrap = "|/usr/sbin/bsmtp -h localhost -f \"\(Bareos\) \" -s \"Bootstrap for Job %j\" giuseppe@giuseppeurso.net"
}

Job {
  Name = "BackupMysql"
  Client = "Mysql"
  JobDefs = "MySqlDefs"
}

Client {
  Name = Mysql
  Address = 192.168.0.1
  Password = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  File Retention = 30 days            # 30 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                      # Prune expired Jobs/Files
}

Storage {
    Name = MysqlBackupStorage
    # N.B. Use a fully qualified name here
    Address = backupper.mylan.com 
    Password = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    Device = FileStorageMysql
    Media Type = File
}

FileSet {
   Name = "MysqlDumps"
   Include {
     Options {
       signature = MD5
     }
     File = "/var/backups/server/mysql"
     File = "/etc/mysql/"
   }
}

Pool {
    Name = MysqlPool
    Pool Type = Backup
    Recycle = yes # Bareos can automatically recycle Volumes
    AutoPrune = yes # Prune expired volumes
    Volume Retention = 365 days
    Maximum Volume Bytes = 2G
    Maximum Volumes = 100
}

This is the web folders part with php scripts, images and so.

JobDefs {
 Name = WebDefs
 Type = Backup
 Level = Incremental
 Accurate = Yes
 Write Bootstrap = "/var/lib/bareos/%c-%n.bsr"
 Fileset = Webcopy
 Pool = WebPool
 Schedule = "WeeklyCycle"
 Storage = Web1BackupStorage
 Messages = Standard
 Priority = 10
 Write Bootstrap = "/var/lib/bareos/%c.bsr"
 Write Bootstrap = "|/usr/sbin/bsmtp -h localhost -f \"\(Bareos\) \" -s \"Bootstrap for Job %j\" giuseppe@giuseppeurso.net"
 }

Job {
 Name = "BackupWeb1"
 Client = "Web1"
 JobDefs = "WebDefs"
 }

Client {
 Name = Web1
 Address = 192.168.0.1
 Password = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 File Retention = 30 days # 30 days
 Job Retention = 6 months # six months
 AutoPrune = yes # Prune expired Jobs/Files
 }

Storage { 
  Name = Web1BackupStorage
  # N.B. Use a fully qualified name here
  Address = backupper.mylan.com
  Password = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  Device = FileStorageWeb
  Media Type = File
 }

FileSet {
 Name = "Webcopy"
    Include {
       Options {
           signature = MD5
        }
        File = "/var/www"
        File = "/etc/apache2/sites-available"
     }
 }

Pool {
 Name = WebPool
 Pool Type = Backup
 Recycle = yes # Bareos can automatically recycle Volumes
 AutoPrune = yes # Prune expired volumes
 Volume Retention = 365 days
 Maximum Volume Bytes = 2G
 Maximum Volumes = 100
 }

 

For the storage part we need to edit:

nano /etc/bareos/bareos-0sd.conf

and add one device for mysql backups and one for web backups:

Device {
 Name = FileStorageMysql
 Media Type = File
 Archive Device = /backups/bareos/storage
 LabelMedia = yes; # lets Bareos label unlabeled media
 Random Access = Yes;
 AutomaticMount = yes; # when device opened, read it
 RemovableMedia = no;
 AlwaysOpen = no;
}
Device {
 Name = FileStorageWeb
 Media Type = File
 Archive Device = /backups/bareos/storage
 LabelMedia = yes; # lets Bareos label unlabeled media
 Random Access = Yes;
 AutomaticMount = yes; # when device opened, read it
 RemovableMedia = no;
 AlwaysOpen = no;
}

Installing the client part

The following part is about installing the bareos client on the machine to be backed up. It will expose one service on a tcp port. We will make sure also that the two machines will talk each other using a private network.

We will call this machine, the one to be backed up, “Client”. As we already did for the server, let’s add the Bareos repository to our software package manager:

URL=http://download.bareos.org/bareos/release/latest/Debian_7.0/
printf "deb $URL /\n" > /etc/apt/sources.list.d/bareos.list

# add package key
wget -q $URL/Release.key -O- | apt-key add -

apt-get update
apt-get install bareos-client

At this point we need to configure roles and authentication between components to make it work and for this purpose the following image stolen from the bareos project website is very useful:

bareos-components-autentication

This image made very easy understanding who is who. In the case of the client we should take into consideration just the bareos-fd part.

At this point i restart the services on the server machine and on the client machine.

On server:

/etc/init.d/bareos-fd restart
/etc/init.d/bareos-sd restart
/etc/init.d/bareos-dir restart

And on Client

/etc/init.d/bareos-fd restart

Then I can use the bconsole to check what’s going on, it’s very easy, just enter into bconsole typing ‘bconsole’ and then ‘help’

bconsole
Connecting to Director eventhorizon:9101
Enter a period to cancel a command.
*help
 Command Description
 ======= ===========
 add Add media to a pool
 autodisplay Autodisplay console messages
 automount Automount after label
 cancel Cancel a job
 create Create DB Pool from resource
 delete Delete volume, pool or job
 disable Disable a job
 enable Enable a job
 estimate Performs FileSet estimate, listing gives full listing
 exit Terminate Bconsole session
 export Export volumes from normal slots to import/export slots
 gui Non-interactive gui mode
 help Print help on specific command
 import Import volumes from import/export slots to normal slots
 label Label a tape
 list List objects from catalog
 llist Full or long list like list command
 messages Display pending messages
 memory Print current memory usage
 mount Mount storage
 move Move slots in an autochanger
 prune Prune records from catalog
 purge Purge records from catalog
 quit Terminate Bconsole session
 query Query catalog
 restore Restore files
 relabel Relabel a tape
 release Release storage
 reload Reload conf file
 rerun Rerun a job
 run Run a job
 status Report status
 setbandwidth Sets bandwidth
 setdebug Sets debug level
 setip Sets new client address -- if authorized
 show Show resource records
 sqlquery Use SQL to query catalog
 time Print current time
 trace Turn on/off trace to file
 unmount Unmount storage
 umount Umount - for old-time Unix guys, see unmount
 update Update volume, pool or stats
 use Use specific catalog
 var Does variable expansion
 version Print Director version
 wait Wait until no jobs are running
When at a prompt, entering a period cancels the command.
You have messages.
*

The web front-end

Let’s find a convenient location where to download webacula and download it.

cd /var/tmp
svn co https://github.com/tim4dev/webacula webacula

This last command will download the entire svn of webacula. So we will use the latest version contained. First thing: make a folder for webacula under /var/www and then copy some folders into this folder

mkdir -p /var/www/webacula
cp -pr /var/tmp/webacula/trunk/html /var/www/webacula/
cp -pr /var/tmp/webacula/trunk/application /var/www/webacula/
cp -pr /var/tmp/webacula/trunk/data /var/www/webacula/
cp -pr /var/tmp/webacula/trunk/docs /var/www/webacula/
cp -pr /var/tmp/webacula/trunk/install /var/www/webacula/
cp -pr /var/tmp/webacula/trunk/languages /var/www/webacula/
cp -pr /var/tmp/webacula/trunk/library /var/www/webacula/
chown -R www-data.www-data /var/www/webacula
usermod -aG bareos www-data
chown root:bareos /usr/sbin/bconsole
chmod u=rw,g=r,o= /etc/bareos/bconsole.conf
cp /var/tmp/webacula/trunk/install/apache/webacula.conf /etc/apache2/sites-available/webacula.conf
sed -i 's/\/usr\/share/\/var\/www/' /etc/apache2/sites-available/webacula.conf
a2ensite webacula.conf
a2enmod rewrite
service apache2 restart
[Sat Dec 28 03:56:51 2013] [warn] module rewrite_module is already loaded, skipping
apache2: Syntax error on line 268 of /etc/apache2/apache2.conf: Syntax error on line 18 of /etc/apache2/sites-enabled/webacula.conf: /etc/apache2/sites-enabled/webacula.conf:18: <Directory> was not closed.
Action 'configtest' failed.
The Apache error log may have more information.
failed!

Don’t worry about the last error, I reported it here because probably a lot of people will have this problem with the tag 5.5.1 of Webacula. There is an easy fix which is adding a ‘<‘ to the /etc/apache2/sites-enabled/webacula.conf file. Quiet easy. this is where to add it:

There is one line starting by

/Directory>

Note the missing opening ‘<‘ , Instead of

</Directory>

This is what we need to change in order to fix the problem. Once done, save and reload Apache2 configuration with:

/etc/init.d/apache2 reload

Now we need to make the user www-data capable to run the bconsole, in order to be able to read and write data. For this reason we need Sudo, so let’s install it and then we can add www-data to the sudoers file with a specific command.

apt-get install sudo
visudo

Add this line under the root one

www-data ALL=NOPASSWD: /usr/sbin/bconsole

At this point we need to download and install the Zend PHP framework minimal edition to make Webacula work. So download ZendFramework-1.12.3-minimal.tar.gz (check for the version number)  from http://www.zend.com/en/company/community/downloads. You will need a login (I know it sucks). Put it on /var/tmp

cd /var/tmp
tar -xzf ZendFramework-1.12.3-minimal.tar.gz
cp -pr /var/tmp/ZendFramework-1.12.3-minimal/library/Zend /var/www/webacula/library/Zend/

edit and configure the following file:

nano /var/www/webacula/application/config.ini

Remember to change

bacula.bconsole = "/sbin/bconsole"

into

bacula.bconsole = "/usr/sbin/bconsole"

And

bacula.bconsolecmd = "-n -c /etc/bacula/bconsole.conf"

into

bacula.bconsolecmd = "-n -c /etc/bareos/bconsole.conf"

Now it’s time to configure a database access for webacula. For this the developers provide us with some scripts. There is a configuration to be edited with database name, database password for root, before running the scripts:

nano /usr/src/webacula/trunk/install/db.conf
cd usr/src/webacula/trunk/install/MySql/
./10_make_tables.sh
./20_acl_make_tables.sh

There is one more thing to be fixed in order to make webacula run. Theoretically we have done now and we “should” be able to run our webapp at the address http://server/webacula. But, depending on your error reporting configuration for php, our result here will vary from a white page and an error page. I had a white one so i had to take a look at the error loga to see what was going on. The error log reported:

[Sat Dec 28 12:48:23 2013] [error] [client 10.0.0.1] PHP Fatal error:  Uncaught exception 'Zend_Exception' with message 'Bacula version mismatch for the Catalog database. Wanted 12, got 2001. ' in /var/www/webacula/html/index.php:186\nStack trace:\n#0 {main}\n  thrown in /var/www/webacula/html/index.php on line 186

This is probably due to the different versioning numbers going on between Bacula and Bareos. You will be tempted to  fix this from the database table “Version” which seems to be there exactly for this reason, but don’t because remember also bareos components use the database and they probably check that field as well (i did it actually, this is why I’m so damn sure). The solution is here https://github.com/bareos/contrib-webacula/blob/bareos-master/application/config.ini and I shouldn’t have used a tag… anyway, just open index.php and look for this line

define('BACULA_VERSION', 12); // Bacula Catalog version

And replace it with this one

define('BACULA_VERSION', 2001); // Bacula Catalog version

Before you start using bareos you have to lable the Volumes using the bconsole. The following is an example made with the configuration explained here:

bconsole 
Connecting to Director eventhorizon:9101
Enter a period to cancel a command.
*label
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
The defined Storage resources are:
 1: File
 2: MysqlBackupStorage
 3: Web1BackupStorage
Select Storage resource (1-3): 1
FileStorage
Defined Pools:
 1: Default
 2: File
 3: Scratch
 4: MysqlPool
 5: WebPool
Select the Pool (1-5): 1
Connecting to Storage daemon File at eventhorizon.giuseppeurso.net:9103 ...
Sending label command for Volume "FileStorage" Slot 0 ...
3000 OK label. VolBytes=211 Volume="FileStorage" Device="FileStorage" (/backups/bareos)
Catalog record for Volume "FileStorage", Slot 0 successfully created.
Requesting to mount FileStorage ...
3906 File device ""FileStorage" (/backups/bareos)" is always mounted.
You have messages.
*label
The defined Storage resources are:
 1: File
 2: MysqlBackupStorage
 3: Web1BackupStorage
Select Storage resource (1-3): 2
Enter new Volume name: MysqlStorage
Defined Pools:
 1: Default
 2: File
 3: Scratch
 4: MysqlPool
 5: WebPool
Select the Pool (1-5): 4
Connecting to Storage daemon MysqlBackupStorage at 192.168.0.2:9103 ...
Sending label command for Volume "MysqlStorage" Slot 0 ...
3000 OK label. VolBytes=214 Volume="MysqlStorage" Device="FileStorageMysql" (/var/lib/bareos/storage)
Catalog record for Volume "MysqlStorage", Slot 0 successfully created.
Requesting to mount FileStorageMysql ...
3906 File device ""FileStorageMysql" (/var/lib/bareos/storage)" is always mounted.
*label
The defined Storage resources are:
 1: File
 2: MysqlBackupStorage
 3: Web1BackupStorage
Select Storage resource (1-3): 1
Enter new Volume name: Web1Storage
Defined Pools:
 1: Default
 2: File
 3: Scratch
 4: MysqlPool
 5: WebPool
Select the Pool (1-5): 5
Connecting to Storage daemon File at eventhorizon.giuseppeurso.net:9103 ...
Sending label command for Volume "Web1Storage" Slot 0 ...
3000 OK label. VolBytes=211 Volume="Web1Storage" Device="FileStorage" (/backups/bareos)
Catalog record for Volume "Web1Storage", Slot 0 successfully created.
Requesting to mount FileStorage ...
3906 File device ""FileStorage" (/backups/bareos)" is always mounted.
*label
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
The defined Storage resources are:
     1: File
     2: MysqlBackupStorage
     3: Web1BackupStorage
Select Storage resource (1-3): 1
Enter new Volume name: FileSTorage
Defined Pools:
     1: Default
     2: File
     3: Scratch
     4: MysqlPool
     5: WebPool
Select the Pool (1-5): 2
Connecting to Storage daemon File at eventhorizon.giuseppeurso.net:9103 ...
Sending label command for Volume "FileSTorage" Slot 0 ...
3000 OK label. VolBytes=208 Volume="FileSTorage" Device="FileStorage" (/backups/bareos)
Catalog record for Volume "FileSTorage", Slot 0  successfully created.
Requesting to mount FileStorage ...
3001 OK mount requested. Device="FileStorage" (/backups/bareos)
*quit

You will get also this error:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'Media.VolParts' in 'field list'

I honestly don’t know what that field is supposed to contain but I want my app to work so I just create that field as a varchar 255 like this:

 mysql -uroot -p -e "use bareos; alter table Media add VolParts varchar (255);"

This should be all. Now point yur browser to the address where webacula is supposed to answer and check if you see the login prompt like this:

webacula_login_form

 

Username is root, the password is the one you choose while configuring the sql install script

That’s it. If you have comments or questions, please just use the form below this article.

Ciao

 

P.S.

I want to add this information: In this post I used fictional names for machines and for machines addresses. So I had some trouble once put this configuration working and I spotted a couple of typos and I had to fix the various passwords in various configuration files.

This image illustrates better:

webacula-after-tuning

This was Installing and configuring Bareos on Debian squeeze/wheezy with web front-end

Incoming search terms:

  • bareos (19)
  • bareos raspberry pi (3)
  • https://blog giuseppeurso net/installing-bareos-bacula-fork-web-front-end-debian-wheezy/ (3)
  • bareos vs bacula (2)
  • bareos ubuntu (2)
  • bacula debian install (1)
  • how to install bareos (1)
  • how to install bareos webui on debian 8 (1)
  • install bareos webui (1)
  • install bareos webui on debian 8 (1)

Debian, managing LVM on a live system.

Playing with LVM on Debian Weehzy.

Resizing or deleting  partitions, gaining space and then redistribute it across the partition you need to resize.

Basically the logical operations are pretty simple and straightforward if taken into a ‘logical’ vie. Of course you have to shrink the partition first and then the volume, while, at the opposite if you want to grow a partition you have to resize the volume in order to be able to contain the bigger partition.

But you can’t do this on a running system. You have to restart into recovery mode. So you will be able to umount partitions as you will be using just the memory.

Particularly this applies to the machine from a dutch provider called Transip  who sells these convenient VPS and more, these have huge a disk space amount (the basic VPS has 50 gigs).

They offer a very nice Control Panel, although in dutch, is very easy to use and it offers advanced features like the chance to set the reverse dns by yourself, or or creating a private network with your virtual machine in order to carry on internal operation such as for example backups, or build your load balancing, your mysql cluster and so. Very very nice, their interface is built by themselves, very smart company.

Anyway, back to my problem, they have this console you can use to install the operating system from scratch and Icouldn’t manage to get into a partition manager that  allowed me to decide how many and how large partition to use. So i had to use one of the predefined setup (“Use entire disk and setup LVM” then “Use separate partition like /usr, / home, /var etc”).  This is a picture of the starting situation:

partition scheme transip

But I don’t really enjoy that kind of partitions and their size. So I’ll let the installation finish and then I’ll start modifying my lvm.

This machine will be used to keep some backups from mysql and some web folders under /var/www, from another machine. For this reason I will install and configure Bareos.

 

With “df -h” i can identify the following partitions, the ones I’m interested in. I want to get some free space from shrinking /home and merging /usr into the root partition (/):

/home 36 Gb
/    322 Mb
/tmp  368Mb
/usr   8.3Gb
/var   2.8 Gb
/blah

First, I want to get rid of /usr and merge it with / (the root partition).

For this reason I will:

cp -pr /usr /home/usr
umount /dev/machine-name/usr
lvremove /dev/machine-name/usr

then with

vgdisplay

I’ll be able to check the actual free disk space gained. You should see at the end of the output a line starting by (look at the picture below):

Free PE / Size.

That is the measure you can increase another partition of, or of the redistributable space. Here I’m going to make backups so I’m going to create a new partition called “/backups” where I will store my backups. This is supposed to be a very big partition, so i’ll start merging and shrinking partitions in order to leave the most of the space available for backups.

So now for example I’ll reduce the filesystem first and then teh volume size for the partition /home.

umount /dev/machine-name/home
e2fsck -f /dev/machine-name/home -p 2G
lvreduce -L 2G /dev/machine-name/home

Then this is what I have after all these operations:

after shrinking and deleting partitions and volumes

 

 

At this point I have still to merge /usr into / (the root partition), but i have to make it bigger first. So I’ll give more space to the logical volume first and then I’ll extend the file system as well.

lvextend -L 3G /dev/machine-name/root
resize2fs -p /dev/machine-name/root 3G
rm -fr /usr
mv /home/usr /

At this point we want to remove the /usr entry from /etc/fstab otherwise we will get errors because that partition doesn’t exist any more on our system. (little advice: since we are going to create a new partition and a new mount point at this point you could consider instead of deleting the entry, to modify it for the new mount point so there will be no need to come back on this file again, but it’s up to you).

 

Then create a new partition , but first the logical volume

lvcreate -L 39G -n backups vg-name

Create then a file system

 

 

mkfs .ext4 /dev/machine-name/backups

Let’s create a mount point

mkdir /backups

and let’s add the partition to /etc/fstab (just copy the line for /var taling care of renaming where needed)

If it’s all ok now we can reboot in a normal way and check whether the new partition mounts correctly etc.

So, reboot and connect with ssh. Then with “df -h” i’ll verify all.

root@eventhorizon:~# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 3.0G 308M 2.5G 11% /
udev 10M 0 10M 0% /dev
tmpfs 101M 232K 101M 1% /run
/dev/mapper/eventhorizon-root 3.0G 308M 2.5G 11% /
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 201M 0 201M 0% /run/shm
/dev/vda1 228M 18M 199M 9% /boot
/dev/mapper/eventhorizon-home 2.0G 153M 1.8G 8% /home
/dev/mapper/eventhorizon-tmp 368M 11M 339M 3% /tmp
/dev/mapper/eventhorizon-backups 39G 176M 37G 1% /backups
/dev/mapper/eventhorizon-var 2.8G 181M 2.5G 7% /var

Great! all is as expected.
Now i can install and configure Bareos, so follow my next posts!!
Giuseppe

Maven3, Javac & Co. On Debian

apache-maven-project-2

Maven3 &the right JDK

I was trying to build Maven3, to build other stuff (hippo cms) and suddenly my system said:

BUILD FAILED
/usr/src/apache-maven-3.0.5/build.xml:223: Unable to find a javac compiler;
com.sun.tools.javac.Main is not on the classpath.
Perhaps JAVA_HOME does not point to the JDK.
It is currently set to "/usr/lib/jvm/java-6-openjdk-amd64/jre"

This means that I’m trying to compile Maven3 without having installed the JAVA SE jdk from Oracle website, but luky you, you can follow this link: [Download Java].

One negative thing is that you have to accept the software license and then you can just download the package within the same browser session, thus you have to SCP the tar.gz archive to your server machine. This is not lazy-sysadmin-friendly indeed. 😛

By the way what i’ve done in order to undertsand where and how this package should have been moved to work fine was to look around for already present java version on my system.
the command:

# which java

gives some info, mine gives:

/usr/bin/java

Then with

# ls -al /usr/bin/java 
lrwxrwxrwx 1 root root 22 Apr 27 10:10 /usr/bin/java -> /etc/alternatives/java

We see a link pointing in /etc/alternatives. Let’s have a look there:

# ls -al /etc/alternatives/java
lrwxrwxrwx 1 root root 46 Apr 27 10:10 /etc/alternatives/java -> /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java

Another link to /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java, and that is the place where to extract our downloaded archive, so

# mv /usr/src/jdk1.7.0_21 /usr/lib/jvm/
# cd /usr/lib/jvm/
# ls -al
total 32
drwxr-xr-x  5 root root  4096 May  4 08:44 .
drwxr-xr-x 46 root root 12288 Apr 27 14:24 ..
lrwxrwxrwx  1 root root    20 Mar  5 10:16 java-1.6.0-openjdk-amd64 -> java-6-openjdk-amd64
-rw-r--r--  1 root root  2387 Mar  5 10:16 .java-1.6.0-openjdk-amd64.jinfo
drwxr-xr-x  5 root root  4096 Apr 27 10:05 java-6-openjdk-amd64
drwxr-xr-x  3 root root  4096 Apr 27 10:05 java-6-openjdk-common
drwxr-xr-x  8 uucp  143  4096 Apr  4 11:32 jdk1.7.0_21

Here we are!. You can see in red (last line of previous code snippet) our jdk dir. The one in blue is the one in use (I don’t know why it is linking somenthing in the same directory…) So,

# cd /etc/alternatives
# ls -al java
lrwxrwxrwx 1 root root 46 Apr 27 10:10 java -> /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java
# rm java
ln -s /usr/lib/jvm/jdk1.7.0_21/jre/bin/java java

Then finally, I could go back into the maven3 source directory and build it with ant… After tons of output and after having downloaded the internet, it builds.

all:

BUILD SUCCESSFUL
Total time: 6 minutes 8 seconds

Bash Script: one tar gz archive from each subfolder.

my treasure, missing link between dogs and unicorns, i'll never get used to such a beauty. My actual best friend. Her nose stinks of a mixture of out-of-date parmigiano and various other cheeses.
my Filippa

Hi all, this is a backup tool to be used in for automated or manual backup. What this small script does is creating one tar.gz archive for each subdirectory found starting from a given directory.

It takes just two arguments: where to start from, where to save the new created archives.

The goal is “KEEP IT SIMPLE”, so you can easily adapt this t your systems. You can for instance, hardcode the two args so you can use it for a specific duty (web folders backup /var/www or whatever).

Here you are:

#!/bin/bash
cd $1
for dir in */
do
base=$(basename "$dir")
tar -czf "$2/${base}.tar.gz" "$dir"
echo "compressing $2/${base}.tar.gz $dir";
done

Example. let’s say that you have a webserver whit an usual directory structure for clients, for example:
/var/www/client1
/var/www/client2
/var/www/clientx
and so.
If you wanted a separated tar gz archive for each client subfolder you should use this script this way:
./multitar /var/www /backups

 

Have a nice weekend

Giuseppe

Raspberry Wifi Static Ip Config

This image shows my raspberry pi with attached an usb wireless lan adapter.

As title says, here i store how i managed to get a fixed ip address on the wireless lan network interface from shell or command line.
It’s very easy and usefull in a number of situations. For example this makes part of another my projects about remoting audio of my laptop using air-whatever and the raspberry as server, but i will describe this soon, in my next posts.

Raspberry Wifi Static Ip Config

Ingredients: Continue reading “Raspberry Wifi Static Ip Config”

Incoming search terms:

  • raspberry pi wifi static ip (23)
  • raspbmc wifi static ip (23)
  • raspberry pi (19)
  • raspberry static ip (17)
  • raspbmc static ip (14)
  • hallon wifi (1)

Constantly Under attack, inquiring discovered huge URSS dns poison

russian dns poison attackWell it turned out it was a mobile application “classmates” produced and distribuited for free by mail.ru the reason for the unusual considereable amount of traffic on one if my ip addresses

what? Continue reading “Constantly Under attack, inquiring discovered huge URSS dns poison”