Simpana 10 – PostgreSQL 8.4 backup on CentOS Linux 5.10 x64 – example

I am going to assume that this is a test deployment and as such will expect that you have installed your CentOS 5.10 x64 Linux the way you want it, and I will follow on from that point on what I needed to perform to get the distribution release of PostgreSQL to work with Simpana 10 PostgreSQL iDA to perform a backup. Of course some assumed knowledge present.

  1. Install the postgresql packages onto your  CentOS client.
    $ sudo yum install postgresql84 postgresql-server postgresql-devel
  2. Startup postgresql server for the first time, you need to run initdb switch instead of start for the first time only.
  3. $ sudo service postgresql initdb
  4. We should also enable the service to run at boot moving forward
    $ sudo chkconfig postgresql on
  5. Before we change the authentication method below, we need to set a password that we know for the postgres user in the postgresql database. To perform this we need to change to the postgres user and connect to postgresql database and update the password for the user to something we know.
    $ sudo su –
    # su – postgres
    $ psql
  6. Now at the postgres prompt update the password for the postgres user, unless you want to make your own. Won’t discuss how, just going to show how to set postgresql user password. Be sure to remember what you set the password too, it will be required later on.
    postgres=# ALTER USER postgres WITH PASSWORD ‘password’;
    ALTER ROLE
    postgres=#q
  7. Postgresql packages distributed with CentOS don’t use md5 password authentication, it defaults to peer/ident based authentication. In this example we will flip this to md5 based authentication, and we will touch on a peer/ident based authentication example in a later post. Perform the changes below to enable md5 authentication.
    $ cd /var/lib/pgsql/data
    $ sudo vi pg_hba.conf
    Find the line at the bottom of the file that looks like the one below;
    local     all     all                ident
    You need to change this to have md5 on the end, i.e. replace ident to be md5 instead. Save the changes.
  8. Now restart postgresql for the changes to take effect. (required)
    $ sudo service postgresql stop
    $ sudo service postgresql start
  9. Now you can test that this has worked by execution as root the command below, and when prompted for the postgres user password authenticate using the password set in step 6.
    # psql -U postgres
    If it worked, you will get the famous postgres=# prompt, in which you can enter q [enter] to quit it.
  10. Next up we now need to enable archive logs. We need to edit the postgres.conf file which on CentOS rpm based install is /var/lib/pgsql/data and the lines we need to add in the Archiving section is below;
    archive_mode = on
    archive_command = ‘cp %p /var/postgresql/archive/%f’
    Save those additions and move on below.
  11. Make sure to create the folders/destination used in the archive_command above and ensure postgres user can write to it etc.
  12. Now restart postgresql for the changes to take effect. (required)
    $ sudo service postgresql stop
    $ sudo service postgresql start
  13. Install the Simpana PostgreSQL iDA.
  14. Once installed refresh the Simpana Console and attempt to create your PostgreSQL instance. See the dialog below for the values I used in this configuration. Of course the username is the postgres user and password we configured in step 6. Note the archive log directory is the one we used in the archive_command string at step 10 too.simpana_10-centos-5.10_x64-postgresql_instance_creation
  15. If everything goes to plan you should have your instance created and now you can do configuration against the DumpBaseBackupSet subclient and/or FSBasedBackupSet subclient. For the difference between what each does, I recommend you review the documentation. As each backupset has its own unique capabilities. See the bottom of the Backup documentation page for explanations.
  16. Assign a Storage Policy to each subclient and run a backup of each to confirm it works.

CommVault Documentation references:

Simpana 10 – Linux client prepost command execution failure

Came across an interesting condition today, which took me a bit of testing to identify why the job would go into a pending state. This one relates to Simpana 10 on a Linux client where you have a File System iDA with a PrePost command being executed. In my test below the script is doing nothing special, it’s merely to have something to execute to show the behavior. I’ve provided it below purely for reference.

[root@jldb1 bin]# cat pre-scan.sh
#!/bin/sh
# test
#

echo $1 $2 $3 $4 $5 $6 $7 $8 $9 >> /root/pre-scan.log
exit 0

Job goes pending and produced the following errors and output below;

JPR (Job Pending Record)
Error Code: [7:75]
Description: Unable to run [/usr/local/bin/pre-scan.sh] on client.
Source: jwcs, Process: startPrePostCmd

simpana_10-linux-prepost-command-execution-failure

[JobManager.log – commserve]

3024  d88   03/27 18:16:26 21  Scheduler  Set pending cause [Unable to run [/usr/local/bin/pre-scan.sh] on the client.                 ]::Client [jwcs] Application [startPrePostCmd] Message Id [117440587] RCID [0] ReservationId [0].  Level [0] flags [0] id [0] overwrite [0] append [0] CustId[0].
3024  118c  03/27 18:16:26 21  Scheduler  Phase [Failed] message received from jwcs.lab.heimic.net] Module [startPrePostCmd] Token [21:3:1] restartPhase [0]
3024  118c  03/27 18:16:26 21  JobSvr Obj Phase [3-Pre Scan] for Backup Job Failed. Backup will continue with phase [Pre Scan].

[startPrePostCmd.log – commserve]

4940  e4c   03/27 20:21:46 ### Init() - Initializing job control [token=21:3:7,cn=jwcs], serverName [jwcs.lab.heimic.net], ControlFlag [1], Job Id [21]
4940  e4c   03/27 20:21:47 ### Cvcl::init() - CVCL: Running in FIPS Mode
4940  e4c   03/27 20:21:48 ### CVJobCtrlLog::registerProcess(): successfully created file [C:Program FilesCommVaultSimpanaBaseJobControl4.940]
4940  e4c   03/27 20:21:48 ### ::main() - jobId 21 - restoreTaskId = 0
4940  e4c   03/27 20:21:48 ### ::main() - jobId 21 - adminTaskId = 0
4940  e4c   03/27 20:21:48 ### ::getBackupCmdAndMachine() - jobId 21 - before construct application id
4940  e4c   03/27 20:21:49 ### ::getBackupCmdAndMachine() - appTypeId = 29
4940  e4c   03/27 20:21:49 ### ::getBackupCmdAndMachine() - jobId 21 - symbolic AppId = 2:20
4940  e4c   03/27 20:21:49 ### ::getBackupCmdAndMachine() - jobId 21 - prePostId = 1
4940  e4c   03/27 20:21:49 ### ::getBackupCmdAndMachine() - jobId 21 - preifind cmd = /usr/local/bin/pre-scan.sh
4940  e4c   03/27 20:21:49 ### ::main() - jobId 21 - commandPath = /usr/local/bin/pre-scan.sh
4940  e4c   03/27 20:21:49 21  ::main() - jobId 21 - before execute cmd
4940  e4c   03/27 20:21:49 21  ::main() - jobId 21 - Use Local System Acct.
4940  e4c   03/27 20:21:49 21  ::main() - jobId 21 - remoteexename = [/usr/local/bin/pre-scan.sh]
4940  e4c   03/27 20:21:49 21  ::main() - jobId 21 - args = [ -bkplevel 1 -attempt 7 -job 21]
4940  e4c   03/27 20:21:49 21  executePrePostCmd() -  Attempting to execute remote command on client [jldb1]..
4940  e4c   03/27 20:21:49 21  executePrePostCmd() - jobId 21 - Received error text from server cvsession [Unknown Error]
4940  e4c   03/27 20:21:49 21  executePrePostCmd() - jobId 21 - Error [0] returned from executeRemoteCommand /usr/local/bin/pre-scan.sh
4940  e4c   03/27 20:21:49 21  EvEvent::setMsgEventArguments() - MsgId[0x0700004b], Arg[1] = [117440623]
4940  e4c   03/27 20:21:49 21  EvEvent::setMsgEventArguments() - MsgId[0x0700004b], Arg[2] = [/usr/local/bin/pre-scan.sh]
4940  e4c   03/27 20:21:49 21  EvEvent::setMsgEventArguments() - MsgId[0x0700004b], Arg[3] = []
4940  e4c   03/27 20:21:49 21  EvEvent::setMsgEventArguments() - [MsgId[0x0700004b][]: [3] Args Pushed, [1] Args expected.
4940  e4c   03/27 20:21:49 21  ::exitHere() - jobId 21 - Exiting due to failure.
4940  e4c   03/27 20:21:49 21  BKP CALLED COMPLETE (PHASE Status::FAIL), 21. Token [21:3:7]
4940  e4c   03/27 20:21:53 21  ::exitHere() - jobId 21 - startPrePostCmd Terminating Event.
4940  238c  03/27 20:21:53 21  CVJobCtrlLog::unregisterProcess(): successfully removed file [C:Program FilesCommVaultSimpanaBaseJobControl4.940]

[cvd.log – client]

30846 427e0940 03/27 20:21:50 ### [CVipcD] Requests from non-CS with hostname [jwcs.lab.heimic.net] and clientname [jwcs] to execute in user entered path are not allowed

I worked out this problem is caused by lack of value in regkey sCSGUID as found in the location below;

/etc/CommVaultRegistry/Galaxy/Instance001/CommServe/.properties

Sample below;

[root@jldb1 ]# cat /etc/CommVaultRegistry/Galaxy/Instance001/CommServe/.properties | more
bCSConnectivityAvailable 1
sCSCLIENTNAME jwcs
sCSGUID
sCSHOSTNAME jwcs.lab.heimic.net
sCSHOSTNAMEinCSDB jwcs.lab.heimic.net

sCSGUID should be populated and its lack of value causes this condition with pre-scan script execution.

Fix:

Easiest method to recreate this regkey value is to do a local uninstall of the simpana services on the client. Revoke the client certificate in Simpana Console via Control Panel – Certificate Administration for the client in question. Followed by a reinstall.

Observation:

Subclients that have no scripts being executed as part of the backup will run fine if this regkey value is missing. You will never see a problem until you add a script. In addition, clients that have a simpana firewall configuration will be broken and subclients without scripts will break too. As the regkey value is used for simpana firewall configuration exchange I believe based on my testing.

Hope you enjoy my post… drop me a comment if you like the content and/or it helps you.

Simpana 10 – Specifying the media parameters for RMAN command line operations – Example

An recent addition to Simpana 10 Oracle iDA over Simpana 9 was the ability to specify Media Parameters for RMAN Command Line Operations, which wasn’t possible in Simpana 9.

Below is an example on its use, and the documentation links from Commvault are 1, 2 an 3.

The client in this example is “jwora1” running Windows 2008 R2 x64 and an Oracle 11gR2 64bit release. Simpana 10 with a SP4 is installed on client and Commserve – “jwcs”.

RMAN Script:

run {
allocate channel ch1 type 'sbt_tape' PARMS="BLKSIZE=262144,ENV=(CVOraSbtParams=C:p.txt,CvClientName=jwora1,CvInstanceName=Instance001)" trace 2;
backup current controlfile;
}

Contents of p.txt file below;

[sp]
SP_Main-jwma1

[mediaagent]
jwma1

Below is a look at the GUI configuration for the Oracle instance “orcl” on client “jwora1” which shows that third party command line backups should use Storage Policy (SP) – “SP_Main-jwcs”. However as you will not by the running of the job using the Media Parameters it will use a different SP and MediaAgent, as defined by the p.txt file I passed.

subclient not configured with any SP
subclient not configured with any SP
orcl properties showing command line backup should use SP - SP_Main-jwcs by default.
orcl properties showing command line backup should use SP – SP_Main-jwcs by default.
orcl properties showing log backups would use SP - SP_Main_jwcs by default
orcl properties showing log backups would use SP – SP_Main_jwcs by default.
sample execution of my rman backup script - current control file backup
sample execution of my rman backup script – current control file backup
Commserve Job Controller showing the running job. Note which MediaAgent is used and SP.
Commserve Job Controller showing the running job. Note which MediaAgent is used and SP.

If you find my posts of value, please send me some feedback. Especially if you find this post and it helps you in your travels.

UPDATE: And to follow on from the example above, the following is also possible too. If you don’t pass the CvClientName and CvInstanceName on the channel allocation, you can pull those too from the parameters file. Sample below of alternative backup script syntax and parameters file contents. All documented on the documentation link provided top of post.

RMAN Script:

run {
allocate channel ch1 type 'sbt_tape' PARMS="BLKSIZE=262144,ENV=(CVOraSbtParams=C:p2.txt)" trace 2;
backup current controlfile;
}

Contents of p2.txt file:

[sp]
SP_Main-jwma1
[mediaagent]
jwma1
[CvClientName]
jwora1
[CvInstanceName]
Instance001

The parameter file can have spaces between the definitions like in the top example, which I prefer, as it makes the file easier to read. Where as the p2.txt file has no extra spaces, which also works but makes it harder to read personally.

Enjoy.

Conflicting OSX Free space between Finder and Disk Utility

I was fixing up a neighbours Macintosh and I suspect due to the disk being used up to 100% it ended up corrupting, and when I run disk verify/repair in Disk Utility it seems to work but we still see the free space reported between Finder and Disk Utility being incorrect. Check out the picture below to see what I mean.

osx-disk-space-oops_21_mar_2014

Never and I repeat, never fill up your boot volume on OSX as you will cause lots of problems. I suspect this machine that had this issue will need to be formatted and reinstalled to correct the condition.

Microsoft OneNote now free on Apple App Store

Noticed overnight that the rumours of Microsoft OneNote becoming free have actual come true. It looks like we can download it. I’ll be installing it as I don’t mind OneNote at all. It’s something I started to use a lot more of in the last 3 years.

I might as well install OneDrive too and attempt to use it as well, although have been a long time user of Dropbox.

Post over here talks about the OneNote release on Macintosh.

OneNote_Free_2014-03-18

System security and sudo

Was reading twitter, as I would typically do when in transit. Came across a really good post on sudo[1] and security. Highly recommend having a read.

Check it out here.

Some excellent information contained in the post.

[1] – Sudo (su “do”) allows a system administrator to delegate authority to give certain users (or groups of users) the ability to run some (or all) commands as root or another user while providing an audit trail of the commands and their arguments.

Linux installation on SSD

I am going to be installing a Linux desktop onto an SSD drive soon, and I have started to research. If you have done this already and found something you needed to do that was not documented, please drop me a line.

My aim is to setup my Intel i3 Mini-ITX system with a Linux Desktop install (Ubuntu probably) and eventually drop in some more storage, since the case and motherboard could handle 4 more SATA devices, not including the optical and SSD boot drive.

Figured I could end up running for additional storage, services and desktop environment for doing other tasks like maybe get VMware Workstation going so I can run my existing virtual machines.

The current duties it does that requires Windows will move onto a laptop I will purchase at the end of this month, or early next month.

Linux system hardening

I really need to sit down and do some research on Linux System Hardening, as I really want to take a look at any kernel hardening that can be done on the system. i.e. I don’t want users to be able to see who else is on the machine via the output of commands like ‘w’ and ‘who’. I remember grsecurity use to be available to do this, so I should look around to see if this is still valid or do we do it another one.

I did find the following web post about SSH hardening which I enjoyed reading here.

If you’ve done Linux System Hardening recently, chime in on the comments with your experience.

Plumbing issue now resolved

Today my step dad and I fixed a problem I discovered post the new pergola slab going in. Days after the area was cleared, I noticed a lot of water near the tap and thought my wife must of over watered the lawn.

Post the slab being done it was more noticeable when pergola installer cut the downpipe. i.e no water came out of the live pipes. Since these should contain water since it goes to a tank. Thought the collar (reducer from 90mm to 100mm pipe) had come off, as it had once before.

We cut out the pipe, and when we put a torch down the water was actually below the point of the collars, which meant the leak was lower.

Dug out the side to access pipe and I immediately spotted the problem. It appears the 45 degree elbow that was under the concrete slab must of been knocked during excavation of the slab area I imagine and this split it on the inside of the elbow. It was so split that it was nearly around the whole pipe diameter. It had else caused some soil to erode around it too. Every time it rained, the system would need to load up, and post rain the system would unload all that water into the rear out this split. No wonder the ground was so wet post rain.

Didn’t take and pictures of the split, as I was too covered in mud to get a camera. However did capture the picture below showing out fix, as you can see not a bad job. Loaded up the down pipes and no leaks. So win for us.

Plumbing_Fix_8_Mar_2014

Now I can get on to the next jobs around the house and yard. So much still to do. Will continue to take pictures. Have another low spot where I am going to lay a surface drain and some arg line/pipe to help stop the area from pooling with water (where the trampoline use to be)

 

New PC laptop purchase

I was going to build up another mini-ITX desktop, however the more I look at it I am better off buying one of those Dell Outlet Inspirion laptops. As  I end up getting a PC that can be used as a desktop replacement and doesn’t need any more hardware purchased. i.e. keyboard/mouse/monitor. So I am monitoring the outlet site the past week watching what Intel i3 Third generation systems come up and the prices.

The PC laptop will probably run Windows 7 x64 Home (or Windows 8.1 if my work VPN finally ends up supporting it). While my existing mini-ITX desktop (i3/16gb ram) will probably be converted to a Ubuntu desktop, so I can implement lxc containers again and some other workload on it. Really want to get tvheadend setup on it and plug in the USB tuner I was using with my Raspberry Pi. Remember the post here, about that?

If I get some time this weekend one evening going to install a temporary hard disk into my desktop and do a Ubuntu 12.04.4 install and do some messing about to ensure the hardware all detects fine and resolution of monitors works fine. The system uses an Intel HD 4000 onboard, so I hope accelerated video works or that could be a problem.

Looking to purchase the laptop the end of this month or early next month.

Finally section of Garden Edging installed

Today I finished off the last section of garden edging I have been installing. Below is a picture to mark the event.

garden-edge-complete-04_mar_2014

Now it’s completed I have a list of other tasks I now must commence in the backyard, so it all starts again on Saturday morning. The rain the past week has highlighted some problem areas which I need to resolve before winter comes around, cause I wouldn’t like to have a wet winter and still have water drainage problems.

I’ve also managed to use the new Makita Cordless Jigsaw and completely love it. It’s just so good having a cordless jigsaw and no need to run leads for quick cuts.

New Voice Services Provider – Telecube

Totally missed it, but a friend alerted me to the fact we have a new voice services provider in play.

Telecube seem to be a new provider and seem to be making a name for themselves.

I signed up on the weekend, and the immediate thing I noticed was the very nice portal for configuration of the services and the fact you can configure up to 10 extensions which is very handy.

You can also check out the Whirlpool thread for them here. If you’d like to sign up, click here.