While Life:

PostgreSQL 8.4 Servers and Warm Standby (Point-In-Time Recovery)

2010-04-30T04:28:00.000-07:00

PostgreSQL is a great database server, but when your dataset is rather large, you want a back-up server with minimal downtime. To achieve this, we will be using the PostgreSQL built in features for PITR (Point in time recovery). This tutorial will teach you how to configure it from the basics starting from installation.

Assumptions

This tutorial assumes you have 2 PostgreSQL 8.4 servers running on a Debian / Ubuntu based Linux distribution. The database versions need to be the same. One can be your machine and other one can be your VM.

Installation

First install postgresql

$ sudo apt-get install postgresql

pgadmin3 is a excellent PostgreSQL client

$ sudo apt-get install pgadmin3

Server is running ?

Try this out:

$ sudo -u postgres psql

It should prompt

postgres=#

Type \password and give a password and type \q to quit and come out. It means its successfully running :)

postgres=# \password
Enter new password: <some password>
Enter again: <enter same password again>

postgres=# \q

if any problem persists. May be your server is not running, start it by using:

$ sudo /etc/init.d/postgresql-8.4 start

You can now login to your pgadmin3 by selecting new connection, with following credentials:

What do we intend to do

Step 1: Configure the Master PostgreSQL server

As stated earlier we will be using PITR (Point In Time Recovery) for this. This creates WAL files, which will be copied to the destination/standby database server. Each WAL file is (by default) 16MB in size.

$ mkdir /archive; chown postgres.postgres /archive
$ vim /etc/postgresql/8.4/main/postgresql.conf

archive_mode = on # for pgpoolII
archive_command = 'cp -i %p /archive%f'
archive_timeout = 300

Next, run the following commands to initiate postgres in backup mode.

$ sudo /etc/init.d/postgresql-8.4 reload;
$ sudo su postgres;
$ psql;
postgres=# SELECT pg_start_backup('backup');
postgres=# \q

Next, copy the files in /var/lib/postgresql/8.4/main to the destination/standby server. I prefer to use rsync for this.

rsync --progress -azv --delete  /var/lib/postgresql/8.4/main/  root@<destinationserver>:/var/lib/postgresql/8.4/main

When this is finished, we will need to stop postgresql's backup mode;

$ sudo su postgres;
$ psql;
postgres=# SELECT pg_stop_backup();
postgres=# \q

Destination/Standby server

Now we need to configure the destination server may be VM in this case. First, create another archive directory. Again, substitute /archive/ for your desired directory. For ease I recommend keeping these directories the same.

$ sudo mkdir /archive; sudo chown postgres.postgres /archive;

we need pg_standby. pg_standby supports creation of a "warm standby" database server

$ apt-get install postgresql-contrib-8.4

Create a symlink to pg_standby, so that its universally available.


$ sudo ln -s /usr/lib/postgresql/8.4/bin/pg_standby /usr/bin/pg_standby

Create a file names recovery here:

$ sudo vim /var/lib/postgresql/8.4/main/recovery.conf

Paste the following line into the newly created file:


restore_command = 'pg_standby -l -d -s 2 -t /tmp/pgsql.trigger /archive %f %p %r 2>>/var/log/postgresql/pg_standby.log'

Next, we will need to rsync the /archive directories across. This can be done from either the master or standby server. The following assumes the destination server:

$ sudo rsync --progress -azv --delete root@<sourceserver>:/archive  /archive

When finished, start postgresql..

$ sudo /etc/init.d/postgresql-8.4 start

When you want to switch over, all you have to do is re-rsync the /archive directory and create a trigger file to start postgresql into normal mode. Please note that from the moment you create the trigger file, new data from the master server will not be added to the standby server.

$ sudo rsync --progress -azv -e ssh --delete root@<sourceserver>:/archive  /archive
$ sudo su -c 'touch /tmp/pgsql.trigger' postgres

You may add a cron jobs to sync your archive directory between your master and slave PostgreSQL servers.

So once file is touched it runs into normal node, with minimal downtime. Hurray!

Railscasts crawler (Download all screencasts easily)

2009-03-14T01:36:00.000-07:00

I wrote this script, as I felt really annoying to download each screencast everytime I needed them. So I ended up writing script to automatically download all screencasts without hassle i.e crawler.

Its in Ruby of course :-) and requires simple Hpricot gem.

If you not having it. Just type this command in your terminal --

$gem install hpricot

--Rest include this script in your /lib folder

--Change the path in the script where you want to download all the screencasts

--Go to your projects development environment (script/console) and run the script by these commands--

video = Railscasts.new #new Railscasts object

video.start #will start downloading all screencasts from Railscasts

Note:

If you stop the script in between manually, it will not download those screencasts that are already in your hard disk.
All logs are maintained in Railsproject/log/railscasts.log.
Handling all exceptions raised

Improvements/Suggestions are appreciated.

Thanks

And yes script:


# Author : Akshay Gupta
#file: railscasts.rb
# First check you have all gems installed. Place the script in /lib folder and run the script.
# I don't have expertize in ruby, please tell how it can be improved.
# change the path accordingly, where you want to save path
# My working env is on MacOS, one need to make some changes if running on Windows

require 'rubygems'
require 'hpricot'
require 'open-uri'
require 'logger'
$log = Logger.new('log/railscasts.log')
$path = "/Users/akshaygupta/railsvideo/railscasts/"
$stop = false


class Railscasts
attr_accessor :url

def initialize
  @@page = 1
  @@url  = "http://railscasts.com/episodes?page="
  start
end

def url
  @url = @@url+@@page.to_s
end

def start
  url
  build_doc
  screencasts_links
  download_screencasts
  next_page
  if !$stop
    start
  else
    puts "Successfully done :) Enjy all the screencasts"
  end
end

def build_doc
  begin
    $log.info("*********Fetching #{@url}***********")
    @doc = Hpricot(open(@url))
  rescue Exception => e
    $log.debug("Problem fetching #{e}")
  end
end


def screencasts_links
  begin
    @download_links = 
      (@doc/".download/a[1]").collect {|a| (a.search("[@href]").first[:href])}
    $log.info(" All Download links on this page :\n #{@download_links}")
  rescue
    $log.info("Problem in download links")
  end
end

def download_screencasts
  @download_links.each do |mov|
    begin
      file = mov.split('/').last
      res = `cd #{$path}; ls | grep "#{file}"`
      if !res
        $log.info("Now downloading file #{file}")
        result = `cd #{$path}; wget "#{mov}"`
        if result
          $log.info("Successfully Downloaded #{file}")
        end
      else
        $log.info("Already downloaded #{file}")
      end
    rescue Exception => e
      $log.info("problem downloding file #{e}")
    end
  end
end

 
  def next_page
    if @@page < 17
      @@page += 1
    else
      $log.info("All screencasts downloaded :-), Mission accomplished!!")
      $stop = true
    end
  end
end

Moments, Life and Remarkability

2008-12-31T12:21:00.000-08:00

Moments pass by as a drop of water, each one of it make Life. Moments vanishes as you start thinking of it to complete your life. But just analyze WHY? Why are we here ? What we are supposed to be?
Some good saying: Live whole-heartedly, help others, do whatever you like with passion, contribute to the miraculous deeds our ancestors had done once, never give up life will always test your patience and endurance. Is it love, lust, success we are living for? Are we going the right way? Are we destined ?

If one ever think of why Einstein, Faraday, Sachin, Gandhi..name anyone..why were they great?How come respect comes out of its own while remembering them.

Simple they discovered something, they were leaders, they were simply different.
How come they were different, they were mere humans.

Your answers started sprouting:

Things were easier at that time. Nah
Easy resources. Not possible.
Plenty of time to try out something new. Not sure? same 24 hours.
Things was unexplored, now everything is there. Same thoughts prevails over centuries.

Eight Secrets Of Life

View SlideShare presentation or Upload your own. (tags: secrets powerpoint)

It seems they did something different, impossible and remarkable. My thoughts say follow those simple guys who were remarkable, contributer and made their life worth living. Moving against the tide is never easy, but the flew, they were not different rather they were much simpler. Isn't?
Its just that they analyzed and walked through the path their heart was leading. It was their passion and inner urge to follow their dream that made them remarkable. They never did it to be famous or for wealth, its the satisfaction they were thriving for. This is greatness.

Now, lets discuss our life, we are brought up educated, equipped with all resources (most of them one don't even need), plenty of time (exclude the time spent on cell, chat, bc and complains about life), I belief everything is more easy, approachable, just one click away thanks to Mr Tim Berners Lee. One dont have to travel years to find where India is, on boat with few crew members. Our great remarkable leaders fulfilled all the resposibilities, they did loved some one, they did married, they did whatever a common man spent time upon. So whats missing ??

Passion, Depth of thought, Direction etc

In one thought we have never thought, why are we here for? It was their remarability that made them respectable, different, famous, successful. You know Apple never fell over Einstein's head, he was just sitting idle under apple tree and looking at calm moon and an Apple and he realized that both are of same size and yet one falls down and one never. So distance can make things look alike but something decreases and he coined the name as Gravity. Remarkable enough for the whole world.

In Modern World one try to dress, talk , walk, become rich but surprisingly none of them takes to the path of remarkability other than creating differences, boundaries. Simplicity is the characterstic of being different.

I believe "Simplicity of character is the natural result of profound thought"

Job vs Life

2008-09-13T11:17:00.000-07:00

Why are we being educated?

Nursery, LKG, UKG, I, II, III, IV, V, VI, VII, VIII, IX, X, XI, XII
What Next ? ...IIT, an year dropped:(, Jamia is although good. I, II, III, IV year. B.Tech Comp(uter)lete. What Next?...MS/MBA/Job?I opted for Job :-)

Being from a middle class family, some how our society gives you an aim to get a "JOB" and work generously to rise and earn decent salaries.

And when I am very near to start with my first job, there is something within myself which is questioning is this the end (as shown by our society) or is this the beginning? Beginning as of...starting of my career, going to company everyday, work and again work. Is this what for I have been educated for so long and awaited to feel the eternal bliss of a Job, what only matters is to earn money, do i also get trapped as millions of people and loose my identity, will it provide me happiness to the rest of my life, will I be able to fulfill my dreams/ambitions within some years from now before marriage specifically.

I felt really disappointing that neither our society nor our education system taught anything in this regard. Always I tried soving Matrix/Differential/Integration problems, mugging history, solving assignments, impress my teachers, having high grade. Unfortunately, it had lead me nowhere and I completely don't know why did I actually studied all the crap which I have already forgotten long time back and to blame myself, I never took the pain to explore beyond the psychological comfort of a job which somehow flourished in my mindset too and I completely felt a class-I guy who enters new classroom (World), and feels completely strange with no mentor this time and as understanding grows, this is stage where again I have to prove myself where millions of people are already competing to grow by hook or by crook. Remember famous dialogue from RDB: "gate k is taraf hum duniya ko nachate hai and gate k us taraf duniya hamein aur duniya ki is bheed me na jaane laakhon DJ piss gaye",indeed very true. We are out of the gate, no step backing, now.

Don't read if you think I'm about to give to an answer or an alternate to situation. What I mean is to understand yourself, organize your thought. Think again and again why have you really been educated? Its only your thought process that can enlighten your path to vision beyond the boundaries.

If I talk about myself, I am closely analyzing all trends of lifestyle, fashion, Gen X, Tech World, Entrepreneurs, Government, India's "Cool" philosophy (will talk more about it later), Mobile, Internet etc., and will launch myself in the direction I'll feel myself more aligned to unleash the mystery of life and to achieve 100 percent of my thought. This might be the reason why Americans/Japanese are leading as they walk in that platform they are actually meant for besides merely following others blindly and unknowingly getting trapped.

Remember, job in any way is a medium (not the end) to reach to your dream/ambitions and this is what you exactly need to explore and then one need to keep the SPARK alive to achieve them. Happy Beginning :-)

Ideas and their Execution..

2007-12-11T10:06:00.000-08:00

Ideas ... ?

Human brain works at much higher frequency than a CPU of one's PC or laptop, without getting unplugged or low battery. Simulations shake hand concurrently without deadlock par than multitasking system can. Hemoglobin transfers data and control signals with FSB with the frequency of light. Brain stores all memories with infinite storage capacity as a Hard-disk without ever getting crashed, Responses which are impulsive, simple, potent and follow all the Laws of Nature, and portray ones thought process which flows out intermittently from our Mind (Cache). Ans if they really solve some problem...I call them IDEAS ... !!!
Hope now one understands my IDEA !!
I'm another idiotic computers guy :)

Individual Globalization....

2007-05-17T17:00:00.000-07:00

Voyage:

The term mostly related with Christopher Coumbus on his voyage in 1492 who sailed with the Nina, the pinta and the Santa Maria in an effort to discover a shorter, more direct route to India to confirm that the world is round not flat!!
U know Y he did the Voyage???Just to encounter a New World, to search Indian Riches: precious metals , silks, spices. Era of The Globalizaton of countries and power.

World:

Then from 19th to 20th century the expansion of world via telephones, media, electronics world, Stock exchange, the PC's, satellite, Multi national Companies headed while being interrupted by the great depresson(colonization and bombing on Hiroshima and Nagasaki) and World Wars I &II the life never stopped and the key factor "Change" resulted in the

Globalization of Companies and their Integration

Flat World:

"The Playing Ground Is Being Leveled: Mr Nilkani, CEO Infosys"
Now the 21st Century and the world shrinking is to nano size and the storage expanding to Terabytes. Asians, especially Indian professional outsourcing the work and Indian IT market and Software graduates encomplishing tons of work while its night in the Developed Countries has set the world for a individual to Expand Its Horizon and

The Era Of INDIVIDUAL Globalization, the newfound power to collaborate and compete globally has now Leveled the GROUND.