More On Automating Your Disaster Recovery Restore

By Rich Loeber,

Some while ago, I wrote a security tip about automating your IBM i disaster recovery restore process. Of the many tips I’ve done over the years, none has drawn as much “fan mail” as this particular tip.

The essence of the tip is an implementation of the OS/400 command LODRUN to automatically restore a full system backup tape. The LODRUN searches the tape for a known program (named QINSTAPP), loads that program into the session QTEMP library and then calls it. Your QINSTAPP program can then load the contents of the tape. The only thing you have to do is create the QINSTAPP program and include it in the backup.

Well, I created that methodology quite a while ago, and a “fan” recently contacted me to let me know that, during testing, he found a problem with it. It seems that the current versions of i/OS don’t like it when the SAVLIB for *ALLUSR or *NONSYS libraries is not the first backup set on the tape. The tip clearly was calling for the QINSTAPP *PGM object to be the first object on the tape, so there was a conflict.

First, I have to commend my “fan” (Keith Scott at Hoover Materials Handling Group) for putting the process to the test. Over the years, I’ve given away hundreds of copies of the shell CL program for QINSTAPP and Keith was the only one to test it and find the problem. No matter where you get your code, testing should ALWAYS be a requirement, especially for something so important as a disaster recovery procedure. I tested this process long ago and it worked then, but it is problematic today.

The news is not all bad, however. With the current versions of i/OS, the LODRUN command supports a SEQNBR parameter where you can use a *SEARCH parameter. This will cause the process to scan the tape until it finds the QINSTAPP saved from QTEMP. At that point, it will load the program and run it.

There is a downside, however. Because the SAVLIB has to come first on the tape, there is a lot of tape searching going on before the actual save gets going. This could add 30 minutes or so to your restore process, depending on the kind of tape you’re using. But, the restoration process is still fully automated, so there’s a strong benefit.

The modified save process should now be changed to save your system in the following sequence:

  • SAVLIB for the libraries you want to save (either *ALLUSR or *NONSYS)
  • SAVOBJ for the QINSTAPP saved from QTEMP
  • AVSECDTA to save your security setup
  • SAVDLO for your shared folder objects
  • SAV for the IFS (other than shared folders)

The QINSTAPP has to be changed to first restore the security data which will sit on the tape right after the QINSTAPP itself. Then, the restore should force a rewind on the tape and restore the libraries. When that’s done, the remaining restores for the IFS will complete the process. When all is said and done, the last step must be a RSTAUT to restore object authorities.

To properly illustrate this, I’ve updated my sample QINSTAPP CL program and also created a companion QDRSAVE CL program to show the system save process in the right sequence. If you’d like to see these sample programs, send me an email (rich@kisco.com) and I’ll send you the shells that I recently created.

Automating Your Disaster Recovery Restore

By Rich Loeber,

Part of the job of a security officer is creating, maintaining and testing your disaster recovery plan. A major part of disaster recovery is recreating your computing environment on a completely different system and this always involves data and program restores. You might be one of the fortunate ones that has access to a comprehensive third party save/restore application that automates the disaster recovery restore process for you. If you’re in this group, this post is not for you. But, if you’re a small to medium sized shop that can’t afford (or won’t consider) one of these products, then you’re on your own to create a workable disaster recovery method.

If you fall into this group, then you should consider implementing a LODRUN program to automate your disaster recovery process. LODRUN is an i/OS command that is normally just used by developers of third party software as a method of controlling the installation process for their software packages. But, LODRUN can easily be used to automate your recovery restore process as well.

When you run the LODRUN program, the command calls a process which looks for a *PGM object at the beginning of the tape with the unique name of QINSTAPP. Once found, it will transfer this *PGM object into the special QTEMP library and then call it, passing along the parameters from the LODRUN command which includes the device name of the tape drive being used.

So, for you to have a controlled restore of your backup tape, all you need to do is create a CL program named QINSTAPP with a single parameter for the tape device name. Then, you can use that CL program to do the controlled restore from the tape onto the new system.

A typical off-site backup tape will contain the following items:

1. A backup of the user profiles
2. A backup of the configuration
3. Backups of the user libraries
4. Backups of the IFS objects

You are probably already familiar with exactly how you do backups for your shop. To use the LODRUN method for an automated restore, all you need to do is create a CL program named QINSTAPP that will restore the objects from the backup tape in the same sequence in which they are recorded on the tape.

Some hints:

1. Change your save program to transfer the QINSTAPP *PGM object to QTEMP and then do a SAVOBJ to save it as the very first object on your backup tape.

2. Be sure that you use the *LEAVE option so that the tape is properly positioned after each restore is completed.

3. Be sure to run the RSTAUT command after everything has been restored to get the authorities correct.

4. Use the CHKTAP command at the very end to rewind and/or unload the tape, this will give you better flexibility when it is time to make changes to your program.

5. Add a clear comment to the CL program you use for creating backups that advises any programmer that works on that program to make sure and mirror any changes in your restore program.

6. When you first create the QINSTAPP CL program, have your save program handy so that you get the sequence of events absolutely correct.

When you’re ready to run the restore, all you’ll need to do is mount the tape and issue the LODRUN command. At that point, you custom restore program will take over and give you a quick, complete and controlled restore of your system on your recovery system. If you’d like to see a sample QINSTAPP program, send me an email (rich@kisco.com) and I’ll send you a shell that I recently created.