3.16. Continuous, automated, ODI repository full rebuilds

3.16.1. Overview

We’ll be building upon the output of demo 1 (or demo 2), so if you haven’t already run through demo 1 (or demo 2), do that now before coming back to this demo.

This demo shows the ODI-SCM solution used to automatically destroy and rebuild an ODI repository from the SCM system. We use the SVN repository that we created in demo 1 as the source SCM system and create a new SVN working copy from it. We then destroy the ODI repository

In a later demo we’ll show this feature, as well an incremental build, being triggered by a process that monitors the SCM system for changes.

Why would anyone want to do this?

To find integrity issues in your ODI code base as early as you can.

3.16.2. Setting Up Additional Tools For This Demo

3.16.2.1. PaperCut

PaperCut is a simple development utility that you can use to show emails being sent, via SMTP, by your application. It does not relay the messages. It just allows you to just see what is being sent the the SMTP service.

Download the application from http://papercut.codeplex.com/. Then unpack the ZIP file contents to a new directory.

Start the application by double clicking on the papercut.exe executable.

_images/30_10_10.png

Note that the default configuration for PaperCut is for it to listen on TCP port 25 (the standard SMTP server TCP port).

3.16.3. Create The Demo Configuration (INI) File

Copy of the pre-configured demo INI file OdiScmAutoRebuildDemo.ini to demo base directory. From the command prompt:

copy "%ODI_SCM_HOME%\Configuration\Demo\Demo3\OdiScmAutoRebuildDemo.ini" C:\OdiScmWalkThrough\

This demo INI file is pre-configured for ODI 11g.

Open the INI file and edit the same entries in the [OracleDI] and [Tools] sections, as for demo 1, for your environment.

3.16.4. Set The Environment

Again, we need to tell ODI-SCM to use the configuration defined in the INI file. From the command prompt:

set ODI_SCM_INI=C:\OdiScmWalkThrough\OdiScmAutoRebuildDemo.ini
call OdiScmEnvSet

3.16.5. Create The Oracle User

Create a new Oracle user (odirepofordemo3), for the new ODI repository, using an ODI-SCM command. From the command prompt:

OdiScmCreateOdiRepositoryDbUser
_images/30_20_10.png

3.16.6. Additional Step For ODI 10g Users

3.16.6.1. Manually Create The Master and Work Repositories

If you’re using ODI 10g then an extra step is required - creating a base empty Master/Work repository export backup.

This is because we can’t use an ODI SDK with ODI 10g to create empty repositories.

Use the values from the INI file and follow the manual process, decribed in Demo 1, to create an empty ODI 10g Master and Work repository.

Use repcreate.bat to start the wizard to create the new Master repository. Bbe sure to use 100 for the ID of the new Master repository.

_images/30_30_10.png

Then connect to the new Master repository, using the Topology UI, and create a new Work repository, with ID 100, named WORKREP, etc.

_images/30_40_10.png

Then:

_images/30_50_10.png

Then:

_images/30_60_10.png

Finally (be sure to use 100 for the ID of the new Work repository):

_images/30_70_10.png

3.16.6.2. Create An Oracle Export Backup of the New Repositories

Create the Oracle export backup of the new repository pair. From the command prompt:

cd "%ODI_SCM_TOOLS_ORACLE_HOME%\bin"

Then:

exp.exe odirepofordemo3/odirepofordemo3@localhost:1521/xe owner=odirepofordemo3 file=C:\OdiScmResources\odirepofordemo3_REPID_100_empty_master_work_%ODI_SCM_ORACLEDI_VERSION%.dmp statistics=none
_images/30_80_10.png

The export tool will terminate. Check that the export file has been created. From the command prompt:

dir C:\OdiScmResources\odirepofordemo3_REPID_100_empty_master_work_*.dmp
_images/30_90_10.png

3.16.7. A Point About The Repository ID

If you’ve just skipped the additional step, required for ODI 10g users, then you might not have noticed that, in this demo, we have are using 100 for the new ODI repository’s ID.

We have used this ID already for the internal ID of the environment 1 ODI repository in demo 1. The reuse of this ID is intentional in this demo.

This is not a practice that should be followed when creating ODI repositories in which new code or code changes will be made. If you you’ll very likely encounter object internal ID conflicts when integrating the code from the different repositories. You really don’t want to go there.

What we’re showing by reusing the ID is that ODI-SCM can safely import code into a repository that is not the original repository with the same ID. This operation is now blocked in ODI 11g, by default.

This ODI-SCM feature allows you to reuse repository IDs to create repositories that are only ever used to import code into, such as those used for build testing.

It also allows you to destroy and create repositories freely as ODI-SCM is able to re-align ODI object internal ID tracking metadata after importing code.

This feature is enabled by setting the configuration (INI) file entry Export Cleans ImportRep Objects=Yes in the [Generate] section. This causes ODI-SCM to remove the original repository identification from objects when exporting then. It was enabled in demo 1, hence the ODI object exports files were created without the original repository identifiers.

3.16.8. Press The Red Button!

Now run the destory/rebuild. From the command prompt:

OdiScmAutoRebuild
_images/30_100_10.png

Check the PaperCut window. You should see an email sent from ODI-SCM notifying you of the start of the rebuild process.

_images/30_110_10.png

When the rebuild process has completed you should see that another notification email has been sent. This email lets you know that the build process has completed and if it succeeded or failed at some point in the process.

_images/30_120_10.png

3.16.9. Rebuilding From the Existing Working Copy

The above process rebuilds the ODI repository by destroying the working copy, recreating it and getting all of the source code from the SCM system.

What if you want to rebuild the ODI repository from the existing working copy?

Well, there’s a simple change to the command line that let’s you do this:

OdiScmAutoRebuild FromWorkingCopy

The FromWorkingCopy argument tells ODI-SCM to not destroy and recreate the working copy, or get all of the source code again from the SCM system.

FromWorkingCopy can be used to override the default of FromSCM (the other valid argument to this command).

Rebuilding from the existing working copy is actually extremely useful when we hook up ODI-SCM to a Continuous Integration server. We’ll show this working in another demo.

3.16.10. What Next?

Well, one good, simple, immediate application would be to run OdiScmAutoRebuild in an infinite loop. You get a fresh full build each time, from the SCM system it runs and you discover build integrity issues soon after they’re introduced to the code base in the SCM system.

Replace PaperCut usage with your enterprise email server, of course!