martinberger.swiss

Database

New ODA X11-L, first job: re-image

8 min read

You unbox a brand-new Oracle Database Appliance X11-L, rack it, configure the network and start provisioning - and it fails. Not on something exotic, but right in the middle of creating the Grid Infrastructure stack. The twist: the appliance reports a perfectly healthy System Version of 19.30.0.0.0, yet provisioning dies on an invalid version.

That is the lesson of this post: on a fresh-from-factory ODA, don’t trust the version the box shows you. The factory image shipped a DCS/system stack that was out of step with the Grid Infrastructure clone bundled with it, and the GI clone carried a version string that the provisioning code can’t even parse. This turned out to be a known bug - Bug 38899107 - and the fix is a re-image.

GI Image is: Patch 30403673: ORACLE DATABASE APPLIANCE 19.30.0.0.0 GI CLONE FOR ODACLI/DCS STACK.

The system - looks perfectly fine

Straight out of the factory the node still carries its default hostname oak, and describe-component reports a clean, consistent 19.30.0.0.0 across the stack. Nothing here hints at a problem:

[root@oak clones]# /opt/oracle/dcs/bin/odacli describe-component
System Version
--------------
19.30.0.0.0

System Node Name
----------------
oak

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date
DCSCONTROLLER                            19.30.0.0.0          up-to-date
DCSCLI                                   19.30.0.0.0          up-to-date
DCSAGENT                                 19.30.0.0.0          up-to-date
DCSADMIN                                 19.30.0.0.0          up-to-date
OS                                       8.10                 up-to-date
ILOM                                     5.1.5.29.r167270     up-to-date
BIOS                                     90050500             up-to-date
LOCAL CONTROLLER FIRMWARE {
     [c8, c9]                            GDB7302Q             up-to-date
     [c14, c15]                          26.46.3048           up-to-date
}
SHARED CONTROLLER FIRMWARE {
     [c0, c5, c6, c7]                    MPPA6R5Q             MPPA8R5Q
     [c1, c2, c3, c4, c10, c11, c12,     9CV1R490             up-to-date
     c13]
}
HMP                                      2.4.10.1.600         up-to-date

Notice there is no GI line and no DB line yet - that is expected on an un-provisioned box. The Grid Infrastructure home only gets created during provisioning, and that is exactly where it breaks.

The symptom - provisioning fails creating the Grid stack

The provisioning job runs through networking, users, groups, certificates and even extracts the GI clone - then collapses at Grid stack creation / Updating GI home version:

[root@oda-solothurn stage]# odacli describe-job -i 284a3ad4-ff95-460c-9e06-927440e02e97

Job details
----------------------------------------------------------------
                     ID:  284a3ad4-ff95-460c-9e06-927440e02e97
            Description:  Provisioning service creation
                 Status:  Failure (To view Error Correlation report, run "odacli describe-job -i 284a3ad4-ff95-460c-9e06-927440e02e97 --ecr" command)
                Created:  May 28, 2026 15:47:50 CEST
                Message:  DCS-10200:Specified version: 19.30.0.0.260120(REL-JAN260130) is invalid.

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
...
Creating GI home directories             May 28, 2026 15:50:09 CEST               May 28, 2026 15:50:09 CEST               Success
Extract GI clone                         May 28, 2026 15:50:09 CEST               May 28, 2026 15:50:39 CEST               Success
Grid home creation                       May 28, 2026 15:50:09 CEST               May 28, 2026 15:50:39 CEST               Success
Storage discovery                        May 28, 2026 15:50:39 CEST               May 28, 2026 15:53:16 CEST               Success
Creating wallet for Root User            May 28, 2026 15:53:16 CEST               May 28, 2026 15:53:18 CEST               Success
Creating wallet for ASM Client           May 28, 2026 15:53:18 CEST               May 28, 2026 15:53:20 CEST               Success
Grid stack creation                      May 28, 2026 15:53:20 CEST               May 28, 2026 15:57:34 CEST               Failure
Provisioning GI with RHP                 May 28, 2026 15:53:20 CEST               May 28, 2026 15:57:30 CEST               Success
Provisioning service creation            May 28, 2026 15:57:31 CEST               May 28, 2026 15:57:34 CEST               Failure
Set CRS ping target                      May 28, 2026 15:57:31 CEST               May 28, 2026 15:57:31 CEST               Success
Updating GI home version                 May 28, 2026 15:57:31 CEST               May 28, 2026 15:57:34 CEST               Failure

The headline message is the tell:

DCS-10200:Specified version: 19.30.0.0.260120(REL-JAN260130) is invalid.

That (REL-JAN260130) suffix is not a version number - and that is the whole problem.

Reading the error

A second symptom shows up in describe-component after the failed attempt. Now there is a GI line, but it carries that unparseable string, and the command itself ends in an internal error:

[root@oda-solothurn stage]# /opt/oracle/dcs/bin/odacli describe-component
...
Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date
GI                                       19.30.0.0            up-to-date
                                         .260120(REL
                                         -JAN260130)
DCSCONTROLLER                            19.30.0.0.0          up-to-date
...
DCS-10001:Internal error encountered: Missing version for 'GRID_VERSION' in the version file.

Two distinct failures, one root cause:

  • DCS-10200 - invalid version. The provisioning task Updating GI home version tries to record the GI home’s version. The GI clone reports itself as 19.30.0.0.260120(REL-JAN260130). DCS parses the build segment as an integer, the (REL-JAN260130) text is not numeric, and it throws.
  • DCS-10001 - missing GRID_VERSION. A knock-on effect: because Updating GI home version never finished, no GRID_VERSION was written to the VERSION file, so a later lookup has nothing to read.

The Error Correlation Report (--ecr) makes the Java-level cause explicit - a plain NumberFormatException:

[root@oda-solothurn dcs]# odacli describe-job -i 284a3ad4-ff95-460c-9e06-927440e02e97 --ecr
ODA Assistant - Error Correlation report
----------------------------------------
          Failed job ID:  284a3ad4-ff95-460c-9e06-927440e02e97
            Description:  Provisioning service creation

Failed Task Messages
--------------------
[Provisioning service creation] -  DCS-10200:Specified version: 19.30.0.0.260120(REL-JAN260130) is invalid.

oda-solothurn Log Messages
----------------------------
  DCS Agent
  ~~~~~~~~~
    Error Logs
    ==========
    [Backup Dcs Admin Rpm] - DCS-12018:Directory /opt/oracle/oak/pkgrepos/dcsadmin/19.30.0.0.0 does not exist.
    [Updating GI home version] - Failed to parse version: 19.30.0.0.260120(REL-JAN260130)
    [Updating GI home version] - java.lang.NumberFormatException: For input string: "260120(REL-JAN260130)"
    [Updating GI home version] - InvalidVersionError
    [Updating GI home version] - DcsException{errorHttpCode=InvalidParameter, msg=Specified version: 19.30.0.0.260120(REL-JAN260130) is invalid., msgId=10200, causedBy=null}
    [Provisioning service creation] - DCS-10200:Specified version: 19.30.0.0.260120(REL-JAN260130) is invalid.
      Error code - DCS-10200
      Cause: The specified version did not match any of the patches available on the system.
      Action: Use 'odacli list-availablepatches' to get the list of available versions. Retry the operation with an available version.

java.lang.NumberFormatException: For input string: "260120(REL-JAN260130)" is the smoking gun. The standard DCS-10200 advice - “run odacli list-availablepatches and retry with an available version” - is a dead end here, because the bad version is not something you supplied. It is baked into the shipped GI clone.

Root cause - the factory image is inconsistent

Look at the DCS VERSION file as shipped. The build is 260121:

[root@oak dcs]# cat VERSION
# This is a system-generated file.
# Do not modify its content.

DCS_ADMIN_VERSION=19.30.0.0.0
DCS_ADMIN_VERSION_BUILD=260121
DCS_COMPONENT_VERSION=19.30.0.0.0
DCS_COMPONENT_VERSION_BUILD=260121
LOCAL_SYSTEM_NOREBOOT_VERSION=19.30.0.0.0
LOCAL_SYSTEM_NOREBOOT_VERSION_BUILD=260121
LOCAL_SYSTEM_VERSION=19.30.0.0.0
LOCAL_SYSTEM_VERSION_BUILD=260121
STORAGE_VERSION=19.30.0.0.0
STORAGE_VERSION_BUILD=260121

So the picture is:

  • The DCS / system stack is build 260121.
  • The GI clone in the image tags itself 19.30.0.0.260120(REL-JAN260130) - a different build, with a non-numeric label appended.

The factory image was assembled with a DCS/system stack and a Grid Infrastructure clone that don’t line up: different build numbers and a GI label that isn’t a valid version at all. When provisioning tries to reconcile the GI home version against that label, it cannot, and the whole create-appliance job fails. The DCS-12018 “dcsadmin pkgrepo directory does not exist” line is collateral from the same mismatch.

This is a known defect. From the Oracle Support SR:

The issue you are facing is due to a bug (Bug 38899107). In certain instances, just downloading the latest GI clone and re-provisioning can work. However, in most cases, reimaging the ODA is the solution.

The fix - re-image and re-provision

Oracle Support’s action plan, and the path that actually worked:

  1. Download the new 19.30 ISO (bare metal image) from My Oracle Support.
  2. Clean up the appliance with the cleanup utility. On a High-Availability system, run it on both nodes (sequentially - finish node 1, then start node 2):
    # /opt/oracle/oak/onecmd/cleanup.pl
  3. Re-image the ODA from the ISO via Oracle ILOM (mount the ISO, set it as the next boot device, power-cycle).
  4. Download the latest GI clone Patch 30403673 for 19.30 from My Oracle Support.
  5. Extract and register the GI clone into the repository - note the build is now 260210, matching the re-imaged system:
    # odacli update-repository -f /u01/stage/odacli-dcs-19.30.0.0.0-260210-GI-19.30.0.0.zip
  6. Provision the appliance (odacli create-appliance or the Web Console).

The “sometimes a fresh GI clone alone is enough” note is real, but as Oracle says, in most cases only a re-image clears it - because the inconsistency is in the shipped image itself, not just the registered clone.

After re-image - the version is whole again

Post re-image, the DCS VERSION file is consistent: the build moves to 260210 across the board, with a parseable version set throughout:

oracle@oda-solothurn:/u01/stage/ [rdbms19] cat /opt/oracle/dcs/VERSION
# This is a system-generated file.
# Do not modify its content.

DCS_ADMIN_VERSION=19.30.0.0.0
DCS_ADMIN_VERSION_BUILD=260210
DCS_COMPONENT_VERSION=19.30.0.0.0
DCS_COMPONENT_VERSION_BUILD=260210
LOCAL_SYSTEM_NOREBOOT_VERSION=19.30.0.0.0
LOCAL_SYSTEM_NOREBOOT_VERSION_BUILD=260210
LOCAL_SYSTEM_VERSION=19.30.0.0.0
LOCAL_SYSTEM_VERSION_BUILD=260210
STORAGE_VERSION=19.30.0.0.0
STORAGE_VERSION_BUILD=260210
GRID_VERSION=19.30.0.0.0
GRID_VERSION_BUILD=000000

With a parseable, consistent version set - and GRID_VERSION recorded once provisioning can proceed - Updating GI home version no longer throws and provisioning completes.

Takeaways

  • A clean System Version proves nothing about the GI clone. describe-component showed a healthy 19.30.0.0.0.
  • NumberFormatException in the ECR is the giveaway. When Updating GI home version fails with DCS-10200 and the log shows a NumberFormatException on a version string, you are looking at a clone/stack mismatch, not a typo in your command.
  • Don’t waste time on list-availablepatches. The generic DCS-10200 advice doesn’t apply when you never specified the version yourself.
  • For a brand-new ODA, plan for a re-image to the latest bundle before going live. It is the cleanest way to guarantee the system stack and the GI/DB clones actually match - exactly what Bug 38899107 broke at the factory.