Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-5134

Error creating Batch JDBC storage in domain

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 10.0.0.Final
    • 10.0.0.Beta1
    • Batch
    • None
    • Hide

      1. start a domain from the default configuration
      2. for main-server-group deploy a JDBC driver for oracle, db2 or sybase and create a data source in full profile using the driver
      3. 

      /profile=full/subsystem=batch-jberet/jdbc-job-repository=jdbc:add(data-source=$DS_NAME)

      4. server-one and server-two race to create the tables, resulting in:

      {
    "outcome" => "failed",
    "result" => undefined,
    "failure-description" => {"WFLYDC0074: Operation failed or was rolled back on all servers. Server failures:" => {"server-group" => {"main-server-group" => {"host" => {"master" => {"server-two" => {"WFLYCTL0080: Failed services" => {"org.wildfy.batch.job.repository.jdbc" => "org.jboss.msc.service.StartException in service org.wildfy.batch.job.repository.jdbc: WFLYBATCH000011: Failed to create JDBC job repository.
    Caused by: javax.batch.operations.BatchRuntimeException: JBERET000624: Failed to create tables for batch job repository database product name Adaptive Server Enterprise with DDL sql/jberet-sybase.ddl
    Caused by: java.sql.BatchUpdateException: JZ0BE: BatchUpdateException: Error occurred while executing batch statement: There is already an object named 'JOB_INSTANCE' in the database.
"}}}}}}}},
    "rolled-back" => true,
    "server-groups" => {"main-server-group" => {"host" => {"master" => {
        "server-one" => {"response" => {
            "outcome" => "failed",
            "rolled-back" => true
        }},
        "server-two" => {"response" => {
            "outcome" => "failed",
            "failure-description" => {"WFLYCTL0080: Failed services" => {"org.wildfy.batch.job.repository.jdbc" => "org.jboss.msc.service.StartException in service org.wildfy.batch.job.repository.jdbc: WFLYBATCH000011: Failed to create JDBC job repository.
    Caused by: javax.batch.operations.BatchRuntimeException: JBERET000624: Failed to create tables for batch job repository database product name Adaptive Server Enterprise with DDL sql/jberet-sybase.ddl
    Caused by: java.sql.BatchUpdateException: JZ0BE: BatchUpdateException: Error occurred while executing batch statement: There is already an object named 'JOB_INSTANCE' in the database.
"}},
            "rolled-back" => true
        }}
    }}}}
}
      Show
      1. start a domain from the default configuration 2. for main-server-group deploy a JDBC driver for oracle, db2 or sybase and create a data source in full profile using the driver 3. /profile=full/subsystem=batch-jberet/jdbc-job-repository=jdbc:add(data-source=$DS_NAME) 4. server-one and server-two race to create the tables, resulting in: { "outcome" => "failed", "result" => undefined, "failure-description" => {"WFLYDC0074: Operation failed or was rolled back on all servers. Server failures:" => {"server-group" => {"main-server-group" => {"host" => {"master" => {"server-two" => {"WFLYCTL0080: Failed services" => {"org.wildfy.batch.job.repository.jdbc" => "org.jboss.msc.service.StartException in service org.wildfy.batch.job.repository.jdbc: WFLYBATCH000011: Failed to create JDBC job repository. Caused by: javax.batch.operations.BatchRuntimeException: JBERET000624: Failed to create tables for batch job repository database product name Adaptive Server Enterprise with DDL sql/jberet-sybase.ddl Caused by: java.sql.BatchUpdateException: JZ0BE: BatchUpdateException: Error occurred while executing batch statement: There is already an object named 'JOB_INSTANCE' in the database. "}}}}}}}}, "rolled-back" => true, "server-groups" => {"main-server-group" => {"host" => {"master" => { "server-one" => {"response" => { "outcome" => "failed", "rolled-back" => true }}, "server-two" => {"response" => { "outcome" => "failed", "failure-description" => {"WFLYCTL0080: Failed services" => {"org.wildfy.batch.job.repository.jdbc" => "org.jboss.msc.service.StartException in service org.wildfy.batch.job.repository.jdbc: WFLYBATCH000011: Failed to create JDBC job repository. Caused by: javax.batch.operations.BatchRuntimeException: JBERET000624: Failed to create tables for batch job repository database product name Adaptive Server Enterprise with DDL sql/jberet-sybase.ddl Caused by: java.sql.BatchUpdateException: JZ0BE: BatchUpdateException: Error occurred while executing batch statement: There is already an object named 'JOB_INSTANCE' in the database. "}}, "rolled-back" => true }} }}}} }
    • Workaround Exists
    • Hide

      Because one node will (always, I think) succeed to create all tables and the rollback of the management operation doesn't rollback the database changes, issuing the command in step 3 for the second time will succeed, because all tables already exist this time.

      Show
      Because one node will (always, I think) succeed to create all tables and the rollback of the management operation doesn't rollback the database changes, issuing the command in step 3 for the second time will succeed, because all tables already exist this time.

    Description

      When multiple servers (eg. a server-group in a JBoss domain) attempt to create a JDBC Batch repository at once, and they share a common database, they race in creating the database tables, because sometimes instance A will create the table PARTITION_EXECUTION after instance B invokes the check whether this table exists (this check is used to decide whether all tables need to be created) - therefore B will decide to create all tables too.
      If the database doesn't support the IF NOT EXISTS clause in CREATE TABLE (db2, oracle, sybase..), it will probably lead to a failure like 'table already exists', which in turn rolls back the creation of the JDBC store on all nodes in the domain, including those where the schema creation succeeded or was skipped. The tables remain physically in the database.

      This is similar to WFLY-5061, but this time, it is caused by having multiple servers in a domain.

      Sharing a common database in domain mode should work because it is useful for restarting failed jobs on different nodes if one node crashes.

      Attachments

        Issue Links

          clones

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-758 Error creating Batch JDBC storage in domain

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is blocked by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-5174 Upgrade jberet-core to 1.2.0.Beta1

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Task - Represents a small unit of work that is not end-user facing. JBERET-185 Add test to verify race condition when concurrent clients creating jdbc repository

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved
          relates to

          Bug - A problem that impairs or prevents the functions of the product. WFLY-5061 Tables for JDBC Batch repository are created twice

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Activity

            People

              cfang@redhat.com Cheng Fang
              jmartisk@redhat.com Jan Martiska
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: