Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6989

Refactor incremental snapshots content in partials files

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major Major
    • 2.3.GA
    • None
    • documentation
    • None
    • False
    • None
    • False

      The partials files that are used in the connector topics for triggering and stopping incremental snapshots must be refactored.
       
      Since the 2.1.4 productization release, with the introduction of the Kafka signaling channel, revisions were made to the partials files that contain the instructions for triggering and stopping incremental snapshots. The following changes are causing downstream build problems that result in content failing to build or failing to render:

      • New include directives were added to pull in the Kafka versions of the instructions for triggering and stopping incremental snapshots.

      A fourth-level header was added to to the partials file for using Kafka to trigger an incremental snapshot. 
       
      Recursive partials relative references 
      To use partials files downstream, they cannot contain include directives that reference another partials file, and they cannot contain any Asciidoc headers. 
       
      The connector docs use the following files to provide the instructions for triggering and stopping incremental snapshots for SQL-based, and noSQL connectors:
       

      • proc-triggering-an-incremental-snapshot-nosql.adoc
      • proc-triggering-an-incremental-snapshot-sql.adoc
      • proc-stopping-an-incremental-snapshot-nosql.adoc
      • proc-stopping-an-incremental-snapshot-sql.adoc

      Each of these files contain an `include` directives at the end of the file that links to another partials files, which contain the Kafka version of the instructions for triggering or stopping a snapshot: 
       
      include::{partialsdir}/modules/all-connectors/proc-stopping-an-incremental-snapshot-kafka.adoc[leveloffset=+3]
      include::{partialsdir}/modules/all-connectors/proc-triggering-an-incremental-snapshot-kafka.adoc[leveloffset=+3]
       
      The partialsdir attribute in the preceding links contain different types of values in the upstream and downstream repos:

      • Upstream, the attribute resolve to absolute paths to files in the Debezium repo.
        As a result, for the upstream Antora docs, these links resolve correctly.
      • Downstream the attribute is defined as a relative path.

      This causes a problem because the parent proc-triggering-*.adoc and proc-stopping-*.adoc files named above are themselves referenced from the different connector files by way of relative paths  through the use of the same attribute.
      This results in a situation in which the incremental snapshots topic in a file such as db2.adoc uses an include directive with a relative path to pull in the content from the partials file proc-triggering-an-incremental-snapshot-sql.adoc.
      This file, in turn, uses an include directive with a relative path to pull in the content from another partials file, proc-triggering-an-incremental-snapshot-kafka.adoc.
       
      When the content is built downstream, the linked content fails to render, because the include directive inserts an extra reference to the /partials directory, so that the link incorrectly resolves to:
       
      upstream/debezium/debezium-2.3/documentation/modules/ROOT/partials/partials/modules/all-connectors/proc-triggering-an-incremental-snapshot-kafka.adoc

      Possible resolution

      a) Simplest path would be to edit the paths in the downstream files, but this is not a long-term solution.
      b) Reference the Kafka-based content directly from the parent connector files.
      This approach requires moving the Title text at the head of the files for the Kafka-based content into the parent connector docs and then following them with the relevant include directives.

       
      Embedded headings in the partials file

      A second problem results from the use of the following 4th level heading in the file `proc-triggering-an-incremental-snapshot-kakfa.adoc`:
       
      ==== Ad hoc incremental snapshots with additional-condition
       
      The content represented by the embedded heading is ignored during the downstream build and fails to render.

      Possible resolution
      Either replace the heading with a title (i.e., .Ad hoc incremental snapshots ...), or place the heading directly in the parent connector files, as is currently done with the Ad hoc snapshots and Incremental snapshots headings.

            broldan@redhat.com Robert Roldan
            broldan@redhat.com Robert Roldan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: