Bug report

Resetting the offset to supplement the historical data, the consumption stops。

Today is 12:00 on June 29, 2022. When I want to supplement the historical data by resetting the debezium offset, the re-consumption of historical data is very slow or there is no consumption. At this time, the debezium status is normal.

 

As shown in the picture below：

 

What Debezium connector do you use and what version?

<mysql 1.9.4>

What is the connector configuration?

<

{
"name": "uds_prod_1h-connector_bak",
"config":

{ "connector.class": "io.debezium.connector.mysql.MySqlConnector", "task.max": "1", "database.hostname": "uds", "database.port": "3306", "database.dbname": "uds", "database.user": "canal", "database.password": "", "database.server.id": "20200629", "database.server.name": "debezium-prod-dd_sdi_ods_uds_uds_prod_1h", "database.include.list": "uds", "table.include.list": "uds.account_domains_mapping,uds.account_failure_kafka_messages,uds.activity_order,uds.administrator,uds.administrator_condition,uds.administrator_role,uds.advertiser_gdt,uds.advertiser_tt,uds.ammeter_apply_order,uds.app_message,uds.app_message_record,uds.app_message_template,uds.app_user_blocklist,uds.app_user_blocklist_transaction,uds.app_user_relationship,uds.banned_word,uds.banner,uds.calendar_task_distribution,uds.campaign,uds.campaign_channel_mapping,uds.campaign_city_mapping,uds.campaign_info,uds.campaign_theme,uds.car_deliver_order,uds.car_delivery_center,uds.car_examine_order,uds.car_examine_precheck,uds.car_order,uds.car_order_accessory,uds.car_order_allowance,uds.car_order_battery_rent,uds.car_order_bill,uds.car_order_contract,uds.car_order_contract_send_record,uds.car_order_delivery_info,uds.car_order_examine_precheck,uds.car_order_ext_info,uds.car_order_follow,uds.car_order_follow_info,uds.car_order_follow_transaction,uds.car_order_loan,uds.car_order_material,uds.car_order_payment,uds.car_order_pic,uds.car_order_pq,uds.car_order_price_confirmation,uds.car_order_produce_status_history,uds.car_order_reconfiguration,uds.car_order_register_enterprise,uds.car_order_register_person,uds.car_order_relation,uds.car_order_schedule,uds.car_order_sign_accessory,uds.car_order_sign_accessory_status_history,uds.car_order_status_history,uds.car_order_vehicle_owner,uds.car_pq,uds.car_pq_region_mapping,uds.car_replacement_order,uds.car_show,uds.chat_group,uds.chat_token,uds.china_region,uds.city_conf,uds.city_delivery_center_mapping,uds.club_event,uds.club_event_conf,uds.club_kafka_message,uds.codex,uds.codex_range,uds.codex_transaction,uds.commodity,uds.community_identity,uds.community_identity_batch_task,uds.community_identity_batch_task_transaction,uds.community_identity_group,uds.company,uds.condition,uds.contacts,uds.coupon,uds.coupon_batch_task,uds.coupon_batch_task_transaction,uds.coupon_box,uds.coupon_box_mapping,uds.coupon_box_send,uds.coupon_box_task,uds.coupon_commodity_mapping,uds.coupon_consume_transaction,uds.coupon_debit,uds.coupon_debit_transaction,uds.coupon_event_type,uds.coupon_give,uds.coupon_issue,uds.coupon_register,uds.coupon_send,uds.coupon_send_check,uds.coupon_send_transaction,uds.crawler_config,uds.crawler_topic,uds.credit,uds.credit_batch_reason,uds.credit_debit,uds.credit_debit_batch_transaction,uds.credit_debit_contacts,uds.credit_debit_transaction_message_mapping,uds.credit_event_type,uds.credit_frozen_transaction,uds.credit_given,uds.credit_manual_operation_reason,uds.credit_record,uds.credit_red_packet,uds.credit_red_packet_transaction,uds.credit_restrict,uds.credit_transaction,uds.credit_transaction_ext,uds.credit_transaction_red_packet_mapping,uds.credit_wbs,uds.custom_field,uds.custom_field_category,uds.custom_field_option,uds.custom_field_user_mapping,uds.deadlocks,uds.delay_unbind_result,uds.department,uds.des_order_mapping,uds.dictionary_field,uds.dictionary_option,uds.dlm_in_charge_city_mapping,uds.drive_booking,uds.e_ticket,uds.e_ticket_relevance,uds.email_message,uds.email_message_record,uds.enterprise,uds.enterprise_region,uds.evaluation_auth,uds.evaluation_dimension,uds.evaluation_scene,uds.evaluation_template,uds.event,uds.event_conf,uds.event_conf_value,uds.event_relationship,uds.exhibition_area_traffic,uds.fellow,uds.fellow_nsp_mapping,uds.fellow_order_mapping,uds.fellow_parttime_department,uds.fellow_receptionist,uds.fellow_receptionist_bak,uds.fellow_share,uds.fellow_user_relation,uds.fellow_user_relationship,uds.fellow_user_relationship_transaction,uds.fellow_user_remark,uds.first_time_drive_car_type_transaction,uds.floor_traffic,uds.follow_notice_config,uds.frontline_staff,uds.frontline_staff_charge_info,uds.gate,uds.gate_event,uds.im_auto_group_member,uds.im_auto_group_rule,uds.im_batch_send_group,uds.im_batch_send_group_member,uds.im_batch_send_group_message,uds.im_message_template,uds.intention_fail_reason,uds.intention_product_mapping,uds.intention_type_config,uds.invitation_code_collection,uds.inviter_relationship,uds.inviter_relationship_global,uds.inviter_relationship_transaction,uds.job,uds.job_execution,uds.leads_activate_record,uds.leads_activate_rule,uds.leads_channel,uds.leads_distribution,uds.leads_distribution_history,uds.leads_distribution_rule,uds.leads_follow,uds.leads_group,uds.leads_group_user_mapping,uds.leads_no_region,uds.leads_recycle_reason,uds.major_cus_ent_contact,uds.major_cus_ent_contact_20220120bak,uds.major_cus_ent_sign,uds.major_cus_enterprise,uds.major_cus_enterprise_20220120bak,uds.major_cus_program,uds.major_cus_requisition,uds.media,uds.message_content,uds.ne_business_history,uds.ne_call_history,uds.nic_code,uds.nic_code_range,uds.nic_code_transaction,uds.nic_ftue,uds.nic_ftue_user,uds.nic_role,uds.nic_role_code,uds.nic_role_code_range,uds.nic_user,uds.nic_user_relation,uds.nickname_remark,uds.nio_house,uds.nio_house_ext_info,uds.nio_house_gate_mapping,uds.nio_house_traffic,uds.nio_house_visitor,uds.nio_value,uds.nio_value_child_event,uds.nio_value_event_conf,uds.nio_value_event_total,uds.nio_value_manual_execute,uds.nio_value_manual_execute_fail,uds.nio_value_manual_operation_log,uds.nsp_user,uds.nsp_user_house_mapping,uds.operation_account,uds.pe_charge_order,uds.pe_mount_order,uds.pe_recon_order,uds.phone_message,uds.phone_message_record,uds.phone_message_sensitive_word,uds.phone_message_template,uds.poor_evaluation_track,uds.purchase_intention,uds.purchase_intention_transaction,uds.questionnaire,uds.questionnaire_answer,uds.questionnaire_question,uds.questionnaire_question_option,uds.questionnaire_question_title,uds.questionnaire_send_record,uds.questionnaire_template,uds.ranks_fixdata,uds.ranks_fixdata_bak,uds.rate_limiter_config,uds.raw_value_kafka_message_failed_log,uds.red_packet_plugin_user,uds.red_packet_plugin_user_log,uds.refund_operation_record,uds.refund_order,uds.refund_order_remark,uds.refund_order_workflow,uds.refund_reason,uds.register_record,uds.relation_car_order_kafka_transaction,uds.replacement_order,uds.replacement_order_status_history,uds.resource,uds.role,uds.role_resource,uds.sales_area,uds.sales_area_delivery_region,uds.sales_area_invoice_region,uds.sales_area_region,uds.sales_upload_log,uds.SALES_WEB_SESSION,uds.SALES_WEB_SESSION_ATTRIBUTES,uds.scan_change_order,uds.scan_charge_order,uds.service_conf,uds.setting,uds.shadow_activity,uds.shadow_activity_participation,uds.shadow_activity_scene,uds.shadow_activity_user_like,uds.shadow_app,uds.shadow_credit_transaction,uds.shadow_invite_history,uds.shadow_scene_task,uds.ship_address,uds.ship_address_old,uds.ship_region,uds.ship_region_old,uds.ship_region_test,uds.ship_region3,uds.so_car_wash_order,uds.so_driving_order,uds.so_operate_order,uds.so_parking_order,uds.sp_order,uds.sp_order_contract,uds.sp_order_contract_send_record,uds.sp_order_invoice_enterprise,uds.sp_order_invoice_person,uds.sp_order_payment,uds.special_value_changelog,uds.stock_car,uds.stock_car_accessory,uds.stock_car_config,uds.stock_car_material,uds.stock_car_operation_record,uds.stock_car_reserve_time,uds.stock_car_status_history,uds.stock_car_transaction,uds.survey,uds.survey_activity,uds.survey_activity_channel,uds.survey_answer,uds.survey_send,uds.survey_template,uds.task_setting,uds.tc_code,uds.temp_leads,uds.temp_register_record,uds.temp_register_records,uds.test_drive_car,uds.test_drive_car_pic,uds.test_drive_order,uds.test_drive_order_history_status,uds.test_drive_participant,uds.test_drive_report,uds.test_drive_site,uds.test_drive_source,uds.third_party_user,uds.third_party_user_exception,uds.traffic_user,uds.transactional_messages,uds.tt_pt_potcus_achieve,uds.tt_pt_potcus_achieve_detail,uds.tt_pt_potcus_achieve_level,uds.unprocessed_value_event,uds.urm_session,uds.user,uds.user_account_info,uds.user_address,uds.user_campaign_source,uds.user_campaign_transaction,uds.user_community_identity_mapping,uds.user_device_info,uds.user_dictionary_field_mapping,uds.user_employee_info,uds.user_enterprise_relation,uds.user_evaluation,uds.user_evaluation_remark,uds.user_ext_info,uds.user_identity_info,uds.user_identity_photos,uds.user_imsi_info,uds.user_info,uds.user_invitation_code,uds.user_label,uds.user_label_mapping,uds.user_location_bak,uds.user_mac_info,uds.user_marketing_info,uds.user_marketing_info_transaction,uds.user_questionnaire,uds.user_questionnaire_answer,uds.user_region,uds.user_relationship,uds.user_relationship_log,uds.user_relationship_remark,uds.user_rights,uds.user_rights_category,uds.user_rights_nio_house_mapping,uds.user_role_transaction,uds.user_sales_area,uds.user_source,uds.user_survey_answer,uds.user_tag,uds.user_tag_group,uds.user_tag_mapping,uds.user_user_rights,uds.user_vehicle_info,uds.value_transaction,uds.vehicle_authorizations,uds.vehicle_delivery_center,uds.vehicle_info,uds.vehicle_maintenance_record,uds.vehicle_maintenance_record_item,uds.vehicle_user_relationship,uds.vehicle_user_relationship_transaction,uds.vom_car_order_kafka_message,uds.wbs,uds.wechat_user,uds.z_activity,uds.z_activity_checkin,uds.z_activity_register,uds.z_books_lending_transaction,uds.z_child_care_transaction,uds.z_community_activity_scene,uds.z_community_group,uds.z_community_group_transaction,uds.z_community_group_user_mapping,uds.z_coupon_consume_transaction,uds.z_nio_account_login_transaction,uds.z_nio_house_checkin_transaction,uds.z_room_book_order_transaction", "database.history.kafka.bootstrap.servers": "kafka-001:9092,kafka-002:9092,kafka-003:9092", "database.history.kafka.topic": "history-debezium-uds_prod_1h_20220629_bak", "database.history.producer.sasl.mechanism": "PLAIN", "database.history.producer.security.protocol": "SASL_PLAINTEXT", "database.history.producer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"debezium\" password=\"V\";", "database.history.consumer.sasl.mechanism": "PLAIN", "database.history.consumer.security.protocol": "SASL_PLAINTEXT", "database.history.consumer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"debezium\" password=\"V\";", "include.schema.changes": "true", "include.query": "true", "snapshot.locking.mode": "none", "snapshot.mode": "schema_only", "topic.creation.default.replication.factor": "3", "topic.creation.default.partitions": "3", "topic.creation.default.compression.type": "lz4", "bigint.unsigned.handling.mode": "precise", "database.history.store.only.captured.tables.ddl": "false", "database.history.skip.unparseable.ddl": "true", "inconsistent.schema.handling.mode": "warn", "database.history.kafka.recovery.poll.interval.ms": "600000", "database.history.kafka.recovery.attempts": "3", "boolean.type": "io.debezium.connector.mysql.converters.TinyIntOneToBooleanConverter", "converters": "boolean" }

}

>

What is the captured database version and mode of depoyment?

(E.g. on-premises, with a specific cloud provider, etc.)

<mariadb>

What behaviour do you expect?

<Successfully captured data >

What behaviour do you see?

<In this way, the data can be successfully captured, but the debezium connector is in a normal state, but the problem of not re-pulling the historical data is caused. There is no obvious error in the log. There are about 60 debezium connectors in the cluster. The server resources are sufficient, and the startup parameters set by the kafka connector are as follows
KAFKA_HEAP_OPTS="Xms8G -Xmx8G -XX:NewSize=2G -XX:MaxNewSize=2G -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xloggc:/data/service/debezium/kafka_2. 13-2.7.0/logs/gc%t.log">

 

 

The phenomenon that exists under the feedback, can you give me some suggestions for this situation or my application situation, thank you very much