Enqueuing full indexation of accounts results in slower enqueuing compared to other indices. This is because the Account index scope has includes of users and cinstances.
rake sphinx:enqueue
This results in the following queries as seen in the log
Account Load (0.6ms) SELECT `accounts`.`id` FROM `accounts` WHERE (`accounts `.`master` = FALSE OR `accounts`.`master` IS NULL) AND `accounts`.`state` != 'sc heduled_for_deletion' AND `accounts`.`provider_account_id` NOT IN (SELECT `accou nts`.`id` FROM `accounts` WHERE `accounts`.`state` = 'scheduled_for_deletion') O RDER BY `accounts`.`id` ASC LIMIT 1000 ↳ lib/tasks/sphinx.rake:24 User Load (0.7ms) SELECT `users`.* FROM `users` WHERE `users`.`account_id` IN (2, 3, 4, 5, 6, 8, 9, 11, 12, 14) ↳ lib/tasks/sphinx.rake:24 Cinstance Load (0.4ms) SELECT `cinstances`.* FROM `cinstances` WHERE `cinstan ces`.`type` IN ('Cinstance') AND `cinstances`.`user_account_id` IN (2, 3, 4, 5, 6, 8, 9, 11, 12, 14) ↳ lib/tasks/sphinx.rake:24
This is necessary for normal operation to avoid N+1 queries. But it unnecessarily slows down enqueued indexation because there we only need account ID and nothing else.
lib/tasks/sphinx.rake:
scope.select(:id).find_in_batches(batch_size: 1000) do |batch|
batch.each do |record|
SphinxIndexationWorker.perform_later(record.class, record.id)
end
progress.call(increment: batch.size)
end
So we need to somehow avoid the extra two queries during enqueue. Apparently Rails is not smart enough to remove them automatically when `select(:id)` is used.
- is related to
-
THREESCALE-8586 improve querying for non-master account and sphinx indexing
-
- Closed
-
- mentioned on
