-
Bug
-
Resolution: Done
-
Undefined
-
None
-
False
-
-
False
-
-
-
Approved
[2674087217] Upstream Reporter: KodieGlosserIBM
Upstream issue status: Closed
Upstream description:
I think I found a potential race condition specifically here (context aware chunking): https://github.com/instructlab/sdg/pull/284 Basically if there is more than 1 knowledge document for git to clone, and it happens to do multiple clones with the same second it will generate the same output dir: document_output_dir = Path(output_dir) / f"documents-{date_suffix}" Which causes SDG to fail since the directory already exists on the git clone.
Generating data on a single knowledge document, things works just fine. Its when we get to multiple I am seeing failures.
Upstream URL: https://github.com/instructlab/sdg/issues/404
- links to
- mentioned on
(41 mentioned on)
