We should create an area where we can store a set of persistent repositories created by various versions of ModeShape. We can then test that each version of software can read/use all repositories created/used with all prior versions of ModeShape.
For example, ModeShape 3.5 should be able to read/use a repository created (and last used) with 3.0, with 3.1, 3.2, 3.3, and 3.4.
We can probably use a git repository under the ModeShape organization on GitHub, but we'll need to figure out:
- What tests do we want to execute on each persistent repository? Most likely at least opening and reading, and perhaps modifying. Also, we probably want to be able to create a backup of it, and we probably want to be able to restore repositories from backups created by previous versions. (These would likely indirectly test all the JSON and BSON reading and writing functionality, too, since the repository stores the documents in BSON form and the backups store the documents in JSON form.)
- Should we plan for separate test modules for each software version, like we do in the performance repository? This is probably the case, since we'll want to run all the tests at once (or at least profiles that run the tests for multiple versions in a single Maven run).
- What will the git structure be? Should the persistent repositories be stored separately from the test modules, or can they be stored within the 'src/test/resources' area?
- What content should be in each persistent repository? It probably needs to be varied enough to cover the possible document formats, but large enough to be useful yet not take too long to run. Should the content be generated, or should we create a notional repository with "real" data (e.g., real images, files, data, etc.) Is there a public data source we can easily adapt?