Uploaded image for project: 'ModeShape'
  1. ModeShape
  2. MODE-2643

Git Connector throws an Exception on large files

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 5.3.0.Final
    • 5.2.0.Final
    • Federation
    • None
    • Hide

      Wildfly 10.1.0 with modeshape 5.2. The setup is done in the standalone.xml file. Modeshape uses both file based storage for persistence and binary. (I have tested it with database storage also and has the same result).

      Checkout a git repository with a big file in it. In my case I used a file from about 126MB. Add the git repository through the git connector to the Modeshape repository.

      Read the big file from Git through the Modeshape repository.

      Show
      Wildfly 10.1.0 with modeshape 5.2. The setup is done in the standalone.xml file. Modeshape uses both file based storage for persistence and binary. (I have tested it with database storage also and has the same result). Checkout a git repository with a big file in it. In my case I used a file from about 126MB. Add the git repository through the git connector to the Modeshape repository. Read the big file from Git through the Modeshape repository.

    Description

      Reading files from git through Modeshape with help from the git connector is working fine for small files but for big files this is failing with the following error:

      Caused by: org.modeshape.persistence.file.FileProviderException: An active transaction is required, but wasn't detected
  at org.modeshape.persistence.file.FileDb.transactionalContent(FileDb.java:251)
  at org.modeshape.persistence.file.FileDb.editContent(FileDb.java:131)
  at org.modeshape.jcr.cache.document.LocalDocumentStore.edit(LocalDocumentStore.java:178)
  at org.modeshape.jcr.federation.FederatedDocumentStore.edit(FederatedDocumentStore.java:348)
  at org.modeshape.jcr.cache.document.DocumentTranslator.incrementBinaryReferenceCount(DocumentTranslator.java:1249)
  at org.modeshape.jcr.cache.document.DocumentTranslator.valueToDocument(DocumentTranslator.java:1222)
  at org.modeshape.jcr.cache.document.DocumentTranslator.setProperty(DocumentTranslator.java:425)
  at org.modeshape.jcr.federation.FederatedDocumentWriter.addProperty(FederatedDocumentWriter.java:131)
  at org.modeshape.connector.git.GitTree.addInformationForPath(GitTree.java:239)
  at org.modeshape.connector.git.GitTree.execute(GitTree.java:128)
  at org.modeshape.connector.git.GitConnector.getDocumentById(GitConnector.java:233)

      What I have seen in the source from the git connector is that it handles huge files differently from small files. Line 230 found here: https://github.com/ModeShape/modeshape/blob/master/connectors/modeshape-connector-git/src/main/java/org/modeshape/connector/git/GitTree.java#L230
      I haven't found out what Git considers a huge file and what not. So far I have had it working with a 46MB file but not with a 126MB file.

      The method values.binaryFrom(fileLoader.openStream()); attempts to create & store a ModeShape binary value for that particular Git value without an existing transaction (the entire flow is simply reading stuff from Git).

      Attachments

        Activity

          People

            hchiorean Horia Chiorean (Inactive)
            pverboom Patrick Verboom (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: