Uploaded image for project: 'ModeShape'
  1. ModeShape
  2. MODE-2643

Git Connector throws an Exception on large files

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 5.3.0.Final
    • 5.2.0.Final
    • Federation
    • None
    • Hide

      Wildfly 10.1.0 with modeshape 5.2. The setup is done in the standalone.xml file. Modeshape uses both file based storage for persistence and binary. (I have tested it with database storage also and has the same result).

      Checkout a git repository with a big file in it. In my case I used a file from about 126MB. Add the git repository through the git connector to the Modeshape repository.

      Read the big file from Git through the Modeshape repository.

      Show
      Wildfly 10.1.0 with modeshape 5.2. The setup is done in the standalone.xml file. Modeshape uses both file based storage for persistence and binary. (I have tested it with database storage also and has the same result). Checkout a git repository with a big file in it. In my case I used a file from about 126MB. Add the git repository through the git connector to the Modeshape repository. Read the big file from Git through the Modeshape repository.

      Reading files from git through Modeshape with help from the git connector is working fine for small files but for big files this is failing with the following error:

      Caused by: org.modeshape.persistence.file.FileProviderException: An active transaction is required, but wasn't detected
  at org.modeshape.persistence.file.FileDb.transactionalContent(FileDb.java:251)
  at org.modeshape.persistence.file.FileDb.editContent(FileDb.java:131)
  at org.modeshape.jcr.cache.document.LocalDocumentStore.edit(LocalDocumentStore.java:178)
  at org.modeshape.jcr.federation.FederatedDocumentStore.edit(FederatedDocumentStore.java:348)
  at org.modeshape.jcr.cache.document.DocumentTranslator.incrementBinaryReferenceCount(DocumentTranslator.java:1249)
  at org.modeshape.jcr.cache.document.DocumentTranslator.valueToDocument(DocumentTranslator.java:1222)
  at org.modeshape.jcr.cache.document.DocumentTranslator.setProperty(DocumentTranslator.java:425)
  at org.modeshape.jcr.federation.FederatedDocumentWriter.addProperty(FederatedDocumentWriter.java:131)
  at org.modeshape.connector.git.GitTree.addInformationForPath(GitTree.java:239)
  at org.modeshape.connector.git.GitTree.execute(GitTree.java:128)
  at org.modeshape.connector.git.GitConnector.getDocumentById(GitConnector.java:233)

      What I have seen in the source from the git connector is that it handles huge files differently from small files. Line 230 found here: https://github.com/ModeShape/modeshape/blob/master/connectors/modeshape-connector-git/src/main/java/org/modeshape/connector/git/GitTree.java#L230
      I haven't found out what Git considers a huge file and what not. So far I have had it working with a 46MB file but not with a 126MB file.

      The method values.binaryFrom(fileLoader.openStream()); attempts to create & store a ModeShape binary value for that particular Git value without an existing transaction (the entire flow is simply reading stuff from Git).

              hchiorean Horia Chiorean (Inactive)
              pverboom Patrick Verboom (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: