-
Enhancement
-
Resolution: Done
-
Major
-
2.5.0.Final
As Randall has noted, it might be very beneficial if the disk connector provided a way for large property values to be stored in a separate area. This would keep the node size on disk smaller, allowing for quicker reads and writes of the node itself and allow large values to be reused between nodes when possible. In this regard, the feature would be very similar to the JpaSource's large value support.
However, since these large values would be stored on disk, they could also leverage the existing FileSystemBinary optimization that lazily loads binary values that are stored in a file. This could greatly reduce the load time for nodes with very large binary properties and, in some circumstances, reduce the memory consumption of the connector as a whole as the entire binary property will no longer need to be loaded into memory when the node is retrieved. Instead, it can be read in as a FileInputStream with only chunks of the value in memory at any particular time.