The ThreadDB database library provides a multithreaded, file mapped container manager. It allows to extend the capabilites of the C++ standard containers to efficiently manage large volumes of application data way beyond of the real system memory.
While memory mapping provides a powerful means of manging large volumes of data in an efficient way it has certain restrictions to be considered. One is the limited support for dynamically increasing files. There are only few really portable ways to achieve this by the cost of reduced throughput performance. Another is the limitation of the virtual address space on 32bit Systems. Here the maximum filesize may not exceed 2Gbyte.
ThreadDB overcomes these limitations by loading only those portions of data in the process address space really required at a time.
