BergDB is a Java/.NET database library designed to make it easy and efficient to store data without leaving the comfort of our beloved source code. I created BergDB for us developers who prefer to focus on our specific task, rather then spend time on database issues.
BergDB is open source; it is released under the AGLP license. The complete license information is included in the latest release. If you require another license, please contact me. Also feel free to contact me with feedback, bug reports and anything related to robust database technology.
BergDB is a simple, secure, fast, unobtrusive database library. It is designed to reduce development time, to provide high transaction throughput and to keep data safe.
1. General features
Open Source. BergDB is released under the AGPL license.
In-process. BergDB is designed for application databases and runs in the same process as the application code for highest possible performance and for reduced complexity.
Widely cross-platform. The database library is available to any JVM or .NET language. It is tested with Java and C# and should work with: Scala, Clojure, Groovy, JRuby, Jython, Rhino, F#, VB.NET and more. BergDB runs on any major operating system including Linux, Mac OS, Android and Windows. It runs on hardware ranging from mobile phones to high-end servers. The on-disk storage format is identical for all platforms and therefore interoperable.
Scalable. BergDB is suitable for applications storing small amounts of data, say 1 kB. Yet, it scales to store terabytes of data in billions of entries.
Time-key-value data model. Data is stored as time-key-value entries. A key is a string, much like a filename in a file system. A value is an object, a string or a raw byte array. An entry also includes the time when it was stored. A value can be retrieved by a key and a lookup time, or by iterating the entries in increasing or decreasing key order starting at a given key and lookup time. Some would call the data model a document store or an object database. The data model is a persistent data structure. All states, not just the last one, are stored and available to query simultaneously.
Schemaless. Values stored in the database are raw byte arrays, Unicode strings, or objects (could also be called "documents"). Objects have an internal structure that is visible to the database (for secondary indexing), but each object have an independent set of named and typed values. No schema is enforced.
Secondary indexes. Besides the primary index, BergDB supports secondary indexing of objects based on object field values. Indexes can be added and removed with no locks and no interruptions of ongoing operations. There are basically three index types: integer, floating point value, and string. Strings can be indexed using a comprehensive set of collations (string orderings) provided by ICU (International Components for Unicode). Indexes can be used with historic queries as well as when querying the latest state. Compound indexes are supported.
BergDB is easy to use even when advanced features are needed.
Code only. The developer use ordinary Java (or any other .NET/JVM language) code to store and retrieve data. There is no query language, no special annotations, no interfaces to implement, no XML settings - just ordinary code.
Simple API. The database runs embedded in your application process for highest possible performance. The database is a simple library with Javadoc API documentation. A Java / .NET object can be stored with a single line of code.
Simple transactions. Atomic transactions are implemented directly in your ordinary programming language by implementing a simple interface.
Zero database administration. Database administration is never needed and there are few settings. There is no need to spend time optimizing settings to reach high performance.
Low-level. BergDB is a simple, low-level database library. It does one thing only: store data. It does not provide access control, ad hoc queries, SQL support, or automatic sharding.
Testability. The database has been designed with testability in mind.
Empty, in-memory test databases can be created quickly and run in your ordinary
automated test code. The time of transactions can be simulated with
a special test clock (
Your data is stored securely with BergDB.
Always consistent. Changes to the database state is always made using atomic transactions with the highest isolation level: serializable. BergDB guarantees that there will never be phantom reads, non-repeatable reads, or dirty reads.
Access to history. All states of the database are available to query simultaneously, not just the last state. Values are not removed or overwritten, they are always kept during the retain period (which is infinite by default). Unlike other databases, there is no need for a special restore-to-point-in-time procedure. All previous states are available simultaneously; just set the lookup time of the database connection and all following queries will query at that time.
Crash proof. BergDB never really crashes even if the process suddenly halts or the power is cut to the computer. The database will still startup the ordinary way. There is never any need for a special recovery procedure.
Encryption. The data may be stored encrypted. Also, every read from storage checks the integrity of the data with a checksum. This is done to detect storage bit errors.
Durability control. Different transactions may require different durability guarantees. With BergDB, it is possible to specify custom durability rules such as: a large payment should be considered durable when the corresponding transaction has been written to the main storage and to at least one remote location. This feature is important for high-value financial transactions.
Replication. The BergDB data is easy to replicate, possibly to a remote location. Replicas can be added and removed while the database is running with no disruption of existing operations.
Live backups. Backups can easily be made while the database is running. Due to the append-only approach of data storage, a complete backup of the database can be made by simply copying the database files; even while it is running.
Transaction throughput. BergDB is designed to be fast for both traditional rotating disk drives and for solid state drives. Rotating disk drives have a seek time of around 10 ms. The way to achieve high throughput of ACID transactions (over 1000 transaction/s) to rotating drives is to store the data append-only. That is, all data is added to the end of the storage. Existing bytes already written are never overwritten. This approach obviously reduces the number of disk seeks and allows BergDB to reach ACID transaction throughputs of 10,000 or even 100,000 transactions per second.
Fast startup. BergDB has a unique design that allows it to have a
short and constant startup time (O(1) in big O notation).
This is important as your data grows. On my laptop (Toshiba Z930), the
typical startup time is around 1.5 seconds. This is for a database of
1 million entries as well as one with 10 million or
100 million entries. The startup time is the same.
I used the class
bergdb.test.big.Huge to test this.
Do not believe me, test on your computer instead.
High in-memory performance. When the active data fits in RAM memory, the BergDB database has a performance that is on par with in-memory databases that are optimized for the case when all data fits in RAM. In a database with 1 million entries, it takes around 1 microsecond in average to get an entry given its primary key on my computer. That is, 1 million random lookups per second for a 1M database. To reproduce: use a computer as fast as mine (Toshiba Z930) and run the class bergdb.test.Get (change the code to create 1 million entries).
Performant concurrency. BergDB uses software transactional memory (STM) to achieve high transaction throughput with minimum blocking. There are no traditional locks at all; BergDB uses optimistic concurrency control and a persistent data structure. Readers are never blocked by writers or other readers. Readers always see a consistent state of the database without degrading the performance of other readers and writers.
BergDB is a small library that tries to get out of your way.
Single jar. For a Java project, BergDB is deployed as a single
jar file. There are no other dependencies except for a Java VM,
version 1.5 or later. The
bergdb.jar file of the last release
is 3 MB. If advanced string collation for secondary indexing is not required,
the much smaller jar file
bergdb-no-icu.jar (0.5 MB) can be used.
Low storage overhead. An empty database (release 2013.2.130707) is stored in one file and is 224 bytes in size. A database with one entry: "/key" -> "value" is stored in a file of size 328 bytes.
No CPU usage when database is idle. BergDB consumes CPU cycles only when it is actually used. There are no long-running background threads that perform administrative tasks. All work is done when the database library is called from ordinary application threads.
Small API. The BergDB Pub API - the standard API - is small and is designed to be easy to use. It is designed to be unobtrusive to your brain.
No global state. BergDB does not use any global state. Yes, for real. It does not use custom classloaders or set UncaughtExceptionHandler:s. It does not use System.out/in/err. There is no use of global logging, environment variables, command line arguments, files with settings. BergDB is controlled entirely through ordinary code. Multiple instances of BergDB can run in the same process without problems.