Immutable memtables

[justinethier]

When a memtable is full, convert it to an immutable memtable and keep it in memory until it can be flushed to disk.

[justinethier]

See:

func (tree *LsmTree) walJob()

Would need to store a list of immutable memtables in the LSM tree instance as well as the "primary" memtable. Or is that just the first one in the list?

[justinethier]

The advantage of this strategy is that we don't have to hold a lock while the immutable memtable is flushed to disk. Basically the sequence of operations is:

• Create immutable memtable (requires lock)
• Fire/wakeup an async job to flush this memtable to disk (create new SST file)
• After flush, lock the tree, move in the new SST file, remove the immutable memtable

One complication is that right now we roll the WAL once the memtable is flushed. We won't be able to do that with this new approach. One solution is to keep 2 WAL's at once. Stop writing to old WAL when memtable becomes immutable, and create new WAL. Then when we remove the immutable memtable we can safely delete the old WAL as that data has now been committed to disk in the new SST file.