tree b1bc55ef61975c7d3e1fa434df7135970b9cd006
parent 93d3c91ae36b0440c6f0f79b16a63a3837b1bb56
author luochen <cluo8@uci.edu> 1587830937 -0700
committer Luo Chen <cluo8@uci.edu> 1588026143 +0000

[ASTERIXDB-2715][STO] Dynamic Memory Component Architecture

- user model changes: yes
- storage format changes: no
- interface changes: yes

Details:
- Introduce a dynamic memory component architecture that uses a global
virtual buffer cache to manage the write memory for all LSM-trees.
- When the overall write memory is nearly full, we flush a dataset
partition at a time using a round-robin way. Additionally, we allow
users to configure the maximum size of filtered memory components
to provide better pruning capability.
- Clean up legacy code for statically allocating write memory to each
dataset.
- Remove the following parameters:
  storage.metadata.memorycomponent.numpages
  storage.max.active.writable.datasets
- Add the following parameters:
  storage.memorycomponent.flush.threshold (default: 0.9)
  storage.filtered.memorycomponent.max.size (default 0)

Change-Id: Ia6a0f4de020acd7af89ef630322526c4be5076e0
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5824
Reviewed-by: Luo Chen <cluo8@uci.edu>
Reviewed-by: Murtadha Hubail <mhubail@apache.org>
Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>