| commit | 0622e8e8febfe508a17dcf50e3044e8d87f1f35d | [log] [tgz] |
|---|---|---|
| author | Yingyi Bu <buyingyi@gmail.com> | Tue May 27 17:46:41 2014 -0700 |
| committer | Ian Maxon <imaxon@uci.edu> | Wed May 28 16:58:39 2014 -0800 |
| tree | 74dd10b80238e4e2ead14b691db81a4713c457be | |
| parent | f53df34d25d8c23e6d2be578df17c5373d85748a [diff] |
Several major changes in hyracks: -- reduced CC/NC communications for reporting partition request and availability; partition request/availability are only reported for the case of send-side materialized (without pipelining) policies in case of task re-attempt. -- changed buffer cache to dynamically allocate memory based on needs instead of pre-allocating -- changed each network channel to lazily allocate memory based on needs, and changed materialized connectors to lazily allocate files based on needs -- changed several major CCNCCFunctions to use non-java serde -- added a sort-based group-by operator which pushes group-by aggregations into an external sort -- make external sort a stable sort 1,3,and 4 is to reduce the job overhead. 2 is to reduce the unecessary NC resource consumptions such as memory and files. 5 and 6 are improvements to runtime operators. One change in algebricks: -- implemented a rule to push group-by aggregation into sort, i.e., using the sort-based gby operator Several important changes in pregelix: -- remove static states in vertex -- direct check halt bit without deserialization -- optimize the sort algorithm by packing yet-another 2-byte normalized key into the tPointers array Change-Id: Id696f9a9f1647b4a025b8b33d20b3a89127c60d6 Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/35 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Till Westmann <westmann@gmail.com>