Implement TokenizeOperator in addition to the changes made by Zachary Heilbron.
Now, the compiler builds the bulk-load plan.
For other details, please refer to the corresponding Hyracks changes.
Change-Id: I646539d88ea2bdc6da4fbf2b6e9460a6189125ff
Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/79
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Young-Seok Kim <kisskys@gmail.com>
diff --git a/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.adm b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.adm
new file mode 100644
index 0000000..5a5cec0
--- /dev/null
+++ b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.adm
@@ -0,0 +1,10 @@
+{"dblpid": "books/acm/kim95/Blakeley95", "title": "OQL[C++] Extending C++ with an Object Query Capability.", "authors": "José A. Blakeley", "misc": "2002-01-03 69-88 Modern Database Systems db/books/collections/kim95.html#Blakeley95 1995" }
+{"dblpid": "books/acm/kim95/ChristodoulakisK95", "title": "Multimedia Information Systems Issues and Approaches.", "authors": "Stavros Christodoulakis Leonidas Koveos", "misc": "2002-01-03 318-337 1995 Modern Database Systems db/books/collections/kim95.html#ChristodoulakisK95" }
+{"dblpid": "books/acm/kim95/DittrichD95", "title": "Where Object-Oriented DBMSs Should Do Better A Critique Based on Early Experiences.", "authors": "Angelika Kotz Dittrich Klaus R. Dittrich", "misc": "2002-01-03 238-254 1995 Modern Database Systems db/books/collections/kim95.html#DittrichD95" }
+{"dblpid": "books/acm/kim95/Goodman95", "title": "An Object-Oriented DBMS War Story Developing a Genome Mapping Database in C++.", "authors": "Nathan Goodman", "misc": "2002-01-03 216-237 1995 Modern Database Systems db/books/collections/kim95.html#Goodman95" }
+{"dblpid": "books/acm/kim95/KelleyGKRG95", "title": "Schema Architecture of the UniSQL/M Multidatabase System", "authors": "William Kelley Sunit K. Gala Won Kim Tom C. Reyes Bruce Graham", "misc": "2004-03-08 Modern Database Systems books/acm/Kim95 621-648 1995 db/books/collections/kim95.html#KelleyGKRG95" }
+{"dblpid": "books/acm/kim95/Kim95", "title": "Introduction to Part 1 Next-Generation Database Technology.", "authors": "Won Kim", "misc": "2002-01-03 5-17 1995 Modern Database Systems db/books/collections/kim95.html#Kim95" }
+{"dblpid": "books/acm/kim95/Kim95b", "title": "Introduction to Part 2 Technology for Interoperating Legacy Databases.", "authors": "Won Kim", "misc": "2002-01-03 515-520 1995 Modern Database Systems db/books/collections/kim95.html#Kim95b" }
+{"dblpid": "books/acm/kim95/KimG95", "title": "Requirements for a Performance Benchmark for Object-Oriented Database Systems.", "authors": "Won Kim Jorge F. Garza", "misc": "2002-01-03 203-215 1995 Modern Database Systems db/books/collections/kim95.html#KimG95" }
+{"dblpid": "books/acm/kim95/Kowalski95", "title": "The POSC Solution to Managing E&P Data.", "authors": "Vincent J. Kowalski", "misc": "2002-01-03 281-301 1995 Modern Database Systems db/books/collections/kim95.html#Kowalski95" }
+{"dblpid": "books/acm/kim95/Lunt95", "title": "Authorization in Object-Oriented Databases.", "authors": "Teresa F. Lunt", "misc": "2002-01-03 130-145 1995 Modern Database Systems db/books/collections/kim95.html#Lunt95" }
diff --git a/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.csv b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.csv
new file mode 100644
index 0000000..d7a9491
--- /dev/null
+++ b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.csv
@@ -0,0 +1,20 @@
+Object SQL - A Language for the Design and Implementation of Object Databases.,Jurgen Annevelink Rafiul Ahad Amelia Carlson Daniel H. Fishman Michael L. Heytens William Kent
+OQL[C++] Extending C++ with an Object Query Capability.,José A. Blakeley
+Transaction Management in Multidatabase Systems.,Yuri Breitbart Hector Garcia-Molina Abraham Silberschatz
+Multimedia Information Systems Issues and Approaches.,Stavros Christodoulakis Leonidas Koveos
+Active Database Systems.,Umeshwar Dayal Eric N. Hanson Jennifer Widom
+Where Object-Oriented DBMSs Should Do Better A Critique Based on Early Experiences.,Test Person
+Distributed Databases.,Hector Garcia-Molina Meichun Hsu
+An Object-Oriented DBMS War Story Developing a Genome Mapping Database in C++.,Nathan Goodman
+Cooperative Transactions for Multiuser Environments.,Gail E. Kaiser
+Schema Architecture of the UniSQL/M Multidatabase System,William Kelley Sunit K. Gala Won Kim Tom C. Reyes Bruce Graham
+Physical Object Management.,Alfons Kemper Guido Moerkotte
+Introduction to Part 1 Next-Generation Database Technology.,Won Kim
+Object-Oriented Database Systems Promises Reality and Future.,Won Kim
+Introduction to Part 2 Technology for Interoperating Legacy Databases.,Won Kim
+On Resolving Schematic Heterogeneity in Multidatabase Systems.,Won Kim Injun Choi Sunit K. Gala Mark Scheevel
+Requirements for a Performance Benchmark for Object-Oriented Database Systems.,Won Kim Jorge F. Garza
+On View Support in Object-Oriented Databases Systems.,Won Kim William Kelley
+The POSC Solution to Managing E&P Data.,Vincent J. Kowalski
+C++ Bindings to an Object Database.,David Krieger Tim Andrews
+Authorization in Object-Oriented Databases.,Teresa F. Lunt
diff --git a/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.txt b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.txt
new file mode 100644
index 0000000..bb9d82a
--- /dev/null
+++ b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk.txt
@@ -0,0 +1,20 @@
+Object SQL - A Language for the Design and Implementation of Object Databases.:Jurgen Annevelink Rafiul Ahad Amelia Carlson Daniel H. Fishman Michael L. Heytens William Kent
+OQL[C++] Extending C++ with an Object Query Capability.:José A. Blakeley
+Transaction Management in Multidatabase Systems.:Yuri Breitbart Hector Garcia-Molina Abraham Silberschatz
+Multimedia Information Systems Issues and Approaches.:Stavros Christodoulakis Leonidas Koveos
+Active Database Systems.:Umeshwar Dayal Eric N. Hanson Jennifer Widom
+Where Object-Oriented DBMSs Should Do Better A Critique Based on Early Experiences.:Test Person
+Distributed Databases.:Hector Garcia-Molina Meichun Hsu
+An Object-Oriented DBMS War Story Developing a Genome Mapping Database in C++.:Nathan Goodman
+Cooperative Transactions for Multiuser Environments.:Gail E. Kaiser
+Schema Architecture of the UniSQL/M Multidatabase System:William Kelley Sunit K. Gala Won Kim Tom C. Reyes Bruce Graham
+Physical Object Management.:Alfons Kemper Guido Moerkotte
+Introduction to Part 1 Next-Generation Database Technology.:Won Kim
+Object-Oriented Database Systems Promises, Reality, and Future.:Won Kim
+Introduction to Part 2 Technology for Interoperating Legacy Databases.:Won Kim
+On Resolving Schematic Heterogeneity in Multidatabase Systems.:Won Kim Injun Choi Sunit K. Gala Mark Scheevel
+Requirements for a Performance Benchmark for Object-Oriented Database Systems.:Won Kim Jorge F. Garza
+On View Support in Object-Oriented Databases Systems.:Won Kim William Kelley
+The POSC Solution to Managing E&P Data.:Vincent J. Kowalski
+C++ Bindings to an Object Database.:David Krieger Tim Andrews
+Authorization in Object-Oriented Databases.:Teresa F. Lunt
diff --git a/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk_including_uuid.adm b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk_including_uuid.adm
new file mode 100644
index 0000000..174e408
--- /dev/null
+++ b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk_including_uuid.adm
@@ -0,0 +1,10 @@
+{"id": "89fb1567-70f8-4e59-87d9-ace64f73daf1", "dblpid": "books/acm/kim95/Blakeley95", "title": "OQL[C++] Extending C++ with an Object Query Capability.", "authors": "José A. Blakeley", "misc": "2002-01-03 69-88 Modern Database Systems db/books/collections/kim95.html#Blakeley95 1995" }
+{"id": "8cea25ab-55f8-467e-929d-94888f754832", "dblpid": "books/acm/kim95/ChristodoulakisK95", "title": "Multimedia Information Systems Issues and Approaches.", "authors": "Stavros Christodoulakis Leonidas Koveos", "misc": "2002-01-03 318-337 1995 Modern Database Systems db/books/collections/kim95.html#ChristodoulakisK95" }
+{"id": "8d59a28c-dfde-404b-a6a4-de2acefbcd88", "dblpid": "books/acm/kim95/DittrichD95", "title": "Where Object-Oriented DBMSs Should Do Better A Critique Based on Early Experiences.", "authors": "Angelika Kotz Dittrich Klaus R. Dittrich", "misc": "2002-01-03 238-254 1995 Modern Database Systems db/books/collections/kim95.html#DittrichD95" }
+{"id": "8d68c3e2-5101-479e-9dbb-5dc59d3d465c", "dblpid": "books/acm/kim95/Goodman95", "title": "An Object-Oriented DBMS War Story Developing a Genome Mapping Database in C++.", "authors": "Nathan Goodman", "misc": "2002-01-03 216-237 1995 Modern Database Systems db/books/collections/kim95.html#Goodman95" }
+{"id": "9171b105-e2e0-4ebd-b7ae-d1c2914b3233", "dblpid": "books/acm/kim95/KelleyGKRG95", "title": "Schema Architecture of the UniSQL/M Multidatabase System", "authors": "William Kelley Sunit K. Gala Won Kim Tom C. Reyes Bruce Graham", "misc": "2004-03-08 Modern Database Systems books/acm/Kim95 621-648 1995 db/books/collections/kim95.html#KelleyGKRG95" }
+{"id": "93e4d3ec-a6c9-4a71-a8a1-d8a827250c72", "dblpid": "books/acm/kim95/Kim95", "title": "Introduction to Part 1 Next-Generation Database Technology.", "authors": "Won Kim", "misc": "2002-01-03 5-17 1995 Modern Database Systems db/books/collections/kim95.html#Kim95" }
+{"id": "9816c851-7232-498b-9eb3-5fed80f46307", "dblpid": "books/acm/kim95/Kim95b", "title": "Introduction to Part 2 Technology for Interoperating Legacy Databases.", "authors": "Won Kim", "misc": "2002-01-03 515-520 1995 Modern Database Systems db/books/collections/kim95.html#Kim95b" }
+{"id": "00f3969c-8427-47cd-9bd3-201cd931af9c", "dblpid": "books/acm/kim95/KimG95", "title": "Requirements for a Performance Benchmark for Object-Oriented Database Systems.", "authors": "Won Kim Jorge F. Garza", "misc": "2002-01-03 203-215 1995 Modern Database Systems db/books/collections/kim95.html#KimG95" }
+{"id": "01d75219-1d25-4a52-a651-d81dca2f9330", "dblpid": "books/acm/kim95/Kowalski95", "title": "The POSC Solution to Managing E&P Data.", "authors": "Vincent J. Kowalski", "misc": "2002-01-03 281-301 1995 Modern Database Systems db/books/collections/kim95.html#Kowalski95" }
+{"id": "02a199ca-bf58-412e-bd9f-60a0c975a8ac", "dblpid": "books/acm/kim95/Lunt95", "title": "Authorization in Object-Oriented Databases.", "authors": "Teresa F. Lunt", "misc": "2002-01-03 130-145 1995 Modern Database Systems db/books/collections/kim95.html#Lunt95" }
diff --git a/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk_including_uuid.csv b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk_including_uuid.csv
new file mode 100644
index 0000000..3ef4fdd
--- /dev/null
+++ b/asterix-app/data/pub-small/dblp-small-id-autogenerated-pk_including_uuid.csv
@@ -0,0 +1,20 @@
+00f3969c-8427-47cd-9bd3-201cd931af9c,Object SQL - A Language for the Design and Implementation of Object Databases.,Jurgen Annevelink Rafiul Ahad Amelia Carlson Daniel H. Fishman Michael L. Heytens William Kent
+01d75219-1d25-4a52-a651-d81dca2f9330,OQL[C++] Extending C++ with an Object Query Capability.,José A. Blakeley
+02a199ca-bf58-412e-bd9f-60a0c975a8ac,Transaction Management in Multidatabase Systems.,Yuri Breitbart Hector Garcia-Molina Abraham Silberschatz
+03465c48-9215-47d1-9dcc-26b919db3879,Multimedia Information Systems Issues and Approaches.,Stavros Christodoulakis Leonidas Koveos
+044caf96-82c6-49d9-9369-b0aaa7d79e6d,Active Database Systems.,Umeshwar Dayal Eric N. Hanson Jennifer Widom
+05b7846c-a59f-4de6-a1fc-2d5bb74118c8,Where Object-Oriented DBMSs Should Do Better A Critique Based on Early Experiences.,Test Person
+06f68dbc-3e76-41cb-a27b-dc63464594ae,Distributed Databases.,Hector Garcia-Molina Meichun Hsu
+0b500ae8-f539-4f27-91dc-ba7974f6b4ca,An Object-Oriented DBMS War Story Developing a Genome Mapping Database in C++.,Nathan Goodman
+0b9f1f30-f1f1-4c4b-a87e-b92e811bfabf,Cooperative Transactions for Multiuser Environments.,Gail E. Kaiser
+0c1cc0d4-af8e-410e-ac81-c37bf99d28b4,Schema Architecture of the UniSQL/M Multidatabase System,William Kelley Sunit K. Gala Won Kim Tom C. Reyes Bruce Graham
+1183e948-2cb0-4155-93fb-6aeeb652b4b2,Physical Object Management.,Alfons Kemper Guido Moerkotte
+11ce8c22-f7cb-4c1c-9b97-5e2119c1c01d,Introduction to Part 1 Next-Generation Database Technology.,Won Kim
+11e1f9f0-41cd-46fd-b8af-5791ecf6e2dd,Object-Oriented Database Systems Promises Reality and Future.,Won Kim
+16231f8c-dcd1-49b6-b1e7-0715b1a46388,Introduction to Part 2 Technology for Interoperating Legacy Databases.,Won Kim
+20a4b67b-496c-4bd9-af81-2de94ca79fe4,On Resolving Schematic Heterogeneity in Multidatabase Systems.,Won Kim Injun Choi Sunit K. Gala Mark Scheevel
+2623fde7-dd96-4100-93c6-8cf0f421dee1,Requirements for a Performance Benchmark for Object-Oriented Database Systems.,Won Kim Jorge F. Garza
+27712335-9540-4b08-b423-69887e4614cc,On View Support in Object-Oriented Databases Systems.,Won Kim William Kelley
+2ad34903-838a-473d-a35f-b8b1b63f688c,The POSC Solution to Managing E&P Data.,Vincent J. Kowalski
+2c045409-db9e-408b-a2dc-b95ed5ec9702,C++ Bindings to an Object Database.,David Krieger Tim Andrews
+2c824d67-f302-4afe-84df-cdc6841723ba,Authorization in Object-Oriented Databases.,Teresa F. Lunt