Gitiles
Code Review Sign In
asterix-gerrit.ics.uci.edu / incubator-asterixdb-site / 5f34114603bb17e7c3493d060d88bad69e418cda / . / content / docs / 0.9.1 / ncservice.html
blob: e2465e64ec59ef2793048bd3b6437337562db46b [file] [log] [blame]
Ian Maxon100cb802017-04-24 18:48:07 -0700[diff] [blame]1<!DOCTYPE html>
2<!--
3 | Generated by Apache Maven Doxia at 2017-04-24
4 | Rendered using Apache Maven Fluido Skin 1.3.0
5-->
6<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
7 <head>
8 <meta charset="UTF-8" />
9 <meta name="viewport" content="width=device-width, initial-scale=1.0" />
10 <meta name="Date-Revision-yyyymmdd" content="20170424" />
11 <meta http-equiv="Content-Language" content="en" />
12 <title>AsterixDB &#x2013; <a id="toc">Table of Contents</a></title>
13 <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
14 <link rel="stylesheet" href="./css/site.css" />
15 <link rel="stylesheet" href="./css/print.css" media="print" />
16
17
18 <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script>
19
20
21
Ian Maxon100cb802017-04-24 18:48:07 -0700[diff] [blame]22
Ian Maxon100cb802017-04-24 18:48:07 -0700[diff] [blame]23
24 </head>
25 <body class="topBarDisabled">
26
27
28
29
30 <div class="container-fluid">
31 <div id="banner">
32 <div class="pull-left">
33 <a href="./" id="bannerLeft">
34 <img src="images/asterixlogo.png" alt="AsterixDB"/>
35 </a>
36 </div>
37 <div class="pull-right"> </div>
38 <div class="clear"><hr/></div>
39 </div>
40
41 <div id="breadcrumbs">
42 <ul class="breadcrumb">
43
44
45 <li id="publishDate">Last Published: 2017-04-24</li>
46
47
48
49 <li id="projectVersion" class="pull-right">Version: 0.9.1</li>
50
51 <li class="divider pull-right">|</li>
52
53 <li class="pull-right"> <a href="index.html" title="Documentation Home">
54 Documentation Home</a>
55 </li>
56
57 </ul>
58 </div>
59
60
61 <div class="row-fluid">
62 <div id="leftColumn" class="span3">
63 <div class="well sidebar-nav">
64
65
66 <ul class="nav nav-list">
67 <li class="nav-header">Get Started - Installation</li>
68
69 <li class="active">
70
71 <a href="#"><i class="none"></i>Option 1: using NCService</a>
72 </li>
73
74 <li>
75
76 <a href="ansible.html" title="Option 2: using Ansible">
77 <i class="none"></i>
78 Option 2: using Ansible</a>
79 </li>
80
81 <li>
82
83 <a href="aws.html" title="Option 3: using Amazon Web Services">
84 <i class="none"></i>
85 Option 3: using Amazon Web Services</a>
86 </li>
87
88 <li>
89
90 <a href="yarn.html" title="Option 4: using YARN">
91 <i class="none"></i>
92 Option 4: using YARN</a>
93 </li>
94
95 <li>
96
97 <a href="install.html" title="Option 5: using Managix (deprecated)">
98 <i class="none"></i>
99 Option 5: using Managix (deprecated)</a>
100 </li>
101 <li class="nav-header">AsterixDB Primer</li>
102
103 <li>
104
105 <a href="sqlpp/primer-sqlpp.html" title="Option 1: using SQL++">
106 <i class="none"></i>
107 Option 1: using SQL++</a>
108 </li>
109
110 <li>
111
112 <a href="aql/primer.html" title="Option 2: using AQL">
113 <i class="none"></i>
114 Option 2: using AQL</a>
115 </li>
116 <li class="nav-header">Data Model</li>
117
118 <li>
119
120 <a href="datamodel.html" title="The Asterix Data Model">
121 <i class="none"></i>
122 The Asterix Data Model</a>
123 </li>
124 <li class="nav-header">Queries - SQL++</li>
125
126 <li>
127
128 <a href="sqlpp/manual.html" title="The SQL++ Query Language">
129 <i class="none"></i>
130 The SQL++ Query Language</a>
131 </li>
132
133 <li>
134
135 <a href="sqlpp/builtins.html" title="Builtin Functions">
136 <i class="none"></i>
137 Builtin Functions</a>
138 </li>
139 <li class="nav-header">Queries - AQL</li>
140
141 <li>
142
143 <a href="aql/manual.html" title="The Asterix Query Language (AQL)">
144 <i class="none"></i>
145 The Asterix Query Language (AQL)</a>
146 </li>
147
148 <li>
149
150 <a href="aql/builtins.html" title="Builtin Functions">
151 <i class="none"></i>
152 Builtin Functions</a>
153 </li>
154 <li class="nav-header">API/SDK</li>
155
156 <li>
157
158 <a href="api.html" title="HTTP API">
159 <i class="none"></i>
160 HTTP API</a>
161 </li>
162
163 <li>
164
165 <a href="csv.html" title="CSV Output">
166 <i class="none"></i>
167 CSV Output</a>
168 </li>
169 <li class="nav-header">Advanced Features</li>
170
171 <li>
172
173 <a href="aql/fulltext.html" title="Support of Full-text Queries">
174 <i class="none"></i>
175 Support of Full-text Queries</a>
176 </li>
177
178 <li>
179
180 <a href="aql/externaldata.html" title="Accessing External Data">
181 <i class="none"></i>
182 Accessing External Data</a>
183 </li>
184
185 <li>
186
187 <a href="feeds/tutorial.html" title="Support for Data Ingestion">
188 <i class="none"></i>
189 Support for Data Ingestion</a>
190 </li>
191
192 <li>
193
194 <a href="udf.html" title="User Defined Functions">
195 <i class="none"></i>
196 User Defined Functions</a>
197 </li>
198
199 <li>
200
201 <a href="aql/filters.html" title="Filter-Based LSM Index Acceleration">
202 <i class="none"></i>
203 Filter-Based LSM Index Acceleration</a>
204 </li>
205
206 <li>
207
208 <a href="aql/similarity.html" title="Support of Similarity Queries">
209 <i class="none"></i>
210 Support of Similarity Queries</a>
211 </li>
212 </ul>
213
214
215
216 <hr class="divider" />
217
218 <div id="poweredBy">
219 <div class="clear"></div>
220 <div class="clear"></div>
221 <div class="clear"></div>
222 <a href="./" title="AsterixDB" class="builtBy">
223 <img class="builtBy" alt="AsterixDB" src="images/asterixlogo.png" />
224 </a>
225 </div>
226 </div>
227 </div>
228
229
230 <div id="bodyColumn" class="span9" >
231
232 <!-- ! Licensed to the Apache Software Foundation (ASF) under one
233 ! or more contributor license agreements. See the NOTICE file
234 ! distributed with this work for additional information
235 ! regarding copyright ownership. The ASF licenses this file
236 ! to you under the Apache License, Version 2.0 (the
237 ! "License"); you may not use this file except in compliance
238 ! with the License. You may obtain a copy of the License at
239 !
240 ! http://www.apache.org/licenses/LICENSE-2.0
241 !
242 ! Unless required by applicable law or agreed to in writing,
243 ! software distributed under the License is distributed on an
244 ! "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
245 ! KIND, either express or implied. See the License for the
246 ! specific language governing permissions and limitations
247 ! under the License.
248 ! --><div class="section">
249<h2><a name="Table_of_Contents"></a><a name="toc" id="toc">Table of Contents</a></h2>
250
251<ul>
252
253<li><a href="#quickstart">Quick Start</a></li>
254
255<li><a href="#Small_cluster">Starting a small single-machine cluster using the NCService</a></li>
256
257<li><a href="#Multi_machine">Deploying AsterixDB via NCService in a multi-machine setup</a></li>
258
259<li><a href="#Parameters">Available Configuration Parameters</a></li>
260</ul>
261<h1><a name="quickstart" id="quickstart">Quick Start</a></h1>
262<p>The fastest way to get set up with a single-machine sample instance of AsterixDB is to use the included sample helper scripts. To do so, in the extracted <tt>asterix-server</tt> directory, navigate to <tt>opt/local/bin/</tt></p>
263
264<div class="source">
265<div class="source">
266<pre>user@localhost:~/
267$cd asterix-server/
268user@localhost:~/asterix-server
269$cd opt/local/bin
270</pre></div></div>
271<p>This folder should contain 4 scripts, two pairs of <tt>.sh</tt> and <tt>.bat</tt> files respectively. <tt>start-sample-cluster.sh</tt> will simply start a basic sample cluster using the configuration files located in <tt>opt/local/conf/</tt>.</p>
272
273<div class="source">
274<div class="source">
275<pre>user@localhost:~/a/o/l/bin
276$./start-sample-cluster.sh
277CLUSTERDIR=/home/user/asterix-server/opt/local
278INSTALLDIR=/home/user/asterix-server
279LOGSDIR=/home/user/asterix-server/samples/opt/logs
280
281INFO: Starting sample cluster...
282INFO: Waiting up to 30 seconds for cluster 127.0.0.1:19002 to be available.
283INFO: Cluster started and is ACTIVE.
284user@localhost:~/a/o/l/bin
285$
286</pre></div></div>
287<p>Now, there should be a running AsterixDB cluster on the machine. To go to the Web Interface, visit <a class="externalLink" href="http://localhost:19001">http://localhost:19001</a></p>
288
289<div class="source">
290
291<div class="source">
292<pre>
293<img src="images/asterixdb_interface.png" alt="The AsterixDB Web Interface" />
294<i>Fig. 1</i>: The AsterixDB Web Interface
295</pre></div>
296</div>
297<h1><a name="Small_cluster" id="Small_cluster">Starting a small single-machine cluster using NCService</a></h1>
298<p>The above cluster was started using a script, but below is a description in detail of how precisely this was achieved. The config files here are analagous to the ones within <tt>samples/local/conf</tt>.</p>
299<p>When running a cluster using the <tt>NCService</tt> there are 3 different kinds of processes involved:</p>
300
301<ul>
302
303<li><tt>NCDriver</tt>, also known as the Node Controller or NC for short. This is the process that does the actual work of queries and data management within the AsterixDB cluster</li>
304
305<li><tt>NCService</tt> configures and starts the <tt>NCDriver</tt> process. It is a simple daemon whose only purpose is to wait for the <tt>CCDriver</tt> process to call upon it to initiate cluster bootup.</li>
306
307<li><tt>CCDriver</tt>, which is the Cluster Controller process, also known as the CC. This process manages the distribution of tasks to all NCs, as well as providing the parameters of each NC to the NCService upon cluster startup. It also hosts the Web interface and query compiler and optimizer.</li>
308</ul>
309<p>The cluster startup follows a particular sequence, which is as follows:</p>
310
311<ol style="list-style-type: decimal">
312
313<li>Each host on which an NC is desired and is mentioned in the configuration, the <tt>NCService</tt> daemon is started first. It will listen and wait for the CC to contact it.</li>
314
315<li>The one host on which the CC is to be placed is started with an appropriate configuration file.</li>
316
317<li>The CC contacts all <tt>NCService</tt> daemons and the <tt>NCService</tt> subsequently starts and <tt>NCDriver</tt> process with the configration supplied by the <tt>CC</tt></li>
318
319<li>Each <tt>NCDriver</tt> then contacts the CC to register itself as started</li>
320</ol>
321<p>This process is briefly illustrated in the diagram below:</p>
322
323<div class="source">
324
325<div class="source">
326<pre>
327<img src="images/ncservice.png" alt="The AsterixDB Web Interface" />
328<i>Fig. 2</i>: NCService startup sequence
329</pre></div>
330</div>
331<p>To start a small cluster consisting of 2 NodeControllers (<tt>red</tt> and <tt>blue</tt>) and 1 ClusterController (<tt>cc</tt>) on a single machine only 2 configuration files are required. The first one is</p>
332<p><tt>blue.conf</tt>:</p>
333
334<div class="source">
335<div class="source">
336<pre>[ncservice]
337port=9091
338</pre></div></div>
339<p>It is a configuration file for the second <tt>NCService</tt>. This contains only the port that the <tt>NCService</tt> of the second NodeControllers listens to as it is non-standard. The first <tt>NCService</tt> does not need a configuration file, as it only uses default parameters. In a distributed environment with 1 NodeController per machine, no <tt>NCService</tt> needs a configuration file.</p>
340<p>The second configuration file is</p>
341<p><tt>cc.conf</tt>:</p>
342
343<div class="source">
344<div class="source">
345<pre>[nc/red]
346txn.log.dir=/tmp/asterix/red/txnlog
347core.dump.dir=/tmp/asterix/red/coredump
348iodevices=/tmp/asterix/red
349
350[nc/blue]
351port=9091
352txn.log.dir=/tmp/asterix/blue/txnlog
353core.dump.dir=/tmp/asterix/blue/coredump
354iodevices=/tmp/asterix/blue
355
356[nc]
357app.class=org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
358storage.subdir=storage
359address=127.0.0.1
360command=asterixnc
361
362[cc]
363address = 127.0.0.1
364console.listen.port = 12345
365</pre></div></div>
366<p>This is the configuration file for the cluster and it contains information that each <tt>NCService</tt> will use when starting the corresponding <tt>NCDriver</tt> as well as information for the <tt>CCDriver</tt>.</p>
367<p>To start the cluster simply use the following steps</p>
368
369<ol style="list-style-type: decimal">
370
371<li>
372<p>Change directory into the asterix-server binary folder</p>
373
374<div class="source">
375<div class="source">
376<pre>user@localhost:~/
377$cd asterix-server/
378user@localhost:~/asterix-server
379$cd samples/local/bin
380</pre></div></div></li>
381
382<li>
383<p>Start the 2 <tt>NCServices</tt> for <tt>red</tt> and <tt>blue</tt>.</p>
384
385<div class="source">
386<div class="source">
387<pre>user@localhost:~/asterix-server
388$bin/asterixncservice -config-file blue.conf &gt; blue-service.log 2&gt;&amp;1 &amp;
389user@localhost:~/asterix-server
390$bin/asterixncservice &gt;red-service.log 2&gt;&amp;1 &amp;
391</pre></div></div></li>
392
393<li>
394<p>Start the <tt>CCDriver</tt>.</p>
395
396<div class="source">
397<div class="source">
398<pre>user@localhost:~/asterix-server
399$bin/asterixcc -config-file cc.conf &gt; cc.log 2&gt;&amp;1 &amp;
400</pre></div></div></li>
401</ol>
402<p>The <tt>CCDriver</tt> will connect to the <tt>NCServices</tt> and thus initiate the configuration and the start of the <tt>NCDrivers</tt>. After running these scripts, <tt>jps</tt> should show a result similar to this:</p>
403
404<div class="source">
405<div class="source">
406<pre>user@localhost:~/asterix-server
407$jps
40813184 NCService
40913200 NCDriver
41013185 NCService
41113186 CCDriver
41213533 Jps
41313198 NCDriver
414</pre></div></div>
415<p>The logs for the <tt>NCDrivers</tt> will be in <tt>$BASEDIR/logs</tt>.</p>
416<p>To stop the cluster again simply run</p>
417
418<div class="source">
419<div class="source">
420<pre>$ kill `jps | egrep '(CDriver|NCService)' | awk '{print $1}'`
421</pre></div></div>
422<p>to kill all processes.</p>
423<h1><a name="Multi_machine" id="Multi_machine">Deploying AsterixDB via NCService in a multi-machine setup</a></h1>
424<p>Deploying on multiple machines only differs in the configuration file and where each process is actually resident. Take for example a deployment on 3 machines, <tt>cacofonix-1</tt>,<tt>cacofonix-2</tt>,and <tt>cacofonix-3</tt>. <tt>cacofonix-1</tt> will be the CC, and <tt>cacofonix-2</tt> and <tt>cacofonix-3</tt> will be the two NCs, respectively. The configuration would be as follows:</p>
425<p><tt>cc.conf</tt>:</p>
426
427<div class="source">
428<div class="source">
429<pre>[nc/red]
430txn.log.dir=/lv_scratch/asterix/red/txnlog
431core.dump.dir=/lv_scratch/asterix/red/coredump
432iodevices=/lv_scratch/asterix/red
433address=cacofonix-2
434
435[nc/blue]
436txn.log.dir=/lv_scratch/asterix/blue/txnlog
437core.dump.dir=/lv_scratch/asterix/blue/coredump
438iodevices=/lv_scratch/asterix/blue
439address=cacofonix-3
440
441[nc]
442app.class=org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
443storagedir=storage
444command=asterixnc
445
446[cc]
447address = cacofonix-1
448</pre></div></div>
449<p>To deploy, first the <tt>asterix-server</tt> binary must be present on each machine. Any method to transfer the archive to each machine will work, but here <tt>scp</tt> will be used for simplicity&#x2019;s sake.</p>
450
451<div class="source">
452<div class="source">
453<pre>user@localhost:~
454$for f in {1,2,3}; scp asterix-server.zip cacofonix-$f:~/; end
455</pre></div></div>
456<p>Then unzip the binary on each machine. First, start the <tt>NCService</tt> processes on each of the slave machines. Any way of getting a shell on the machine is fine, be it physical or via <tt>ssh</tt>.</p>
457
458<div class="source">
459<div class="source">
460<pre>user@cacofonix-2 12:41:42 ~/asterix-server/
461$ bin/asterixncservice &gt; red-service.log 2&gt;&amp;1 &amp;
462
463
464user@cacofonix-3 12:41:42 ~/asterix-server/
465$ bin/asterixncservice &gt; blue-service.log 2&gt;&amp;1 &amp;
466</pre></div></div>
467<p>Now that each <tt>NCService</tt> is waiting, the CC can be started.</p>
468
469<div class="source">
470<div class="source">
471<pre>user@cacofonix-1 12:41:42 ~/asterix-server/
472$ bin/asterixcc -config-file cc.conf &gt; cc.log 2&gt;&amp;1 &amp;
473</pre></div></div>
474<p>The cluster should now be started and the Web UI available on the CC host at the default port.</p>
475<h1><a name="Parameters" id="Parameters">Available Configuration Parameters</a></h1>
476<p>The following parameters are for the master process, under the &#x201c;[cc]&#x201d; section.</p>
477
478<table border="0" class="table table-striped">
479 <thead>
480
481<tr class="a">
482
483<th>Section </th>
484
485<th>Parameter </th>
486
487<th>Meaning </th>
488
489<th>Default </th>
490 </tr>
491 </thead>
492 <tbody>
493
494<tr class="b">
495
496<td>cc </td>
497
498<td>active.port </td>
499
500<td>The listen port of the active server </td>
501
502<td>19003 </td>
503 </tr>
504
505<tr class="a">
506
507<td>cc </td>
508
509<td>address </td>
510
511<td>Default bind address for all services on this cluster controller </td>
512
513<td>127.0.0.1 </td>
514 </tr>
515
516<tr class="b">
517
518<td>cc </td>
519
520<td>api.port </td>
521
522<td>The listen port of the API server </td>
523
524<td>19002 </td>
525 </tr>
526
527<tr class="a">
528
529<td>cc </td>
530
531<td>app.class </td>
532
533<td>Application CC main class </td>
534
535<td>org.apache.asterix.hyracks.bootstrap.CCApplication </td>
536 </tr>
537
538<tr class="b">
539
540<td>cc </td>
541
542<td>client.listen.address </td>
543
544<td>Sets the IP Address to listen for connections from clients </td>
545
546<td>same as address </td>
547 </tr>
548
549<tr class="a">
550
551<td>cc </td>
552
553<td>client.listen.port </td>
554
555<td>Sets the port to listen for connections from clients </td>
556
557<td>1098 </td>
558 </tr>
559
560<tr class="b">
561
562<td>cc </td>
563
564<td>cluster.listen.address </td>
565
566<td>Sets the IP Address to listen for connections from NCs </td>
567
568<td>same as address </td>
569 </tr>
570
571<tr class="a">
572
573<td>cc </td>
574
575<td>cluster.listen.port </td>
576
577<td>Sets the port to listen for connections from node controllers </td>
578
579<td>1099 </td>
580 </tr>
581
582<tr class="b">
583
584<td>cc </td>
585
586<td>cluster.public.address </td>
587
588<td>Address that NCs should use to contact this CC </td>
589
590<td>same as cluster.listen.address </td>
591 </tr>
592
593<tr class="a">
594
595<td>cc </td>
596
597<td>cluster.public.port </td>
598
599<td>Port that NCs should use to contact this CC </td>
600
601<td>same as cluster.listen.port </td>
602 </tr>
603
604<tr class="b">
605
606<td>cc </td>
607
608<td>cluster.topology </td>
609
610<td>Sets the XML file that defines the cluster topology </td>
611
612<td>&lt;undefined&gt; </td>
613 </tr>
614
615<tr class="a">
616
617<td>cc </td>
618
619<td>console.listen.address </td>
620
621<td>Sets the listen address for the Cluster Controller </td>
622
623<td>same as address </td>
624 </tr>
625
626<tr class="b">
627
628<td>cc </td>
629
630<td>console.listen.port </td>
631
632<td>Sets the http port for the Cluster Controller) </td>
633
634<td>16001 </td>
635 </tr>
636
637<tr class="a">
638
639<td>cc </td>
640
641<td>heartbeat.max.misses </td>
642
643<td>Sets the maximum number of missed heartbeats before a node is marked as dead </td>
644
645<td>5 </td>
646 </tr>
647
648<tr class="b">
649
650<td>cc </td>
651
652<td>heartbeat.period </td>
653
654<td>Sets the time duration between two heartbeats from each node controller in milliseconds </td>
655
656<td>10000 </td>
657 </tr>
658
659<tr class="a">
660
661<td>cc </td>
662
663<td>job.history.size </td>
664
665<td>Limits the number of historical jobs remembered by the system to the specified value </td>
666
667<td>10 </td>
668 </tr>
669
670<tr class="b">
671
672<td>cc </td>
673
674<td>job.manager.class </td>
675
676<td>Specify the implementation class name for the job manager </td>
677
678<td>org.apache.hyracks.control.cc.job.JobManager </td>
679 </tr>
680
681<tr class="a">
682
683<td>cc </td>
684
685<td>job.queue.capacity </td>
686
687<td>The maximum number of jobs to queue before rejecting new jobs </td>
688
689<td>4096 </td>
690 </tr>
691
692<tr class="b">
693
694<td>cc </td>
695
696<td>job.queue.class </td>
697
698<td>Specify the implementation class name for the job queue </td>
699
700<td>org.apache.hyracks.control.cc.scheduler.FIFOJobQueue </td>
701 </tr>
702
703<tr class="a">
704
705<td>cc </td>
706
707<td>profile.dump.period </td>
708
709<td>Sets the time duration between two profile dumps from each node controller in milliseconds; 0 to disable </td>
710
711<td>0 </td>
712 </tr>
713
714<tr class="b">
715
716<td>cc </td>
717
718<td>result.sweep.threshold </td>
719
720<td>The duration within which an instance of the result cleanup should be invoked in milliseconds </td>
721
722<td>60000 </td>
723 </tr>
724
725<tr class="a">
726
727<td>cc </td>
728
729<td>result.ttl </td>
730
731<td>Limits the amount of time results for asynchronous jobs should be retained by the system in milliseconds </td>
732
733<td>86400000 </td>
734 </tr>
735
736<tr class="b">
737
738<td>cc </td>
739
740<td>root.dir </td>
741
742<td>Sets the root folder used for file operations </td>
743
744<td>${java.io.tmpdir}/asterixdb/ClusterControllerService </td>
745 </tr>
746
747<tr class="a">
748
749<td>cc </td>
750
751<td>web.port </td>
752
753<td>The listen port of the legacy query interface </td>
754
755<td>19001 </td>
756 </tr>
757
758<tr class="b">
759
760<td>cc </td>
761
762<td>web.queryinterface.port </td>
763
764<td>The listen port of the query web interface </td>
765
766<td>19006 </td>
767 </tr>
768 </tbody>
769</table>
770<p>The following parameters for slave processes, under &#x201c;[nc]&#x201d; sections.</p>
771
772<table border="0" class="table table-striped">
773 <thead>
774
775<tr class="a">
776
777<th>Section </th>
778
779<th>Parameter </th>
780
781<th>Meaning </th>
782
783<th>Default </th>
784 </tr>
785 </thead>
786 <tbody>
787
788<tr class="b">
789
790<td>nc </td>
791
792<td>address </td>
793
794<td>Default IP Address to bind listeners on this NC. All services will bind on this address unless a service-specific listen address is supplied. </td>
795
796<td>127.0.0.1 </td>
797 </tr>
798
799<tr class="a">
800
801<td>nc </td>
802
803<td>app.class </td>
804
805<td>Application NC Main Class </td>
806
807<td>org.apache.asterix.hyracks.bootstrap.NCApplication </td>
808 </tr>
809
810<tr class="b">
811
812<td>nc </td>
813
814<td>cluster.address </td>
815
816<td>Cluster Controller address (required unless specified in config file) </td>
817
818<td>&lt;undefined&gt; </td>
819 </tr>
820
821<tr class="a">
822
823<td>nc </td>
824
825<td>cluster.connect.retries </td>
826
827<td>Number of attempts to contact CC before giving up </td>
828
829<td>5 </td>
830 </tr>
831
832<tr class="b">
833
834<td>nc </td>
835
836<td>cluster.listen.address </td>
837
838<td>IP Address to bind cluster listener on this NC </td>
839
840<td>same as address </td>
841 </tr>
842
843<tr class="a">
844
845<td>nc </td>
846
847<td>cluster.listen.port </td>
848
849<td>IP port to bind cluster listener </td>
850
851<td>0 </td>
852 </tr>
853
854<tr class="b">
855
856<td>nc </td>
857
858<td>cluster.port </td>
859
860<td>Cluster Controller port </td>
861
862<td>1099 </td>
863 </tr>
864
865<tr class="a">
866
867<td>nc </td>
868
869<td>cluster.public.address </td>
870
871<td>Public IP Address to announce cluster listener </td>
872
873<td>same as public.address </td>
874 </tr>
875
876<tr class="b">
877
878<td>nc </td>
879
880<td>cluster.public.port </td>
881
882<td>Public IP port to announce cluster listener </td>
883
884<td>same as cluster.listen.port </td>
885 </tr>
886
887<tr class="a">
888
889<td>nc </td>
890
891<td>command </td>
892
893<td>Command NCService should invoke to start the NCDriver </td>
894
895<td>hyracksnc </td>
896 </tr>
897
898<tr class="b">
899
900<td>nc </td>
901
902<td>core.dump.dir </td>
903
904<td>The directory where node core dumps should be written </td>
905
906<td>${java.io.tmpdir}/asterixdb/coredump </td>
907 </tr>
908
909<tr class="a">
910
911<td>nc </td>
912
913<td>data.listen.address </td>
914
915<td>IP Address to bind data listener </td>
916
917<td>same as address </td>
918 </tr>
919
920<tr class="b">
921
922<td>nc </td>
923
924<td>data.listen.port </td>
925
926<td>IP port to bind data listener </td>
927
928<td>0 </td>
929 </tr>
930
931<tr class="a">
932
933<td>nc </td>
934
935<td>data.public.address </td>
936
937<td>Public IP Address to announce data listener </td>
938
939<td>same as public.address </td>
940 </tr>
941
942<tr class="b">
943
944<td>nc </td>
945
946<td>data.public.port </td>
947
948<td>Public IP port to announce data listener </td>
949
950<td>same as data.listen.port </td>
951 </tr>
952
953<tr class="a">
954
955<td>nc </td>
956
957<td>iodevices </td>
958
959<td>Comma separated list of IO Device mount points </td>
960
961<td>${java.io.tmpdir}/asterixdb/iodevice </td>
962 </tr>
963
964<tr class="b">
965
966<td>nc </td>
967
968<td>jvm.args </td>
969
970<td>JVM args to pass to the NCDriver </td>
971
972<td>&lt;undefined&gt; </td>
973 </tr>
974
975<tr class="a">
976
977<td>nc </td>
978
979<td>messaging.listen.address </td>
980
981<td>IP Address to bind messaging listener </td>
982
983<td>same as address </td>
984 </tr>
985
986<tr class="b">
987
988<td>nc </td>
989
990<td>messaging.listen.port </td>
991
992<td>IP port to bind messaging listener </td>
993
994<td>0 </td>
995 </tr>
996
997<tr class="a">
998
999<td>nc </td>
1000
1001<td>messaging.public.address </td>
1002
1003<td>Public IP Address to announce messaging listener </td>
1004
1005<td>same as public.address </td>
1006 </tr>
1007
1008<tr class="b">
1009
1010<td>nc </td>
1011
1012<td>messaging.public.port </td>
1013
1014<td>Public IP port to announce messaging listener </td>
1015
1016<td>same as messaging.listen.port </td>
1017 </tr>
1018
1019<tr class="a">
1020
1021<td>nc </td>
1022
1023<td>ncservice.address </td>
1024
1025<td>Address the CC should use to contact the NCService associated with this NC </td>
1026
1027<td>same as public.address </td>
1028 </tr>
1029
1030<tr class="b">
1031
1032<td>nc </td>
1033
1034<td>ncservice.pid </td>
1035
1036<td>PID of the NCService which launched this NCDriver </td>
1037
1038<td>-1 </td>
1039 </tr>
1040
1041<tr class="a">
1042
1043<td>nc </td>
1044
1045<td>ncservice.port </td>
1046
1047<td>Port the CC should use to contact the NCService associated with this NC </td>
1048
1049<td>9090 </td>
1050 </tr>
1051
1052<tr class="b">
1053
1054<td>nc </td>
1055
1056<td>net.buffer.count </td>
1057
1058<td>Number of network buffers per input/output channel </td>
1059
1060<td>1 </td>
1061 </tr>
1062
1063<tr class="a">
1064
1065<td>nc </td>
1066
1067<td>net.thread.count </td>
1068
1069<td>Number of threads to use for Network I/O </td>
1070
1071<td>1 </td>
1072 </tr>
1073
1074<tr class="b">
1075
1076<td>nc </td>
1077
1078<td>public.address </td>
1079
1080<td>Default public address that other processes should use to contact this NC. All services will advertise this address unless a service-specific public address is supplied. </td>
1081
1082<td>same as address </td>
1083 </tr>
1084
1085<tr class="a">
1086
1087<td>nc </td>
1088
1089<td>result.listen.address </td>
1090
1091<td>IP Address to bind dataset result distribution listener </td>
1092
1093<td>same as address </td>
1094 </tr>
1095
1096<tr class="b">
1097
1098<td>nc </td>
1099
1100<td>result.listen.port </td>
1101
1102<td>IP port to bind dataset result distribution listener </td>
1103
1104<td>0 </td>
1105 </tr>
1106
1107<tr class="a">
1108
1109<td>nc </td>
1110
1111<td>result.manager.memory </td>
1112
1113<td>Memory usable for result caching at this Node Controller in bytes </td>
1114
1115<td>-1 (-1 B) </td>
1116 </tr>
1117
1118<tr class="b">
1119
1120<td>nc </td>
1121
1122<td>result.public.address </td>
1123
1124<td>Public IP Address to announce dataset result distribution listener </td>
1125
1126<td>same as public.address </td>
1127 </tr>
1128
1129<tr class="a">
1130
1131<td>nc </td>
1132
1133<td>result.public.port </td>
1134
1135<td>Public IP port to announce dataset result distribution listener </td>
1136
1137<td>same as result.listen.port </td>
1138 </tr>
1139
1140<tr class="b">
1141
1142<td>nc </td>
1143
1144<td>result.sweep.threshold </td>
1145
1146<td>The duration within which an instance of the result cleanup should be invoked in milliseconds </td>
1147
1148<td>60000 </td>
1149 </tr>
1150
1151<tr class="a">
1152
1153<td>nc </td>
1154
1155<td>result.ttl </td>
1156
1157<td>Limits the amount of time results for asynchronous jobs should be retained by the system in milliseconds </td>
1158
1159<td>86400000 </td>
1160 </tr>
1161
1162<tr class="b">
1163
1164<td>nc </td>
1165
1166<td>storage.buffercache.maxopenfiles </td>
1167
1168<td>The maximum number of open files in the buffer cache </td>
1169
1170<td>2147483647 </td>
1171 </tr>
1172
1173<tr class="a">
1174
1175<td>nc </td>
1176
1177<td>storage.buffercache.pagesize </td>
1178
1179<td>The page size in bytes for pages in the buffer cache </td>
1180
1181<td>131072 (128 kB) </td>
1182 </tr>
1183
1184<tr class="b">
1185
1186<td>nc </td>
1187
1188<td>storage.buffercache.size </td>
1189
1190<td>The size of memory allocated to the disk buffer cache. The value should be a multiple of the buffer cache page size. </td>
1191
1192<td>715915264 (682.75 MB) </td>
1193 </tr>
1194
1195<tr class="a">
1196
1197<td>nc </td>
1198
1199<td>storage.lsm.bloomfilter.falsepositiverate </td>
1200
1201<td>The maximum acceptable false positive rate for bloom filters associated with LSM indexes </td>
1202
1203<td>0.01 </td>
1204 </tr>
1205
1206<tr class="b">
1207
1208<td>nc </td>
1209
1210<td>storage.memorycomponent.globalbudget </td>
1211
1212<td>The size of memory allocated to the memory components. The value should be a multiple of the memory component page size </td>
1213
1214<td>715915264 (682.75 MB) </td>
1215 </tr>
1216
1217<tr class="a">
1218
1219<td>nc </td>
1220
1221<td>storage.memorycomponent.numcomponents </td>
1222
1223<td>The number of memory components to be used per lsm index </td>
1224
1225<td>2 </td>
1226 </tr>
1227
1228<tr class="b">
1229
1230<td>nc </td>
1231
1232<td>storage.memorycomponent.numpages </td>
1233
1234<td>The number of pages to allocate for a memory component. This budget is shared by all the memory components of the primary index and all its secondary indexes across all I/O devices on a node. Note: in-memory components usually has fill factor of 75% since the pages are 75% full and the remaining 25% is un-utilized </td>
1235
1236<td>1/16th of the storage.memorycomponent.globalbudget value </td>
1237 </tr>
1238
1239<tr class="a">
1240
1241<td>nc </td>
1242
1243<td>storage.memorycomponent.pagesize </td>
1244
1245<td>The page size in bytes for pages allocated to memory components </td>
1246
1247<td>131072 (128 kB) </td>
1248 </tr>
1249
1250<tr class="b">
1251
1252<td>nc </td>
1253
1254<td>storage.metadata.memorycomponent.numpages </td>
1255
1256<td>The number of pages to allocate for a metadata memory component </td>
1257
1258<td>1/64th of the storage.memorycomponent.globalbudget value or 256, whichever is larger </td>
1259 </tr>
1260
1261<tr class="a">
1262
1263<td>nc </td>
1264
1265<td>storage.subdir </td>
1266
1267<td>The subdirectory name under each iodevice used for storage </td>
1268
1269<td>storage </td>
1270 </tr>
1271
1272<tr class="b">
1273
1274<td>nc </td>
1275
1276<td>txn.log.dir </td>
1277
1278<td>The directory where transaction logs should be stored </td>
1279
1280<td>${java.io.tmpdir}/asterixdb/txn-log </td>
1281 </tr>
1282 </tbody>
1283</table>
1284<p>The following parameters are configured under the &#x201c;[common]&#x201d; section.</p>
1285
1286<table border="0" class="table table-striped">
1287 <thead>
1288
1289<tr class="a">
1290
1291<th>Section </th>
1292
1293<th>Parameter </th>
1294
1295<th>Meaning </th>
1296
1297<th>Default </th>
1298 </tr>
1299 </thead>
1300 <tbody>
1301
1302<tr class="b">
1303
1304<td>common </td>
1305
1306<td>active.memory.global.budget </td>
1307
1308<td>The memory budget (in bytes) for the active runtime </td>
1309
1310<td>67108864 (64 MB) </td>
1311 </tr>
1312
1313<tr class="a">
1314
1315<td>common </td>
1316
1317<td>compiler.framesize </td>
1318
1319<td>The page size (in bytes) for computation </td>
1320
1321<td>32768 (32 kB) </td>
1322 </tr>
1323
1324<tr class="b">
1325
1326<td>common </td>
1327
1328<td>compiler.groupmemory </td>
1329
1330<td>The memory budget (in bytes) for a group by operator instance in a partition </td>
1331
1332<td>33554432 (32 MB) </td>
1333 </tr>
1334
1335<tr class="a">
1336
1337<td>common </td>
1338
1339<td>compiler.joinmemory </td>
1340
1341<td>The memory budget (in bytes) for a join operator instance in a partition </td>
1342
1343<td>33554432 (32 MB) </td>
1344 </tr>
1345
1346<tr class="b">
1347
1348<td>common </td>
1349
1350<td>compiler.parallelism </td>
1351
1352<td>The degree of parallelism for query execution. Zero means to use the storage parallelism as the query execution parallelism, while other integer values dictate the number of query execution parallel partitions. The system will fall back to use the number of all available CPU cores in the cluster as the degree of parallelism if the number set by a user is too large or too small </td>
1353
1354<td>0 </td>
1355 </tr>
1356
1357<tr class="a">
1358
1359<td>common </td>
1360
1361<td>compiler.sortmemory </td>
1362
1363<td>The memory budget (in bytes) for a sort operator instance in a partition </td>
1364
1365<td>33554432 (32 MB) </td>
1366 </tr>
1367
1368<tr class="b">
1369
1370<td>common </td>
1371
1372<td>instance.name </td>
1373
1374<td>The name of this cluster instance </td>
1375
1376<td>DEFAULT_INSTANCE </td>
1377 </tr>
1378
1379<tr class="a">
1380
1381<td>common </td>
1382
1383<td>log.level </td>
1384
1385<td>The logging level for master and slave processes </td>
1386
1387<td>WARNING </td>
1388 </tr>
1389
1390<tr class="b">
1391
1392<td>common </td>
1393
1394<td>max.wait.active.cluster </td>
1395
1396<td>The max pending time (in seconds) for cluster startup. After the threshold, if the cluster still is not up and running, it is considered unavailable </td>
1397
1398<td>60 </td>
1399 </tr>
1400
1401<tr class="a">
1402
1403<td>common </td>
1404
1405<td>messaging.frame.count </td>
1406
1407<td>Number of reusable frames for NC to NC messaging </td>
1408
1409<td>512 </td>
1410 </tr>
1411
1412<tr class="b">
1413
1414<td>common </td>
1415
1416<td>messaging.frame.size </td>
1417
1418<td>The frame size to be used for NC to NC messaging </td>
1419
1420<td>4096 (4 kB) </td>
1421 </tr>
1422
1423<tr class="a">
1424
1425<td>common </td>
1426
1427<td>metadata.callback.port </td>
1428
1429<td>IP port to bind metadata callback listener (0 = random port) </td>
1430
1431<td>0 </td>
1432 </tr>
1433
1434<tr class="b">
1435
1436<td>common </td>
1437
1438<td>metadata.listen.port </td>
1439
1440<td>IP port to bind metadata listener (0 = random port) </td>
1441
1442<td>0 </td>
1443 </tr>
1444
1445<tr class="a">
1446
1447<td>common </td>
1448
1449<td>metadata.node </td>
1450
1451<td>the node which should serve as the metadata node </td>
1452
1453<td>&lt;undefined&gt; </td>
1454 </tr>
1455
1456<tr class="b">
1457
1458<td>common </td>
1459
1460<td>metadata.registration.timeout.secs </td>
1461
1462<td>how long in seconds to wait for the metadata node to register with the CC </td>
1463
1464<td>60 </td>
1465 </tr>
1466
1467<tr class="a">
1468
1469<td>common </td>
1470
1471<td>replication.log.batchsize </td>
1472
1473<td>The size in bytes to replicate in each batch </td>
1474
1475<td>4096 (4 kB) </td>
1476 </tr>
1477
1478<tr class="b">
1479
1480<td>common </td>
1481
1482<td>replication.log.buffer.numpages </td>
1483
1484<td>The number of log buffer pages </td>
1485
1486<td>8 </td>
1487 </tr>
1488
1489<tr class="a">
1490
1491<td>common </td>
1492
1493<td>replication.log.buffer.pagesize </td>
1494
1495<td>The size in bytes of each log buffer page </td>
1496
1497<td>131072 (128 kB) </td>
1498 </tr>
1499
1500<tr class="b">
1501
1502<td>common </td>
1503
1504<td>replication.max.remote.recovery.attempts </td>
1505
1506<td>The maximum number of times to attempt to recover from a replica on failure before giving up </td>
1507
1508<td>5 </td>
1509 </tr>
1510
1511<tr class="a">
1512
1513<td>common </td>
1514
1515<td>replication.timeout </td>
1516
1517<td>The time in seconds to timeout when trying to contact a replica, before assuming it is dead </td>
1518
1519<td>15 </td>
1520 </tr>
1521
1522<tr class="b">
1523
1524<td>common </td>
1525
1526<td>txn.commitprofiler.enabled </td>
1527
1528<td>Enable output of commit profiler logs </td>
1529
1530<td>false </td>
1531 </tr>
1532
1533<tr class="a">
1534
1535<td>common </td>
1536
1537<td>txn.commitprofiler.reportinterval </td>
1538
1539<td>Interval (in seconds) to report commit profiler logs </td>
1540
1541<td>5 </td>
1542 </tr>
1543
1544<tr class="b">
1545
1546<td>common </td>
1547
1548<td>txn.job.recovery.memorysize </td>
1549
1550<td>The memory budget (in bytes) used for recovery </td>
1551
1552<td>67108864 (64 MB) </td>
1553 </tr>
1554
1555<tr class="a">
1556
1557<td>common </td>
1558
1559<td>txn.lock.escalationthreshold </td>
1560
1561<td>The maximum number of entity locks to obtain before upgrading to a dataset lock </td>
1562
1563<td>1000 </td>
1564 </tr>
1565
1566<tr class="b">
1567
1568<td>common </td>
1569
1570<td>txn.lock.shrinktimer </td>
1571
1572<td>The time (in milliseconds) where under utilization of resources will trigger a shrink phase </td>
1573
1574<td>5000 </td>
1575 </tr>
1576
1577<tr class="a">
1578
1579<td>common </td>
1580
1581<td>txn.lock.timeout.sweepthreshold </td>
1582
1583<td>Interval (in milliseconds) for checking lock timeout </td>
1584
1585<td>10000 </td>
1586 </tr>
1587
1588<tr class="b">
1589
1590<td>common </td>
1591
1592<td>txn.lock.timeout.waitthreshold </td>
1593
1594<td>Time out (in milliseconds) of waiting for a lock </td>
1595
1596<td>60000 </td>
1597 </tr>
1598
1599<tr class="a">
1600
1601<td>common </td>
1602
1603<td>txn.log.buffer.numpages </td>
1604
1605<td>The number of pages in the transaction log tail </td>
1606
1607<td>8 </td>
1608 </tr>
1609
1610<tr class="b">
1611
1612<td>common </td>
1613
1614<td>txn.log.buffer.pagesize </td>
1615
1616<td>The page size (in bytes) for transaction log buffer </td>
1617
1618<td>131072 (128 kB) </td>
1619 </tr>
1620
1621<tr class="a">
1622
1623<td>common </td>
1624
1625<td>txn.log.checkpoint.history </td>
1626
1627<td>The number of checkpoints to keep in the transaction log </td>
1628
1629<td>0 </td>
1630 </tr>
1631
1632<tr class="b">
1633
1634<td>common </td>
1635
1636<td>txn.log.checkpoint.lsnthreshold </td>
1637
1638<td>The checkpoint threshold (in terms of LSNs (log sequence numbers) that have been written to the transaction log, i.e., the length of the transaction log) for transaction logs </td>
1639
1640<td>67108864 (64 MB) </td>
1641 </tr>
1642
1643<tr class="a">
1644
1645<td>common </td>
1646
1647<td>txn.log.checkpoint.pollfrequency </td>
1648
1649<td>The frequency (in seconds) the checkpoint thread should check to see if a checkpoint should be written </td>
1650
1651<td>120 </td>
1652 </tr>
1653
1654<tr class="b">
1655
1656<td>common </td>
1657
1658<td>txn.log.partitionsize </td>
1659
1660<td>The maximum size (in bytes) of each transaction log file </td>
1661
1662<td>268435456 (256 MB) </td>
1663 </tr>
1664 </tbody>
1665</table>
1666<p>For the optional NCService process configuration file, the following parameters, under &#x201c;[ncservice]&#x201d; section.</p>
1667
1668<table border="0" class="table table-striped">
1669 <thead>
1670
1671<tr class="a">
1672
1673<th>Parameter </th>
1674
1675<th>Meaning </th>
1676
1677<th>Default </th>
1678 </tr>
1679 </thead>
1680 <tbody>
1681
1682<tr class="b">
1683
1684<td>address </td>
1685
1686<td>The address the NCService listens on for commands from the CC </td>
1687
1688<td>(all addresses) </td>
1689 </tr>
1690
1691<tr class="a">
1692
1693<td>port </td>
1694
1695<td>The port for the NCService listens on for commands from the CC </td>
1696
1697<td>9090 </td>
1698 </tr>
1699
1700<tr class="b">
1701
1702<td>logdir </td>
1703
1704<td>Directory where NCService logs should be written (&#x2018;-&#x2019; indicates that output should go to stdout) </td>
1705
1706<td>${app.home}/logs (${user.home} if &#x2018;app.home&#x2019; not present in NCService Java system properties. </td>
1707 </tr>
1708 </tbody>
1709</table></div>
1710 </div>
1711 </div>
1712 </div>
1713
1714 <hr/>
1715
1716 <footer>
1717 <div class="container-fluid">
1718 <div class="row span12">Copyright &copy; 2017
1719 <a href="https://www.apache.org/">The Apache Software Foundation</a>.
1720 All Rights Reserved.
1721
1722 </div>
1723
1724 <?xml version="1.0" encoding="UTF-8"?>
1725<div class="row-fluid">Apache AsterixDB, AsterixDB, Apache, the Apache
1726 feather logo, and the Apache AsterixDB project logo are either
1727 registered trademarks or trademarks of The Apache Software
1728 Foundation in the United States and other countries.
1729 All other marks mentioned may be trademarks or registered
1730 trademarks of their respective owners.</div>
1731
1732
1733 </div>
1734 </footer>
1735 </body>
1736</html>