Gitiles
Code Review Sign In
asterix-gerrit.ics.uci.edu / incubator-asterixdb-site / 3dffc852fe39126bf912ccf0db28c4c8ead64931 / . / dev-setup.html
blob: ab9aa27772d1799b9cb17edef5ea14611ce76d4a [file] [log] [blame]
Ian Maxon3dffc852015-07-13 20:17:09 -0700[diff] [blame^]1<!DOCTYPE html>
2<html lang="en">
3 <head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7 <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
8 <meta name="description" content="">
9 <meta name="author" content="">
10 <link rel="icon" href="https://asterixdb.incubator.apache.org/favicon.ico">
11
12 <title>Contributing to AsterixDB</title>
13
14 <!-- Bootstrap core CSS -->
15 <link href="https://asterixdb.incubator.apache.org/css/bootstrap.min.css" rel="stylesheet">
16 <!-- Bootstrap theme -->
17
18 <!-- Custom styles for this template -->
19 <link href="https://asterixdb.incubator.apache.org/css/theme.css" rel="stylesheet">
20
21 <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
22 <!--[if lt IE 9]>
23 <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
24 <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
25 <![endif]-->
26 <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css">
27 </head>
28
29 <div class="container theme-showcase" role="main">
30 <div class="masthead">
31 <p class="lead">
32 <a href="https://asterixdb.incubator.apache.org/index.html"><img src="https://asterixdb.incubator.apache.org/img/asterixdb.png" style="height:75px; width:auto; vertical-align:bottom; margin-top:10px;"/></a>
33 </p>
34</div>
35
36<nav class="navbar navbar-default">
37 <div class="container">
38 <div class="navbar-header">
39 <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target=".navbar-collapse">
40 <span class="sr-only">Toggle navigation</span>
41 <span class="icon-bar"></span>
42 <span class="icon-bar"></span>
43 <span class="icon-bar"></span>
44 </button>
45 <a class="navbar-brand" href="https://asterixdb.incubator.apache.org/index.html">Overview</a>
46 </div>
47 <div class="navbar-collapse collapse">
48 <ul class="nav navbar-nav">
49 <li><a href="https://asterixdb.incubator.apache.org/download.html">Download</a></li>
50 <li><a href="http://asterixdb.ics.uci.edu/documentation/index.html">Documentation <i class="fa fa-external-link fa-sm"></i></a></li>
51 <li><a href="https://asterixdb.incubator.apache.org/about.html">About</a></li>
52 <li><a href="https://asterixdb.incubator.apache.org/community.html">Community</a></li>
53 <li class="dropdown">
54 <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Contributing <span class="caret"></span></a>
55 <ul class="dropdown-menu" role="menu">
56 <li class="dropdown-header">For New Contributors</li>
57 <li><a href="https://asterixdb.incubator.apache.org/dev-setup.html">Development Setup</a></li>
58 <li class="divider"></li>
59 <li class="dropdown-header">For Committers</li>
60 <li><a href="https://asterixdb.incubator.apache.org/pushing.html">Pushing changes</a></li>
61 </ul>
62 </li>
63 </ul>
64 </div><!--/.nav-collapse -->
65 </div>
66 </nav>
67
68
69 <div class="row">
70<div class="col-md-8 col-centered">
71<h2>Contributing to AsterixDB</h2>
72<hr />
73
74<ul id="markdown-toc">
75 <li><a href="#introduction" id="markdown-toc-introduction">Introduction</a></li>
76 <li><a href="#setting-up-an-asterix-development-environment-in-eclipse" id="markdown-toc-setting-up-an-asterix-development-environment-in-eclipse">Setting up an Asterix Development environment in Eclipse</a> <ul>
77 <li><a href="#prerequisites" id="markdown-toc-prerequisites">Prerequisites</a></li>
78 <li><a href="#steps" id="markdown-toc-steps">Steps</a></li>
79 </ul>
80 </li>
81 <li><a href="#signing-up-for-gerrit" id="markdown-toc-signing-up-for-gerrit">Signing Up for Gerrit</a></li>
82 <li><a href="#configuring-your-local-working-environment-for-gerrit" id="markdown-toc-configuring-your-local-working-environment-for-gerrit">Configuring your local working environment for Gerrit</a> <ul>
83 <li><a href="#one-time-tasks" id="markdown-toc-one-time-tasks">One-time tasks</a></li>
84 <li><a href="#once-per-repository-tasks" id="markdown-toc-once-per-repository-tasks">Once-per-repository tasks</a></li>
85 </ul>
86 </li>
87 <li><a href="#making-changes---working-method" id="markdown-toc-making-changes---working-method">Making Changes - working method</a></li>
88 <li><a href="#making-more-changes" id="markdown-toc-making-more-changes">Making More Changes</a></li>
89 <li><a href="#using-jenkins-with-gerrit" id="markdown-toc-using-jenkins-with-gerrit">Using Jenkins with Gerrit</a> <ul>
90 <li><a href="#verification" id="markdown-toc-verification">Verification</a></li>
91 <li><a href="#retrigggering-builds-and-triggering-builds-manually" id="markdown-toc-retrigggering-builds-and-triggering-builds-manually">Retrigggering builds and triggering builds manually</a></li>
92 <li><a href="#cross-project-dependencies" id="markdown-toc-cross-project-dependencies">Cross-project Dependencies</a></li>
93 </ul>
94 </li>
95</ul>
96
97<hr />
98
99<h2 id="introduction">Introduction</h2>
100<p>We warmly welcome any contributions to the AsterixDB or related (Hyracks, Pregelix) projects. A great way to start contributing is to pick a bug labelled starter in JIRA and submit a patch for it, to get acquainted with our codebase and development process.</p>
101
102<hr />
103
104<h2 id="setting-up-an-asterix-development-environment-in-eclipse">Setting up an Asterix Development environment in Eclipse</h2>
105
106<h3 id="prerequisites">Prerequisites</h3>
107<ul>
108 <li>A suitable *nix environment (Linux, OSX)</li>
109 <li>JDK 1.7+</li>
110 <li>A relatively recent version of Eclipse</li>
111</ul>
112
113<h3 id="steps">Steps</h3>
114
115<ol>
116 <li>
117 <p>Check out Hyracks and Asterix master in two folders via git in the command line. Assume that the path is <code>$HOME/workspace</code>.</p>
118
119 <pre><code> cd $HOME/workspace/
120 git clone https://github.com/apache/incubator-asterixdb/
121 git clone https://github.com/apache/incubator-asterixdb-hyracks/
122</code></pre>
123
124 <p>You will now have <code>$HOME/workspace/incubator-asterixdb/</code> and <code>$HOME/workspace/incubator-asterixdb-hyracks/</code>.</p>
125 </li>
126 <li>
127 <p>Go to the hyracks and install it’s artifacts to the local Maven repository by executing the following commands:</p>
128
129 <pre><code> cd $HOME/workspace/hyracks/
130 mvn install -DskipTests
131</code></pre>
132 </li>
133 <li>
134 <p>Go to the asterixdb folder and perform the same action:</p>
135
136 <pre><code> cd $HOME/workspace/asterixdb/asterix-maven-plugins/
137 mvn install -DskipTests
138 cd $HOME/workspace/asterixdb/
139 mvn install -DskipTests
140</code></pre>
141 </li>
142 <li>In Eclipse, import hyracks as an existing Maven Project.
143 <ul>
144 <li><code>File -&gt; Import -&gt; Maven -&gt; Existing Maven Projects -&gt; Next</code></li>
145 <li>Specify the Root directory as <code>$HOME/workspace/hyracks</code> and then click Next until Finish is enabled.</li>
146 <li>If Eclipse tries to install the <code>m2e</code> (Maven To Eclipse) connector, let it do so.</li>
147 <li>There might be some plugin errors; however, that is not a big deal. Wait until the job finishes.</li>
148 <li>Then, click Finish.</li>
149 </ul>
150 </li>
151 <li>Then import asterixdb as an existing Maven Project.
152 <ul>
153 <li><code>File -&gt; Import -&gt; Maven -&gt; Existing Maven Projects -&gt; Next</code></li>
154 <li>Specify root directory as <code>$HOME/workspace/asterixdb</code> and then click Next until Finish is enabled.</li>
155 <li>Eclipse may try installing <code>m2e</code> or other plugins again; let it do so.</li>
156 <li>Then, click Finish.</li>
157 </ul>
158 </li>
159 <li>Fix Eclipse’s build path to include compile-time generated sources.
160 <ul>
161 <li>Right click the project where a red X mark is shown(e.g. <code>asterix-algebra</code>. Then resolve by applying the following:
162 <ul>
163 <li>Right click asterix-algebra. Click Build Path and Configure Build Path. Click Add Folder.</li>
164 <li>Under the <code>target -&gt; generated sources</code>, check the parent folder of the <code>edu</code> folder and click OK.</li>
165 <li>For example, if the directory structure is “target - generated-sources - javacc - edu - uci …”, check the javacc directory and click OK. Then X mark will dissapear.</li>
166 </ul>
167 </li>
168 <li>Repeat this step to all projects which show a red X mark except “asterix-fuzzyjoin” and “asterix-transactions”.</li>
169 <li>It may be the case that only “asterix-algebra” and “asterix-runtime” will require these steps.</li>
170 </ul>
171 </li>
172 <li>Set up Eclipse code formatting rules
173 <ul>
174 <li>Download files <a href="http://wiki.asterixdb.googlecode.com/git/AsterixCodeFormatProfile.xml">AsterixCodeFormatProfile.xml</a> <a href="http://wiki.asterixdb.googlecode.com/git/AsterixCleanupFormatProfile.xml">AsterixCleanupFormatProfile.xml</a></li>
175 <li>Import profiles into Eclipse
176 <ul>
177 <li>Preferences -&gt; Java -&gt; Code style -&gt; Formatter -&gt; Import -&gt; Select AsterixCodeFormatProfile.xml</li>
178 <li>Preferences -&gt; Java -&gt; Code style -&gt; Clean up -&gt; Import -&gt; Select AsterixCleanupFormatProfile.xml</li>
179 <li>Preferences -&gt; Java -&gt; Editor -&gt; Save actions -&gt; Perform the selected action on save -&gt; Format source code</li>
180 </ul>
181 </li>
182 </ul>
183 </li>
184</ol>
185
186<hr />
187
188<h2 id="signing-up-for-gerrit">Signing Up for Gerrit</h2>
189
190<p>You should only need to perform the following steps once.</p>
191
192<p>Our Gerrit server is <a href="https://asterix-gerrit.ics.uci.edu/">here</a></p>
193
194<ol>
195 <li>Visit the above URL, and click on “Register” in the upper right.</li>
196 <li>Gerrit uses OpenID; you may create an account using any OpenID provider, or with a Yahoo! account.</li>
197 <li>Click on “Sign in with a Yahoo! ID”. You will be taken to a page to log in to Yahoo!.</li>
198 <li>From there, you will get a page asking you to allow asterix-gerrit.ics.uci.edu to access your email and basic information. Approve this request.</li>
199 <li>You will be directed back to Gerrit to complete the account signup. Make sure to fill out this page! It is necessary to allow you to push code changes to Gerrit for review.</li>
200 <li>For “Full Name” and “Preferred Email”, enter what you like. If you would like to register a different email address such as an @uci.edu address, you may do so. This is where Gerrit will send you notifications about code review requests, voting, and so on.</li>
201 <li>For “Select a unique username”, choose a short alphanumeric ID. This will be your ssh username for git access. Be sure to click “Select Username” to verify that the name is unique and allowed.</li>
202 <li>Also provide a ssh public key here. There is information on the page on creating one if you do not have one. Click “Add” when done.
203 * Note that you can create a new ssh public key just for Gerrit if you like; if you do so you will need to modify your .ssh/config file to ensure the right key is selected.</li>
204 <li>Click “Continue” at the bottom of the page.</li>
205 <li>
206 <p>Finally, verify that you have everything set up: From a command-line, type
207<code>
208ssh -p 29418 username@asterix-gerrit.ics.uci.edu
209</code></p>
210 </li>
211 <li>You should see</li>
212</ol>
213
214<pre><code> **** Welcome to Gerrit Code Review ****
215
216 Hi Full Name, you have successfully connected over SSH.
217
218 Unfortunately, interactive shells are disabled.
219 To clone a hosted Git repository, use:
220
221 git clone ssh://username@asterix-gerrit.ics.uci.edu:29418/REPOSITORY_NAME.git
222
223</code></pre>
224
225<p>At this point, please send an email to Chris or Ian, specifying the email address of your new account, and ask to be added to the “AsterixDB Devs” group. You may push change proposals to Gerrit, and vote +1/-1 Code Review on any proposals without being in AsterixDB Devs, but you cannot vote +2 on anything to allow a change to be merged.</p>
226
227<hr />
228
229<h2 id="configuring-your-local-working-environment-for-gerrit">Configuring your local working environment for Gerrit</h2>
230
231<h3 id="one-time-tasks">One-time tasks</h3>
232
233<ol>
234 <li>At the command-line, run the following. Replace “username” with the user name you chose in step 7 above.
235<code>
236git config --global gerrit.url ssh://username@asterix-gerrit.ics.uci.edu:29418/
237</code></li>
238 <li>Download my “git-gerrit” utility library:
239<code>
240git clone https://github.com/ceejatec/git-gerrit
241</code></li>
242 <li>Ensure that the <code>git-gerrit/git-gerrit</code> script is on your PATH. You could copy it to somewhere, but it would be best to leave it inside the github clone so you can easily get updates.</li>
243</ol>
244
245<h3 id="once-per-repository-tasks">Once-per-repository tasks</h3>
246
247<ol>
248 <li>
249 <p>To work on (say) Asterix, first clone the Google Code repository (if you already have a local clone, great!).</p>
250
251 <p>git clone https://github.com/apache/incubator-asterixdb</p>
252 </li>
253 <li>
254 <p><code>cd</code> into the clone repo directory, and then run the following command to create the “gerrit” remote.</p>
255
256 <p>git gerrit init</p>
257 </li>
258</ol>
259
260<hr />
261
262<h2 id="making-changes---working-method">Making Changes - working method</h2>
263
264<ol>
265 <li>
266 <p>When you want to start working on a bug, feature, etc, first make a local <code>git</code> branch. Never work directly on <code>master</code>! <code>master</code> should always be a pure mirror of <code>origin/master</code>, ie, Google Code.</p>
267
268 <pre><code> git checkout -b my_branch
269</code></pre>
270 </li>
271 <li>Make your changes, test them, etc. Feel free to <code>git commit</code> as often as you like.</li>
272 <li>
273 <p><strong>Optional</strong>: If you like, you can push your branch up to Google Code, either to share it with others or as a backup. You may do this at whatever point in time you like.</p>
274
275 <pre><code> git push origin my_branch
276</code></pre>
277 </li>
278 <li>
279 <p>Every so often, you should update your local <code>master</code> mirror, and then merge that onto your working branch. This will prevent your branch from falling too far out of date, and ensure that your code review proposals will merge successfully with <code>master</code>. There are a number of ways to do this, but <code>git-gerrit</code> provides a convenience function:</p>
280
281 <pre><code> git gerrit update
282</code></pre>
283 </li>
284 <li>
285 <p>When you are ready to submit changes for code review, first ensure that you have committed everything locally that is necessary (<code>git status</code> should report “nothing to commit, working directory clean”). This is also a good time to update (see step 4). Then run:</p>
286
287 <pre><code> git gerrit submit
288</code></pre>
289 </li>
290 <li>This will pop open your editor to invite you to create a good commit message. This will be the single commit message which will be the only one to appear in the project’s master git history. Take the time to make it clear. The editor will contain the log messages of everything you committed on your branch as a reminder, but generally you will want to delete all this and replace it with a comprehensive message. Also: As noted in the initial message, the last line of the buffer will contain a <code>Change-Id</code> field. Do not delete that line! It is used by Gerrit to identify this particular merge proposal.</li>
291 <li>
292 <p>When you save your commit message, git-gerrit will push all of the changes from your working branch up to Gerrit. Assuming no errors, you should see output similar to the following:</p>
293
294 <pre><code> remote: Resolving deltas: 100% (1/1)
295 remote: Processing changes: new: 1, refs: 1, done
296 remote:
297 remote: New Changes:
298 remote: http://fulliautomatix.ics.uci.edu:8443/30
299 remote:
300 To ssh://ceej@fulliautomatix.ics.uci.edu:29418/ceej-gerrit-test
301 * [new branch] HEAD -&gt; refs/for/master
302</code></pre>
303 </li>
304 <li>That URL under “New Changes” is your code review! Send it to others to request reviews.</li>
305 <li>If you get any negative code reviews and need to make changes, you can just repeat steps 2 - 6 of the working method. Your local branch will still have all the history you put there, if you need to revert changes or look back and see what you did, etc.</li>
306 <li>When you repeat step 6, you will notice two things: First, git-gerrit keeps the change message for you, including the Change-Id, so you don’t have to re-invent it every time. Second, the output from <code>git gerrit submit</code> will not include the URL of the review this time. I’m not sure why; I wish it did. But if you re-visit the old URL in your browser, you should see an additional “Patch Set” containing your revised changes for people to review.</li>
307</ol>
308
309<h2 id="making-more-changes">Making More Changes</h2>
310
311<p>You may have as many feature branches as you want locally, and each may be at any point in the review cycle.</p>
312
313<p>Once you are done with a change (that is, it has been Code Reviewed and approved in Gerrit and merged), it is probably best to start work on the next big thing on a new branch. However, git-gerrit attempts to be smart if you choose to re-use a branch. For example, it should notice if the “current” Change-Id for a branch has been merged to master, and it will create a new Change-Id and a new fresh commit message the next time you run <code>git gerrit submit</code>.</p>
314
315<p>In that case, you may notice that the list of changes on the branch still contains older commits which have been merged. That is due to the unfortunate way Gerrit works; it requires squashing to a single git commit for review, which loses the association with the original commits in your history.</p>
316
317<p>It is also possible that you may want to re-use a branch and git-gerrit fails to notice that the older change has been merged - or perhaps the change was Abandoned on Gerrit and not merged, and you want to continue working on the same branch to create a new proposal. In this case, run
318<code>
319git gerrit new
320</code>
321on your branch. This will force git-gerrit to create a new Change-Id and commit message the next time you run <code>git gerrit submit</code>.</p>
322
323<hr />
324
325<h2 id="using-jenkins-with-gerrit">Using Jenkins with Gerrit</h2>
326
327<h3 id="verification">Verification</h3>
328
329<p>As of right now, whenever a patch set is submitted to Gerrit (i.e. whenever a user performs ‘git gerrit submit’) are automatically picked up by Jenkins, which runs <code>mvn verify</code> to test the patch fully. Once the build finishes, Jenkins comments on the patchset and votes with the result (+1 for all tests passing, -1 for anything less). Ideally, and in most cases, verifocation of a patchset reqires no intervention. However there are a few exceptions to this as detailed below.</p>
330
331<h3 id="retrigggering-builds-and-triggering-builds-manually">Retrigggering builds and triggering builds manually</h3>
332
333<h4 id="retriggering">Retriggering</h4>
334<p>Occasionally, there are builds in which tests failed, but perhaps not for a reason that has anything to do with the proposed patch’s changes. For example, on occasion the Integration tests can have issues with ports on the build server already being bound. The simple work around to this is to try building again.</p>
335
336<p>One way to perform this is by simply visiting the link that is posted on Gerrit by Jenkins, and hitting the ‘Retrigger’ link on the left-hand side of the page. This will try retesting the build with the exact same parameters as last time, hopefully with a different result.</p>
337
338<h4 id="manual-trigger">Manual Trigger</h4>
339<p>Builds of Gerrit patches can also be fully manually triggered from Jenkins. This can be desirable if a change is now dependent on a Hyracks change (via the Topic field feature described below), or if the patch is a draft or is not triggered automatically for any other reason. To manually trigger a build of a patchset, visit the Jenkins front-page, and click the ‘Query and Trigger Gerrit patches’ link. This should link to a page allowing you to manually trigger Gerrit patches. In the search field any query that can be searched in the Gerrit web interface can be used, but the default query of all open patchsets is likely fine. Hit the ‘Search’ button, and then click the checkbox that represents the patch that needs to be built. Then simply click ‘Trigger Selected’ and the patch will start building.</p>
340
341<h3 id="cross-project-dependencies">Cross-project Dependencies</h3>
342
343<p>It is sometimes the case that a change in one related project alters interfaces or expected results that an upstream project relies on. The converse can be true as well, such as an AsterixDB change that needs a Hyracks feature. In these sorts of situations, there is a mechanism to describe to Jenkins this dependenc of one project’s patchset to a particular Hyracks Change, by using the ‘Topic’ field in Gerrit. To utilize
344this feature, perform the following steps:</p>
345
346<ol>
347 <li>First submit the Hyracks change which the upstream (AsterixDB,etc..) change depends on if you have not done so already</li>
348 <li>In the Hyracks change details on the Gerrit web interface, take note of the refspec in the <em>Download</em> field of the patchset details. It will look something like <code>refs/changes/07/207/1</code>. This is what uniquely identifies the Hyracks change in Gerrit.</li>
349 <li>Submit the dependent change now, if you have not done so already. In the Gerrit web interface, view the change’s details. In the upper left-hand corner, there will be an empty field in a table, labeled <code>Topic</code>, with a notepad icon right-justified in the second column of the table. Click the notepad Icon, and in the resulting dialog box, enter in the refspec (<code>refs/changes/07/207/1</code>) as the Topic value.</li>
350 <li>Now, manually trigger the dependent change in Jenkins (see previous section for details)</li>
351</ol>
352
353<hr />
354
355</div>
356</div>
357
358
359 <hr />
360
361 <footer>
362 <div class="row">
363 <div class="col-md-6 col-md-offset-3 text-center">
364 <p>&copy; Copyright 2015 The Apache Software foundation. All Rights Reserved. </p>
365 <p>Apache AsterixDB, Apache, and the Apache feather logo are trademarks of the Apache Software Foundation</p>
366 </div>
367 <div class="col-md-1 text-right">
368 <img src="https://asterixdb.incubator.apache.org/img/egg-logo.png" style="height:auto; width:200px; vertical-align:bottom;"/>
369 </div>
370 </div>
371 </footer>
372 </div> <!-- /container -->
373
374
375 <!-- Bootstrap core JavaScript
376 ================================================== -->
377 <!-- Placed at the end of the document so the pages load faster -->
378 <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>
379 <script src="https://asterixdb.incubator.apache.org/js/bootstrap.min.js"></script>
380 </body>
381</html>