Move to org.bdgenomics.utils 0.2.0 release #664

fnothaft · 2015-04-30T23:27:08Z

Resolves #536, #581, #603, #657. In anticipation of the 0.17.0 release, this moves to the org.bdgenomics.utils 0.2.0 release. This leads to:

Unblocks Fix imports for RealignIndels to allow instrumentation to work #654.
Moves from the org.bdgenomics.utils.parquet.io package to the org.bdgenomics.utils.io package
Eliminates the CLI classes that had already moved to utils.
Adds the infrastructure for building Scala 2.10 and 2.11 artifacts. This includes infrastructure for releasing both artifacts, so the ADAM 0.17.0 release will have both 2.10 and 2.11 artifacts! Yay!

AmplabJenkins · 2015-04-30T23:35:38Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/691/

Build result: FAILURE

GitHub pull request #664 of commit ed7a98f automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/664/merge^{commit} # timeout=10 > git branch -a --contains 3066f146a1794ec8541556a1f976abd800469192 # timeout=10 > git rev-parse remotes/origin/pr/664/merge^{commit} # timeout=10Checking out Revision 3066f146a1794ec8541556a1f976abd800469192 (origin/pr/664/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 3066f146a1794ec8541556a1f976abd800469192First time build. Skipping changelog.Triggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 1.0.4,centos completed with result FAILUREADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 2.3.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2015-05-01T00:05:29Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/692/
Test PASSed.

…quet to utils-io.

…ala 2.11.

AmplabJenkins · 2015-05-01T17:00:35Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/695/
Test PASSed.

ryan-williams · 2015-05-03T00:08:38Z

pom.xml

  <description>A fast, scalable genome analysis system</description>
  <url>http://bdgenomics.org/</url>

  <properties>
    <java.version>1.7</java.version>
-    <scala.version>2.10.4</scala.version>


why take these out, OOC? just feel like it's cleaner to not interpolate these variables below?

It's part of the dual-build process. Specifically, we "flatten" the POM by hardcoding defaults to Scala 2.10 (scala.version = 2.10.4, scala.artifact.suffix = 2.10). Then, when we want to build for Scala 2.11, we run the scripts/move_to_scala_2.11.sh script which just uses sed to swap 2.10.4 with 2.11.4 and 2.10 with 2.11. Not beautiful, but gets the job done.

If Maven allowed you to put variables in the artifact names, you wouldn't need to do this. However, as I found via experimentation, there's "good" reasons that Maven doesn't allow you to have variables in artifact names. If you required all published POM's to be "flattened", then variables in an unpublished POM would be OK.

ah ok, I think I get it, and that the piece I was missing was "Maven doesn't allow variables in the <artifactId> field", which necessitates this change. Thanks for the info!

So, Maven doesn't strictly disallow variables in the <artifactId> field (it just gives you a warning that "this might break things"), but then I did bigdatagenomics/utils@47e325a, and tried to cut a release of that.

Let's just say it broke in spectacular and vengeful ways. After I made that commit, I spent a long time wondering asking reflecting on the mistakes in my life that led me to write 47e325a. The commit seemed so innocent at the time, but alas, my youthful ignorance (of mvn) led me astray.

Very interesting. I am out of my Maven depth, but now somewhat curious about the failure modes if it's at all interesting; no need to go into further detail if not. Seems innocuous enough, as you said.

And, confirming one piece of subtext: this is all happening because you are trying to release different artifacts for 2.10 vs. 2.11 for the first time, right?

it's never used… fixes bigdatagenomics#662 Moves to utils:0.2.1.

AmplabJenkins · 2015-05-03T16:06:48Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/699/
Test PASSed.

laserson · 2015-05-13T19:10:38Z

+1, lgtm, I'm good to merge @fnothaft

laserson · 2015-05-13T19:15:05Z

er, after rebasing, of course

ryan-williams · 2015-05-26T21:26:15Z

bump! this would be nice to have in

laserson · 2015-05-26T22:04:15Z

@fnothaft let me know if you want me to rebase this for you...

ryan-williams · 2015-05-26T22:11:22Z

@laserson I actually just sent fnothaft/adam#6 to do so, albeit with a merge of origin/master into this branch, fnothaft/utils-0.2.0 :)

laserson · 2015-05-26T22:13:44Z

Well, feel free to just apply the changes, git commit --amend, and either
submit your own PR with @fnothaft's commit or just manually merge it in.

......................................
Uri Laserson
+1 917 742 8019
[email protected]

On Tue, May 26, 2015 at 3:11 PM, Ryan Williams [email protected]
wrote:

@laserson https://github.com/laserson I actually just sent
fnothaft#6 fnothaft#6 to do so,
albeit with a merge of origin/master into this branch,
fnothaft/utils-0.2.0 :)

—
Reply to this email directly or view it on GitHub
#664 (comment).

fnothaft · 2015-05-26T23:01:25Z

Sorry, I was OOO at UCSF for meetings today. I will rebase this now.

ryan-williams · 2015-05-26T23:06:35Z

no worries, @fnothaft, thanks

laserson · 2015-05-26T23:08:04Z

Merged through @ryan-williams's PR (#678). Thanks guys!

fnothaft · 2015-05-26T23:08:32Z

Thanks @laserson and @ryan-williams!

fnothaft added this to the 0.17.0 milestone Apr 30, 2015

fnothaft force-pushed the utils-0.2.0 branch from ed7a98f to f27c54d Compare April 30, 2015 23:56

fnothaft added 3 commits May 1, 2015 09:47

[ADAM-657, 581] Move to org.bdgenomics.utils:0.2.0 and from utils-par…

95f4749

…quet to utils-io.

[ADAM-536] Add infrastructure for cross building to Scala 2.10 and Sc…

0e1ef10

…ala 2.11.

[ADAM-603] Migrate CLI classes to BDG.

6aa2198

fnothaft force-pushed the utils-0.2.0 branch from f27c54d to 6aa2198 Compare May 1, 2015 16:48

ryan-williams reviewed May 3, 2015
View reviewed changes

This was referenced May 3, 2015

kill job argument to BDGSparkCommand bigdatagenomics/utils#38

Merged

kill Job arg to ADAMSparkCommand #670

Closed

kill Job arg to ADAMSparkCommand

0059c85

it's never used… fixes bigdatagenomics#662 Moves to utils:0.2.1.

ryan-williams mentioned this pull request May 26, 2015

upgrade to bdg utils 0.2.1 #678

Merged

laserson merged commit 0059c85 into bigdatagenomics:master May 26, 2015

ryan-williams mentioned this pull request May 26, 2015

scala 2.11 #536

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move to org.bdgenomics.utils 0.2.0 release #664

Move to org.bdgenomics.utils 0.2.0 release #664

fnothaft commented Apr 30, 2015

AmplabJenkins commented Apr 30, 2015

AmplabJenkins commented May 1, 2015

AmplabJenkins commented May 1, 2015

ryan-williams May 3, 2015

fnothaft May 3, 2015

ryan-williams May 3, 2015

fnothaft May 3, 2015

ryan-williams May 3, 2015

AmplabJenkins commented May 3, 2015

laserson commented May 13, 2015

laserson commented May 13, 2015

ryan-williams commented May 26, 2015

laserson commented May 26, 2015

ryan-williams commented May 26, 2015

laserson commented May 26, 2015

fnothaft commented May 26, 2015

ryan-williams commented May 26, 2015

laserson commented May 26, 2015

fnothaft commented May 26, 2015

Move to org.bdgenomics.utils 0.2.0 release #664

Move to org.bdgenomics.utils 0.2.0 release #664

Conversation

fnothaft commented Apr 30, 2015

AmplabJenkins commented Apr 30, 2015

Build result: FAILURE

AmplabJenkins commented May 1, 2015

AmplabJenkins commented May 1, 2015

ryan-williams May 3, 2015

Choose a reason for hiding this comment

fnothaft May 3, 2015

Choose a reason for hiding this comment

ryan-williams May 3, 2015

Choose a reason for hiding this comment

fnothaft May 3, 2015

Choose a reason for hiding this comment

ryan-williams May 3, 2015

Choose a reason for hiding this comment

AmplabJenkins commented May 3, 2015

laserson commented May 13, 2015

laserson commented May 13, 2015

ryan-williams commented May 26, 2015

laserson commented May 26, 2015

ryan-williams commented May 26, 2015

laserson commented May 26, 2015

fnothaft commented May 26, 2015

ryan-williams commented May 26, 2015

laserson commented May 26, 2015

fnothaft commented May 26, 2015