Merge branch '1-7-2'

CHANGELOG.mdown

@@ -1,6 +1,66 @@
 # Roxy Changes
 This file describes changes made to Roxy per release
 
+## 1.7.2
+
+### New Features
+* [#410](https://github.com/marklogic/roxy/issues/410) Test .sjs modules using SJS unit testing
+* [#404](https://github.com/marklogic/roxy/issues/404) Remove orphaned amps at wipe
+* [#397](https://github.com/marklogic/roxy/issues/397) Dynamically suppress error-format if ML8
+* [#395](https://github.com/marklogic/roxy/issues/395) Automatically adjust rewriter based on ml version
+* [#380](https://github.com/marklogic/roxy/issues/380) Ignore replica forests when bootstrapping to standalone
+* [#376](https://github.com/marklogic/roxy/issues/376) Suppressing prompts
+* [#375](https://github.com/marklogic/roxy/issues/375) Create ML8 Roxy release
+* [#374](https://github.com/marklogic/roxy/issues/374) RFE: show app_specific.rb methods in help output
+* [#371](https://github.com/marklogic/roxy/issues/371) Deploy modules not honoring modules root
+* [#366](https://github.com/marklogic/roxy/issues/366) Added bootstrap/wipe for internal replicas
+* [#365](https://github.com/marklogic/roxy/issues/365) Add howto command to show Roxy wiki pages
+* [#364](https://github.com/marklogic/roxy/issues/364) Suppress triggers/schemas in ml-config if eq modules
+* [#362](https://github.com/marklogic/roxy/issues/362) Support a new type of application for those which are neither REST nor MVC
+* [#359](https://github.com/marklogic/roxy/issues/359) Added separate mlcp user/pass properties
+* [#337](https://github.com/marklogic/roxy/pull/337) Adding roxy property to ignore certain files or folders
+* [#335](https://github.com/marklogic/roxy/issues/335) Raise exception to OS level for integratability
+* [#332](https://github.com/marklogic/roxy/issues/332) Make group support work for ML 5
+* [#330](https://github.com/marklogic/roxy/issues/330) Update ml new to select server-version 8 (and drop 4)
+* [#328](https://github.com/marklogic/roxy/issues/328) Add group/server/database settings to help
+* [#313](https://github.com/marklogic/roxy/issues/313) exposed validate command
+* [#312](https://github.com/marklogic/roxy/issues/312) added support for group and host properties, backwards compatible
+* [#231](https://github.com/marklogic/roxy/issues/231) separate deploy modules from rest extensions and transforms enhancement
+* [#223](https://github.com/marklogic/roxy/issues/223) Allow capturing modules for any app-type capture
+* [#78](https://github.com/marklogic/roxy/issues/78) Integrated with MLCP
+
+### Resolved Issues
+* [#428](https://github.com/marklogic/roxy/issues/428) JSON sending in the request body is not being converted into map:map in ML8
+* [#412](https://github.com/marklogic/roxy/issues/412) Update copyright
+* [#405](https://github.com/marklogic/roxy/issues/405) Make sure field-value-searches works at field level
+* [#400](https://github.com/marklogic/roxy/issues/400) Win8.1 ML7 Roxy rest problem
+* [#399](https://github.com/marklogic/roxy/issues/399) ML8 with Roxy on Win8.1
+* [#398](https://github.com/marklogic/roxy/issues/398) Ignore empty settings in version-check
+* [#391](https://github.com/marklogic/roxy/issues/391) Corb doesn't run against ML8
+* [#383](https://github.com/marklogic/roxy/issues/383) Reassigning replicas doesn't spread replicas
+* [#379](https://github.com/marklogic/roxy/issues/379) Replication forests not created once forests exists
+* [#378](https://github.com/marklogic/roxy/issues/378) Forest replication doesn't work unless forests-per-host
+* [#377](https://github.com/marklogic/roxy/issues/377) Deploy rest ignores .sjs extensions
+* [#370](https://github.com/marklogic/roxy/issues/370) deploy modules throws 404 error after upgrade to ML8
+* [#369](https://github.com/marklogic/roxy/issues/369) Set exit code when setup:do-setup fails
+* [#361](https://github.com/marklogic/roxy/issues/361) Is granular replica forest configuration possible?
+* [#355](https://github.com/marklogic/roxy/issues/355) Sample ssl certificate fails
+* [#353](https://github.com/marklogic/roxy/issues/353) Capture configs is unable to capture users and roles when specified by name
+* [#348](https://github.com/marklogic/roxy/issues/348) ml-config.sample.xml causes roxy to not bootstrap app servers in dev branch
+* [#339](https://github.com/marklogic/roxy/issues/339) self-test is failing
+* [#336](https://github.com/marklogic/roxy/issues/336) ML8 doesn't support REST property error-format
+* [#334](https://github.com/marklogic/roxy/issues/334) MLCP path with spaces fails
+* [#333](https://github.com/marklogic/roxy/issues/333) ml script breaks in gitbash (Windows)
+* [#329](https://github.com/marklogic/roxy/issues/329) 'filesystem' db created when when 'filesystem' is specified as the 'modules-db'?
+* [#324](https://github.com/marklogic/roxy/issues/324) Command line unit test (ml local test) fails with 404 not found error
+* [#323](https://github.com/marklogic/roxy/issues/323) Clear content breaks if (some) forests are read-only
+* [#322](https://github.com/marklogic/roxy/issues/322) Bootstrap looses retire status of forest
+* [#317](https://github.com/marklogic/roxy/issues/317) restart command not working when variable are provide from command line
+* [#355](https://github.com/marklogic/roxy/issues/355) Sample ssl certificate fails
+* [#282](https://github.com/marklogic/roxy/issues/282) Orphaned Amps Fix
+* [#280](https://github.com/marklogic/roxy/issues/280) Corb.jar not found
+* [#265](https://github.com/marklogic/roxy/issues/265) Failing tests in untouched hybrid-project
+
 ## 1.7.1
 
 ### New Features

README.mdown

@@ -95,7 +95,7 @@ Download the [ml](https://github.com/marklogic/roxy/raw/master/ml) file
 1. Open a command prompt in the root folder of Roxy.
 2. Run ml init to create sample configuration files.  
   *You must specify the --server-version option with a value of 4, 5, 6 or 7*.  
-  *You must specify the --app-type with a value or rest, hybrid, or mvc*.
+  *You must specify the --app-type with a value or bare, rest, hybrid, or mvc*.
 
   `$ ml init app-name --server-version=7 --app-type=rest`
 3. Modify deploy/build.properties with your application's settings.

deploy/app_specific.rb

@@ -45,3 +45,33 @@ class ServerConfig
   #   # but it can be called without an environment
   # end
 end
+
+#
+# Uncomment, and adjust below code to get help about your app_specific
+# commands included into Roxy help. (ml -h)
+#
+
+#class Help
+#  def self.app_specific
+#    <<-DOC.strip_heredoc
+#
+#      App-specific commands:
+#        example       Installs app-specific alerting
+#    DOC
+#  end
+#
+#  def self.example
+#    <<-DOC.strip_heredoc
+#      Usage: ml {env} example [args] [options]
+#      
+#      Runs a special example task against given environment.
+#      
+#      Arguments:
+#        this    Do this
+#        that    Do that
+#        
+#      Options:
+#        --whatever=value
+#    DOC
+#  end
+#end

deploy/default.properties

@@ -21,6 +21,7 @@ group=Default
 
 #
 # The type of application. Choices are:
+# bare: a bare Roxy app project
 # mvc: a normal, XQuery-based Roxy MVC app
 # rest: an app based on the ML6 REST API
 # hybrid: an app that uses Roxy rewriting and the ML6 REST API
@@ -60,6 +61,7 @@ password=admin
 #
 app-name=roxy
 modules-root=/
+modules-prefix=/
 
 # The role that is given permissions and execute privileges
 app-role=${app-name}-role
@@ -199,8 +201,15 @@ local-server=localhost
 #
 mlcp-home=/usr/local/mlcp
 mlcp-vmargs=-Xmx512m
+mlcp-user=${user}
+mlcp-password=${password}
 
 #
 # Temporary fix to support MarkLogic 8 EA2
 #
 evaler-port=7998
+
+#
+# List of MarkLogic system databases, for replication
+#
+system-dbs=App-Services,Documents,Extensions,Fab,Last-Login,Meters,Modules,Schemas,Security,Triggers

deploy/lib/Help.rb

@@ -1,6 +1,6 @@
 class Help
   def self.usage
-    <<-DOC.strip_heredoc
+    help = <<-DOC.strip_heredoc
 
       Usage:
         ml [ENVIRONMENT] COMMAND [ARGS]
@@ -20,11 +20,12 @@ def self.usage
 
       Bootstrapping commands (with environment):
         bootstrap     Configures your application on the given environment
-        capture       Captures the source code of an existing App Builder application
+        capture       Captures the source code and if applicable the REST configuration of an existing application
         clean         Removes all files from the cpf, modules, or content databases on the given environment
         credentials   Configures user and password for the given environment
         info          Returns settings for the given environment
         restart       Restarts the given environment
+        validate      Compare your ml-config against the given environment
         wipe          Removes your application from the given environment
 
       Deployment/Data commands (with environment):
@@ -33,11 +34,23 @@ def self.usage
         load          Loads a file or folder into the given environment
         mlcp          Runs MLCP against the given environment
         recordloader  Runs RecordLoader against the given environment
+        settings      Lists all supported settings for a given environment
         test          Runs xquery unit tests against the given environment
         xqsync        Runs XQSync against the given environment
+    DOC
+
+    help += app_specific || ''
+
+    help += <<-DOC.strip_heredoc
 
       All commands can be run with -h or --help for more information.
     DOC
+
+    help
+  end
+
+  def self.app_specific
+    #stub
   end
 
   def self.create
@@ -191,14 +204,15 @@ def self.initcpf
 
   def self.restart
     <<-DOC.strip_heredoc
-      Usage: ml {env} restart [group] [options]
+      Usage: ml {env} restart [{groupname}|cluster] [options]
 
       General options:
         -v, [--verbose]  # Verbose output
 
       Restart the MarkLogic process in the given environment on each host in the
       specified group. If no group is specified, restart the MarkLogic process
-      on each host in the group to which the target host belongs.
+      on each host in the group to which the target host belongs. Use 'cluster'
+      to restart all hosts within the cluster to which the target belongs.
     DOC
   end
 
@@ -459,11 +473,17 @@ def self.upgrade
   def self.capture
     <<-DOC.strip_heredoc
       Usage: ml {env} capture --modules-db=[name of modules database]
+		Captures the source for an existing application
+		
+	  modules-db: (required)
+        The modules database of the application.
+	  
+	  ml {env} capture --app-builder=[name of Application Builder-based application]
         Captures the source and REST API configuration for an existing
         Application Builder-based application.
 
-      modules-db: (required)
-        The modules database of the App Builder application.
+      app-builder: (required)
+        The name of the App Builder application.
     DOC
   end
 

deploy/lib/MLClient.rb

@@ -1,5 +1,5 @@
 ###############################################################################
-# Copyright 2012 MarkLogic Corporation
+# Copyright 2012-2015 MarkLogic Corporation
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -21,11 +21,17 @@
 require 'uri'
 
 class MLClient
+  def MLClient.no_prompt=(no_prompt)
+    @@no_prompt = no_prompt
+  end
+
   def initialize(options)
     @ml_username = options[:user_name]
     @ml_password = options[:password].xquery_unsafe
     @logger = options[:logger] || logger
     @request = {}
+
+    @@no_prompt = options[:no_prompt]
   end
 
   def MLClient.logger()
@@ -72,7 +78,7 @@ def build_request_params(url, verb)
     }
   end
 
-  def go(url, verb, headers = {}, params = nil, body = nil)
+  def go(url, verb, headers = {}, params = nil, body = nil, xcc = false)
     logger.debug(%Q{[#{verb.upcase}]\t#{url}})
     password_prompt
     request_params = build_request_params(url, verb)
@@ -91,6 +97,7 @@ def go(url, verb, headers = {}, params = nil, body = nil)
       request_params[:request].body = body
     end
 
+    request_params[:request].use_xcc(xcc)
     response = get_http.request request_params
     response.value
     response
@@ -103,18 +110,34 @@ def url_encode(str)
   end
 
   def prompt(*args)
-    print(*args)
-    gets.strip
+    if @@no_prompt
+      raise ExitException.new("--no-prompt parameter prevents prompting for input")
+    else
+      print(*args)
+      gets.strip
+    end
   end
 
   def password_prompt
+    if (@ml_username == "") then
+      if @@no_prompt
+        raise ExitException.new("--no-prompt parameter prevents prompting for username")
+      else
+        print "Login for admin user: "
+        @ml_username = gets.chomp
+      end
+    end
     if (@ml_password == "") then
-      if STDIN.respond_to?(:noecho)
-      print "Password for admin user: "
-      @ml_password = STDIN.noecho(&:gets).chomp
-      print "\n"
+      if @@no_prompt
+        raise ExitException.new("--no-prompt parameter prevents prompting for password")
       else
-        raise ExitException.new("Upgrade to Ruby >= 1.9 for password prompting on the shell. Or you can set password= in your properties file")
+        if STDIN.respond_to?(:noecho)
+          print "Password for #{@ml_username} user: "
+          @ml_password = STDIN.noecho(&:gets).chomp
+          print "\n"
+        else
+          raise ExitException.new("Upgrade to Ruby >= 1.9 for password prompting on the shell. Or you can set password= in your properties file")
+        end
       end
     end
   end

deploy/lib/RoxyHttp.rb

@@ -32,7 +32,7 @@ def set_path(path)
       @path = path
     end
   end
-
+  
   module HTTPHeader
     @@nonce_count = -1
     CNONCE = Digest::MD5.hexdigest "%x" % (Time.now.to_i + rand(65535))
@@ -318,6 +318,7 @@ def start(request_params)
           @http.ca_file         = ca_file
         end
       end
+
       # open connection
       @http.start
     end

deploy/lib/framework.rb

@@ -1,5 +1,5 @@
 ###############################################################################
-# Copyright 2012 MarkLogic Corporation
+# Copyright 2012-2015 MarkLogic Corporation
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -32,6 +32,8 @@ def initialize(options)
       else
         @template_base = File.expand_path('../templates', __FILE__)
       end
+
+      @no_prompt = options[:no_prompt]
     end
 
     def create
@@ -75,8 +77,12 @@ def create
           if force == false
             view_type = (format != nil && format != "none") ? ((format == "json") ? "a" : "an") + " #{format} view" : "no view"
             print "\nAbout to create a #{view}() function in the #{controller}.xqy controller with #{view_type}. Proceed?\n> "
-            answer = gets().downcase.strip
-            return if answer != "y" && answer != "yes"
+            if @no_prompt
+              raise ExitException.new("--no-prompt parameter prevents prompting for input. Use -f to force.")
+            else
+              answer = gets().downcase.strip
+              return if answer != "y" && answer != "yes"
+            end
           end
 
           create_object('controller', controller, view, controller)