Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Docker image for Silverpeas 6.4 #16998

Merged
merged 2 commits into from
Jun 24, 2024

Conversation

mmoqui
Copy link
Contributor

@mmoqui mmoqui commented Jun 17, 2024

Update silverpeas Docker library to take into account Silverpeas 6.4

@mmoqui mmoqui requested a review from a team as a code owner June 17, 2024 09:41

This comment has been minimized.

@whalelines
Copy link
Contributor

+  if [ -f ${SILVERPEAS_HOME}/configuration/config.properties ]; then
+    echo "The configuration file ${SILVERPEAS_HOME}/configuration/config.properties already exists. Does nothing"
+    return
+  fi

Best to quote the argument to -f (and no need to use curly braces here)

if [ -f "$SILVERPEAS_HOME/configuration/config.properties" ]; then
+# Migrate the JCR from Apache Jackrabbit 2 to Apache Jackrabbit Oak
+# For doing we have to find out where the JCR home directory is located.
+migrate_jcr() {
+  # figure out the data home directory (by default it is into the Silverpeas home directory)
+  data_home=`grep "SILVERPEAS_DATA_HOME=" ${SILVERPEAS_HOME}/configuration/config.properties  | cut -d '=' -f 2 | xargs`

Best to quote here too

data_home=`grep "SILVERPEAS_DATA_HOME=" "$SILVERPEAS_HOME/configuration/config.properties"  | cut -d '=' -f 2 | xargs`
+  # figure out now the JCR home directory (by default it is located into the data home directory)
+  jcr_home=`grep "JCR_HOME[ ]*=" ${SILVERPEAS_HOME}/configuration/config.properties  | cut -d '=' -f 2 | xargs`

This regular expression looks more like a glob pattern. What are you trying to match? Also, quote/brace as above.

+  if [ "Z${jcr_home}" = "Z" ]; then
+    jcr_home="${data_home}/jcr"
+  else    
+    jcr_home=`sed -e "s/SILVERPEAS_DATA_HOME/data_home/g" <<< "${jcr_home}"`

Do you want SILVERPEAS_DATA_HOME replaced with the literal string data_home or the value of the data_home variable?

+    jcr_home=`eval echo -e "${jcr_home}"`

Perhaps the value replacement happens here (SILVERPEAS_DATA_HOME is always preceded by $ in $jcr_home)?

+  jcr_dir=`dirname ${jcr_home}`
+  if [ -d "${jcr_dir}/jackrabbit" ] && [ ! -d "${jcr_dir}/jcr/segmentstore" ]; then
+    echo "Migrate the JCR from Apache Jackrabbit 2 to Apache Jackrabbit Oak..."
+    /opt/oak-migration/oak-migrate.sh "${jcr_dir}/jackrabbit" "${jcr_dir}/jcr"
+    test $? -eq 0 || exit 1
+  fi

Typically it is better to test the exit value of a command directly

if ! /opt/oak-migration/oak-migrate.sh "$jcr_dir/jackrabbit" "$jcr_dir/jcr"; then
  exit 1
fi

@tianon
Copy link
Member

tianon commented Jun 17, 2024

COPY src/oak-migrate.zip /opt/

This is definitely a big eyebrow-raiser - can you elaborate on it? Why commit the .zip to Git? If it's a canonical upstream artifact, shouldn't it be downloaded from somewhere instead or maybe even be part of the silverpeas distribution itself? (If it isn't, I'm struggling to see a strong reason for it to be stored in Git as a zip file. 😅)

+  && ./silverpeas assemble || eval "cat ../log/build-* && exit 1" \

I don't think you need eval here, and probably meant to use { ...; } or ( ... ) instead? Also mixing && and || in a long chain like this is likely to cause problems -- I'd suggest adding set -eux (at the very least set -e) to the RUN block and converting all the && to ; instead, but alternatively, you should be able to get clever with {} or () to make sure the precedence works correctly.

@mmoqui
Copy link
Contributor Author

mmoqui commented Jun 18, 2024

Best to quote the argument to -f (and no need to use curly braces here)

Ok for quoting vars referring a path (in case there are some specific characters like spaces; it shouldn't be happen here).

This regular expression looks more like a glob pattern. What are you trying to match? Also, quote/brace as above.

The goal is to get the value of the JCR_HOME var in the config.properties configuration file. This var refers the path of the JCR home directory for Silverpeas (JCR stands for Java Content Repository, a kind of a document-oriented database structured into a tree).

Do you want SILVERPEAS_DATA_HOME replaced with the literal string data_home or the value of the data_home variable?

The value of the data_home variable. The JCR_HOME value can be made up of another variable, SILVERPEAS_DATA_HOME. The value of latter was fetched previously in the script (in the first lines of the migrate_jcr function). The goal is to expand (id est replace) this var by the value of data_home.

Perhaps the value replacement happens here (SILVERPEAS_DATA_HOME is always preceded by $ in $jcr_home)?

Yes, the variable expansion is done here by the eval statement. (Yes, SILVERPEAS_DATA_HOME is always prefixed by either $ or is wrapped by ${})

This is definitely a big eyebrow-raiser - can you elaborate on it? Why commit the .zip to Git? I

Yes, you're right.

I don't think you need eval here, and probably meant to use { ...; } or ( ... ) instead? [...]

Ok

@whalelines
Copy link
Contributor

Best to quote the argument to -f (and no need to use curly braces here)

Ok for quoting vars referring a path (in case there are some specific characters like spaces; it shouldn't be happen here).

Best to approach this defensively, over time shouldn't can soften to didn't think it would.

jcr_home=`grep "JCR_HOME[ ]*=" ${SILVERPEAS_HOME}/configuration/config.properties  | cut -d '=' -f 2 | xargs`

This regular expression looks more like a glob pattern. What are you trying to match? Also, quote/brace as above.

The goal is to get the value of the JCR_HOME var in the config.properties configuration file.

I looked at the /opt/silverpeas/configuration/sample_config.properties in the current silverpeas:latest image. It had this line:

#JCR_HOME = ${SILVERPEAS_DATA_HOME}/jcr

So it seems the regular expression is using [ ]* to capture zero or more space characters. The square brackets are unnecessary. Also, do you want to replace the value regardless of whether the line is commented out? Perhaps this command is more in line with your intention?

jcr_home=$(grep "^JCR_HOME *=" ${SILVERPEAS_HOME}/configuration/config.properties  | cut -d '=' -f 2 | xargs`

Also, it looks like you are using xargs to strip whitespace from the value since cut will leave the spaces since it it splitting on the equal sign. This may not work the way you expect:

$ echo 'foo = bar ' | cut -d '=' -f 2 | xargs -I % echo X%X
Xbar X

(the above test was run in silverpeas:latest)

@mmoqui
Copy link
Contributor Author

mmoqui commented Jun 19, 2024

Also, do you want to replace the value regardless of whether the line is commented out? Perhaps this command is more in line with your intention?

the config_sample.properties file is a sample configuration file in which all the supported properties are defined with their default value. When such a property isn't declared or redefined in the config.properties file, then the configuration task uses the hard-coded default value of the property. The regexp here is to get the value of a possible declared JCR_HOME property in config.properties. If not defined, then the default value is used.

This may not work the way you expect: [...]

The white spaces between the first character of the value and the = sign are stripped. Any tailing spaces are taken into account because those characters are considered to be part of the value (as it is in Java).

This comment has been minimized.

@whalelines
Copy link
Contributor

+  if [ -f ${SILVERPEAS_HOME}/configuration/config.properties ]; then
+    echo "The configuration file ${SILVERPEAS_HOME}/configuration/config.properties already exists. Does nothing"
+    return
+  fi

The argument to -f is missing double quotes.


[ ]* is still non-canonical, * is.


for f in "${SILVERPEAS_HOME}/log/build-*"; do

Have the asterisk inside the quotes will prevent glob expansion. I think you want

for f in "$SILVERPEAS_HOME"/log/build-*; do

@yosifkit
Copy link
Member

RUN \
...\
  && wget -nc https://www.silverpeas.org/files/oak-migrate.zip \
  && mkdir -p /opt/oak-migration \
  && unzip oak-migrate.zip -d /opt/oak-migration/ \

Any files downloaded should be verified as much as possible (https://github.com/docker-library/official-images/blob/48dadc42dc8aab072c754bb7811f0a63c16499e5/README.md#image-build). Using https and embedding a checksum in the Dockerfile can be used when no signature is published. This will provide a natural build cache bust when the remote artifact changes (and prevent an accidental fetch of a newer artifact when the image is rebuilt for a base image update).

This comment has been minimized.

Copy link

Diff for 859eb2f:
diff --git a/_bashbrew-cat b/_bashbrew-cat
index 9e67d04..4432d21 100644
--- a/_bashbrew-cat
+++ b/_bashbrew-cat
@@ -1,6 +1,10 @@
 Maintainers: Miguel Moquillon <[email protected]> (@mmoqui)
 GitRepo: https://github.com/Silverpeas/docker-silverpeas-prod.git
 
-Tags: 6.3.5, latest
+Tags: 6.3.5
 GitFetch: refs/heads/6.3.x
 GitCommit: 126477050a7890b45fd57eebe471bfdcff20643d
+
+Tags: 6.4, latest
+GitFetch: refs/heads/6.4.x
+GitCommit: c3054e40add18fd254afdd07a2b206be6a4d3530
diff --git a/_bashbrew-list b/_bashbrew-list
index 28e063a..93dc03e 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -1,2 +1,3 @@
 silverpeas:6.3.5
+silverpeas:6.4
 silverpeas:latest
diff --git a/silverpeas_latest/Dockerfile b/silverpeas_6.3.5/Dockerfile
similarity index 100%
copy from silverpeas_latest/Dockerfile
copy to silverpeas_6.3.5/Dockerfile
diff --git a/silverpeas_latest/src/converter.groovy b/silverpeas_6.3.5/src/converter.groovy
similarity index 100%
copy from silverpeas_latest/src/converter.groovy
copy to silverpeas_6.3.5/src/converter.groovy
diff --git a/silverpeas_latest/src/run.sh b/silverpeas_6.3.5/src/run.sh
similarity index 100%
copy from silverpeas_latest/src/run.sh
copy to silverpeas_6.3.5/src/run.sh
diff --git a/silverpeas_latest/src/settings.xml b/silverpeas_6.3.5/src/settings.xml
similarity index 100%
copy from silverpeas_latest/src/settings.xml
copy to silverpeas_6.3.5/src/settings.xml
diff --git a/silverpeas_latest/src/silverpeas.gradle b/silverpeas_6.3.5/src/silverpeas.gradle
similarity index 100%
copy from silverpeas_latest/src/silverpeas.gradle
copy to silverpeas_6.3.5/src/silverpeas.gradle
diff --git a/silverpeas_latest/Dockerfile b/silverpeas_latest/Dockerfile
index 968bc09..194f2dd 100644
--- a/silverpeas_latest/Dockerfile
+++ b/silverpeas_latest/Dockerfile
@@ -1,4 +1,4 @@
-FROM ubuntu:focal
+FROM ubuntu:jammy
 
 MAINTAINER Miguel Moquillon "[email protected]"
 
@@ -11,8 +11,8 @@ ENV TERM=xterm
 # Installation of LibreOffice, ImageMagick, Ghostscript, and then
 # the dependencies required to run SWFTools and PDF2JSON
 RUN apt-get update \
-  && apt-get install -y tzdata \ 
-  && apt-get install -y \
+  && apt-get install -y --no-install-recommends tzdata \ 
+  && apt-get install -y --no-install-recommends \
     apt-utils \
     iputils-ping \
     curl \
@@ -37,13 +37,13 @@ RUN apt-get update \
 
 # Fetch and install SWFTools
 RUN wget -nc https://www.silverpeas.org/files/swftools-bin-0.9.2.zip \
-  && echo 'd40bd091c84bde2872f2733a3c767b3a686c8e8477a3af3a96ef347cf05c5e43 *swftools-bin-0.9.2.zip' | sha256sum - \
+  && echo 'd40bd091c84bde2872f2733a3c767b3a686c8e8477a3af3a96ef347cf05c5e43 swftools-bin-0.9.2.zip' | sha256sum -c --status - \
   && unzip swftools-bin-0.9.2.zip -d / \
   && rm swftools-bin-0.9.2.zip
 
 # Fetch and install PDF2JSON
 RUN wget -nc https://www.silverpeas.org/files/pdf2json-bin-0.68.zip \
-  && echo 'eec849cdd75224f9d44c0999ed1fbe8764a773d8ab0cf7fff4bf922ab81c9f84 *pdf2json-bin-0.68.zip' | sha256sum - \
+  && echo 'eec849cdd75224f9d44c0999ed1fbe8764a773d8ab0cf7fff4bf922ab81c9f84 pdf2json-bin-0.68.zip' | sha256sum -c --status - \
   && unzip pdf2json-bin-0.68.zip -d / \
   && rm pdf2json-bin-0.68.zip
 
@@ -92,11 +92,11 @@ ENV JAVA_HOME /docker-java-home
 ENV SILVERPEAS_HOME /opt/silverpeas
 ENV JBOSS_HOME /opt/wildfly
 
-ENV SILVERPEAS_VERSION=6.3.5
-ENV WILDFLY_VERSION=26.1.1
-LABEL name="Silverpeas 6.3.5" description="Image to install and to run Silverpeas 6.3.5" vendor="Silverpeas" version="6.3.5" build=1
+ENV SILVERPEAS_VERSION=6.4
+ENV WILDFLY_VERSION=26.1.3
+LABEL name="Silverpeas 6.4" description="Image to install and to run Silverpeas 6.4" vendor="Silverpeas" version="6.4" build=1
 
-# Fetch both Silverpeas and Wildfly and unpack them into /opt
+# Fetch both Silverpeas, Wildfly, and the JCR migration script and unpack them into /opt
 RUN wget -nc https://www.silverpeas.org/files/silverpeas-${SILVERPEAS_VERSION}-wildfly${WILDFLY_VERSION%.?.?}.zip \
   && wget -nc https://www.silverpeas.org/files/silverpeas-${SILVERPEAS_VERSION}-wildfly${WILDFLY_VERSION%.?.?}.zip.asc \
   && gpg --keyserver keys.openpgp.org --recv-keys 3F4657EF9C591F2FEA458FEBC19391EB3DF442B6 \
@@ -106,6 +106,11 @@ RUN wget -nc https://www.silverpeas.org/files/silverpeas-${SILVERPEAS_VERSION}-w
   && unzip wildfly-${WILDFLY_VERSION}.Final.zip -d /opt \
   && mv /opt/silverpeas-${SILVERPEAS_VERSION}-wildfly${WILDFLY_VERSION%.?.?} /opt/silverpeas \
   && mv /opt/wildfly-${WILDFLY_VERSION}.Final /opt/wildfly \
+  && wget -nc https://www.silverpeas.org/files/oak-migrate.zip \
+  && echo '02d21f69004f2d9e634e82ec062d94521bd6bc0385d7c0ddf9af261cb63afdbb oak-migrate.zip' | sha256sum -c --status - \
+  && mkdir -p /opt/oak-migration \
+  && unzip oak-migrate.zip -d /opt/oak-migration/ \
+  && chmod +x /opt/oak-migration/oak-migrate.sh \
   && rm *.zip \
   && mkdir -p /root/.m2
 
@@ -124,10 +129,12 @@ COPY src/run.sh /opt/
 COPY src/converter.groovy ${SILVERPEAS_HOME}/configuration/silverpeas/
 
 # Assemble Silverpeas
-RUN sed -i -e "s/SILVERPEAS_VERSION/${SILVERPEAS_VERSION}/g" ${SILVERPEAS_HOME}/bin/silverpeas.gradle \
-  && ./silverpeas construct \
-  && rm ../log/build-* \
-  && touch .install
+RUN set -eux; \
+  sed -i -e "s/SILVERPEAS_VERSION/${SILVERPEAS_VERSION}/g" ${SILVERPEAS_HOME}/bin/silverpeas.gradle; \
+  echo "Construct Silverpeas ${SILVERPEAS_VERSION}"; \
+  ./silverpeas assemble || (cat ../log/build-* && exit 1); \
+  rm ../log/build-*; \
+  touch .install;
 
 #
 # Expose image entries. By default, when running, the container will set up Silverpeas and Wildfly
@@ -141,5 +148,5 @@ EXPOSE 8000 9990
 # the data, the properties and the workflow definitions that are produced in Silverpeas.
 VOLUME ["/opt/silverpeas/log", "/opt/silverpeas/data", "/opt/silverpeas/properties", "/opt/silverpeas/xmlcomponents/workflows"]
 
-# What to execute by default when running the container
+# What to execute by default when running the container.
 CMD ["/opt/run.sh"]
diff --git a/silverpeas_latest/src/run.sh b/silverpeas_latest/src/run.sh
index fe81856..7ab13c0 100755
--- a/silverpeas_latest/src/run.sh
+++ b/silverpeas_latest/src/run.sh
@@ -6,7 +6,14 @@ set -e
 # of Silverpeas
 #
 
+# Creates the Silverpeas global configuration file config.properties from the environment variables
+# set in the Docker image
 pre_install() {
+  if [ -f "${SILVERPEAS_HOME}/configuration/config.properties" ]; then
+    echo "The configuration file ${SILVERPEAS_HOME}/configuration/config.properties already exists. Does nothing"
+    return
+  fi
+
   dbtype=${DB_SERVERTYPE:-POSTGRESQL}
   dbserver=${DB_SERVER:-database}
   dbport=${DB_PORT}
@@ -17,21 +24,26 @@ pre_install() {
   if [ ! "Z${dbpassword}" = "Z" ]; then
     echo "Generate ${SILVERPEAS_HOME}/configuration/config.properties..."
     cat > ${SILVERPEAS_HOME}/configuration/config.properties <<-EOF
+SILVERPEAS_USER_LANGUAGE=en
+SILVERPEAS_CONTENT_LANGUAGES=en
+
 DB_SERVERTYPE = $dbtype
 DB_SERVER = $dbserver
 DB_NAME = $dbname
 DB_USER = $dbuser
 DB_PASSWORD = $dbpassword
 EOF
-    test "Z${dbport}" = "Z" || echo "DB_PORT_$dbtype = $dbport" >> ${SILVERPEAS_HOME}/configuration/config.properties
+    test "Z${dbport}" = "Z" || echo "DB_PORT_$dbtype = $dbport" >> "${SILVERPEAS_HOME}/configuration/config.properties"
   fi
 }
 
+# Start Silverpeas
 start_silverpeas() {
   echo "Start Silverpeas..."
-  exec ${JBOSS_HOME}/bin/standalone.sh -b 0.0.0.0 -c standalone-full.xml
+  exec "${JBOSS_HOME}/bin/standalone.sh" -b 0.0.0.0 -c standalone-full.xml
 }
 
+# Stop Silverpeas
 stop_silverpeas() {
   echo "Stop Silverpeas..."
   ./silverpeas stop
@@ -41,21 +53,52 @@ stop_silverpeas() {
   fi
 }
 
-trap 'stop_silverpeas' SIGTERM
+# Migrate the JCR from Apache Jackrabbit 2 to Apache Jackrabbit Oak
+# For doing we have to find out where the JCR home directory is located.
+migrate_jcr() {
+  # figure out the data home directory (by default it is into the Silverpeas home directory)
+  data_home=`grep "SILVERPEAS_DATA_HOME[ ]*=" "${SILVERPEAS_HOME}/configuration/config.properties"  | cut -d '=' -f 2 | xargs`
+  if [ "Z${data_home}" = "Z" ]; then
+    data_home="${SILVERPEAS_HOME}/data"
+  else
+    data_home=`sed -e "s/{env./{/g" <<< "${data_home}"`
+    data_home=`eval echo -e "${data_home}"`
+  fi
+
+  # figure out now the JCR home directory (by default it is located into the data home directory)
+  jcr_home=`grep "JCR_HOME[ ]*=" "${SILVERPEAS_HOME}/configuration/config.properties"  | cut -d '=' -f 2 | xargs`
+  if [ "Z${jcr_home}" = "Z" ]; then
+    jcr_home="${data_home}/jcr"
+  else    
+    jcr_home=`sed -e "s/SILVERPEAS_DATA_HOME/data_home/g" <<< "${jcr_home}"`
+    jcr_home=`eval echo -e "${jcr_home}"`
+  fi
+
+  jcr_dir=`dirname ${jcr_home}`
+  if [ -d "${jcr_dir}/jackrabbit" ] && [ ! -d "${jcr_dir}/jcr/segmentstore" ]; then
+    echo "Migrate the JCR from Apache Jackrabbit 2 to Apache Jackrabbit Oak..."
+    if ! /opt/oak-migration/oak-migrate.sh "$jcr_dir/jackrabbit" "$jcr_dir/jcr"; then
+      exit 1
+    fi
+  fi
+}
+
+trap 'stop_silverpeas' SIGTERM SIGKILL SIGQUIT
 
-if [ -f ${SILVERPEAS_HOME}/bin/.install ]; then
+if [ -f "${SILVERPEAS_HOME}/bin/.install" ]; then
   pre_install
-  if [ -f ${SILVERPEAS_HOME}/configuration/config.properties ]; then
-    echo "First start: set up Silverpeas ..."
+  if [ -f "${SILVERPEAS_HOME}/configuration/config.properties" ]; then
+    echo "First start: set up Silverpeas..."
     set +e
     ./silverpeas install
     if [ $? -eq 0 ]; then
-       rm ${SILVERPEAS_HOME}/bin/.install
+       rm "${SILVERPEAS_HOME}/bin/.install"
+       migrate_jcr
        test ${PING_ON} = 1 && curl -k -i --head https://www.silverpeas.org/ping
     else
       echo "Error while setting up Silverpeas"
       echo
-      for f in ${SILVERPEAS_HOME}/log/build-*; do
+      for f in "${SILVERPEAS_HOME}"/log/build-*; do
         cat $f
       done
       exit 1
@@ -67,7 +110,7 @@ if [ -f ${SILVERPEAS_HOME}/bin/.install ]; then
   fi
 fi
 
-if [ -f ${SILVERPEAS_HOME}/configuration/config.properties ] && [ ! -e ${SILVERPEAS_HOME}/bin/.install ]; then
+if [ -f "${SILVERPEAS_HOME}/configuration/config.properties" ] && [ ! -e "${SILVERPEAS_HOME}/bin/.install" ]; then
   start_silverpeas
 else
   echo "A failure has occurred in the setting up of Silverpeas! No start!"

Relevant Maintainers:

Copy link
Member

@tianon tianon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those eval echo -e "${data_home}" style variable assignments still make me nervous, but in practice they're probably going to be mostly OK unless someone's config file has some really unexpected content/structure.

(The trap also only applies during the runtime of the startup script, which is correct but worth noting.)

@tianon tianon merged commit 50336c7 into docker-library:master Jun 24, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants