Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.10 #1144

Merged
merged 31 commits into from
Nov 10, 2023
Merged

0.10 #1144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6293d8a
Correct options descriptions: add missing start word
mexicarne Oct 4, 2022
6757a91
docker restapi set to slim
BlackOrder Dec 19, 2022
d6f5fc2
python with jemalloc
BlackOrder Dec 19, 2022
3e7e370
cleanup
BlackOrder Dec 19, 2022
39ee472
dynamic sub-services host/port
BlackOrder Dec 19, 2022
9981e45
if path is url, use urllib.request to get
BlackOrder Jan 1, 2023
1fd3529
InvalidScriptSyntax message inclue url example
BlackOrder Jan 1, 2023
987be21
handle only http/s, ftp, and file
BlackOrder Jan 2, 2023
b620681
add note to docs about remote path
BlackOrder Jan 2, 2023
8372cc5
reversed error message
BlackOrder Jan 2, 2023
8c26145
Merge pull request #1101 from BlackOrder/docker-dynamic-subservices
farirat Jan 10, 2023
9321573
Merge branch '0.10' into docker-restapi-jemalloc
farirat Jan 10, 2023
798ac30
Merge pull request #1100 from BlackOrder/docker-restapi-jemalloc
farirat Jan 11, 2023
03b6a4c
detect path type with urlparse
BlackOrder Jan 12, 2023
3b7433b
cast logprivacy to boolean
BlackOrder Dec 26, 2022
dc0c5c7
Quickfix for setting default values for multipart messages.
Kisuke-CZE Oct 20, 2023
ff9424a
Adjust logging. Log message DLR, SRC_ADDR and DST_ADDR according to l…
Kisuke-CZE Oct 24, 2023
58f8993
Adjust logging. Log message DLR, SRC_ADDR and DST_ADDR according to l…
Kisuke-CZE Oct 24, 2023
7fff3f2
Revert "Adjust logging. Log message DLR, SRC_ADDR and DST_ADDR accord…
Kisuke-CZE Oct 24, 2023
e81a3c1
reuse _pdu for clarity
farirat Nov 7, 2023
08f965f
version bump
farirat Nov 7, 2023
3be523c
Merge pull request #1141 from Kisuke-CZE/fix-multipart-logging
farirat Nov 7, 2023
ea08012
Merge pull request #1105 from BlackOrder/logprivacy
farirat Nov 7, 2023
c9e2eeb
Merge pull request #1140 from Kisuke-CZE/master
farirat Nov 7, 2023
7028913
Merge pull request #1077 from mexicarne/correct-descriptions
farirat Nov 7, 2023
d331036
Merge branch '0.10' into cli_interceptors_accepts_url
farirat Nov 7, 2023
b4f516e
avoiding imports from inside code for better readability and to allow…
farirat Nov 7, 2023
8a41ef3
avoiding imports from inside code for better readability and to allow…
farirat Nov 7, 2023
f74cc31
Merge pull request #1102 from BlackOrder/cli_interceptors_accepts_url
farirat Nov 7, 2023
ef4ccdd
Merge branch '0.10' into 0.10.14
farirat Nov 7, 2023
b48de91
Merge pull request #1143 from jookies/0.10.14
farirat Nov 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 24 additions & 18 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,14 @@ RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Run python with jemalloc
ENV LD_PRELOAD /usr/lib/x86_64-linux-gnu/libjemalloc.so.2

# Install Jasmin SMS gateway
RUN mkdir -p /etc/jasmin/resource \
/etc/jasmin/store \
/var/log/jasmin \
&& chown jasmin:jasmin /etc/jasmin/store \
/var/log/jasmin
ENV ROOT_PATH /
ENV CONFIG_PATH /etc/jasmin
ENV RESOURCE_PATH /etc/jasmin/resource
ENV STORE_PATH /etc/jasmin/store
ENV LOG_PATH /var/log/jasmin

RUN mkdir -p ${RESOURCE_PATH} ${STORE_PATH} ${LOG_PATH}
RUN chown jasmin:jasmin ${CONFIG_PATH} ${RESOURCE_PATH} ${STORE_PATH} ${LOG_PATH}

WORKDIR /build

Expand All @@ -35,21 +37,25 @@ RUN pip install .

ENV UNICODEMAP_JP unicode-ascii

ENV ROOT_PATH /
ENV CONFIG_PATH /etc/jasmin
ENV RESOURCE_PATH /etc/jasmin/resource
ENV STORE_PATH /etc/jasmin/store
ENV LOG_PATH /var/log/jasmin

COPY misc/config/*.cfg ${CONFIG_PATH}/
COPY misc/config/resource ${RESOURCE_PATH}

WORKDIR /usr/jasmin

# Change binding host for jcli, redis, and amqp
RUN sed -i '/\[jcli\]/a bind=0.0.0.0' /etc/jasmin/jasmin.cfg
RUN sed -i '/\[redis-client\]/a host=redis' /etc/jasmin/jasmin.cfg
RUN sed -i '/\[amqp-broker\]/a host=rabbitmq' /etc/jasmin/jasmin.cfg
WORKDIR /etc/jasmin

# Default Redis and RabbitMQ connections
ENV AMQP_BROKER_HOST 'rabbitmq'
ENV AMQP_BROKER_PORT 5672
ENV REDIS_CLIENT_HOST 'redis'
ENV REDIS_CLIENT_PORT 6379

# Change binding host for jcli
RUN sed -i '/\[jcli\]/a bind=0.0.0.0' ${CONFIG_PATH}/jasmin.cfg
# Change binding port for redis, and amqp
RUN sed -i "/\[redis-client\]/a port=$REDIS_CLIENT_PORT" ${CONFIG_PATH}/jasmin.cfg
RUN sed -i "/\[amqp-broker\]/a port=$AMQP_BROKER_PORT" ${CONFIG_PATH}/jasmin.cfg
# Change binding host for redis, and amqp
RUN sed -i "/\[redis-client\]/a host=$REDIS_CLIENT_HOST" ${CONFIG_PATH}/jasmin.cfg
RUN sed -i "/\[amqp-broker\]/a host=$AMQP_BROKER_HOST" ${CONFIG_PATH}/jasmin.cfg

EXPOSE 2775 8990 1401
VOLUME ["/var/log/jasmin", "/etc/jasmin", "/etc/jasmin/store"]
Expand Down
36 changes: 25 additions & 11 deletions docker/Dockerfile.restapi
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3-buster
FROM python:3.11-slim-bullseye

MAINTAINER Jookies LTD <[email protected]>

Expand All @@ -9,22 +9,24 @@ RUN groupadd -r jasmin && useradd -r -g jasmin jasmin
RUN apt-get update && apt-get install -y \
libffi-dev \
libssl-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Run python with jemalloc
# More on this:
# - https://zapier.com/engineering/celery-python-jemalloc/
# - https://paste.pics/581cc286226407ab0be400b94951a7d9
libjemalloc2

# For RestAPI MODE
ENV RESTAPI_MODE 0
ENV RESTAPI_OLD_HTTP_HOST '127.0.0.1'
RUN apt-get clean && rm -rf /var/lib/apt/lists/*

ENV ENABLE_PUBLISH_SUBMIT_SM_RESP 0
# Run python with jemalloc
ENV LD_PRELOAD /usr/lib/x86_64-linux-gnu/libjemalloc.so.2

ENV ROOT_PATH /
ENV CONFIG_PATH /etc/jasmin
ENV RESOURCE_PATH /etc/jasmin/resource
ENV STORE_PATH /etc/jasmin/store
ENV RESOURCE_PATH ${CONFIG_PATH}/resource
ENV STORE_PATH ${CONFIG_PATH}/store
ENV LOG_PATH /var/log/jasmin

RUN mkdir -p ${CONFIG_PATH} ${RESOURCE_PATH} ${STORE_PATH} ${LOG_PATH}
RUN mkdir -p ${RESOURCE_PATH} ${STORE_PATH} ${LOG_PATH}
RUN chown jasmin:jasmin ${CONFIG_PATH} ${RESOURCE_PATH} ${STORE_PATH} ${LOG_PATH}

WORKDIR /build
Expand All @@ -39,10 +41,22 @@ RUN pip install gunicorn
ENV UNICODEMAP_JP unicode-ascii

COPY misc/config/*.cfg ${CONFIG_PATH}/
COPY misc/config/resource/*.xml ${RESOURCE_PATH}/
COPY misc/config/resource ${RESOURCE_PATH}

WORKDIR /etc/jasmin

# Default Redis and RabbitMQ connections
ENV AMQP_BROKER_HOST 'rabbitmq'
ENV AMQP_BROKER_PORT 5672
ENV REDIS_CLIENT_HOST 'redis'
ENV REDIS_CLIENT_PORT 6379

# For RestAPI MODE
ENV RESTAPI_MODE 0
ENV RESTAPI_OLD_HTTP_HOST '127.0.0.1'

ENV ENABLE_PUBLISH_SUBMIT_SM_RESP 0

# Change binding host for jcli
RUN sed -i '/\[jcli\]/a bind=0.0.0.0' ${CONFIG_PATH}/jasmin.cfg
# Change binding port for redis, and amqp
Expand Down
10 changes: 10 additions & 0 deletions docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
#!/bin/bash
set -e

# Change binding host:port for redis, and amqp
sed -i "/\[redis-client\]/,/host=/ s/host=.*/host=$REDIS_CLIENT_HOST/" ${CONFIG_PATH}/jasmin.cfg
sed -i "/\[redis-client\]/,/port=/ s/port=.*/port=$REDIS_CLIENT_PORT/" ${CONFIG_PATH}/jasmin.cfg
sed -i "/\[amqp-broker\]/,/host=/ s/host=.*/host=$AMQP_BROKER_HOST/" ${CONFIG_PATH}/jasmin.cfg
sed -i "/\[amqp-broker\]/,/port=/ s/port=.*/port=$AMQP_BROKER_PORT/" ${CONFIG_PATH}/jasmin.cfg

echo 'Cleaning lock files'
rm -f /tmp/*.lock


if [ "$2" = "--enable-interceptor-client" ]; then
echo 'Starting interceptord'
interceptord.py &
Expand Down
12 changes: 6 additions & 6 deletions docker/restapi-docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
set -e

# Change binding host:port for redis, and amqp
sed -i "/\[redis-client\]/,/host=/ s/host=.*/host=$REDIS_CLIENT_HOST/" /etc/jasmin/jasmin.cfg
sed -i "/\[redis-client\]/,/port=/ s/port=.*/port=$REDIS_CLIENT_PORT/" /etc/jasmin/jasmin.cfg
sed -i "/\[amqp-broker\]/,/host=/ s/host=.*/host=$AMQP_BROKER_HOST/" /etc/jasmin/jasmin.cfg
sed -i "/\[amqp-broker\]/,/port=/ s/port=.*/port=$AMQP_BROKER_PORT/" /etc/jasmin/jasmin.cfg
sed -i "/\[redis-client\]/,/host=/ s/host=.*/host=$REDIS_CLIENT_HOST/" ${CONFIG_PATH}/jasmin.cfg
sed -i "/\[redis-client\]/,/port=/ s/port=.*/port=$REDIS_CLIENT_PORT/" ${CONFIG_PATH}/jasmin.cfg
sed -i "/\[amqp-broker\]/,/host=/ s/host=.*/host=$AMQP_BROKER_HOST/" ${CONFIG_PATH}/jasmin.cfg
sed -i "/\[amqp-broker\]/,/port=/ s/port=.*/port=$AMQP_BROKER_PORT/" ${CONFIG_PATH}/jasmin.cfg

echo 'Cleaning lock files'
rm -f /tmp/*.lock
Expand All @@ -23,11 +23,11 @@ else
if [ "$ENABLE_PUBLISH_SUBMIT_SM_RESP" = 1 ]; then
# Enable publish_submit_sm_resp
echo 'Enabling publish_submit_sm_resp'
sed -i "s/.*publish_submit_sm_resp\s*=.*/publish_submit_sm_resp=True/g" /etc/jasmin/jasmin.cfg
sed -i "s/.*publish_submit_sm_resp\s*=.*/publish_submit_sm_resp=True/g" ${CONFIG_PATH}/jasmin.cfg
else
# Disable publish_submit_sm_resp
echo 'Disabling publish_submit_sm_resp'
sed -i "s/.*publish_submit_sm_resp\s*=.*/publish_submit_sm_resp=False/g" /etc/jasmin/jasmin.cfg
sed -i "s/.*publish_submit_sm_resp\s*=.*/publish_submit_sm_resp=False/g" ${CONFIG_PATH}/jasmin.cfg
fi

if [ "$2" = "--enable-interceptor-client" ]; then
Expand Down
2 changes: 1 addition & 1 deletion jasmin/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

MAJOR = 0
MINOR = 10
PATCH = 13
PATCH = 14
META = ''


Expand Down
6 changes: 3 additions & 3 deletions jasmin/bin/jasmind.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ class Options(usage.Options):
['enable-dlr-thrower', None, 'Enable DLR Thrower service (not recommended: start the dlrd daemon instead)'],
['enable-dlr-lookup', None, 'Enable DLR Lookup service (not recommended: start the dlrlookupd daemon instead)'],
# @TODO: deliver-thrower must be executed as a standalone process, just like dlr-thrower
['disable-deliver-thrower', None, 'Do not DeliverSm Thrower service'],
['disable-http-api', None, 'Do not HTTP API'],
['disable-jcli', None, 'Do not jCli console'],
['disable-deliver-thrower', None, 'Do not start DeliverSm Thrower service'],
['disable-http-api', None, 'Do not start HTTP API'],
['disable-jcli', None, 'Do not start jCli console'],
['enable-interceptor-client', None, 'Start Interceptor client'],
]

Expand Down
12 changes: 7 additions & 5 deletions jasmin/managers/listeners.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,10 +291,12 @@ def submit_sm_resp_event(self, r, amqpMessage):
splitMethod = 'sar'
elif UDHI_INDICATOR_SET and r.request.params['short_message'][:3] == b'\x05\x00\x03':
splitMethod = 'udh'

# create _pdu before splitting, so we can use it in both branches (logging the message according the last part of message)
_pdu = r.request

# Concatenate short_message
if splitMethod is not None:
_pdu = r.request
if splitMethod == 'sar':
short_message = _pdu.params['short_message']
else:
Expand All @@ -311,7 +313,7 @@ def submit_sm_resp_event(self, r, amqpMessage):
if submit_sm_resp_bill is not None and submit_sm_resp_bill.getTotalAmounts() > 0:
total_bill_amount += submit_sm_resp_bill.getTotalAmounts()
else:
short_message = r.request.params['short_message']
short_message = _pdu.params['short_message']

# Do not log text for privacy reasons
# Added in #691
Expand All @@ -328,12 +330,12 @@ def submit_sm_resp_event(self, r, amqpMessage):
r.response.params['message_id'],
r.response.status,
amqpMessage.content.properties['priority'],
r.request.params['registered_delivery'].receipt,
_pdu.params['registered_delivery'].receipt,
'none' if ('headers' not in amqpMessage.content.properties or
'expiration' not in amqpMessage.content.properties['headers'])
else amqpMessage.content.properties['headers']['expiration'],
r.request.params['source_addr'],
r.request.params['destination_addr'],
_pdu.params['source_addr'],
_pdu.params['destination_addr'],
logged_content)
else:
# Message must be retried ?
Expand Down
18 changes: 15 additions & 3 deletions jasmin/protocols/cli/mointerceptorm.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import re
import inspect
import pickle
import urllib.parse, urllib.request, urllib.error
from jasmin.protocols.cli.managers import PersistableManager, Session
from jasmin.protocols.cli.filtersm import MOFILTERS
from jasmin.routing.jasminApi import MOInterceptorScript
Expand Down Expand Up @@ -133,9 +134,20 @@ def parse_args_and_call_with_instance(self, *args, **kwargs):
stype, script_path = validate_typed_script(arg)

if stype == 'python3':
# Open file and get its content
with open(script_path, 'r') as content_file:
pyCode = content_file.read()
pathscheme = urllib.parse.urlparse(script_path).scheme
if pathscheme == '':
# Open file and get its content
with open(script_path, 'r') as content_file:
pyCode = content_file.read()
elif pathscheme in ['https', 'http', 'ftp', 'file']:
try:
with urllib.request.urlopen(script_path) as content_file:
pyCode = content_file.read().decode('utf-8')
except urllib.error.URLError as e:
# Handle errors that may occur while reading the file from a URL
return self.protocol.sendData('[URL]: %s' % str(e))
else:
raise NotImplementedError("Not implemented yet !")

# Test compilation of the script
compile(pyCode, '', 'exec')
Expand Down
18 changes: 15 additions & 3 deletions jasmin/protocols/cli/mtinterceptorm.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import re
import inspect
import pickle
import urllib.parse, urllib.request, urllib.error
from jasmin.protocols.cli.managers import PersistableManager, Session
from jasmin.protocols.cli.filtersm import MTFILTERS
from jasmin.routing.jasminApi import MTInterceptorScript
Expand Down Expand Up @@ -133,9 +134,20 @@ def parse_args_and_call_with_instance(self, *args, **kwargs):
stype, script_path = validate_typed_script(arg)

if stype == 'python3':
# Open file and get its content
with open(script_path, 'r') as content_file:
pyCode = content_file.read()
pathscheme = urllib.parse.urlparse(script_path).scheme
if pathscheme == '':
# Open file and get its content
with open(script_path, 'r') as content_file:
pyCode = content_file.read()
elif pathscheme in ['https', 'http', 'ftp', 'file']:
try:
with urllib.request.urlopen(script_path) as content_file:
pyCode = content_file.read().decode('utf-8')
except urllib.error.URLError as e:
# Handle errors that may occur while reading the file from a URL
return self.protocol.sendData('[URL]: %s' % str(e))
else:
raise NotImplementedError("Not implemented yet !")

# Test compilation of the script
compile(pyCode, '', 'exec')
Expand Down
2 changes: 1 addition & 1 deletion jasmin/protocols/cli/smppccm.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def castInputToBuiltInType(key, value):
return replace_if_present_flap_value_map[value]
elif key == 'priority':
return priority_flag_value_map[value]
elif key in ['con_fail_retry', 'con_loss_retry', 'ssl']:
elif key in ['con_fail_retry', 'con_loss_retry', 'ssl', 'logprivacy']:
if value == 'yes':
return True
elif value == 'no':
Expand Down
7 changes: 7 additions & 0 deletions jasmin/protocols/http/endpoints/send.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,13 @@ def route_routable(self, updated_request):

# Update SubmitSmPDU by default values from user MtMessagingCredential
SubmitSmPDU = v.updatePDUWithUserDefaults(SubmitSmPDU)

# Force same default values on subPDU while multipart
_pdu = SubmitSmPDU
while hasattr(_pdu, 'nextPdu'):
_pdu = _pdu.nextPdu
_pdu = v.updatePDUWithUserDefaults(_pdu)


# Prepare for interception then routing
routedConnector = None # init
Expand Down
6 changes: 6 additions & 0 deletions jasmin/protocols/smpp/factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,12 @@ def submit_sm_event_interceptor(self, system_id, *args):

# Update SubmitSmPDU by default values from user MtMessagingCredential
SubmitSmPDU = v.updatePDUWithUserDefaults(SubmitSmPDU)

# Force same default values on subPDU while multipart
_pdu = SubmitSmPDU
while hasattr(_pdu, 'nextPdu'):
_pdu = _pdu.nextPdu
_pdu = v.updatePDUWithUserDefaults(_pdu)

if self.RouterPB is None:
self.log.error('(submit_sm_event_interceptor/%s) RouterPB not set: submit_sm will not be routed',
Expand Down
4 changes: 2 additions & 2 deletions misc/doc/sources/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

# General information about the project.
project = 'Jasmin SMS Gateway'
copyright = '2014-2022, Jasmin'
copyright = '2014-2023, Jasmin'

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
Expand All @@ -38,7 +38,7 @@
# The short X.Y version.
version = "0.10"
# The full version, including alpha/beta/rc tags.
release = "0.10.13"
release = "0.10.14"

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down
6 changes: 6 additions & 0 deletions misc/doc/sources/management/jcli/modules.rst
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,12 @@ Here's an example of adding a **DefaultInterceptor** to a python script::

.. note:: As of now, only **python3** script is permitted.

.. note:: The path to the script can be any of the fallowing:

* **python3(/path/to/script.py)** or **python3(file://path/to/script.py)**: The path must be absolute, relative path is not supported
* **python3(https://example.com/path/to/script.py)**: The script is a remote python3 script. The script will be
downloaded and copied to Jasmin core. Accepts http, https, and ftp protocols.

.. note:: Pay attention that the given script is copied to Jasmin core, do not expect Jasmin to refresh the script
code when you update it, you'll need to redefine the *mointerceptor* rule again so Jasmin will refresh the script.

Expand Down
2 changes: 1 addition & 1 deletion nfpm.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: "jasmin-sms-gateway"
arch: "amd64"
platform: "linux"
version: "v0.10.13"
version: "v0.10.14"
section: "default"
priority: "extra"
maintainer: "Jookies LTD <[email protected]>"
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ def parse_requirements(filename):

setup(
name="jasmin",
version='0.10.13',
version='0.10.14',
author="Jookies LTD",
author_email="[email protected]",
url="https://www.jasminsms.com",
Expand Down
Loading