Release notes for RunMyJobs releases 9.2.9.0 through to 9.2.9.1

Before: When creating a JDBC process definition, with a parameter JDBCEnableParameterSubstitution, then a constraint was automatically created for this parameter. If the constraint definition did not exist, it was created. When this process definition is then exported and imported into another environment, where this constraint definition does not yet exist, the import could fail. A workaround is to create manually a temporary JDBC process definition with a parameter JDBCEnableParameterSubstitution, as this will create the constraint definition. After that the temporary JDBC process definition can be dropped again.

After: Now, the constraint definition is created at startup. Therefore the constraint definition is available when such a process definition is imported. 

Before: Since 9.2.8.0 and in very specific circumstances, it is possible that a process could be scheduled a minute late. E.g. when a chain call of a chain is put on hold, immediately after the chain is submitted to run immediately, then there is a small chance that the chain gets scheduled a minute late.

After: this timing issue is fixed.

Before*:* HTTP 30x redirection messages were incorrectly logged under debug logging as failed responses:
"HTTP server failed response with response code 302"
Also, no distinction was made between HTTP and HTTPS calls in the debug logs.

After*:* 30x messages are not reported as failed responses anymore. Instead are they reported as:
"HTTP server responded with URL redirection code 302"
Also, a distinction is made in the log messages between HTTP and HTTPS.

Before: It was possible to duplicate System_FTP_mget_Parallel and System_FTP_mput_Parallel, even though nothing could be changed on the resulting duplicate.

After: To prevent confusion, it is no longer possible to duplicate these definitions.

Before: Calling getCurrentValueString() on a Date or Number parameter would cause a ClassCastException.

After: Calling getCurrentValueString() on a Date or Number parameter will return the same value as getInValueString() or getOutValueString depending on whether the parameter is an IN parameter or not, respectively. This makes is consistent with the other getCurrentValue*() methods.

Before: The information of components that are initialised by lifecycle manager was exposed.

After: Do not expose the information of components that are initialised by lifecycle manager.

Before: Under some circumstances it was possible for a resilient process to finish before the system had finished starting up. If this happened, and a second process was waiting for the first process, then a delay of up to one hour could occur before the completion of the process was noticed by the waiting process.

After: This has been fixed so that the waiter will always immediately see the completion of the child process.

Before: When selecting a queue provider for read, the internal routing table would be recalculated, even though the queue provider had not changed. This could occur when setting a field to have the same value that it already has.

After: The internal routing table is only recalculated if a change to the routing information occurs. This avoids unnecessary work from being done.

Before: Under some circumstances, loading translations for the user interface could be drastically slowed down.

After: Loading of translations for the user interface is made faster in all cases.

Before: Default filter "All closed queues" shows queues which are held, which is confusing.

After: The filter has been renamed to: "All Held Queues".

Before: After filling in the responsibility it could take a while that you see the next field filled. The reason is that all fields are checked to pre-fill default data.

After: It is possible to influence the behavior pre-filling mapped parameters - keep the recent behavior and prefill all mapped parameters, only pre-fill related mandatory mapped parameters or no pre-fill

Before: When using REL expressions to pass in values in OraApps job parameters this could fail when the job is used in a chain.

After: The REL expression is evaluated before passing the value to the job constraint for checking.

Before: PeopleSoft jobs can't run when the PeopleSoft system is created in any partition other than GLOBAL.

After: PeopleSoft jobs can now run when the PeopleSoft system is created in any partition.

Before: When starting PeopleSoft PSJob processes, only generic parameters could be set. You could not set any specific parameter value for a PSJob subitem.

After: When importing PSJob definition from PeopleSoft the subitem parameters are imported and shown on separate tabs. Now, for all these processes that are run by the PSJob definition parameters can be set individually.

Before: It was only possible to use 1 distribution id when submitting a job to PeopleSoft.

After: You can now use multiple distribution ids separated by either a , (comma) or ; (semicolon).

Before: Replacement expressions are not working with empty strings in SOAP definitions if variable value is null.

After: Replacement expressions are also applied on empty strings when the variable is null.

NOTE: A replacement expression for a variable that doesn't exist will be replaced with the original expression. For example, if your process has a parameter P1 with no value, a parameter P2 with the value test< then the replacement:

<body>
  <P1>${P1}</P1>
  <P2>${P2:xml}</P2>
  <P3>${P3}</P3>
  <empty>${}</empty>
</body>

will result in:

<body>
  <P1></P1>
  <P2>test&lt;</P2>
  <P3>${P3}</P3>
  <empty>${}</empty>
</body>

Before: An NullPointerException could be thrown when we are not able to find a correct credential for the basic authentication.

After: When we do not find a correct credential we will now set the return code of process to be the return code of the SOAP request (e.g. 401)

Before: JDBC jobs running against a MariaDB database failed with a java.sql.SQLSyntaxErrorException: invalid callable syntax. must be like

After: JDBC jobs can now execute SQL statements against a MariaDB database.

Before: It was not possible to apply a base64 encoding rule to a parameter replacement.

After: It is now possible to encode a parameter replacement as base64 using the base64 rule. Example: ${parameter_name:base64}. This rule can also be chained with other escaping rules, for example ${parameter_name:xml:base64}.

Before: Before version 9.2.8, parameter values in HTTP/SOAP requests are not escaped. From version 9.2.8 onwards, parameters are escaped with XML.

After: You now can specify how parameter values should be escaped. Without specifying an escaping rule, there is no escaping of the parameter value.

There are 4 options to escape a parameter value:

1. xml – escape using XML 1.1 character escaping rules, the result will also use character entities for any non-ASCII character
2. json - escape the parameter so that it can be included inside of a JSON string
3. url - escape using URL escaping, assuming a UTF-8 character set
4. html - escape using HTML character escaping rules

To specify an escape rule you need to add an option on the parameter substitution, e.g. ${parameter_name:xml}.

You can specify multiple escape rules on 1 parameter by appending them after each other. They will be handled in sequence of specifying, e.g. ${parameter_name:xml:json} will first escape using XML entities, and then escape that result using json, so an input of ' would become \' whereas ${parameter_name:json:xml} would convert a ' into \', as it would first convert into JSON ( \' ), and then convert that using XML entities.

Before: Mail process definitions failed when sending to multiple recipients.

After: Mail process definitions can send to multiple recipients, separated by either , or ;.

Before: The PeopleSoft connector reads the WSDL in to determine which SOAP calls can be made. The connector was based on reading WSDL 1.1.

After: The PeopleSoft connector can now read WSDL 1.1 and 2.0 files to determine the SOAP calls that can be used.

Before: The native library files are not accessible when running as a Windows Service.

After: The appropriate directory is now on the java.library.path when running as a Windows Service.

Before: The version of the Eclipse plugin was not increased, causing Eclipse to ignore the new version.

After: The version has been increased, allowing Eclipse to download the version.

Before: External database authentication extension could become very slow and/or run out of memory with a large amount of users and roles.

After: The external database authentication extension has been rewritten and now reacts fast as well uses minimal server memory.

Before: Under some circumstances, if there were a lot of updates, Data Transformer jobs were not able to find their configuration.

After: This has been fixed.

Before: Extension points could hold on to class loaders even if the extension point was no longer used.

After: The class loader is no longer referenced when not in use.

Before: Using a connection pool with AS400 connection could fail when the secure gateway fails to connect. The connections were not removed from the connection pool when the connection was not established.

After: When the connection that is used is returned to the connection pool we check if it is still connected. If so we return the connection, if not we remove the connection from the pool

Before:
Catalog interface for downloading extensions was not available as a separate download.

After:
Catalog interface (catalog-admin.car) is now available in the Software 'Download CAR Files' section of the UI.

Before: A SOAPRequest call called within RedwoodScript could only be used to produce a SOAP call. However previously it could also be used to produce a HTTP call.

After: Only when the appropriate values are set  (e.g. SOAPAction, Accept etc.) these values are set on the SOAPRequest. It means that when not set it can be used as a HTTP request.

Before: Transport requests via CTS+ were always created with RFC user as owner

After: Transport request via CTS+ can be created for the other user with new job definition parameter SAP user.

Before: it was not possible to create unique custom indexes on custom tables.

After: it is possible to create unique custom indexes on custom tables.

Before: function module BAPI_XBP_GET_APPLICATION_RC has always been called when retrieving joblog
After: function module BAPI_XBP_GET_APPLICATION_RC will be called only if FL_SHOWLOG is set when retrieving joblog

Before: Managing PI channels was not possible due to the jobs failing with an XML parse exception.

After: It is possible to manage PI channels.

Before:

• When deleting a Process Definition from the its details page, an error reporting a ClassCastException would show.
• When deleting a Process Definition related object (Chain, Report, ...) from its overview page, the object would still be visible in the overview with a name prefixed by "System_Internal_" until the page was refreshed.
• When deleting a Process Definition or related object from its details page, the object would still be visible in the overview with a name prefixed by "System_Internal_" until the page was refreshed.
• When deleting a Process Definition or related object, the object would still temporarily be visible using the "show hidden objects" filter.

After: This is all no longer the case, the ClassCastException is no longer shown, objects get removed from the overview and wont show when using the "show hidden objects" filter.

Before: When an error occurs during runtime in the jobworker the errormessage/stacktrace overwrites the log file.

After: We append the message/stacktrace to the error log (stderr.log)

Before: Under certain specific timings, it was possible that a Process Chain could run future steps before they should get started. This chance is higher if the chain has preconditions and/or final status handlers.

After: The Process Chain runs correctly in these situations.

Before: A "Duplicate Key" exception could occur when multiple UserMessages were created based on the same UserMessage definition.

After: This has been fixed.

Before: DashBoard dials for both the Monitoring dashboard and HouseKeeping dashboard when clicked, opened the home page instead of filtered overviews of e.g. processes. Additionally the filter menu (top-right corner) did not work anymore.

After: Both Monitoring and HouseKeeping dashboards work properly again.

Before: High CPU usage on spool host when retrieving spool list in PDF/BIN format
After: No high CPU usage on spool host when retrieving spool list in PDF/BIN format

Before: The Novum theme was missing some images that caused buttons not be be clickable

After: All images are now provided, making the buttons visible and clickable again.

Before: P_LAUFD and P_LAUFI in Mass Activity RDIP have been mapped to wrong fields. As such parameter values have not been set properly on the Mass Activity runs.

After: Map P_LAUFD and P_LAUFI in Mass Activity RDIP to correct fields.

Before: Configuration options: UI.HelpVersion and UI.ProductInfoDoc were mandatory, not setting them caused logged ERRORS during installation or build if not set.

After: Made UI.HelpVersion and UI.ProductInfoDoc optional in redwood/scheduler/ui/src/xml/configurationoptions.xml config file

Before: Splitter is transparent therefore not always clear for the user were it is located.

After: We accentuated the splitter background color so it is more visible were it is located.

Before: Job count in joblog.log and joblog.dat could be null.

After: Job count in joblog will be set properly.

Before: Killing a batch SAP job did not kill its child jobs

After: Killing a batch SAP job kills its child jobs, when /configuration/sap/xbp/EnableKillXBPChildJobsWithParent = true

Before: When more than one MailConnector was active on the same mail account they conflicted because of how the remote id was generated. Due to this only one of them handled an incoming email and started a corresponding job. In addition when a mail server automatically changed the header of an already processed email it was processed again by the MailConnector each time the header was changed.

After: When more than one MailConnector is now active on the same mail account they no longer conflict because of how the remote id is generated now. Due to this now each of them handles an incoming email and starts a corresponding job. In addition when a mail server now automatically changes the header of an already processed email it is not processed again by the MailConnector each time the header is changed when it is excluded by adding the connection setting "redwood.mail.id.exclude.header" and listing the corresponding headers to be excluded separated by "," or ";".

Mandatory fields with are now marked with * for Job Chain Editor Raise/Wait Events.

Before: When running System_ReportRun on a Report process definition that did not have a limit defined, it would fail with a NullPointerException, and if a limit had been defined, it would be used, even if it exceeded the system wide report limit of 500,000 rows.

After: Whenever a Report process definition is run, it will be limited to 500,000 rows, whether run directly or via System_ReportRun. If run via System_ReportRun, then it will not fail with a NullPointerException, even if the limit is not set.

Before: When starting PeopleSoft jobs that create child jobs a NullPointerException could occur with the message that child jobs can't be created. The childjob definition had to be imported before a parent job with these childjobs could be used. When the job definitions are not imported we are searching for a default job definition "System_PeopleSoft_MonitorJob". The search for this job definition caused a NullPointer Exception.

After: The "System_PeopleSoft_MonitorJob" is searched in the correct Application.

Before: Under some circumstances, a user with limited security permissions would get an NPE when attempting to attach a file to a UserMessage. Specifically, if the user could not see the System process server, this would occur.

After: This now works without error.

Before: When the process server of the SAP system located in a partition different from the one of  the SAP system, SAP_GenerateCcmsJobDefinition fails with a NPE.

After: When the process server of the SAP system located in a partition different from the one of the SAP system, SAP_GenerateCcmsJobDefinition completed with a message that ABAP program and variant cannot be imported.

Before: When importing a CCMS jobs, the recipient type on the generated process definition has been always set as Internet User.

After: When importing a CCMS job, recipient type on the generated process definition is set to the actual recipient type of the SAP job.

Before: Platform agent installer still showed obsolete (and non-functioning) option '-a|--enable-update'
After: Installer option '-a|--enable-update' has been removed from the 'Usage' information. When the option is used anyway, the installer will raise a warning message mentioning that the option is obsolete and has been removed.

Before: SAP BOBj jobs may go into error when the connection breaks
After: SAP BOBj jobs don't go into error when the connection breaks

Before: SAP transactions spool in XLSX format is corrupt when retrieving from a cloud instance.

After: CSV file will be generated when retrieving spool in XLSX format. It is converted to XLSX format in Finance Automation.

Before: Second SAP spool could not be retrieved when process definition default output format is xls

After: Second SAP spool can be retrieved properly when process definition default output format is xls

Before: When sending mails, and the server supports TLS upgrade, the TLS upgrade would fail, causing the mail to not be sent, and the mail process to error.

After: Mail can now be sent when the mail server supports TLS upgrades.

Before: The update process for the Jubilee Bank Holiday fix didn't correctly check if the hotfix had already been installed, leading to a failure during startup.

After: The Jubilee Bank Holiday update now correct detects if the hotfix has already been installed, and so no longer fails upon startup.

This includes the fixes for the following security issues released in 9.0.62, and announced later:

• CVE-2021-43980: Information disclosure

Code was back-ported to Tomcat 9.0.47 onwards exposed a long standing (but extremely hard to trigger) concurrency bug in Apache Tomcat 10.1.0 to 10.1.0-M12, 10.0.0-M1 to 10.0.18, 9.0.0-M1 to 9.0.60 and 8.5.0 to 8.5.77 that could cause client connections to share an Http11Processor instance resulting in responses, or part responses, to be received by the wrong client.

Update moment.js, a JavaScript library, to version 2.29.3 to overcome CVE-2022-24785. This update protects against unauthorised file access (a "path traversal" vulnerability).

Upgrades PostgreSQL JDBC drivers to 42.5.0 to overcome *CVE-2022-31197* which allows for potential SQL injection in RunMyJobs.

Note: The product is not affect by this CVE, as it never calls the ResultSet.refreshRow() method, however the library is being updated out of an abundance of caution.

Before: Platform agent included OpenSSL 1.1.1o.

After: Platform agent now includes OpenSSL 1.1.1q which contains the following CVE fixes:

• CVE-2022-2068 Fixed additional bugs in the c_rehash script which was not properly sanitising shell metacharacters to prevent command injection [Moderate severity
• CVE-2022-2097 Fixed AES OCB failure to encrypt some bytes on 32-bit x86 platformsModerate severity

Before: User message parameter UserMessage_NotifyMethod was set to direction "in" and could not be changed during the lifetime of a workflow.

After: User message parameter UserMessage_NotifyMethod is now set to direction "in/out" and can be changed during the lifetime of a workflow.

Before: Under some circumstances when logging in during a system upgrade, the upgrade would fail with a NoRowsUpdatedException, requiring the system to be restarted.

After: Changes have been made to retry if a NoRowsUpdatedException occurs during the upgrade, this means that the startup may take a little longer, but the system won't fail to startup.

Before: Specifying the CC or BCC headers in a mail definition would not send the mail to these addresses.

After: Setting these headers works as expected.

Before: SAP jobs with variant names with leading or trailing blanks could not be started.

After: SAP jobs with variant names with leading or trailing blanks can be started.

Before: In some circumstances it was possible for process definitions of types other than RedwoodScript and Java to be treated as Java code, and compiled if the library attached to the definition was changed. This would then fail the compilation, and thus prevent the update of the library.

After: Now only RedwoodScript and Java processes will be treated as Java code.

Before: Wild-carded file events could generate a flood of Operator Messages when they encounter files that could not be moved and files that are still in use and therefore cannot be moved yet.
After: Wild-carded file events will generate a single Operator Message at each scan interval for every file encountered that could not be moved.

Before: On Windows, when initial startup and configuration of the platform agent is very fast, the agent could run into an error during switch-over to a new log file where the name of the newly generated log file is identical to that of the initial log file. The following error message would then be reported: "The process cannot access the file because it is being used by another process."
After: The newly generated log file name is verified to be unique.

Before: Jobs could occasionally fail with error message: "Cannot get passwd information for UID 975311234: Success'.
After: This error is raised after system call 'getpwuid()' returns no result for the provided (network) user-id. The system call is now retried a couple of times to prevent intermittent failures. If the call still returns failure after the third retry attempt, the error is raised.

Before: A platform installation path, ending with a slash character '/' resulted in a duplicate PATHS entry in the scheduler script after a version upgrade. This could result in a repetitive startup sequence for platform agents.

After: A trailing slash character in the existing PATHS setting is ignored for existing installations.
For new platform agent installations, a possible trailing slash character is removed from the destination path argument during the installation.

Before:
The Platform Agent running on Ubuntu did not adequately recognize the ssh authenticity confirmation message of an agentless remote host. This resulted in the job being retried indefinitely.

After:
The Platform Agent running on Ubuntu recognizes the ssh authenticity confirmation message of an agentless remote host.

Before: On Windows, when initial startup and configuration of the platform agent is very fast, the agent could run into an error during switch-over to a new log file where the name of the newly generated log file is identical to that of the initial log file. The following error message would then be reported: "The process cannot access the file because it is being used by another process."
After: The newly generated log file name is verified to be unique. This is fixing an earlier attempt to make this robust, which did not do the job very well.

Before: If Platform Agent trace logging was enabled and the log file became large so it needed switching to a new log file then the Platform Agent was likely to crash.

After: When Platform Agent trace logging is enabled and the log file needs switching over, then the Platform Agent performs normally.

Before: If a no_proxy file or environment variable NO_PROXY was configured that did not include localhost, then the Platform Agent was unable to run jobs.

After: The Platform Agent can successfully run jobs even when a no_proxy file or NO_PROXY environment variable failed to include localhost.

Before: If a batch of files for a file event contained several files that could not be moved, then the back-off time was doubled for each file found in error. rather than just once for the whole batch. Or if a batch of files contained errors, but the last file processed was successful then backing off was stopped, causing looping Operator Messages.

After: A batch of files for a file event is now handled in a consistent fashion, preventing looping Operator Messages or alternatively backing off that is too aggressive.

Before:
Platform  agent would fail to startup if the persistent message channel contained an invalid or empty message.
The log would show the following error message:
JCS-116017: The Platform Agent service could not successfully be started. Reason: Fatal issue encountered in persistent message queue(s)

After:
Platform agent does a number of retry attempts to move away invalid/empty messages in the persistent channel and continues starting up successfully.

Before: Connecting with User SYS with SQLPLUS command fails on windows. The connection creates incorrect connection string syntax.

After: The connection string is now created correctly (with "as sysdba" at the end of the connection string)

Before: A resource problem on the platform agent host could result in a large amount of error messages and Operator Messages like "retrying failed fork: Resource temporarily unavailable".

After: Retry attempts due to resource problems will generate warnings and no Operator Messages anymore. When the maximum number of retry attempts is exceeded, an error will be raised and displayed in the Operator Messages.

Before: Timeout values on platform-agent for Sun Solaris, set for SSL read/write timeout, could end up as incorrect very large numbers. Setting Loglevel=trace would show these incorrect values as e.g.:

TRACE 2022-03-22 23:34:18,530 CET 14269-getmessages #3 opsys.socket - fd=8/10.31.2.179:36525->10.31.2.137:443: timeout set to 21474836480s

TRACE 2022-03-22 23:34:18,530 CET 14269-getmessages #3 opsys.socket - fd=8/10.31.2.179:36525->10.31.2.137:443:: waiting for write, timeout in 21474836480000ms

After: The timeout values are correctly set. E.g.: timeout set to 5s

Before: Platform agent included OpenSSL 1.1.1l.

After: Platform agent now includes OpenSSL 1.1.1n.

The following CVE fixes are included in 1.1.1n.

CVE-2022-0778: Infinite loop in BN_mod_sqrt() reachable when parsing certificates High severity

Note: This issue does not affect the scheduler, however the library is being upgraded out of caution. (We do not accept "user input" certificates).

Before: Platform agent included OpenSSL 1.1.1n.

After: Platform agent now includes OpenSSL 1.1.1o.

The following CVEs are fixed in OpenSSL 1.1.1o:

1. CVE-2022-1473 - Low severity - OPENSSL_LH_flush() function has a slow memory leak if it decodes lots of certs.
2. CVE-2022-1434 - Low severity - Only affects the OpenSSL 3.0 implementation of the RC4-MD5 cipher suite.
3. CVE-2022-1343 - Moderate severity - OCSP_basic_verify issue when the OCSP_NOCHECKS flag is used.
4. CVE-2022-1292 - Moderate severity - The c_rehash script does not properly sanitise shell metacharacters to prevent command injection.

Note: These issues do not affect the scheduler, however the library is being upgraded out of caution.

Before: jrfc did not correctly handle invalid UTF (Unicode transformation format) sequences.
In the case of an invalid UTF sequence received by jrfc, we would see hundreds of 'Invalid byte xx at y bytes into UTF8 sequence' for the same offset (ie looping) that will then exceed our internal buffer sizes, because later in the logs when the output is processed we see hundreds of 'StringBuffer maximum capacity (> 512Mb)' exceeded FATAL log messages.

After: The UTF code used by jrfc, and by the Platform Agent in general, now correctly handles the detection of an invalid UTF sequence and no longer loops.

Before: If a Windows Platform Agent encountered a non-transient WMI error, such as WBEM_E_ACCESS_DENIED, whilst monitoring an agentless job,, then the WMI operation would be repeatedly attempted with no limit. This would result in a very large number of job log messages for the Redwood Platform Server.

After: If a Windows Platform Agent encounters a WMI error whilst monitoring an agentless job, then the WMI operation is retried a limited number of times, in case the condition is transitory (such as WBEM_E_SERVER_TOO_BUSY). If the WMI error continues to occur, then the job fails and a small number of Operator Messages will be generated.

Before: to get the list of installer options for the Windows platform agent installer, you had to know what the option was to get the help list. Also the installer might do a partial install when unknown options are present.

After: the windows installer will show the help dialogue whenever an unknown option is passed.

Before: Starting multiple platform-agent instances with 'scheduler start' using the 'ksh93' shell, could result in an infinite starting loop, raising errors like:
"A process still exists for instance 'test', please verify that it is stopped and remove its pid file '/opt/redwood/agent/etc/pid/test/test.ppid' before starting it again."

After: The instances start normally.

When there is no secure gateway candidate available and as such no platform agent is available to act as the secure gateway, RunMyJobs SaaS systems can no longer send out alerts themselves, although customer contacts would like to be notified of this event.

A new feature has been introduced that automatically sends an alert via the cloud portal to all customer notification contacts when the secure gateway is not available. Also a notification will be send out when the connection is back online.

An exception is made when the system is in an update or restart phase, in that case, alerts will be suppressed.

Before: It was not possible to set CTS+ attributes
After: Enable to set CTS+ attributes with 2 new parameters CTS_PROJECT and TMWFLOW

Before: BOBj job could go into status Error due to comminication errors with the SAP BOBj system but completed in SAP BOBj.

After: Communication errors (eg. interrupted connection to the SAP BOBj system) will be treated as transient, though, keeping the job in status Running until they are resolved.

Before: SAP job cannot be killed when its output is being retrieved by spool host and the spool host is not running.

After: SAP job can be killed.

Before: Cannot retrieve spool output in HTML format, and large spool output in PDF format via the Spool Host.

After: Large spool output in PDF format can be retrieved via the Spool Host for RMJ and RMF, spool output in HTML can be retrieved via the Spool Host for RMF.

Before: When using the new implementation of SAP BOBJ connector, list of values for SAP BOBJ jobs with single value, interval value and string value could not be parsed properly.

After: List of values for SAP BOBJ jobs with single value, interval value and string value can be parsed properly.

Before: SAP mass activity processes may terminate prematurely before their sub-processes are finished in SAP.

After: SAP mass activity processes wait until their sub-processes are finished in SAP before terminating.

Before: If LANGUAGE parameter is not set in the job definition, multistep SAP jobs always use English as step language, otherwise it will use the the provided value

After: If LANGUAGE parameter is not set in the job definition, multistep SAP jobs will use SAP system default language as step language, otherwise it will use the the provided value

Before: Cannot retrieve SAP joblog in pipe delimited format via the Spool Host when FL_SPOOL is set.

After: SAP joblog can now be retrieved in pipe delimited format via the Spool Host.

SAP JCo has been updated to the latest 3.1.6 release.

Before: If a BW process is skipped in SAP, the remote status of the corresponding process on the central server would not be updated to reflect this.

After: If a BW process is skipped in SAP, the remote status of the corresponding process on the central server will be updated to reflect this.

Before:
$ character is not allowed in AS400 JobName. Only getting the logfile from an AS400 Job was not possible.
It was not possible to retrieve only the logfile for a job.

After:
$ character is allowed in the JobName.
Retrieving only the logfile is now possible by setting two parameters on the job:

• OutputFileRetrieval=false
• IndependentQPJOBLOG=true

It is also possible to configure this for all jobs, by setting the corresponding ProcessServer parameters:

• As400JobOutputFileRetrieval
• As400JobIndependentQPJOBLOG

Before: When there is no route to the process server when the process server is started, you may still get an error and the PS goes to shutdown.

After: When starting the ProcessServer the AS400 service, if there is no route to the proces server, it will continue to try and connect to the AS400 system until it is available.

Before: If you have a very small interval for file events (less then 2 seconds) it could be that when deleting a file event definition a NullPointerException was thrown in the AS400 event monitor.

After: The AS400 event monitor now holds all information that is needed for an event monitoring. After the monitoring the event can be removed without problems.

Before: Every job uses 2 AS400 connections. Every connection caused a small memory leak internally in the IBM code.

After: Implementing a connection pool so every job takes connections out of the connection pool.

Note: Due to an internal issue in the AS400 library, every used AS400 system/user combination will use a small amount of memory. In normal use this will not be an issue, as the amount of memory used is limited. However, if tens of thousands of system/user combinations are used, this adds up, and the library does not free this memory. We are investigating with IBM how to improve this situation.

Before: After a restart of the AS400 ProcessServer jobs in status "Console" were not monitored anymore. The jobs must be changed to a final status manually.

After: Jobs in status "Console" will be picked up after a restart and checked for status changes.

Before: When the AS400 starts up in the cloud and the Secure Gateway is not up yet a lot of operator messages are produced with an unknown host exception. 

After: When duplicate messages are created within a certain timeframe (Default 5 minutes) only one message is written.

Before: A missing Content-Type header could cause the browser to not process the SSO login request properly when using proxy servers.

After: This has been fixed.

Before: There were some missing database indexes that, in some situations, caused the system to slow down.

After: Extra indexes have been added to help in this situation

Before: When configuring the system with the Admin server, if an invalid database connection was entered initially, later initialisation steps may fail.

After: The invalid database connection details will not interfere with the requirements for the initialisation steps, and installation can continue.

Before: Mail_Attachments specified on UserMessageJobs (Workflows) weren't forwarded as Mail attachments.

After: Mail_Attachments specified on UserMessageJobs (Workflows) are now forwarded as Mail attachments.

before: As part of counting the number of processes to keep, steps were wrongfully added to this count.

 after: This has been corrected.

Before: when deleting a job chain and an audit rule was set up to match the name of objects, the audit rule was not triggered and no audit entries were present in the trail

After: when deleting a job chain and an audit rule is set up to match the name of objects, the audit entries are created in the trail

Before: If you create a http request wit method GET and a body was specified the request was automatically translated to a POST request.

After: A message is given when a body is specified when you create a http request with method GET or HEAD. The body is ignored.

Before: On databases that have case sensitive table names and fields, streaming Documents and upgrading old RedwoodScript definitions may fail to retrieve the data due to unknown table/field exceptions being thrown.

After: The case used for the queries is consistent with the table creation and will now correctly find the tables and fields in case sensitive database.

Before: Empty optional RTX elements were written to the RTX output.

After: Empty optional RTX elements are omitted from RTX output.

Added the french language to the AS400 connector.

Before: An exception is thrown when the JSON output contains JSONArray's.

After: All JSON elements can be parsed.

Before: A fix was added to 9.2.8.9 that allowed an AS400 to retry connection if the system could not connect. This retried the connection immediately, potentially leading to a high CPU usage while waiting for the AS400 system to become available.

After: The system now waits for a second between retry attempts.

Before: Adding workdays to a specific date with REL was only possible through a cumbersome workaround.

After: Two new REL functions have been added to provide a convenient way to add days of different time windows to a specific date:

• addOpenDays(date, timeWindow, days)
• addOpenDaysFromNow(timeWindow, days)

Before:- EventDefinition Related Object summary for Job Chain Call Wait and Raise events now show the expression field, this should be the call and event definition combined

• Job Chain Call Wait and Raise events show pages are missing a couple of important fields:Event Definition and Job Chain Call

After: - EventDefinition Related Object summary for Job Chain Call Wait and Raise events show call and event definition combined

• Job Chain Call Wait and Raise events Event Definition and Job Chain Call fields added
• Job Chain Call show page has been extended with Locks, Raise and Wait events fields

Before: Under certain circumstances objects were being retained in memory where they were not needed where short lived user sessions were being used.

After: These objects are no longer retained, reducing retained memory usage.

Add parameter map to the custom job context that contains all the job parameters
Add default implementations for getparametervalue, setparametervalue and
getParameters
GetParameterValue(String name) - takes parameter name as input, returns an Object as the value. Will return all parameter types (in, inout, out)
SetParameterValue(String name, Object value) - takes name and value as parameters, will set the value of the matching out or inout parameter if it exists. Will ignore if the parameter is an input param
getParameters: will return a Map with the parametername and value.
Add a minimum polling interval value (5 seconds) that the user cannot override to protect against destructive polling intervals set by the user
Add desired jobStatus Field on customJobContext that can be set. The framework will take care of setting the status on the cronacle job appropriately
If the job status field is empty, but a return code is set, use the current behavior to finalize the job (0 = completed, anything else = error)
If the job status field is set but the return code is empty, infer the return code from the status (Completed = 0, Error = 1)
Ensure console status is supported. If a job goes into console status, updateStatus will continue to be called, and it will be up to the developer to decide when to change the status again to finalize the job.
Expose remoteJobData on customJobContext so it can be set either from execute or updateStatus
NOTE: As a result of this, the signature of updateStatus will now just get the customJobContext instead of the remoteJobData
If execute returns a Finalized status, do not monitor that job and do not call update status - end it immediately

Before: Under certain circumstances local REL resolving would fail when being part of a job chain step.

After: This has been fixed.

Before: With the loop over step functionality, when the target call definition got updated before the loop kicks in, an exception will be raised that a branched definition cannot be duplicated.
After: The loop procedure will always update the target definition to the latest master version before looping over it.
Additionally, the call duplication procedure will perform the activity in batches to not overflow the system when the table to loop over is big, or when the call is a big job chain.

Before: In rare circumstances it was possible for two processes to attempt to create a directory at the same time, one of these would fail, potentially causing a process to not be run.

After: The creation of the directory is properly checked, and if two processes attempt to create the same directory at the same time, this will no longer fail.

Before: Using the java.util.logging classes in Redwood Script could result in leaking memory.

After: This has been fixed and logging will no longer leak memory.

Before: api-tools.jar event returned the eventid as the return code which made existing customer scripts fail as this was a change of behaviour.

After: api-tools.jar now returns 0 for success.
The event id is reported as "Successfully raised event with ID 85" for example.
Unless the new -silent argument is specified.
This matches the behaviour of jtool event.

Before: A potential deadlock could occur running a lot of jobs in parallel, and calling one of SchedulerSession.waitForJob(), SchedulerSession.waitForJobs(), or SchedulerSession.waitForAllChildren().

After: This has been fixed.

Before: Under certain specific timings, it was possible that a Process Chain could run future steps before they should get started. This chance is higher if the chain has preconditions and/or final status handlers.

After: The Process Chain runs correctly in these situations.

Before: When an error occurred during writing to the database, an error was logged in the log file, even though the error would not be meaningful after the fact.

After: This is no longer logged as error but as debug, reducing clutter in log files, but still accessible via dynamic trace.

Added new REL function String.replace

Before: Every jar file was written to disk when it was needed for execution, even multiple times when the same file was part of multiple libraries.

After: This is no longer the case, reducing the footprint on disk when executing Redwood Script with multiple different libraries.

Before: reading a job file residing on a platform agent would not close the socket immediately, but only after the Java garbage collector ran. Opening many (thousands) of files in rapid succession could result in starvation (out of open files or sockets).

After: reading a job file that resides on a platform agent will now immediately close the socket in such a way that there is no remaining open socket (not even in CLOSE_WAIT or TIME_WAIT) on client (java server) or server (platform agent).

A warning on startup of tomcat has been made more user-friendly. It previously logged an error about the DataManager not availabe, but that is correct as on startup Cronacle is not yet available. Instead it now logs a warning explaining that if this message is logged during startup of tomcat, the message can be safely ignored.

Note that the message only occurs when tomcat is purging expired sessions (from a previous run).

Before: Slowdown could be observed in systems with a lot of Events.

After: Optimization has been done in queries in the EventComponent, including extra indexes.

Before: When the scheduler has a large number of queues and/or process servers, greater than about 100 of each, then startup could take a long time.

After: The process has been optimised to ensure that startup occurs quickly, regardless of the number of active process servers or queues.

Before: Using a Time.expression REL to truncate a value would result in a DateTimeException.

After: RELs containing Time.expression that include value truncation work successfully.

Before: The Platinum Jubilee Bank Holiday would not happen on the date of 3 June 2022. The Spring Bank Holiday would happen on the date of 30 May 2022

After: The Platinum Jubilee Bank Holiday was added to happen once on the date of 3 June 2022. The Spring Bank Holiday was moved only in 2022 from 30 May 2022 to 2 June 2022

Before: The following maintenance jobs all started at the fixed times based on a whole hour. These times seem to be the times that most of the jobs are scheduled, causing peaks in the job schedules:

• System_UpdateJobStatistics
• System_ProcessKeepClauses
• System_DeleteJobFiles
• System_Ignored_Alert_Reporter
• System_Aggregate_History

After: The maintenance jobs above are now scheduled on random times, but with the same interval as before, reducing the peaks.

The randomisations happens only once, and happens the first time that the system starts up after installing this version.

The randomisations of the above mentioned maintenance jobs happens only if they have the default 15 minutes set as time interval in between executions or if no time interval is set.

Upgrades the version of the PostgreSQL JDBC driver used in RunMyJobs and other products to 42.4.0, this fixes some minor issues in the driver.

Before: RedwoodScript Scripts using waitForJobs API functions, with infinite wait, stop waiting after one hour unless the process finishes before then.

After: Using waitForJobs with infinite wait will wait until the process has finished.

NOTE: This regression was introduced in 9.2.4.9, 9.2.6.6, and 9.2.8.1.

Before: Several issues running AS400 Jobs

1. Errors occurred when submitting AS400 jobs because an unexpected connection failure occurs.
2. Jobs in Console are not set to Running when all operator messages are replied. This only occurs when the operator message is replied on AS400
3. When the Secure Gateway is not up and running and the AS400 process server is restarted the jobs could go in undispatched status.

After:

At startup we will better check if we can reach out to the AS400. We leave the Process Server in status Connecting while we can’t reach out to the AS400. If we do find the AS400 and we loose the connection while re-monitoring the jobs that were already started we don’t check the connection anymore.

The job is set to Running when all operator messages are replied.

We will retry (default 5 times) the login if we get an unexpected connection failure. This can be overruled on Process Server level (AS400JobSubmitRetry) or Job level (SubmitRetry)

Before: An error could occur when upgrading due to a missing type related to case sensitivity issues with the database.

After: The upgrade will now succeed.

Before: In certain circumstances with occasional use, Redwood Script definitions that have not be updated recently may fail to run due to a NullPointException when trying to retrieve the underlying resources.

After: The old definitions will now correctly be upgrade to the new internal representation and can be run successfully.

Before: It was possible for an import to fail and prevent the user from logging in due to unresolved grants

After: Unresolved grants will no longer prevent the user from logging in.

Before: Internal management of threads could result in attempting to reused the internal ThreadGroup once it has been discarded.

After: The shutdown of the ThreadGroup will no longer happen and allow for future reuse.

Upgrades the version of the fasterxml Jackson products used in all products to 2.13.2, to overcome CVE-2020-36518 which could potentially allow a denial of service by uploading specially crafted JSON documents.

Before: the diagram view of a chain process would display winter time if that process was submitted during winter time, but executed during summer time.
After: the diagram displays the summer time.

Before: The generated name of a duplicated Role could be modified but would not be persisted.

After: Role names can be changed during creation, also when duplicating Roles.

Before: users could type an arbitrarily long value in the name field of the root node in the Process Chain Editor. Names over 80 characters in length cause an error message when pressing save.
After: users can no longer provide a name that is over 80 characters in length.

Before: The wait-event icon was not always visible on the process in the Chain Editor.
After: The wait-event icon behaves just like the raise-event icon: it is visible when
the process is selected or hovered.

Before: the "Edit wait events" and, to a lesser degree, the "Edit precondition" icons were not rendered within the confines of their box (Process Chain Call).
After: the spacing between all the icons of the box has been reduced thus reducing the overall width of the row of icons and bringing them all back on board.

Before:

1. Changing the Process Definition of a node from the property panel of the JobChain editor, would not always update the tabs on the top of the window.
2. Clicking on an entry in the list of available Process Definitions (in an attempt to change the Process Definition as described before), would select the Process Definition, but keep the list of values open

After:

1. Tabs are immediately updated after the Process Definition is changed.
2. The list of Process Definitions closes after clicking an entry.

Before: Prior to 9.2.1.0, it was possible to configure login page to show a message on the login screen by setting the /configuration/security/title registry entry. This functionality was removed in 9.2.1.0.

After: The functionality to add a message to the login screen has been restored. This can be configured by setting the ui.LoginMessage configuration item, eg by setting the /configuration/jcs/UI/LoginMessage registry entry, or by setting the legacy registry entry /configuration/security/title.

Before: the "..." button of the "When expression" input field in the Process Chain Editor were hidden due to an issue with the page layout.
After: the buttons are available. To access them, click on a Call, expand the Raise Events or Wait Events section, press the plus icon if the section is empty, click on the bottom input labelled "When expression". You should now see a button on the right-hand side of the input.

Before: On some edit pages, after user interaction, specifically crafted input could be sent to the server in a way that posed a security risk.

After: We changed the way we handle server-client communication in those cases to remove the security risk.

Before: After executing a search (Intellisearch) on an overview, the overview is filtered on the search, a temporary "queryfilter" is created for it and that filter is selected as the current overview filter. However on various Firefox versions, the temporary "queryfilter" is not correctly selected, leaving the previously selected queryfilter set as selected (even though the filtering based on the search does happen).

After: This is now fixed and Firefox behaves in the same way as other browsers.

Before: Due to the processing required to enable the "Collect Job Output" menu, accessing processes could be slower if the scheduler has many process servers configured.

After: This menu option is removed, and thus the speed of selecting processes in the overview is improved.

Before: When a Chain Call definition had Raise and/or Wait events defined, they were not visible in the Chain Call detail panel of the Chain Editor.

After: Definition Wait and Raise events are now also visible on the Chain Call detail panel, but will be read-only. To change those go the the respective Definition tabs.

Before: When submitting a process, using a submit frame without a time window, the timezone of the start time was not changed to the timezone of the submit frame.

After: It is set correctly now

Upgrades the version of CKEditor4 used in Redwood Platform to 4.18.0 to overcome CVE-2022-24728  and CVE-2022-24729 which potentially allowed users to execute arbitrary javascript by uploading documents with specially crafted HTML, and tricking another user into opening them in the editor.

Before: when a session timeout (due to inactivity) was configured (using the registry entry /configuration/sessiontimeout) and such a timeout occured for a user, the following would happen:

• In the main "control center" ui: an "unknown error occurred when handling a request to the server" popup would show and the same popup would show on any subsequest request to the server
• In applications and extension points that are replacing the mainpage: no message would immediately show, but subsequent server requests would fail and the ui would show a message stating that it could not parse the response.

After: now instead, in both the main "control center" ui and in applications and extension points that are replacing the mainpage, a popup will show notifying to the user that they have been logged out due to inactivity and after that the user will be redirected to the "logged out"/"log in" page.

A new button ‘Help me automate’ was added to the main page to lower the boundary to request help for adding new processes.

Before: User and Role detail pages would only show the User/Role description.

After: User and Role detail pages show both name and description

Before: calling ResultSet.getString() method on a blob type field returned a string representation of a byte array.

After: calling ResultSet.getString() method on a blob type field returns the actual string decoded from the blob

Before: There is no option to monitor Redwood Script threads (either from Processes, Triggers, Shell etc).

After: It is now possible via the support servlet to see which Redwood Script threads are running, who started them, how long they've been running and to either show the process associated with it, or to interrupt the thread (in case of shell threads)

Before: If a process definition was modified, and then submitted by multiple threads in parallel, then every thread would attempt to do the work to create the new definition. This could result in a lot of unnecessary work, as only one of these could succeed, and the other threads would then have to reload the data. This is a general issue, but was most often seen in FCA loops, which would create a new thread for every iteration of data, which would all get submitted at the same time.

After: This is resolved by using an internal cache, and if multiple threads are waiting for the same prepare, then they will wait until the first one has completed, to use its result.

Before: Disabling an Extension Point could cause certain Processes to fail with "missing Configuration Option" errors.

After: This has been fixed.

Before: Not all JDBC drivers were up to date.

After: Updated the JDBC drivers to the latest versions. The new versions are:

• db2: 11.5.7.0
• mssql: 9.4.1
• oracle: 21.4.0.0.1
• postgresql: 42.3.1
• jt400: 10.7

The version of Tomcat that Redwood Platform is based on has been updated to 9.0.58. This includes the fixes for the following security issues released in 9.0.58, and announced later:

• CVE-2022-23181: Local Privilege Escalation

Note: The product doesn't use FileStore to store sessions, and is unaffected by the above vulnerability, this is being updated out of an abundance of care.

SAP JCo version 3.1.3 has issues with MSHOST connections and a memory leak in JCoServer. SAP has fixed the issue with SAP JCo 3.1.5. Default SAP JCo version has been set to 3.1.5.

Before: When an Import was delegated to a child process, the Import overview would not show the output from the actual import.

After: The Import overview always shows the relevant output.

Before: it was not possible to create a custom object (table) and set full permissions for it on a role in one go during an upgrade

After: it is possible to create grants right after custom object creation, within the same persist

Before: Our custom menu appears instead of the password suggestions in the password field.
After: Password suggestions option is enabled in the password field.

The following pertains to the diagram view of the process monitor. This view displays the runtime perspective of chain processes.
Before: if a step is configured not to wait for some process, this process is a chain (a.k.a. a subchain), and this chain is expanded, then the final element of this chain is a line.
After: the final element of the subchain is a smiley (and the smiley is smaller than it used to be)

Upgraded the ckeditor to the newest version, to address a vulnerability issue that exists in the previous version.

Before: In some circumstances it was possible to pass a specially crafted XML document to the SOAP API that would allow you to read any file on the system that the scheduler has access to.

After: This is resolved.

Before: User login audit trails would show the (reverse) proxy ip address for clients.

After: An attempt is made to find the original client ip address on the request to store in the audit trail.

Note:** this change has introduced a backwards imcompatible change on the AuditSubjectLogin API where the setters have been removed. Although chances are low this is used in the field, be aware of this change.

Before: AutoSuggest control in Cronacle UI did not appropriately escape HTML characters, meaning that specially crafted names could potentially result in a cross site scripting attack vector.

After: HTML encoding is provided to autoSuggest control in Cronacle UI.

Cloud only

Before: The default glibc memory allocator in Linux interacts very badly with Java, causing significant memory overallocation. In our cloud environment, this then caused environments to use too much memory, which resulted in them getting erroneously killed by the Linux Out Of Memory Killer (OOMK).

After: The memory allocator has been changed such that memory usage is much more consistent over time and load. Internal testing, which was previously able to reliably cause Linux to kill the environment with its OOMK, now passes and the environments do not see these issues, nor do they see memory increases over time.

Note: For on-premise customers, the memory allocator that we are are using is tcmalloc.

The version of Tomcat that Redwood Platform is based on has been updated to 9.0.54. This includes the fixes for the following security issues released in 9.0.54, and announced later:

• CVE-2021-42340: Denial of Service when using websockets

Note: The product doesn't use web sockets in a way that would be affected by the above denial of service, this is being updated out of an abundance of care.

Before:

• It was not allowed to set the "Keep Process In Status", in the Retention Tab, of the job definitions System_Defaults_System or System_Defaults_Partition.

• When defining a "Keep Process In Status" as 0, then this meant to keep all jobs, the same as not filling it in.

After:

• It is now allowed to set the "Keep Process In Status" of the job definitions System_Defaults_System or System_Defaults_Partition.

• When defining a "Keep Process In Status" as 0, then this now means to keep NO jobs, i.e. ALL THE JOBS WILL BE REMOVED.

IMPORTANT! This means that Jobs of JobDefinitions having a "Keep Process In Status" of 0 days will be deleted immediately after the upgrade of the product.

Make sure that the "Keep Process In Status" of these Job Definitions are changed to empty, BEFORE upgrading the product.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-44228.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

For on premise customers who are not able to apply this update then Redwood strongly recommends that you configure the logging to mitigate this by adding -Dlog4j2.formatMsgNoLookups=true to the startup configurations. You will need to restart your server for this to take effect.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-44228.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

Note that the -Dlog4j2.formatMsgNoLookups=true flag is not effective against the latest exploit, however the default logging configuration does not use the vulnerable pattern layouts.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-45105.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

Note that the -Dlog4j2.formatMsgNoLookups=true flag is not effective against the latest exploit, however the default logging configuration does not use the vulnerable pattern layouts.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.

Before: Logging within the product is done using Log4J. A new CVE has been reported in version of Log4j in use, CVE-2021-44832.

After: Log4J has been upgraded to a version that is not vulnerable to this exploit.

This vulnerability requires that the log4j2.yaml file be modified, and that the server be restarted, for the system to be vulnerable. We recommend that customers protect this logging configuration to ensure that it can not be modified.

Note: Due to the configuration of our cloud services, the published attacks are not effective. Nonetheless, it is recommended that customers upgrade to this version as soon as possible to ensure that any improvements in the attack techniques will be ineffective.