MuleSoft Technical Guides

Guides for MuleSoft Developers.

JSON Logger in MuleSoft

JSON Logger in MuleSoft

Logging is one of the most fundamental yet overlooked element of any application. Logging serves several purposes like tracking a particular transaction, debugging any errors, creating dashboards, and checking the process of different application processes. MuleSoft uses the log4j2 logging standard for logging events. Some of the best practices in logging include using a transaction id, timestamp, log level and relevant message to get the information. Today we will be looking at one of the other logging ways by using a connector: JSON-Logger.

As the name suggests, the JSON logger prints the JSON format logs, which can help a lot when logging into any external log analytic tool such as ELK or Splunk. It logs the relevant information and is helpful in the customization of content and other fields. JSON logger also supports the functionality for sending data to AMQ or JMS. To use a JSON logger in your application, we need to follow the specific steps explained below:

  1. First things first, you need to clone the repository on your local system. Since JSON logger is a custom connector and not provided by MuleSoft, we need to push the logger to Anypoint exchange to use it.

 

  1. To push the logger to MuleSoft Anypoint exchange, get the Org Id for your Organization from the MuleSoft Anypoint platform. Go to Access Management >  Business Group and Retrieve Org Id. In case you do not have access, you can use Anypoint Platform API to retrieve Org Id

Business Group

  1. Add your Anypoint username and password in settings.xml. Since we will be using mule maven plugin to deploy to exchange, make sure you add entry in settings.xml inside servers tag.

<servers>
<server>
            <id>anypoint-exchange-v2</id>
            <username>username</username>
            <password>*****</password>
        </server>
	</servers>
  1. After adding the credentials, go to the cloned directory and run the shell script with Org Id as a parameter:

  1. This script will be responsible for pushing your connector to MuleSoft Anypoint business exchange. You can see in exchange.

JSON Logger Exchange

  1. After pushing the JSON logger to exchange, we can use it in our applications using the dependency in pom.xml. You can get the dependency information from the exchange.

JSON Logger Studio

  1. The connector is handy in the application. Once you add it to the application, you can set the configuration. One can mask any particular field in payload and can disable the content depending on the business case.

JSON Logger Default Configuration

  1. You can set the message and print the payload depending on the use case. Besides, the MuleSoft developer can set the priority, and trace-point can be set for monitoring.

  1. Here is an application information snippet

JSON Logger App

  1. An Example entry by JSON logger looks like this :

{
  "correlationId" : "1ed87460-6ae1-11eb-a037-6c2b59777f19",
  "message" : "Request for Id: ++ 147",
  "tracePoint" : "AFTER_TRANSFORM",
  "priority" : "INFO",
  "elapsed" : 3415,
  "locationInfo" : {
    "lineInFile" : "18",
    "component" : "json-logger:logger",
    "fileName" : "sample-app.xml",
    "rootContainer" : "sample-appFlow"
  },
  "timestamp" : "2021-02-09T14:14:31.989Z",
  "content" : {
    "payload" : {
      "userId" : 8,
      "id" : 147,
      "title" : "eum itaque quod reprehenderit et facilis dolor autem ut",
      "completed" : true
    }
  },
  "applicationName" : "sample-elk-app",
  "applicationVersion" : "1.0",
  "environment" : "local",
  "threadName" : "[MuleRuntime].uber.07: [sample-elk-app].sample-appFlow.BLOCKING @1e0641d"
}

 

Setup Okta as Identity provider on MuleSoft Anypoint Platform

Configure Okta with MuleSoft Anypoint plaform

MuleSoft Anypoint Platform can be configured for Single Sign-On (SSO) using Okta, OpenAM or PingFederate. SSO is useful to authenticate and access multiple applications/websites by logging in only once. Identity Management can be configured using one of the below SSO standards:

  1. OpenID Connect
  2. SAML 2.0

Configuring Okta

1. Create an account on Okta if you do not have one already.

2. Once you log in, create a new application by clicking on the Application menu tab.

okta-idp-create-application-add

Select Web on the next screen and click next.

3. On the next screen, we have to provide details like application name, redirect URI etc.

 

okta-idp-create-application-form

Give a name to your application. Provide https://anypoint.mulesoft.com against Base URIs or leave it blank. Login Redirect URI is of following format – https://anypoint.mulesoft.com/accounts/login/{{domain}}/redirect

Note: {{domain}} is organization-specific, to retrieve that, login to Anypoint Platform -> Access Management -> Organization > Click on the organization name and copy the Organization Domain.

4. Finish the process and take note of the client ID and Secret displayed at the next page’s bottom.

 

 

okta-idp-app-id-secret

5. From the top menu bar, go to API -> Authorisation Servers

 

okta-idp-auth-server

 

Click on default.

6. Clicking on Metadata URI will open a new tab with JSON payload listing Authorization and token endpoint, which will be used in the platform to set up the Identity Provider.

okta-idp-auth-server-uri

Configuring MuleSoft AnyPoint Platform

1. Log in to the platform, and navigate to Access Management > Identity Provider

okta-idp-platform-sso

2. On the next screen, click on Use manual registration and enter the Okta application’s client Id and secret.

okta-idp-platform-config

To test the setup

Once the MuleSoft Anypoint platform and Okta setup are done, we can test the SSO functionality by browsing URL https://anypoint.mulesoft.com/accounts/login/{{domain}} which will be redirected to the Okta login page instead of Anypoint platform one.

Adding User

New users can be added to the Anypoint platform from Okta instead of inviting them from Access Management.

To add new user login to your Okta account, navigate to Users -> People -> Add Person

okta-idp-add-user

Provide the required details. The added user can navigate to the login URL and sign in.

 

One-way and Two-way TLS and their implementation in MuleSoft

One-way and Two-way TLS and their implementation in MuleSoft

Introduction:

In this article, we will focus on the role of one-way and two-way TLS in API Integrations and their implementation in MuleSoft integration. Transport Layer Security, or TLS, is a widely adopted security protocol designed to facilitate privacy and data security for communications over the Internet. A primary use case of TLS is encrypting the communication between web applications and servers.

Keystore and Truststore

Before discussing the implementation of one-way and two-way TLS in MuleSoft, let’s first discuss Keystore and Truestore. In short, a Keystore or Truststore is a file that contains certificates; Keytool facilitates the generation of these files and is a part of the JDK, or you can use a GUI tool like Keystore explorer.

Keystore

  • A Keystore contains private keys and associated certificates having the public key
  • KeyStore has its dedicated password
  • Each entry of a private key/certificate combination in Keystore has its own dedicated password.
  • The owner/Server retrieves a certificate from its KeyStore and presents it to the other side.

Truststore

  • A trust store has a list of all certificates which client trust
  • These could be certificates of Trusted CA and self-signed certificates of trusted parties
  • So when a server presents its certificate, the Client will verify it using the certificates stored in TrustStore
  • A password also protects TrustStore.

One Way TLS

In oneway TLS authentication (Server Certificate Authentication), only the Client validates the server; the server does not verify the client application. When implementing oneway TLS authentication, the server application shares its public certificate with the Client.

  • Generate Keystore for server:

You can generate the Keystore for the server using the Java Keytool tool preinstalled with the JDK. To generate the Keytool, open your terminal/cmd and enter the following command, which would generate Keystore with RSA 4096 bit keypair saved in location C:/Documents/test/server-keystore.jks and with an alias as mule-server

keytool -genkey -alias mule-server -keysize 4096 -keyalg RSA -keystore C:/Documents/test/server-keystore.jks

After executing the above, it will password, first name, last name, common name and other details, including the key password.

  • Export the public certificates from the Keystore and add them to Client Truststore

To export the public certificate execute the command

keytool -export -alias mule-server -keystore C:/Documents/test/server-keystore.jks

 -file C:/Documents/test/server_public.crt

Export Certificate

This will export the public certificates that can now be imported into the Truststore.

  • Import the server public certificate in the client Trustore

The below command will generate the Truestore and import the server’s public certificates that we exported in the previous step into the generated Truststore

keytool -import -alias mule-client-public -keystore C:/Documents/test/client-truststore.jks -file C:/Documents/test/server_public.crt

Two-way TLS import Server Public Certificate

  • Add Truststore and Kestore to mule listener and requester

Server/listener configuration

  • Copy Truststore and Keystore in src/main/resources folder
  • Configure the port as 8082 in the listener config or define it as port
  • Add the configuration for the Keystore you generated in TLS config of the Listener (alias can be left blank if your Keystore has one 1 keypair), the type should be set as jks.

HTTP Listener Config

  • Client/Requester Configuration:
    • Set HTTPS as protocol and host as the server host (localhost) and server port 8082 (defaults to 443 and could be left blank for a website/web application)
    • Click on TLS configuration option and select edit inline
    • Add the Truststore name and password.

This is it! You just set up a TLS connection with our HTTP listener acting as a server and the requester acting as a Client. To test the application, you can create a HTTP listener sending a request to our server endpoint; You can refer to the image; the client HTTP listener accepts HTTP request here so we can hit with the postman or any other client without any issue

Flow MuleSoft

 

Two-way TLS

In two-way SSL authentication, the client application verifies the server application’s identity, and then the server application verifies the identity of the client application. Both parties share their public certificates, and then validation is performed. Two-way SSL authentication works with a mutual handshake by exchanging the certificates.

Implementing Two Way TLS in a MuleSoft Application

Implementation of two way TLS is similar to the implementation of one way TLS. However, here we would be configuring the Keystore and Truststore in both the Listener (server) and Requester (Client).

In this scenario, the Keystore will consist of their respective key pairs for server and Client and the server Truststore consisting of the Client public certificates and the Client Truststore consisting of the server’s public certificates.

  • Generate Keystore for server:

Execute the following command and write the details and Keystore password like you did for 1 way TLS

You can ignore the warning that comes after executing the query

keytool -genkey -alias mule-server -keysize 4096 -keyalg RSA -keystore C:\Users\jayank\Documents\2wayTLS\server-keystore.jks

You can ignore the warning that comes after executing the query

  • Export the public certificates from the server Keystore and add them to Client Truststore

The following command will export the server public certificate:

keytool -export -alias mule-server -keystore C:\Users\jayank\Documents\2wayTLS\server-keystore.jks -file C:\Users\jayank\Documents\2wayTLS\server_public.crt

Export the public certificates from client Keystore

To generate the client Truststore and add the server public certificate into it, execute:

keytool -import -alias mule-client-public -keystore C:\Users\jayank\Documents\2wayTLS\client-truststore.jks -file C:\Users\jayank\Documents\2wayTLS\server_public.crt

import them into the servers truststore

  • Generate the Client Keystore

keytool -genkey -alias mule-client -keysize 4096 -keyalg RSA -keystore C:\Users\jayank\Documents\2wayTLS\client-keystore.jks

  • Export the public certificates from Client Keystore and import them into the servers Truststore  

keytool -export -alias mule-client -keystore

C:\Users\jayank\Documents\2wayTLS\client-keystore.jks -file

C:\Users\jayank\Documents\2wayTLS\client_public.crt

 

Export the public certificates from client Keystore

keytool -import -alias mule-server-public -keystore

C:\Users\jayank\Documents\2wayTLS\server-truststore.jks -file

C:\Users\jayank\Documents\2wayTLS\client_public.crt

Two-way TLS import Server Public Certificate

  • Configure HTTP Listener (Server) with server Keystore and Truststore

Now, if you have followed all the above steps correctly, you will have 4 jks files in your directory 2 Keystores and 2 Truststores 1 each for Client and server.

Now add the server Keystore and server Truststore in HTTP listener with the password and alias (for server) you used while generating those.

  • Configure HTTP Requester (Client) with client Keystore and Truststore

Add the Client-Truststore and Client-Keystore in the HTTP request TLS configuration similar to you did for the server/listener.

Troubleshooting Tips:

  • You can use a test HTTP listener connection to call the requester (Client), which will call the Listener (server) refer to 1 TLS instructions for the same.
  • Add -M-Djavax.net.debug=ssl in MuleSoft Anypoint Studio as an argument or add a runtime property as javax.net.debug=ssl in  Mulesoft CloudHub to see the complete TLS debug logs for your MuleSoft API. This will allow you to see where exactly in the TLS handshaking process your request is failing, do remember to turn them off immediately as these logs are hefty.

TLS arg

 

Thanks for reading. Hope this will help all of you MuleSoft developers in implementing one-way and two-way TLS.

Find More MuleSoft best practices at Caelius Consulting Resource Centre.

Date formatting in DataWeave 2.0

Date formatting in DataWeave 2.0

We often encounter scenarios to convert dates from one format to another, and I find the date formatting a massive task in itself.  Date conversion becomes essential in a state of affairs where there is a need to sync two databases/systems. It’s not an easy task since most databases/systems have their specific formats. Maybe the client wants a date in some particular format; hence, the responsibility to convert from one format to another falls upon Mule or, say, falls upon us, MuleSoft developers. We’ll explore on conveniently change the date formatting in DataWeave.

Let’s start simple and understand the native date types in DataWeave Format.

  1. Date: Date represented in yyyy-MM-dd format. E.g: 2020-10-17
  2. DateTime: Conjunction of Date and time in yyyy-MM-ddTHH:mm:ss format with timezone. E.g: 2020-10-17T23:57:59+05:30
  3. LocalDateTime: Conjunction of Date and time in yyyy-MM-ddTHH:mm:ss format without timezone.E.g: 2020-10-17T23:57:59
  4. LocalTime: Time in the current timezone.E.g: 23:57:59
  5. Timezone: Time relative to GMT. Must start with a + or – . E.g: +05:30
  6. Period: Represents an amount of time in following form P[n]Y[n]M[n]DT[n]H[n]M[n]S. It consists of two parts – Date period, the part before T and Duration, the part after T

Below is the DataWeave script to check the native date types in DataWeave:

{
    Date: typeOf(|2020-10-17|),
    DateTime: typeOf(|2020-10-17T23:57:59+05:30|),
    LocalDateTime: typeOf(|2020-10-17T23:57:59|),
    LocalTime: typeOf(|23:57:59|),
    TimeZone: typeOf(|+05:30|),
    Period: typeOf(|P1Y|)
}

 

Below is the Output for the script above:

{
  "Date": "Date",
  "DateTime": "DateTime",
  "LocalDateTime": "LocalDateTime",
  "LocalTime": "LocalTime",
  "TimeZone": "TimeZone",
  "Period": "Period"
}

 

To get the current time of the system, Dataweave offers now() function.
Printing now() will output:

"2021-02-11T15:04:31.277+05:30"

 

The details like day, month, year etc., are facilitated by now() function.

{
    "Current Date and Time": now(),
    "Year": now().year,
    "Quarter": now().quarter,
    "Month": now().month,
    "Day": now().day,
    "Hour": now().hour,
    "Minutes": now().minutes,
    "Seconds": now().seconds,
    "Day Of Week": now().dayOfWeek,
    "Day Of Year": now().dayOfYear
}

 

Below is the output for the script above:

{
 "Current Date and Time": "2021-02-11T09:37:14.406Z",
 "Year": 2021,
 "Quarter": 1,
 "Month": 2,
 "Day": 11,
 "Hour": 9,
 "Minutes": 37,
 "Seconds": 14,
 "Day Of Week": 4,
 "Day Of Year": 42
}

 

Formatting Dates:

DataWeave can help us attain dates in different formats than what now() returns. Please note that the below script is just an example of the different date formats and doesn’t encompass all the possible scenarios.

%dw 2.0
output application/json
---
{
   "Current Date and Time": now(),
   "LocalDateTime": now() as LocalDateTime,
   "Date in dd-MM-yyyy format": now() as Date {format: "dd-MM-yyyy"},
   "Date in dd-MMM-yyyy format": now() as Date {format: "dd-MMM-yyyy"},
   "Date in dd/MM/yyyy format": now() as Date {format: "dd/MM/yyyy"},
   "Date in dd/MMM/yyyy format": now() as Date {format: "dd/MMM/yyyy"},
   "Date in dd E, MMM, yyyy format": now() as Date {format: "dd E, MMM, yyyy"},
   "Date in yyyy/MMM/dd format": now() as Date {format: "yyyy/MMM/dd"},
   "Date in dd/MMM/yyyy format": now() as Date {format: "dd/MMM/yyyy"},
   //Instead of HH and hh, kk and KK can be used respectively
   "Date in 24hr Format" : now() as String {format : "dd-MM-yyyy HH:mm:ss"},
   "Date in 12hr Format" : now() as String {format : "dd-MM-yyyy hh:mm:ss a"}  
}

 

Output:

{
 "Current Date and Time": "2021-02-11T09:38:40.959Z",
 "LocalDateTime": "2021-02-11T09:38:40.959",
 "Date in dd-MM-yyyy format": "11-02-2021",
 "Date in dd-MMM-yyyy format": "11-Feb-2021",
 "Date in dd/MM/yyyy format": "11/02/2021",
 "Date in dd/MMM/yyyy format": "11/Feb/2021",
 "Date in dd E, MMM, yyyy format": "11 Thu, Feb, 2021",
 "Date in yyyy/MMM/dd format": "2021/Feb/11",
 "Date in dd/MMM/yyyy format": "11/Feb/2021",
 "Date in 24hr Format": "11-02-2021 09:38:40",
 "Date in 12hr Format": "11-02-2021 09:38:40 AM"
}

 

Here, we saw a few examples of formatting system dates and dates received as part of payload into different formats and how to extract information from dates.

*There can be many more scenarios for date conversions other than the one in this MuleSoft technical guide.