MuleSoft Technical Guides Archives - MuleSoft Anypoint Studio Developer blogs I Caelius Consulting

Guides for MuleSoft Developers.

Message Logging Policy in MuleSoft

MuleSoft Integration Team June 30, 2021June 30, 2021 MuleSoft Technical Guides No Comments

The Message Logging policy logs custom messages from incoming requests, responses from the backend, or information from other policies applied to the same API endpoint.

Steps to implement the Message Logging Policy:

Select the Message Logging Policy from the Apply New Policy in the API Manager.

Now specify the type of content you want to log. Fill the Parameters accordingly :

Message	A DataWeave expression to use for extracting the message to be logged.
Conditional	A DataWeave Boolean expression used to specify whether a message is to be logged
Category	A prefix in the log message that indicates the package and class where the log is executed. If no category is set, the default value is org.mule.runtime.logging.policy-<policy id>.
Level	Defines the level of message to log: INFO, WARN, ERROR, or DEBUG
Before Calling API	Set to cause your logging policy to log messages before the API endpoint is called, taking into account the order in which this policy is applied
After Calling API	Set to cause your logging policy to log messages after the API endpoint is called, taking into account the order in which this policy is applied

Suppose we want to log the payload; accordingly, we need to set the parameters as shown in the image below:

Give the request to the API.

Now open the Logs of the API.

Anypoint Platform -> Runtime Manager -> API -> Logs,

We can see the payload in the logs, which is similar to what we got while requesting the API.

Thanks for reading. Learn more from certified MuleSoft Experts here at Caelius Consulting Resource Centre

JSON Threat Protection Policy in MuleSoft

MuleSoft Integration Team June 7, 2021June 8, 2021 MuleSoft Technical Guides No Comments

JSON Threat Protection Policy in MuleSoft is a protection policy for the APIs to defend them from miscellaneous HTTP requests which are inflated to bring down the APIs server. For example, a request can contain an inflated request in which they can increase the JSON depth, object name and values, and array length, which may result in memory or performance issues. Therefore, such requests should be denied at the entry-level only.

This policy usually defines the proper structure for the JSON request we will receive, e.g., Depth, Object name and value length, and Array size.

Steps to implement the JSON Threat Policy:

Select the JSON THREAT POLICY, and it’s a version from the Apply New Policy in the API Manager.

2. Configure the various properties:

Maximum Container Depth – Specifies the maximum nested depth. JSON allows you to nest the containers (object and array) in any order to any depth
Maximum String Value Length – Specifies the maximum length of a string
Maximum Object Entry Name Length – Specifies the maximum string length of an object’s entry name
Maximum Object Entry Count – Specifies the maximum number of entries in an object
Maximum Array Element Count – Specifies the maximum number of elements in an array

3. Now when we make a post request an error message is being shown, depicting that wrong properties are being passed according to the configured policy.

4. We need to set the properties in the following way:

Maximum Container Depth – 3 – which is root -> address -> (addressLine1, addressLine2, city, state,zipCode, country)
Maximum String Value Length – 15 – e.g. max length value in JSON – molly@mue.com
Maximum Object Entry Name Length – 12 – maximum string length of an object’s entry name in JSON – addressLine1
Maximum Object Entry Count – -1 – Specifies that there can be n number of entries in an object.
Maximum Array Element Count – 2

{
  "customerID": "1fe1c22",
  "firstName": "Molly",
  "lastName": "Mule",
  "displayName": "Molly ",
  "address": {
    "addressLine1": "123 Street",
    "addressLine2": "Apt.#3D",
    "city": "San Francisco",
    "state": "California",
    "zipCode": "94110",
    "country": "United States"
  },
  "phone": "415-000-0000",
  "email": "molly@mue.com",
  "ssn": "321-654-0987",
  "dateOfBirth": "1990-09-04"
}

5. Now give the post request and we are able to access the API.

Thanks for reading, find more MuleSoft best practices at Caelius Consulting Resource Centre.

DataWeave for reading XML

MuleSoft Integration Team May 14, 2021May 14, 2021 MuleSoft Technical Guides No Comments

The XML data structure is mapped to DataWeave objects containing other objects, strings, or null values.

All Mule applications have an XML file that specifies the resources that compose the application. Schemas define the configurable attributes of these resources that are referenced in the XML configuration file. This is how a Mule application both validates and illustrates its functional components and their configuration.

Sample XML data:

Throughout this article we will use the following XML data snippet:

<Books:myns xmlns:Books="http://training.mulesoft.com/Books">
<bookstore>
<book category="COOKING">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="CHILDREN">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
<price>29.99</price>
</book>
<book category="WEB">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>
</Books:myns>

1. Converting XML to JSON:

JSON data format, values map one-to-one with DataWeave values.

DataWeave expression:

      %dw 2.0
   output application/json
   ---
   Payload

DataWeave Output:

    {
 "bookstore": {
   "book": {
     "title": "Everyday Italian",
     "author": "Giada De Laurentiis",
     "year": "2005",
     "price": "30.00"
   },
   "book": {
     "title": "Harry Potter",
     "author": "J K. Rowling",
     "year": "2005",
     "price": "29.99"
   },
   "book": {
     "title": "Learning XML",
     "author": "Erik T. Ray",
     "year": "2003",
     "price": "39.95"
   }
 }
}

2. Extract the Namespace:

DataWeave fully supports XML namespaces. It is a mechanism that avoids name conflicts by differentiating XML elements or attributes that may have identical names but different definitions. To extract the namespace from an XML element use the hash symbol “#.”

DataWeave Expression:

%dw 2.0
                   output application/json
                   ---
                   payload.myns.#

DataWeave Output:

                   "http://training.mulesoft.com/Books"

3. Transforming XML fields:

When transforming XML to JSON you may run into a couple of difficulties:

XML only allows one root element, while JSON allows multiple.
There is no array concept in XML (only repeated elements). However, the JSON data structure is based on arrays and objects.

To overcome these issues, DataWeave provides the following solutions:

To transform XML repeated elements into a JSON array, use the asterisk “*” DataWeave selector to create it.
As JSON could have multiple root elements, you don’t have to do anything specific to generate the output.

DataWeave expression:

           %dw 2.0
   output application/json
   ---
payload.bookstore.*book

DataWeave output:

         [
 {
   "title": "Everyday Italian",
   "author": "Giada De Laurentiis",
   "year": "2005",
   "price": "30.00"
 },
 {
   "title": "Harry Potter",
   "author": "J K. Rowling",
   "year": "2005",
   "price": "29.99"
 },
 {
   "title": "Learning XML",
   "author": "Erik T. Ray",
   "year": "2003",
   "price": "39.95"
 }
]

To specify your output format, you need to iterate over the items in your Array. That can be achieved with the map function.

DataWeave expression:

      %dw 2.0
   output application/json
   ---
   payload.bookstore.*book map
  {
    title: $["title"],
    author: $["author"],
    year: $.year,
    price:$.price
  }

DataWeave output:

     [
 {
   "title": "Everyday Italian",
   "author": "Giada De Laurentiis",
   "year": "2005",
   "price": "30.00"
 },
 {
   "title": "Harry Potter",
   "author": "J K. Rowling",
   "year": "2005",
   "price": "29.99"
 },
 {
   "title": "Learning XML",
   "author": "Erik T. Ray",
   "year": "2003",
   "price": "39.95"
 }
]

4. Transforming XML attributes:

DataWeave provides a mechanism that allows the reading of attributes from an XML element using the @ symbol followed by the key.

DataWeave expression:

       %dw 2.0
    output application/json
    ---
    payload.bookstore.*book map
  {
     title: $["title"],
     author: $["author"],
     year: $.year,
     price:$.price,
     category:$.@category
  }

DataWeave output:

                        [
    {
       "title": "Everyday Italian",
       "author": "Giada De Laurentiis",
       "year": "2005",
       "price": "30.00",
       "category": "COOKING"
    },
  {
      "title": "Harry Potter",
      "author": "J K. Rowling",
      "year": "2005",
      "price": "29.99",
      "category": "CHILDREN"
  },
 {
    "title": "Learning XML",
    "author": "Erik T. Ray",
    "year": "2003",
    "price": "39.95",
    "category": "WEB"
  }
 ]

Scatter Gather in Mule 4

MuleSoft Integration Team May 7, 2021May 7, 2021 MuleSoft Technical Guides No Comments

Scatter Gather Pattern helps MuleSoft developers distribute different tasks to achieve parallel processing of tasks/messages/events & then finally aggregate the responses as a single response.

The Scatter-Gather component receives a Mule event and sends a reference of this Mule event to each processing route.
Each of the processing routes starts executing in parallel then returns a Mule event. This is because the Scatter-Gather component executes each route in parallel, not sequentially.
After all processing routes have finished execution, the Scatter-Gather component creates a new Mule event that combines all resulting Mule events from each route. It then passes the new Mule event to the next component in the flow.

Scatter Gather Example:

In Scatter-Gather flow, we have three routes: Flow Reference Component(Flow Reference routes the Mule event to another flow or sub-flow). One set Payload Transformer(The Set Payload component lets you update the payload of the message).

Both sub-flow contains a Database select query operation which gives us data from the Database. This transform message component converts the data from the Database into JSON format and a logger which will track the logs.

1) Route 1 contains a Flow reference of a sub-flow (db-flow1)

Configuration of select query operation –

Configuration XML:

<sub-flow name="db-flow1" doc:id="bf4acf02-bd0d-4156-9980-6aad5b1e908b" >
		<db:select doc:name="Select" doc:id="38eff10a-1141-4b16-aeb4-c3ed78eeae66" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="d3590e43-a0c1-42e9-b6d5-562db1bec17d" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="342df3a5-b836-4a4c-a775-06bde82712d4" message="#[payload]"/>
	</sub-flow>

2) Route 2 contains a Flow reference of a sub flow (db-flow2)

Configuration of Select Query :

Configuration XML:

<sub-flow name="db-flow2" doc:id="f88987da-ea4c-44f5-bc39-0096d78eaa72" >
		<db:select doc:name="Select" doc:id="930efe51-8e73-40f2-9184-8174be3e9d01" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product1]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="ec45dbb2-92eb-43e2-836b-84029b0f4290" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="b100cf5e-809e-4d09-bfba-05f8114b748d" message="#[payload]"/>
	</sub-flow>

3) Route 3 contains a Set Payload component

Configuration of Set Payload:

Configuration XML:

<route >
				<set-payload value="scatter-gather flow payload" doc:name="Set Payload" doc:id="f8c0d364-044e-4e59-b31d-3fb4e8805466" />
				<logger level="INFO" doc:name="Logger" doc:id="62563f04-a79c-4cef-9ab9-5eea9e17637a" message="#[payload]"/>
			</route>

Tables in Database:

Product Table

2. Product1 Table

Flow XML:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns:db="http://www.mulesoft.org/schema/mule/db"
	xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
	<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="dc87c934-3f63-4e54-a7a6-57b9852c6e2a" >
		<http:listener-connection host="0.0.0.0" port="8081" />
	</http:listener-config>
	<db:config name="Database_Config1" doc:name="Database Config" doc:id="99af9304-0aa1-484d-986c-be9022aa3d21" >
		<db:my-sql-connection host="localhost" port="3306" user="root" password="Caelius@123" database="muletrainingdb" />
	</db:config>
	<flow name="scatter-gather" doc:id="b7ea3fed-f402-4359-8a0d-c7a952903010" >
		<http:listener doc:name="Listener" doc:id="8e71879a-1711-4c3d-b4a1-1effa23cdec0" config-ref="HTTP_Listener_config" path="/test"/>
		<scatter-gather doc:name="Scatter-Gather" doc:id="f79ea679-6ffc-432c-a8e5-c56c9a6ff025" >
			<route >
				<flow-ref doc:name="db-flow1" doc:id="da378a2d-6584-4682-be11-746cb1b443a6" name="db-flow1"/>
			</route>
			<route >
				<flow-ref doc:name="db-flow2" doc:id="765de27e-c4ac-4c0b-bced-58f49f2bc460" name="db-flow2"/>
			</route>
			<route >
				<set-payload value="scatter-gather flow payload" doc:name="Set Payload" doc:id="f8c0d364-044e-4e59-b31d-3fb4e8805466" />
				<logger level="INFO" doc:name="Logger" doc:id="62563f04-a79c-4cef-9ab9-5eea9e17637a" message="#[payload]"/>
			</route>
		</scatter-gather>
		<ee:transform doc:name="Transform Message" doc:id="df9303a4-c1dd-4eec-a731-a9358859ed18">
					<ee:message>
						<ee:set-payload><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
					</ee:message>
				</ee:transform>
	</flow>
	<sub-flow name="db-flow1" doc:id="bf4acf02-bd0d-4156-9980-6aad5b1e908b" >
		<db:select doc:name="Select" doc:id="38eff10a-1141-4b16-aeb4-c3ed78eeae66" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="d3590e43-a0c1-42e9-b6d5-562db1bec17d" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="342df3a5-b836-4a4c-a775-06bde82712d4" message="#[payload]"/>
	</sub-flow>
	<sub-flow name="db-flow2" doc:id="f88987da-ea4c-44f5-bc39-0096d78eaa72" >
		<db:select doc:name="Select" doc:id="930efe51-8e73-40f2-9184-8174be3e9d01" config-ref="Database_Config1">
			<db:sql ><![CDATA[select * from product1]]></db:sql>
		</db:select>
		<ee:transform doc:name="Transform Message" doc:id="ec45dbb2-92eb-43e2-836b-84029b0f4290" >
			<ee:message >
				<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
			</ee:message>
		</ee:transform>
		<logger level="INFO" doc:name="Logger" doc:id="b100cf5e-809e-4d09-bfba-05f8114b748d" message="#[payload]"/>
	</sub-flow>
</mule>

Output Combining all routes:

  {
    "0": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": [
            {
                "original_price": 1000.0,
                "product_id": 1,
                "name": "Hp Pavilion laptop",
                "offer_valid_until": "2016-06-27T10:45:56",
                "description": "Hp Laptop ",
                "brand_name": "HP",
                "offer_price": 1000.0
            },
            {
                "original_price": 3000.0,
                "product_id": 2,
                "name": "Macbook Pro laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Apple Laptop ",
                "brand_name": "Apple",
                "offer_price": 3000.0
            },
            {
                "original_price": 2000.0,
                "product_id": 3,
                "name": "Mac Book  laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Apple Laptop ",
                "brand_name": "Apple",
                "offer_price": 2000.0
            },
            {
                "original_price": 4000.0,
                "product_id": 4,
                "name": "IBM laptop",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "IBM Laptop ",
                "brand_name": "IBM",
                "offer_price": 4000.0
            },
            {
                "original_price": 1000.0,
                "product_id": 5,
                "name": "MotoX Mobile",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Motorola Mobile ",
                "brand_name": "Motorola",
                "offer_price": 1000.0
            },
            {
                "original_price": 5000.0,
                "product_id": 6,
                "name": "Samsung Note 5",
                "offer_valid_until": "2016-06-27T10:45:57",
                "description": "Samsung Mobile ",
                "brand_name": "Samsung",
                "offer_price": 5000.0
            }
        ],
        "outboundPropertyNames": [],
        "attributes": null
    },
    "1": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": [
            {
                "Name": "hp",
                "id": 1001
            },
            {
                "Name": "asus",
                "id": 1002
            },
            {
                "Name": "dell",
                "id": 1003
            },
            {
                "Name": "apple",
                "id": 1004
            }
        ],
        "outboundPropertyNames": [],
        "attributes": null
    },
    "2": {
        "inboundAttachmentNames": [],
        "exceptionPayload": null,
        "inboundPropertyNames": [],
        "outboundAttachmentNames": [],
        "payload": "scatter-gather flow payload",
        "outboundPropertyNames": [],
        "attributes": {
            "headers": {
                "user-agent": "PostmanRuntime/7.28.0",
                "accept": "*/*",
                "postman-token": "01f2bd2d-5eaa-4d9f-bd6a-164ab590be81",
                "host": "localhost:8081",
                "accept-encoding": "gzip, deflate, br",
                "connection": "keep-alive"
            },
            "clientCertificate": null,
            "method": "GET",
            "scheme": "http",
            "queryParams": {},
            "requestUri": "/test",
            "queryString": "",
            "version": "HTTP/1.1",
            "maskedRequestPath": null,
            "listenerPath": "/test",
            "localAddress": "/127.0.0.1:8081",
            "relativePath": "/test",
            "uriParams": {},
            "rawRequestUri": "/test",
            "rawRequestPath": "/test",
            "remoteAddress": "/127.0.0.1:54523",
            "requestPath": "/test"
        }
    }
}

All the MuleSoft Anypoint platform users can use this guide to implement scatter-gather pattern in their Mule projects.

Find more MuleSoft technical guides at Caelius Consulting Resource Centre.

MuleSoft Technical Guides

Message Logging Policy in MuleSoft

JSON Threat Protection Policy in MuleSoft

DataWeave for reading XML

Scatter Gather in Mule 4

Recent Posts

Recent Comments

Group Company