Mapping sets overview
A mapping set is a set of mappings that transforms data from one schema to another. This document provides information on how mapping sets are comprised, including input schema, output schema, and mappings.
Getting started
This overview requires a working understanding of the following components of ÃÛ¶¹ÊÓƵ Experience Platform:
- Data Prep: Data Prep allows data engineers to map, transform, and validate data to and from Experience Data Model (XDM).
- Dataflows: Dataflows are a representation of data jobs that move data across Platform. Dataflows are configured across different services, helping move data from source connectors to target datasets, to Identity and Profile, and to Destinations.
- ÃÛ¶¹ÊÓƵ Experience Platform Data Ingestion: The methods by which data can be sent to Experience Platform.
- Experience Data Model (XDM) System: The standardized framework by which Experience Platform organizes customer experience data.
Mapping set syntax
A mapping set is comprised of an ID, name, input schema, output schema, and a list of associated mappings.
The following JSON is an example of a typical mapping set:
{
"id": "cbb0da769faa48fcb29e026a924ba29d",
"name": "Demo Mapping Set",
"inputSchema": {
"id": "a167ff2947ff447ebd8bcf7ef6756232",
"version": 0
},
"outputSchema": {
"schemaRef": {
"id": "https://ns.adobe.com/{TENANT_ID}/schemas/6dd1768be928c36d58ad4897219bb52d491671f966084bc0",
"contentType": "application/vnd.adobe.xed-full+json;version=1"
}
},
"mappings": [
{
"sourceType": "ATTRIBUTE",
"source": "Id",
"destination": "_id",
"name": "Id",
"description": "Identifier field"
},
{
"sourceType": "ATTRIBUTE",
"source": "FirstName",
"destination": "person.name.firstName"
},
{
"sourceType": "ATTRIBUTE",
"source": "LastName",
"destination": "person.name.lastName"
}
]
}
idnameinputSchemaoutputSchemamappingssourceTypeFor each listed mapping, its sourceType attribute indicates the type of source that is to be mapped. Can be one of ATTRIBUTE, STATIC, or EXPRESSION:
ATTRIBUTEis used for any value found in the source path.STATICis used for values injected into the destination path. This value remains constant and is not affected by the source schema.EXPRESSIONis used for an expression, which will be resolved during runtime. A list of available expressions can be found in the mapping functions guide.
sourcesource attribute indicates the field that you want to map. More information about how to configure your source can be found in the sources overview.destinationdestination attribute indicates the field, or the path to the field, where the value extracted from the source field will be placed. More information on how to configure your destinations can be found in the destination overview.mappings.namemappings.descriptionConfiguring mapping sources
In a mapping, the source can be a field, expression, or a static value. Based on the source type given, the value can be extracted in various ways.
Field in columnar data
When mapping a field in columnar data, such as a CSV file, use the ATTRIBUTE source type. If the field contains . within its name, use \ to escape the value. An example of this mapping can be found below:
Sample CSV file:
Full.Name, Email
John Smith, js@example.com
Sample mapping
{
"source": "Full.Name",
"destination": "pi.name",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": {
"name": "John Smith"
}
}
Field in nested data
When mapping a field in nested data, such as a JSON file, use the ATTRIBUTE source type. If the field contains . within its name, use \ to escape the value. An example of this mapping can be found below:
Sample JSON file
{
"customerInfo": {
"name": "John Smith",
"email": "js@example.com"
}
}
Sample mapping
{
"source": "customerInfo.name",
"destination": "pi.name",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": {
"name": "John Smith"
}
}
Field within an array
When mapping a field within an array, you can retrieve a specific value by using an index. To do this, use the ATTRIBUTE source type and the index of the value you want to map. An example of this mapping can be found below:
Sample JSON file
{
"customerInfo": {
"emails": [
{
"name": "John Smith",
"email": "js@example.com"
},
{
"name": "Jane Smith",
"email": "jane@example.com"
}
]
}
}
Sample mapping
{
"source": "customerInfo.emails[0].email",
"destination": "pi.email",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": {
"email": "js@example.com"
}
}
Array to array or object to object
Using the ATTRIBUTE source type, you can also directly map an array to an array or an object to an object. An example of this mapping can be found below:
Sample JSON file
{
"customerInfo": {
"emails": [
{
"name": "John Smith",
"email": "js@example.com"
},
{
"name": "Jane Smith",
"email": "jane@example.com"
}
]
}
}
Sample mapping
{
"source": "customerInfo.emails",
"destination": "pi.emailList",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": {
"emailList": [
{
"name": "John Smith",
"email": "js@example.com"
},
{
"name": "Jane Smith",
"email": "jane@example.com"
}
]
}
}
Iterative operations on arrays
Using the ATTRIBUTE source type, you can iteratively loop through arrays and map them to a target schema by using a wildcard index ([*]). An example of this mapping can be found below:
Sample JSON file
{
"customerInfo": {
"emails": [
{
"name": "John Smith",
"email": "js@example.com"
},
{
"name": "Jane Smith",
"email": "jane@example.com"
}
]
}
}
Sample mapping
{
"source": "customerInfo.emails[*].name",
"destination": "pi[*].names",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": [
{
"names": {
"name": "John Smith"
}
},
{
"names": {
"name": "Jane Smith"
}
}
]
}
Constant value
If you want to map a constant, or a static value, use the STATIC source type. When using the STATIC source type, the source represents the hard-coded value that you want to assign to the destination. An example of this mapping can be found below:
Sample JSON file
{
"name": "John Smith",
"email": "js@example.com"
}
Sample mapping
{
"source": "CUSTOMER",
"destination": "userType",
"sourceType": "STATIC"
}
Transformed data
{
"userType:": "CUSTOMER"
}
Expressions
If you want to map an expression, use the EXPRESSION source type. A list of accepted functions can be found in the mapping functions guide. When using the EXPRESSION source type, the source represents the function you want to resolve. An example of this mapping can be found below:
Sample JSON file
{
"firstName": "John",
"lastName": "Smith",
"email": "js@example.com"
}
Sample mapping
{
"source": "concat(upper(lastName), upper(firstName), now())",
"destination": "pi.created",
"sourceType": "EXPRESSION"
}
Transformed data
{
"pi": {
"created": "SMITHJOHNFri Sep 25 15:17:31 PDT 2020"
}
}
Configuring mapping destinations
In a mapping, the destination is the location where the value extracted from the source will be inserted.
Field at the root level
When you want to map the source value to the root level of your transformed data, follow the example below:
Sample JSON file
{
"customerInfo": {
"name": "John Smith",
"email": "js@example.com"
}
}
Sample mapping
{
"source": "customerInfo.name",
"destination": "name",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"name": "John Smith"
}
Nested field
When you want to map the source value to a nested field in your transformed data, follow the example below:
Sample JSON file
{
"name": "John Smith",
"email": "js@example.com"
}
Sample mapping
{
"source": "name",
"destination": "pi.name",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": {
"name": "John Smith"
}
}
Field at a specific array index
When you want to map the source value to a specific index in an array in your transformed data, follow the example below:
Sample JSON file
{
"customerInfo": {
"name": "John Smith",
"email": "js@example.com"
}
}
Sample mapping
{
"source": "customerInfo.name",
"destination": "piList[0]",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"piList": ["John Smith"]
}
Iterative array operation
When you want to iteratively loop through arrays and map the values to the target, you can use a wildcard index ([*]). An example of this can be seen below:
{
"customerInfo": {
"emails": [
{
"name": "John Smith",
"email": "js@example.com"
},
{
"name": "Jane Smith",
"email": "jane@example.com"
}
]
}
}
Sample mapping
{
"source": "customerInfo.emails[*].name",
"destination": "pi[*].names",
"sourceType": "ATTRIBUTE"
}
Transformed data
{
"pi": [
{
"names": {
"name": "John Smith"
}
},
{
"names": {
"name": "Jane Smith"
}
}
]
}
Next steps
By reading this document, you should now understand how mapping sets are constructed, including how to configure individual mappings within a mapping set. For more information on other Data Prep features, please read the Data Prep overview. To learn how to use mapping sets within the Data Prep API, please read the Data Prep developer guide.