ÃÛ¶¹ÊÓƵ Experience Manager Assets developer use cases, APIs, and reference material assets-cloud-service-apis
The article contains recommendations, reference materials, and resources for developers of Assets as a Cloud Service. It includes new asset upload module, API reference, and information about the support provided in post-processing workflows.
Experience Manager Assets APIs and operations use-cases-and-apis
Assets as a Cloud Service provides several APIs to programmatically interact with digital assets. Each API supports specific use cases, as mentioned in the table below. The Assets user interface, Experience Manager desktop app and ÃÛ¶¹ÊÓƵ Asset Link support all or some of the operations.
Asset upload asset-upload
In Experience Manager as a Cloud Service, you can directly upload the assets to the cloud storage using HTTP API. The steps to upload a binary file are below. Execute these steps in an external application and not within the Experience Manager JVM.
- Submit an HTTP request. It informs Experience Manager deployment of your intent to upload a new binary.
- PUT the contents of the binary to one or more URIs provided by the initiation request.
- Submit an HTTP request to inform the server that the contents of the binary were successfully uploaded.
The approach provides a scalable and more performant handling of asset uploads. The differences as compared to Experience Manager 6.5 are:
- Binaries do not go through Experience Manager, which is now simply coordinating the upload process with the binary cloud storage configured for the deployment.
- Binary cloud storage works with a Content Delivery Network (CDN) or Edge network. A CDN selects an upload endpoint that is nearer for a client. When data travels a shorter distance to a nearby endpoint, the upload performance and user experience improve, especially for geographically distributed teams.
Initiate upload initiate-upload
Submit an HTTP POST request to the desired folder. Assets are created or updated in this folder. Include the selector .initiateUpload.json
to indicate that the request is to initiate upload of a binary file. For example, the path to the folder where the asset should be created is /assets/folder
. The POST request is POST https://[aem_server]:[port]/content/dam/assets/folder.initiateUpload.json
.
The content type of the request body should be application/x-www-form-urlencoded
form data, containing the following fields:
(string) fileName
: Required. The name of the asset as it appears in Experience Manager.(number) fileSize
: Required. The file size, in bytes, of the asset being uploaded.
A single request can be used to initiate uploads for multiple binaries, as long as each binary contains the required fields. If successful, the request responds with a 201
status code and a body containing JSON data in the following format:
{
"completeURI": "(string)",
"folderPath": "(string)",
"files": [
{
"fileName": "(string)",
"mimeType": "(string)",
"uploadToken": "(string)",
"uploadURIs": [
"(string)"
],
"minPartSize": (number),
"maxPartSize": (number)
}
]
}
completeURI
(string): Call this URI when the binary finishes uploading. The URI can be an absolute or relative URI, and clients should be able to handle either. That is, the value can be"https://[aem_server]:[port]/content/dam.completeUpload.json"
or"/content/dam.completeUpload.json"
See complete upload.folderPath
(string): Full path to the folder in which the binary is uploaded.(files)
(array): A list of elements whose length and order match the length and order of the list of binary information provided in the initiate request.fileName
(string): The name of the corresponding binary, as supplied in the initiate request. This value should be included in the complete request.mimeType
(string): The mime type of the corresponding binary, as supplied in the initiate request. This value should be included in the complete request.uploadToken
(string): An upload token for the corresponding binary. This value should be included in the complete request.uploadURIs
(array): A list of strings whose values are full URIs to which the binary’s content should be uploaded (see Upload binary).minPartSize
(number): The minimum length, in bytes, of data that may be provided to any one of theuploadURIs
, if there is more than one URI.maxPartSize
(number): The maximum length, in bytes, of data that may be provided to any one of theuploadURIs
, if there is more than one URI.
Upload binary upload-binary
The output of initiating an upload includes one or more upload URI values. If more than one URI is provided, the client may split the binary into parts and make PUT requests of each part to the provided upload URIs, in order. If you choose to split the binary into parts, adhere to the following guidelines:
- Each part, with the exception of the last, must be of a size greater than or equal to
minPartSize
. - Each part must be of a size less than or equal to
maxPartSize
. - If the size of your binary exceeds
maxPartSize
, split the binary into parts to upload it. - You are not required to use all URIs.
If the size of your binary is less than or equal to maxPartSize
, you may instead upload the entire binary to a single upload URI. If more than one upload URI is provided, use the first one and ignore the rest. You are not required to use all URIs.
CDN edge nodes help accelerate the requested upload of binaries.
The easiest way to accomplish this is to use the value of maxPartSize
as your part size. The API contract guarantees that there are sufficient upload URIs to upload your binary if you use this value as your part size. To do this, split the binary into parts of size maxPartSize
, using one URI for each part, in order. The final part can be of any size less than or equal to maxPartSize
. For example, assume the total size of the binary is 20,000 bytes, the minPartSize
is 5,000 bytes, maxPartSize
is 8,000 bytes, and the number of upload URIs is 5. Execute the following steps:
- Upload the first 8,000 bytes of the binary using the first upload URI.
- Upload the second 8,000 bytes of the binary using the second upload URI.
- Upload the last 4,000 bytes of the binary using the third upload URI. Since this is the final part, it does not need to be larger than
minPartSize
. - You do not need to use the last two upload URIs. You can ignore them.
A common error is to calculate the part size based on the number of upload URIs provided by the API. The API contract does not guarantee that this approach works, and may actually result in part sizes that are outside the range between minPartSize
and maxPartSize
. This can result in binary upload failures.
Again, the easiest and safest way is to simply use parts of size equal to maxPartSize
.
If the upload is successful, the server responds to each request with a 201
status code.
Complete upload complete-upload
After all the parts of a binary file are uploaded, submit an HTTP POST request to the complete URI provided by the initiation data. The content type of the request body should be application/x-www-form-urlencoded
form data, containing the following fields.
fileName
mimeType
uploadToken
createVersion
True
and an asset with the specified name exists, then Experience Manager creates a new version of the asset.versionLabel
versionComment
replace
True
and an asset with the specified name exists, Experience Manager deletes the asset then re-create it.uploadDuration
fileSize
createVersion
nor replace
is specified, then Experience Manager updates the asset’s current version with the new binary.Like the initiate process, the complete request data may contain information for more than one file.
The process of uploading a binary is not done until the complete URL is invoked for the file. An asset is processed after the upload process is complete. Processing does not start even if the asset’s binary file is uploaded completely but upload process is not completed. If upload is successful, the server responds with a 200
status code.
Example Shell Script to upload assets to AEM as a Cloud Service upload-assets-shell-script
The multistep upload process for direct binary access within AEM as a Cloud Service is illustrated in the following example shell-script aem-upload.sh
:
#!/bin/bash
# Check if pv is installed
if ! command -v pv &> /dev/null; then
echo "Error: 'pv' command not found. Please install it before running the script."
exit 1
fi
# Check if jq is installed
if ! command -v jq &> /dev/null; then
echo "Error: 'jq' command not found. Please install it before running the script."
exit 1
fi
# Set DEBUG to true to enable debug statements
DEBUG=true
# Function for printing debug statements
function debug() {
if [ "${DEBUG}" = true ]; then
echo "[DEBUG] $1"
fi
}
# Function to check if a file exists
function file_exists() {
[ -e "$1" ]
}
# Function to check if a path is a directory
function is_directory() {
[ -d "$1" ]
}
# Check if the required number of parameters are provided
if [ "$#" -ne 4 ]; then
echo "Usage: $0 <aem-url> <asset-folder> <file-to-upload> <bearer-token>"
exit 1
fi
AEM_URL="$1"
ASSET_FOLDER="$2"
FILE_TO_UPLOAD="$3"
BEARER_TOKEN="$4"
# Extracting file name or folder name from the file path
NAME=$(basename "${FILE_TO_UPLOAD}")
# Step 1: Check if "file-to-upload" is a folder
if is_directory "${FILE_TO_UPLOAD}"; then
echo "Uploading files from the folder recursively..."
# Recursively upload files in the folder
find "${FILE_TO_UPLOAD}" -type f | while read -r FILE_PATH; do
FILE_NAME=$(basename "${FILE_PATH}")
debug "Uploading file: ${FILE_PATH}"
# You can choose to initiate upload for each file here
# For simplicity, let's assume you use the same ASSET_FOLDER for all files
./aem-upload.sh "${AEM_URL}" "${ASSET_FOLDER}" "${FILE_PATH}" "${BEARER_TOKEN}"
done
else
# "file-to-upload" is a single file
FILE_NAME="${NAME}"
# Step 2: Calculate File Size
FILE_SIZE=$(stat -c %s "${FILE_TO_UPLOAD}")
# Step 3: Initiate Upload
INITIATE_UPLOAD_ENDPOINT="${AEM_URL}/content/dam/${ASSET_FOLDER}.initiateUpload.json"
debug "Initiating upload..."
debug "Initiate Upload Endpoint: ${INITIATE_UPLOAD_ENDPOINT}"
debug "File Name: ${FILE_NAME}"
debug "File Size: ${FILE_SIZE}"
INITIATE_UPLOAD_RESPONSE=$(curl -X POST \
-H "Authorization: Bearer ${BEARER_TOKEN}" \
-H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" \
-d "fileName=${FILE_NAME}" \
-d "fileSize=${FILE_SIZE}" \
${INITIATE_UPLOAD_ENDPOINT})
# Continue with the rest of the script...
fi
# Check if the response body contains the specified HTML content for a 404 error
if echo "${INITIATE_UPLOAD_RESPONSE}" | grep -q "<title>404 Specified folder not found</title>"; then
echo "Folder not found. Creating the folder..."
# Attempt to create the folder
CREATE_FOLDER_ENDPOINT="${AEM_URL}/api/assets/${ASSET_FOLDER}"
debug "Creating folder..."
debug "Create Folder Endpoint: ${CREATE_FOLDER_ENDPOINT}"
CREATE_FOLDER_RESPONSE=$(curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${BEARER_TOKEN}" \
-d '{"class":"'${ASSET_FOLDER}'","properties":{"title":"'${ASSET_FOLDER}'"}}' \
${CREATE_FOLDER_ENDPOINT})
# Check the response code and inform the user accordingly
STATUS_CODE_CREATE_FOLDER=$(echo "${CREATE_FOLDER_RESPONSE}" | jq -r '.properties."status.code"')
case ${STATUS_CODE_CREATE_FOLDER} in
201)
echo "Folder created successfully. Initiating upload again..."
# Retry Initiate Upload after creating the folder
INITIATE_UPLOAD_RESPONSE=$(curl -X POST \
-H "Authorization: Bearer ${BEARER_TOKEN}" \
-H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" \
-d "fileName=${FILE_NAME}" \
-d "fileSize=${FILE_SIZE}" \
${INITIATE_UPLOAD_ENDPOINT})
;;
409)
echo "Error: Folder already exists."
;;
412)
echo "Error: Precondition failed. Root collection cannot be found or accessed."
exit 1
;;
500)
echo "Error: Internal Server Error. Something went wrong."
exit 1
;;
*)
echo "Error: Unexpected response code ${STATUS_CODE_CREATE_FOLDER}"
exit 1
;;
esac
fi
# Extracting values from the response
FOLDER_PATH=$(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.folderPath')
UPLOAD_URIS=($(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.files[0].uploadURIs[]'))
UPLOAD_TOKEN=$(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.files[0].uploadToken')
MIME_TYPE=$(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.files[0].mimeType')
MIN_PART_SIZE=$(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.files[0].minPartSize')
MAX_PART_SIZE=$(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.files[0].maxPartSize')
COMPLETE_URI=$(echo "${INITIATE_UPLOAD_RESPONSE}" | jq -r '.completeURI')
# Extracting "Affinity-cookie" from the response headers
AFFINITY_COOKIE=$(echo "${INITIATE_UPLOAD_RESPONSE}" | grep -i 'Affinity-cookie' | awk '{print $2}')
debug "Folder Path: ${FOLDER_PATH}"
debug "Upload Token: ${UPLOAD_TOKEN}"
debug "MIME Type: ${MIME_TYPE}"
debug "Min Part Size: ${MIN_PART_SIZE}"
debug "Max Part Size: ${MAX_PART_SIZE}"
debug "Complete URI: ${COMPLETE_URI}"
debug "Affinity Cookie: ${AFFINITY_COOKIE}"
if $DEBUG; then
i=1
for UPLOAD_URI in "${UPLOAD_URIS[@]}"; do
debug "Upload URI $i: "$UPLOAD_URI
i=$((i+1))
done
fi
# Calculate the number of parts needed
NUM_PARTS=$(( (FILE_SIZE + MAX_PART_SIZE - 1) / MAX_PART_SIZE ))
debug "Number of Parts: $NUM_PARTS"
# Calculate the part size for the last chunk
LAST_PART_SIZE=$(( FILE_SIZE % MAX_PART_SIZE ))
if [ "${LAST_PART_SIZE}" -eq 0 ]; then
LAST_PART_SIZE=${MAX_PART_SIZE}
fi
# Step 4: Upload binary to the blob store in parts
PART_NUMBER=1
for i in $(seq 1 $NUM_PARTS); do
PART_SIZE=${MAX_PART_SIZE}
if [ ${PART_NUMBER} -eq ${NUM_PARTS} ]; then
PART_SIZE=${LAST_PART_SIZE}
debug "Last part size: ${PART_SIZE}"
fi
PART_FILE="/tmp/${FILE_NAME}_part${PART_NUMBER}"
# Creating part file
SKIP=$((PART_NUMBER - 1))
SKIP=$((MAX_PART_SIZE * SKIP))
dd if="${FILE_TO_UPLOAD}" of="${PART_FILE}" bs="${PART_SIZE}" skip="${SKIP}" count="${PART_SIZE}" iflag=skip_bytes,count_bytes > /dev/null 2>&1
debug "Creating part file: ${PART_FILE} with size ${PART_SIZE}, skipping first ${SKIP} bytes."
UPLOAD_URI=${UPLOAD_URIS[$PART_NUMBER-1]}
debug "Uploading part ${PART_NUMBER}..."
debug "Part Size: $PART_SIZE"
debug "Part File: ${PART_FILE}"
debug "Part File Size: $(stat -c %s "${PART_FILE}")"
debug "Upload URI: ${UPLOAD_URI}"
# Upload the part in the background
if command -v pv &> /dev/null; then
pv "${PART_FILE}" | curl --progress-bar -X PUT --data-binary "@-" "${UPLOAD_URI}" &
else
curl -# -X PUT --data-binary "@${PART_FILE}" "${UPLOAD_URI}" &
fi
PART_NUMBER=$((PART_NUMBER + 1))
done
# Wait for all background processes to finish
wait
# Step 5: Complete the upload in AEM
COMPLETE_UPLOAD_ENDPOINT="${AEM_URL}${COMPLETE_URI}"
debug "Completing the upload..."
debug "Complete Upload Endpoint: ${COMPLETE_UPLOAD_ENDPOINT}"
RESPONSE=$(curl -X POST \
-H "Authorization: Bearer ${BEARER_TOKEN}" \
-H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" \
-H "Affinity-cookie: ${AFFINITY_COOKIE}" \
--data-urlencode "uploadToken=${UPLOAD_TOKEN}" \
--data-urlencode "fileName=${FILE_NAME}" \
--data-urlencode "mimeType=${MIME_TYPE}" \
"${COMPLETE_UPLOAD_ENDPOINT}")
debug $RESPONSE
echo "File upload completed successfully."
Open-source upload library open-source-upload-library
To learn more about the upload algorithms or to build your own upload scripts and tools, ÃÛ¶¹ÊÓƵ provides open-source libraries and tools:
- .
- .
Deprecated asset upload APIs deprecated-asset-upload-api
The new upload method is supported only for ÃÛ¶¹ÊÓƵ Experience Manager as a Cloud Service. The APIs from ÃÛ¶¹ÊÓƵ Experience Manager 6.5 are deprecated. The methods related to upload or update assets or renditions (any binary upload) are deprecated in the following APIs:
- Experience Manager Assets HTTP API
AssetManager
Java API, likeAssetManager.createAsset(..)
,AssetManager.createAssetForBinary(..)
,AssetManager.getAssetForBinary(..)
,AssetManager.removeAssetForBinary(..)
,AssetManager.createOrUpdateAsset(..)
,AssetManager.createOrReplaceAsset(..)
Asset processing and post-processing workflows post-processing-workflows
In Experience Manager, the asset processing is based on Processing Profiles configuration that uses asset microservices. Processing does not require developer extensions.
For post-processing workflow configuration, use the standard workflows with extensions with custom steps.
Support of workflow steps in post-processing workflow post-processing-workflows-steps
If you upgrade from a previous version of Experience Manager, you can use asset microservices to process assets. The cloud-native asset microservices are simpler to configure and use. A few workflow steps used in the DAM Update Asset workflow in the previous version are not supported. For more information about supported classes, see the .
The following technical workflow models are either replaced by asset microservices or the support is not available:
com.day.cq.dam.cameraraw.process.CameraRawHandlingProcess
com.day.cq.dam.core.process.CommandLineProcess
com.day.cq.dam.pdfrasterizer.process.PdfRasterizerHandlingProcess
com.day.cq.dam.core.process.AddPropertyWorkflowProcess
com.day.cq.dam.core.process.CreateSubAssetsProcess
com.day.cq.dam.core.process.DownloadAssetProcess
com.day.cq.dam.word.process.ExtractImagesProcess
com.day.cq.dam.word.process.ExtractPlainProcess
com.day.cq.dam.ids.impl.process.IDSJobProcess
com.day.cq.dam.indd.process.INDDMediaExtractProcess
com.day.cq.dam.indd.process.INDDPageExtractProcess
com.day.cq.dam.core.impl.lightbox.LightboxUpdateAssetProcess
com.day.cq.dam.pim.impl.sourcing.upload.process.ProductAssetsUploadProcess
com.day.cq.dam.core.process.SendDownloadAssetEmailProcess
com.day.cq.dam.similaritysearch.internal.workflow.smarttags.StartTrainingProcess
com.day.cq.dam.similaritysearch.internal.workflow.smarttags.TransferTrainingDataProcess
com.day.cq.dam.switchengine.process.SwitchEngineHandlingProcess
com.day.cq.dam.core.process.GateKeeperProcess
com.day.cq.dam.s7dam.common.process.DMEncodeVideoWorkflowCompletedProcess
com.day.cq.dam.core.process.DeleteImagePreviewProcess
com.day.cq.dam.video.FFMpegTranscodeProcess
com.day.cq.dam.core.process.ThumbnailProcess
com.day.cq.dam.video.FFMpegThumbnailProcess
com.day.cq.dam.core.process.CreateWebEnabledImageProcess
com.day.cq.dam.core.process.CreatePdfPreviewProcess
com.day.cq.dam.s7dam.common.process.VideoUserUploadedThumbnailProcess
com.day.cq.dam.s7dam.common.process.VideoThumbnailDownloadProcess
com.day.cq.dam.s7dam.common.process.VideoProxyServiceProcess
com.day.cq.dam.scene7.impl.process.Scene7UploadProcess
com.day.cq.dam.s7dam.common.process.S7VideoThumbnailProcess
com.day.cq.dam.core.process.MetadataProcessorProcess
com.day.cq.dam.core.process.AssetOffloadingProcess
com.adobe.cq.dam.dm.process.workflow.DMImageProcess
See also