Using Bulk Import
The Bulk Import tool in AEM as a Cloud Service allows administrators to import assets in bulk from cloud storage in a safe and efficient manner.
Learn how to import numerous files into AEM Assets using the Asset view Bulk Import, with Dropbox serving as the example cloud storage provider for a clear and easy-to-follow integration process.
| embed |
|---|
|
|
| accordion |
|---|
| Transcript |
| AEM’s Bulk Import feature is ideal for transferring large volumes of assets into AEM. It’s perfect for migrations and occasional large-scale imports. These configurations can be saved, suitable for one time or repeated use. For instance, you might use one configuration for a single asset migration, and another for regular asset updates. Let’s set up a Bulk Import to bring in assets for new weekend adventures from our shared Dropbox. In AEM Assets, we already have several adventures, but now we want to add more. And here’s how we do it. In AEM Assets, go to Settings, Bulk Import. And here, we can either create a new import configuration, or modify an existing one. Since we don’t have any, let’s go ahead and create a new one. First, choose the cloud storage provider where the assets will be imported from. AEM supports a variety of cloud storage providers, but we’ll use Dropbox for this example. If you haven’t configured Dropbox already, stick around till the end of the video for guidance. And remember, cloud storage setup varies by provider, so check ÃÛ¶¹ÊÓƵ’s and the provider’s documentation. After selecting Dropbox, let’s name this import configuration. We’ll choose something descriptive, reflecting the content and the source. Next, enter the credentials for your Dropbox app account. You’ll find these in the Dropbox Developer Console for your app, under App Key for Client ID, and App Secret for the Client Secret. For Source Folder Name, enter the folder in Dropbox you wish to import from. This folder is relative to the root shared folder associated with your Dropbox app. If you leave this blank, AEM will import all folders and assets from the root of the shared Dropbox folder. In our case, let’s import everything from the Weekend Adventures app folder in Dropbox. Optionally, you can choose to delete the imported files from the cloud storage provider after import. And this can be useful for migrations or scheduled imports to keep the cloud storage clean. And let’s enable this option. Import mode determines the action if an imported asset already exists in AEM. We’ll just set this to replace to override any existing asset. You can also specify a CSV metadata file for updating asset metadata in AEM. This is optional, but particularly useful for migrations. We’ll skip it for now. Click next to authorize AEM’s access to Dropbox. And now we need to decide where in AEM assets to store the imported files. Let’s choose Weekend English Adventures, so the imported adventure folders live alongside our existing adventure assets. We can also set filters to manage what gets imported, like filtering by file path, size, or MIME type. So for instance, we can restrict and only import JPEG and PNG files by selecting those MIME types. And finally, we can save the import configuration. And we have three choices. Without immediate execution, or we can run it instantly, or we can run it later. With our new configuration saved under settings, bulk imports, and saved imports, let’s make sure the assets to import into AEM assets are in Dropbox. Typically the uploading of assets to Dropbox will be done by an agency, or some other user or process that doesn’t have access to AEM assets, since you can always upload files and folders into AEM assets directly. But for this video, we’ll head over to Dropbox, the apps folder in Dropbox, and then the shared Weekend Adventures folder. Drag and drop the desired adventure files here, and note that it’s important to organize your assets in the input source, ensuring the folder structure and naming is what you want in AEM assets, as these will be imported as is. Back in AEM, select our saved configuration for the Weekend Adventure Dropbox import, and we can validate our connection to make sure AEM and Dropbox can talk to one another. We can run the import directly, or what I like to usually do is do a dry run to preview the import details and see what’s going to be imported. After confirming the dry run details, we can proceed with the import. We see the status of our ongoing import at the top, and since we performed a dry run, we know that this import takes about 20 seconds. But of course, depending on the import and the number of files and file size being imported, this time will vary. Once completed, the import is listed under executed imports. The view assets option for the import will take you directly to the imported files in AEM assets. And there you go, you’ve successfully imported new adventure images into AEM assets from Dropbox, all organized and ready to use. Before we go, let me quickly guide you through the setting up of Dropbox for use with AEM assets’ bulk import feature. It’s important to note that the setup process varies for different cloud storage devices, so always consult the provider’s documentation. To set up Dropbox, start by logging into developer.dropbox.com and creating a new app. Choose scoped access, and then select the type of access you need. Typically, app folder is the best choice, as it offers controlled and limited access to your Dropbox. However, you can adjust this based on your specific requirements. Next, name your app. We’ll use Weekend Adventures for this demonstration, but keep in mind that this value, by default, is used as the shared folder in Dropbox under apps. However, you can always change this folder name in the next step if needed. With the app created, you’ll need to adjust a few settings. Note that your app is initially in development mode, and you’ll want to switch it to production status when you’re ready for production use, but for this video we’ll simply leave it in development mode. Under Permission Type, we’ll want to add the following permissions. We’ll want to make sure files metadata read, files content read, and files content write are all selected. The app folder name is where your imported assets will come from in Dropbox. By default, it’s the name of the app, which in our case is Weekend Adventures, but feel free to change this as needed. Take note of the app key and app secret. These are crucial for configuring the Dropbox bulk import configuration in AMAssets. And finally, set up the OAuth2 redirect URI. This is the URL Dropbox redirects to after authentication. Enter https://experience.adobe.com in this field. With these steps completed, your Dropbox is ready for connecting to AMAssets bulk import. Just remember to switch your app’s status to production for actual use. And this wraps up our video on AMAssets bulk import. I hope this video makes your asset management smoother and more efficient, and thanks for watching. |
| embed |
|---|
|
|
| accordion |
|---|
| Transcript |
|
AEM’s bulk import is a great way to move volumes of assets into AEM without the need for custom coding and is recommended for migrations and occasional bulk ingestions. Bulk import, available under tools, assets, bulk import lets you safely and efficiently import folders and assets from either Azure blob storage or Amazon S3 containers. Import configurations can be saved, making them great for both single or repeat use. For example, an import configuration could be used for a one-time bulk asset migration and another could be run repeatedly over time to periodically pull in new assets. Let’s create a bulk import to pull in assets from a back catalog of photo-shoots. First, we have to select which cloud storage provider we’ll import assets from, and provide the necessary coordinates and credentials. I’ve already uploaded some images to an Azure blob storage account organized by folder. So we have a root photo-shoot folder in the container and in that folder for a specific photo-shoot, and within that files is all sub folders and sub files. So I’ll add the information from my Azure blob storage container, which has the photo-shoot files to import. The source folder that you specify the path in the cloud storage from which to import assets. So let’s scope this to the containers photo-shoots folder. Next, we can optionally filter assets in or out of the import by file size. This can be helpful in avoiding importing junk files generated by operating systems or devices that might have accidentally been copied to the cloud storage. Similarly, MIME types can be used to include or exclude files to import. And again, this can be a useful way to help ensure we only import relevant files. The import mode defines the behavior of how the import should handle importing assets. If something already exists in AEM in the same location allowing import to be run repeatedly without ill effect on existing assets. And lastly, we need to specify which folder in AEM the assets will be imported into. Upon saving, the configuration displays in the list. Existing configurations can be edited which will pull up in the configuration form we just filled out or deleted completely. They can also be checked for validity which ensures the cloud storage credentials are valid and AEM is able to connect to the configured cloud storage. Perform a dry run, which provides a summary of what will be imported along with an estimate of how long will take. Note that we’re only importing a handful of assets to illustrate bulk import. But typically this capability is reserved for large bulk migrations or occasional bulk ingestions of assets. And finally run, which executes the import process. Once the import has been run, a new option of job status presents in the top bar, allowing you to see the details of the job. This tends to be most useful for checking on large imports or understanding if and why an import might’ve failed. The view assets option opens up the configured assets target folder. So we can quickly navigate and review the important assets. Note that the import process, not only ingests the files themselves, but also maintain the folder structure defined by a cloud storage keeping the import of assets tidy and organized. Imported assets, of course are processed by AEM in the usual manner, applying any configured processing profiles and presets to them. - |
| note tip |
|---|
| TIP |
| The input sources in this video only show Azure Blob Storage and Amazon S3; however the available sources continue to grow over time. For a complete list of supported input sources, please refer to the available options in product, or documentation. |
Schedule bulk import
Bulk Import supports scheduled execution of configurations, including:
- Single execution at a defined date and time
- Period executions every hour, day or week
